documentation.html (33081B)
1 {% extends "base.html" %} 2 {% set page_title = 'Documentation' %} 3 {% set page_description = 'Full endpoint reference for cdn.amit.is-a.dev - what each route does, what it needs, what it returns, and how it works.' %} 4 {% block page_class %}prose{% endblock %} 5 6 {% block content %} 7 <h1>Documentation</h1> 8 <p class="lede">Every route this service exposes: what it's for, what it needs, what it returns, what makes it fail, and 9 how it actually works under the hood. This page is generated from reading the live source, not from memory or 10 guesswork — if something here is wrong, the code is the tie-breaker, and this page should get fixed to match it.</p> 11 12 <div class="toc"> 13 <p class="toc-title">Jump to</p> 14 <ol> 15 <li><a href="#browsing">Browsing & viewing</a></li> 16 <li><a href="#gh-sync">GitHub Actions sync</a></li> 17 <li><a href="#uploading">Uploading files</a></li> 18 <li><a href="#shortener">URL shortener</a></li> 19 <li><a href="#rate-limits">Rate limits</a></li> 20 <li><a href="#downloading">Downloading & zips</a></li> 21 <li><a href="#batches">Multi-file batches</a></li> 22 <li><a href="#search">Search</a></li> 23 <li><a href="#stats">Stats</a></li> 24 <li><a href="#raw-domain">The raw domain</a></li> 25 <li><a href="#utility">Utility</a></li> 26 <li><a href="#architecture">How storage works</a></li> 27 </ol> 28 </div> 29 30 <section id="browsing"> 31 <h2>Browsing & viewing</h2> 32 33 <div class="endpoint"> 34 <div class="route-row"><span class="method get">GET</span><span class="route-path">/{path}</span></div> 35 <p>The catch-all route behind the whole browsing UI on <code>cdn.amit.is-a.dev</code>. What it renders depends on 36 what <code>path</code> resolves to:</p> 37 <table> 38 <tr><th>If path is...</th><th class="desc">Renders</th></tr> 39 <tr><td>empty (<code>/</code>)</td><td class="desc">Root folder listing, plus the homepage hero/notice.</td></tr> 40 <tr><td>an existing file</td><td class="desc">A file detail page (preview where possible, size, copy-link/copy-raw buttons, download).</td></tr> 41 <tr><td>an existing folder</td><td class="desc">A folder listing (files and sub-folders, with search-this-folder and "Download all" zip).</td></tr> 42 <tr><td>nonexistent</td><td class="desc">404 page.</td></tr> 43 </table> 44 <h3>How it decides</h3> 45 <p>It calls <code>get_file_info(path)</code> first (a HEAD request against the Hugging Face bucket's 46 <code>resolve</code> URL for that path). If that says the path exists as a file, it renders the file 47 page. Otherwise it calls <code>list_directory(path)</code> (a tree listing from the HF bucket API); a non-empty 48 result renders the folder listing, an empty result on a non-root path is a 404.</p> 49 <h3>Errors</h3> 50 <ul class="err-list"><li><span class="err-code">404</span> Path doesn't exist as a file or a non-empty folder.</li></ul> 51 </div> 52 53 <div class="endpoint"> 54 <div class="route-row"><span class="method get">GET</span><span class="route-path">/new</span> <span class="method get">GET</span><span class="route-path">/find</span></div> 55 <p><code>/new</code> renders the upload UI (drag-and-drop → calls <code>/api/put</code> under the hood). 56 <code>/find</code> renders the search UI (calls <code>/api/find</code> as you type). Both are just the same 57 <code>index.html</code> template rendered with a different <code>page</code> context value — no separate 58 backend logic beyond that.</p> 59 </div> 60 </section> 61 62 <section id="gh-sync"> 63 <h2>GitHub Actions sync</h2> 64 65 <div class="endpoint"> 66 <div class="route-row"><span class="method get">GET</span><span class="route-path">/gh-sync</span></div> 67 <p>Renders the human-readable setup guide for the feature below, including a copy-pasteable workflow file. 68 No parameters, no backend logic beyond rendering the template.</p> 69 </div> 70 71 <div class="endpoint" id="post-gh-sync"> 72 <div class="route-row"><span class="method post">POST</span><span class="route-path">/api/gh-sync</span></div> 73 <p>Lets a GitHub Actions workflow push a repo's files to the CDN on every push, landing at 74 <code>/<github-username>/<repo-name>/</code> — with no PAT or shared secret required. Full setup 75 instructions (including the exact workflow YAML to copy) are at <a href="/gh-sync">/gh-sync</a>.</p> 76 77 <h3>Request</h3> 78 <p><code>multipart/form-data</code> with two fields:</p> 79 <table> 80 <tr><th>Field</th><th class="desc">What it is</th></tr> 81 <tr><td><code>token</code></td><td class="desc">A GitHub Actions OIDC identity token, requested by the calling workflow via <code>core.getIDToken('cdn.amit.is-a.dev')</code>. This is the <em>only</em> source of identity this endpoint trusts.</td></tr> 82 <tr><td><code>archive</code></td><td class="desc">A <code>.tar.gz</code> of the repo's contents (the workflow builds this with <code>tar --exclude='.git' -czf ...</code>).</td></tr> 83 </table> 84 85 <h3>How it works, step by step</h3> 86 <ol style="color:var(--muted)"> 87 <li>Verifies the token's signature against GitHub's own public keys (<code>token.actions.githubusercontent.com/.well-known/jwks</code>), checks it isn't expired, and checks the <code>aud</code> claim matches this service.</li> 88 <li>Reads <code>repository_owner</code>, <code>repository</code> and the numeric <code>repository_owner_id</code> straight out of the <em>verified</em> token claims — this is the destination folder and the identity of the sender, and it is never taken from anything else in the request. A workflow file cannot forge these claims because GitHub itself stamps them for the run.</li> 89 <li>Refuses banned visitors and banned GitHub accounts, then applies the <strong>approval rule</strong>: the <em>first</em> sync from a GitHub account is accepted automatically; after that, the account must have been approved by the CDN owner or the sync is refused with <code>403</code>. Accounts are tracked by GitHub's numeric owner ID, so renaming an account or re-registering an old username does not bypass this. The CDN owner's own account is always allowed.</li> 90 91 <li>Rejects the archive with <code>413</code> if the compressed <code>.tar.gz</code> is larger than <strong>200 MB</strong>, before reading any of it into memory.</li> 92 <li>Extracts the tar archive into a temp directory, rejecting anything that tries to write outside that directory (path traversal) or that is a symlink.</li> 93 <li>Rejects the <strong>whole sync</strong> if any file is a video, detected by extension <em>and</em> by file signature (so renaming a video does not help). Nothing is uploaded in that case.</li> 94 <li>Uploads that folder to the Hugging Face storage bucket with the destination and the delete scope both limited to exactly <code><owner>/<repo>/</code> — so the call can create, update, or delete files only inside that one folder, never anywhere else. Files deleted from the source repo are removed from the CDN folder too (this is a mirror, not an accumulate-forever upload). The bucket keeps no version history, so those removals are permanent.</li> 95 </ol> 96 97 <h3>Response <code>200</code></h3> 98 <pre>{ 99 "synced_to": "jaasx009/College", 100 "cdn_url": "https://cdn.amit.is-a.dev/jaasx009/College/", 101 "raw_url_prefix": "https://raw.cdn.amit.is-a.dev/jaasx009/College/", 102 "note": "cdn_url is the browsable folder listing. raw_url_prefix isn't a link by itself - append an individual filename to it to get that file's direct raw URL.", 103 "ref": "refs/heads/main" 104 }</pre> 105 106 <h3>Errors</h3> 107 <ul class="err-list"> 108 <li><span class="err-code">401</span> Token missing, expired, badly signed, wrong audience, or its owner/repository claims don't line up (malformed token).</li> 109 <li><span class="err-code">400</span> Archive isn't a readable <code>.tar.gz</code>, or it contains an unsafe path / a symlink.</li> 110 <li><span class="err-code">403</span> The sender's IP or GitHub account is banned, <em>or</em> this account's first sync was already accepted and further syncs need approval from the CDN owner. The message says which.</li> 111 <li><span class="err-code">413</span> The compressed archive is larger than 200 MB. Nothing is synced.</li> 112 <li><span class="err-code">415</span> The archive contains a video file. The message names the file and nothing is synced.</li> 113 114 </ul> 115 116 <h3>Limits, currently</h3> 117 <p>Any file type except video, and no limit on the number of files. <strong>GitHub sync has no rate limit</strong>, but the compressed <code>.tar.gz</code> can be at most 118 <strong>200 MB</strong>. What protects it instead is the sign-in identity, the approval rule, bans and the video rule above. See <a href="#rate-limits">Rate limits</a>.</p> 119 </div> 120 </section> 121 122 <section id="uploading"> 123 <h2>Uploading files</h2> 124 125 <div class="endpoint"> 126 <div class="route-row"><span class="method post">POST</span><span class="route-path">/api/put</span></div> 127 <p>Direct multipart file upload — what the <a href="/new">/new</a> page's drag-and-drop calls. For uploading 128 from a GitHub repo automatically, use <a href="#post-gh-sync">/api/gh-sync</a> instead; this endpoint is for 129 one-off manual uploads.</p> 130 <h3>Request</h3> 131 <table> 132 <tr><th>Field</th><th class="desc">What it is</th></tr> 133 <tr><td><code>files</code></td><td class="desc">One or more files (multipart, repeatable field).</td></tr> 134 <tr><td><code>folder</code></td><td class="desc">Optional. Must be exactly <code>uploads</code> or <code>third-party</code> — these are the only two folders this endpoint is allowed to write to. Defaults to <code>uploads</code>.</td></tr> 135 </table> 136 <h3>How it works</h3> 137 <p>Each file is streamed to a temporary file named only by a random ID (your filename is never used in a filesystem path), 138 then checked in this order: banned IP, VPN/proxy address, video (by extension or file signature), and the 139 upload limit (200 MB per hour per client IP). It is then stored in the bucket at 140 <code><folder>/<8-char-uuid>-<filename></code>. Any filename is accepted: directory parts are dropped, 141 accents are simplified, and anything other than letters, digits, <code>.</code>, <code>_</code> and <code>-</code> 142 becomes an underscore (<code>my file (1).png</code> is stored as <code>ab12cd34-my_file_1.png</code>). 143 The random prefix means two uploads can never collide, and the filename portion is still readable. If 2 or more files were uploaded together, a <em>batch</em> is also created 144 (see <a href="#batches">Batches</a> below) so the whole set can be shared/downloaded as one link.</p> 145 <h3>Response <code>200</code></h3> 146 <pre>{ 147 "files": [ 148 {"filename": "photo.png", "hf_path": "uploads/ab12cd34-photo.png", 149 "size": 48213, "cdn_url": "...", "raw_url": "..."} 150 ], 151 "batch_id": "a1b2c3d4e5", // only present if 2+ files 152 "batch_url": "https://cdn.amit.is-a.dev/pack/a1b2c3d4e5" 153 }</pre> 154 <h3>Errors</h3> 155 <ul class="err-list"> 156 <li><span class="err-code">400</span> <code>folder</code> isn't <code>uploads</code> or <code>third-party</code>.</li> 157 <li><span class="err-code">403</span> Your IP is banned, or the upload comes from a VPN/proxy address.</li> 158 <li><span class="err-code">415</span> The file is a video (<code>.mp4</code>, <code>.mov</code>, <code>.webm</code>, <code>.mkv</code> and similar, or recognised by its contents). Videos are not accepted on this CDN.</li> 159 <li><span class="err-code">413</span> A single file is larger than 200 MB (waiting won't help; split it).</li> 160 <li><span class="err-code">429</span> You have used your 200 MB for the hour. See <a href="#rate-limits">Rate limits</a>.</li> 161 </ul> 162 </div> 163 164 <div class="endpoint"> 165 <div class="route-row"><span class="method post">POST</span><span class="route-path">/api/put-url</span></div> 166 <p>Uploads a file the server fetches from a URL you give it, instead of one you send directly. Same destination 167 rules and checks (ban, VPN/proxy, video) as <code>/api/put</code>, but with its own smaller allowance: <strong>50 MB per hour per IP</strong>, 168 separate from your file-upload allowance. The file name is taken from the URL or the 169 response's <code>Content-Disposition</code> and sanitised the same way.</p> 170 <h3>Request</h3> 171 <p>JSON body:</p> 172 <pre>{ "url": "https://example.com/file.png", "folder": "uploads" }</pre> 173 <h3>How it works / safety</h3> 174 <p>Only <code>https://</code> URLs are accepted. Before fetching, the hostname is resolved and every returned 175 IP is checked — private, loopback, link-local, reserved, multicast, or unspecified addresses are all 176 rejected. This is an SSRF guard: without it, this endpoint could otherwise be used to make the server fetch 177 internal/local network addresses on the caller's behalf. The download is streamed with a size cap enforced 178 both from the <code>Content-Length</code> header (if present) and live as bytes come in, so a server lying 179 about its own content-length can't bypass the cap.</p> 180 <h3>Response <code>200</code></h3> 181 <p>Same single-file shape as one file from <code>/api/put</code>'s response (no batch, since it's always exactly one file).</p> 182 <h3>Errors</h3> 183 <ul class="err-list"> 184 <li><span class="err-code">400</span> Not <code>https://</code>, hostname doesn't resolve, resolves to a private/internal address, or the source returned a non-200.</li> 185 <li><span class="err-code">403</span> Your IP is banned, or the request comes from a VPN/proxy address.</li> 186 <li><span class="err-code">413</span> The file is larger than 50 MB, the most that can be fetched from a link. The download is stopped as soon as it passes the limit.</li> 187 <li><span class="err-code">415</span> The downloaded file is a video.</li> 188 <li><span class="err-code">429</span> You have used your 50 MB for the hour for uploads from a link. See <a href="#rate-limits">Rate limits</a>.</li> 189 </ul> 190 </div> 191 </section> 192 193 <section id="shortener"> 194 <h2>URL shortener</h2> 195 196 <div class="endpoint"> 197 <div class="route-row"><span class="method get">GET</span><span class="route-path">/shorten</span></div> 198 <p>The page for creating short links. It calls the API below.</p> 199 </div> 200 201 <div class="endpoint"> 202 <div class="route-row"><span class="method post">POST</span><span class="route-path">/api/shorten</span></div> 203 <p>Turns a long <code>https://</code> URL into a short <code>/mask/<id></code> link on this domain.</p> 204 <h3>Request</h3> 205 <pre>{ "url": "https://example.com/some/long/page" }</pre> 206 <h3>How it works / safety</h3> 207 <p>Only <code>https://</code> URLs of at most 2048 characters are accepted. The hostname is resolved and every 208 address is checked — private, loopback, link-local and other internal addresses are rejected, the same SSRF 209 guard used by <code>/api/put-url</code>. Each client IP is limited to <strong>10 requests per minute and 50 per hour</strong>; 210 the limit is counted before the hostname lookup, so rejected attempts count too. The destination is stored as a small 211 object in the bucket under a random 8-character ID.</p> 212 <h3>Response <code>200</code></h3> 213 <pre>{ 214 "id": "abc123XY", 215 "masked_url": "https://cdn.amit.is-a.dev/mask/abc123XY" 216 }</pre> 217 <h3>Errors</h3> 218 <ul class="err-list"> 219 <li><span class="err-code">400</span> Not <code>https://</code>, longer than 2048 characters, invalid, host doesn't resolve, or it points to a private/internal address.</li> 220 <li><span class="err-code">403</span> Your IP is banned.</li> 221 <li><span class="err-code">429</span> More than 10 requests in a minute or 50 in an hour from your IP. See <a href="#rate-limits">Rate limits</a>.</li> 222 </ul> 223 </div> 224 225 <div class="endpoint"> 226 <div class="route-row"><span class="method get">GET</span><span class="route-path">/mask/{short_id}</span></div> 227 <p>Redirects to the stored destination with a <code>307</code> and a <code>Referrer-Policy: no-referrer</code> header, so the 228 destination site doesn't see which page the visitor came from.</p> 229 <h3>Errors</h3> 230 <ul class="err-list"> 231 <li><span class="err-code">404</span> No short link with that ID.</li> 232 </ul> 233 </div> 234 </section> 235 236 <section id="rate-limits"> 237 <h2>Rate limits</h2> 238 <p class="lede">Limits keep the storage free for everyone and stop one visitor from filling it. They apply only to things that 239 <em>write</em>: uploading files, uploading from a link, and shortening a link. Browsing, search, viewing, downloads, zips and 240 GitHub sync are not rate-limited by this app.</p> 241 242 <h3>At a glance</h3> 243 <table> 244 <tr><th>What</th><th class="desc">Limit</th></tr> 245 <tr><td style="white-space:nowrap"><code>/api/put</code></td><td class="desc"><strong>200 MB per hour</strong>, counted in bytes, per IP. No per-minute limit. A single file can be at most 200 MB.</td></tr> 246 <tr><td style="white-space:nowrap"><code>/api/put-url</code></td><td class="desc"><strong>50 MB per hour</strong>, counted in bytes, per IP. This is a separate allowance from file uploads. A single file can be at most 50 MB.</td></tr> 247 <tr><td style="white-space:nowrap"><code>/api/shorten</code></td><td class="desc"><strong>10 per minute</strong> and <strong>50 per hour</strong>, counted in requests, per IP.</td></tr> 248 <tr><td style="white-space:nowrap"><code>/api/gh-sync</code></td><td class="desc"><strong>No rate limit.</strong> The compressed archive can be at most 200 MB.</td></tr> 249 </table> 250 251 <h3>How the limits work</h3> 252 <ul style="color:var(--muted)"> 253 <li><strong>Sliding window.</strong> Your usage is totalled over the <em>last 60 minutes</em> (the shortener also looks at the last 60 seconds). There is no clock-time reset: old uploads age out one by one, so your allowance returns gradually.</li> 254 <li><strong>Per IP.</strong> The limit follows the address your request really comes from. People on the same network (office, school, mobile carrier) share one allowance. Uploads from VPN or proxy addresses are refused outright, whatever your usage.</li> 255 <li><strong>Bytes, not files.</strong> Ten 1 MB files use the same allowance as one 10 MB file. Each file is checked when it arrives.</li> 256 <li><strong>Two separate allowances.</strong> Uploading from a link never uses up your 200 MB of file uploads, and file uploads never use up your 50 MB for links.</li> 257 <li><strong>Biggest single file.</strong> 200 MB for file uploads and 50 MB from a link. A file over that is refused with <code>413</code> rather than <code>429</code>, because waiting would not help. From a link, the download stops as soon as it passes 50 MB.</li> 258 <li><strong>Batches stop at the limit.</strong> If you upload several files in one request and the limit is reached partway, the request fails with <code>429</code>. Files already stored earlier in that request stay stored.</li> 259 <li><strong>What does not count.</strong> Uploads refused before the limit check (banned IP, VPN/proxy address, video) do not use your allowance. Shortener requests <em>do</em> count even when the URL turns out to be invalid, because the limit is checked before the link is examined.</li> 260 <li><strong>GitHub sync is not rate-limited,</strong> but the compressed archive can be at most 200 MB. It is protected by the sign-in identity, the approval rule, bans and the video rule (see <a href="#gh-sync">GitHub Actions sync</a>).</li> 261 <li><strong>In memory.</strong> Counters are kept in the server's memory, so a restart or redeploy clears them.</li> 262 263 </ul> 264 265 <h3>When you hit a limit</h3> 266 <p>The response is <code>429</code> (or <code>413</code> for a file that is too big) with a JSON message. There is no <code>Retry-After</code> header, so use the message to see which limit you hit:</p> 267 <pre>{ "detail": "Upload rate limit exceeded: 200.00 MB/hour. Try again later." } 268 { "detail": "URL upload rate limit exceeded: 50.00 MB/hour. Try again later." } 269 { "detail": "Rate limit exceeded: 10 requests/minute. Try again shortly." } 270 { "detail": "Rate limit exceeded: 50 requests/hour. Try again later." } 271 { "detail": "File too large (limit 200.00 MB per file)." } 272 { "detail": "Archive too large (limit 200.00 MB after compression)." }</pre> 273 <p>After a <em>minute</em> message, wait up to a minute. After an <em>hour</em> message, wait until earlier uploads are an hour old. For <em>file too large</em>, waiting doesn't help: make the file smaller or split it.</p> 274 275 <h3>Staying under the limits</h3> 276 <ul style="color:var(--muted)"> 277 <li>Upload big batches in smaller groups, and leave time between them if you are close to 200 MB in an hour.</li> 278 <li>Compress or resize large files before uploading.</li> 279 <li>Have a whole project to move? <a href="#gh-sync">GitHub sync</a> has no rate limit.</li> 280 </ul> 281 </section> 282 283 <section id="downloading"> 284 <h2>Downloading & zips</h2> 285 286 <div class="endpoint"> 287 <div class="route-row"><span class="method get">GET</span><span class="route-path">/api/get/{path}</span></div> 288 <p>Forces a browser download of a single file (sends <code>Content-Disposition: attachment</code>), unlike 289 viewing the raw domain which lets the browser render the file inline if it can. Streams the file through 290 this server rather than redirecting to Hugging Face directly, and supports <code>Range</code> requests 291 (so video/audio seeking and resumable downloads work).</p> 292 <h3>Errors</h3> 293 <ul class="err-list"><li><span class="err-code">404</span> File doesn't exist.</li></ul> 294 </div> 295 296 <div class="endpoint"> 297 <div class="route-row"><span class="method get">GET</span><span class="route-path">/api/pack-info/{path}</span> <span class="method get">GET</span><span class="route-path">/api/pack/{path}</span></div> 298 <p><code>pack-info</code> returns the file count and total size a folder would zip to, without actually 299 building the zip — used by the UI to show a size estimate before you commit to downloading. <code>pack</code> 300 builds and streams the actual zip.</p> 301 <h3>How it works</h3> 302 <p>Lists every file under <code>path</code> recursively via the HF API, fetches up to 8 of them concurrently, 303 and streams them into a zip in memory (paths inside the zip are relative to the folder you asked for, not 304 absolute). There's a hard limit of <strong>300 files / 200 MB total</strong> per zip — past that it 305 refuses rather than trying to build an enormous zip in memory.</p> 306 <h3>Errors</h3> 307 <ul class="err-list"> 308 <li><span class="err-code">404</span> Folder is empty or doesn't exist.</li> 309 <li><span class="err-code">413</span> Folder exceeds the 300-file / 200 MB zip limit.</li> 310 </ul> 311 </div> 312 </section> 313 314 <section id="batches"> 315 <h2>Multi-file batches</h2> 316 <p>When <a href="#uploading">/api/put</a> is given 2 or more files in one request, it also writes a small 317 "batch manifest" (just the list of resulting files) under a random 10-character batch ID, so that whole set 318 can be shared and managed as a single link instead of separate ones.</p> 319 <div class="endpoint"> 320 <div class="route-row"><span class="method get">GET</span><span class="route-path">/pack/{batch_id}</span></div> 321 <p>Renders a page listing every file in that batch, with per-file and "download all as zip" options.</p> 322 <h3>Errors</h3><ul class="err-list"><li><span class="err-code">404</span> No batch with that ID.</li></ul> 323 </div> 324 <div class="endpoint"> 325 <div class="route-row"><span class="method get">GET</span><span class="route-path">/api/pack-info-batch/{batch_id}</span> <span class="method get">GET</span><span class="route-path">/api/pack-batch/{batch_id}</span></div> 326 <p>Same idea as <code>/api/pack-info</code> / <code>/api/pack</code> above, but operating over the batch's file 327 list instead of a folder path. No 300-file/500MB check here since a batch is inherently one upload's worth 328 of files, not an arbitrary folder walk.</p> 329 <h3>Errors</h3> 330 <ul class="err-list"><li><span class="err-code">404</span> Batch doesn't exist, or exists but is empty.</li></ul> 331 </div> 332 </section> 333 334 <section id="search"> 335 <h2>Search</h2> 336 <div class="endpoint"> 337 <div class="route-row"><span class="method get">GET</span><span class="route-path">/api/find?q=...</span></div> 338 <p>Powers the <a href="/find">/find</a> page. Takes a query string <code>q</code>, returns matching files 339 across the whole repo.</p> 340 <h3>Response <code>200</code></h3> 341 <pre>{ "results": [ { "...": "file entries" } ] }</pre> 342 <p>An empty or missing <code>q</code> is valid input, not an error — it just returns however 343 <code>search_files("")</code> resolves that (effectively little/nothing to match against, no exception thrown).</p> 344 </div> 345 </section> 346 347 <section id="stats"> 348 <h2>Stats</h2> 349 350 <div class="endpoint"> 351 <div class="route-row"><span class="method get">GET</span><span class="route-path">/stats</span></div> 352 <p>A page of charts about what is on the CDN: file types (by count or by size), the most common extensions, a 353 file-size histogram, the biggest folders, files added per month, and the largest files. It loads its data from 354 <code>/api/stats</code> and draws everything with plain CSS and inline SVG, so the page stays tiny.</p> 355 </div> 356 357 <div class="endpoint"> 358 <div class="route-row"><span class="method get">GET</span><span class="route-path">/api/stats</span></div> 359 <p>The JSON behind the page. It contains aggregate numbers only — no IP addresses and no upload history. It is 360 computed from the same cached file listing used for browsing and search and recomputed at most every 10 minutes, so 361 calling it never adds load on the bucket. The response is sent with <code>Cache-Control: public, max-age=300</code>.</p> 362 <h3>Response <code>200</code></h3> 363 <pre>{ 364 "empty": false, 365 "generated": 1791270000, // unix time 366 "totals": { "files": 900, "bytes": 1060758477, "avg": 1178620, "median": 171280, "types": 14 }, 367 "categories": [ { "key": "images", "label": "Images", "files": 421, "bytes": 123456 } ], 368 "extensions": [ { "ext": "png", "files": 236, "bytes": 49283712 } ], // top 10 by count 369 "extensions_more": 72, // files in rarer types 370 "sizes": [ { "label": "< 10 KB", "files": 129 } ], 371 "folders": [ { "name": "uploads", "files": 375, "bytes": 564133888 } ], // top 8 by size 372 "largest": [ { "name": "a.zip", "path": "uploads/ab12cd34-a.zip", "size": 45088768 } ], 373 "months": [ { "m": "2026-09", "files": 71, "bytes": 1234567 } ] // last 12 months 374 }</pre> 375 <p>If the file listing can't be read, it returns the last good result, or <code>{ "empty": true }</code> if there is none yet.</p> 376 </div> 377 </section> 378 379 <section id="raw-domain"> 380 <h2>The raw domain</h2> 381 <div class="endpoint"> 382 <div class="route-row"><span class="method get">GET</span><span class="route-path">/{path}</span> <span class="domain-tag">raw.cdn.amit.is-a.dev</span></div> 383 <p>A separate hostname, not a path prefix on the main domain. Requests here are proxied straight through to 384 the underlying file in the Hugging Face bucket (its <code>resolve</code> URL), with no HTML wrapper — this is 385 meant for direct file links (<code><img src></code>, downloads, embeds), not browsing.</p> 386 <h3>If the path is a folder, not a file</h3> 387 <p>The direct file fetch 404s (Hugging Face's <code>resolve</code> only serves blobs). Before giving up, this 388 checks whether that path is actually a non-empty folder — if so, it 307-redirects to the equivalent listing 389 page on the main domain (<code>cdn.amit.is-a.dev/<path>/</code>) instead of just saying not-found. 390 A genuinely nonexistent path still 404s.</p> 391 <h3>Errors</h3> 392 <ul class="err-list"><li><span class="err-code">404</span> Path doesn't exist as a file or a non-empty folder. Response body is plain text, not the HTML 404 page (deliberately - this is meant to be machine/embed-friendly, not a browsing experience).</li></ul> 393 </div> 394 <div class="endpoint"> 395 <div class="route-row"><span class="method get">GET</span><span class="route-path">/raw/{path}</span> <span class="domain-tag">cdn.amit.is-a.dev</span></div> 396 <p>A convenience prefix on the <em>main</em> domain that just 307-redirects to the real raw domain — i.e. 397 <code>cdn.amit.is-a.dev/raw/foo.png</code> → <code>raw.cdn.amit.is-a.dev/foo.png</code>. Not itself a proxy.</p> 398 </div> 399 </section> 400 401 <section id="utility"> 402 <h2>Utility</h2> 403 <div class="endpoint"> 404 <div class="route-row"><span class="method get">GET</span><span class="route-path">/ping</span></div> 405 <p>Health check. Always returns <code>200 "Server is awake!"</code> as plain text. Used to keep the instance 406 warm on host platforms that spin down idle services — has no other side effects.</p> 407 </div> 408 <div class="endpoint"> 409 <div class="route-row"><span class="method get">GET</span><span class="route-path">/static/{filename}</span> <span class="method get">GET</span><span class="route-path">/static/icons/{filename}</span> <span class="method get">GET</span><span class="route-path">/favicon.ico</span></div> 410 <p>Serve the site's own static assets (CSS-adjacent files, icons, favicon) straight off disk. Not related to 411 anything stored in the CDN's file repo — these ship with the app itself.</p> 412 </div> 413 </section> 414 415 <section id="architecture"> 416 <h2>How storage actually works</h2> 417 <p class="lede">Everything above talks about "the CDN" as one thing, but there's no custom database or file 418 server behind it — every file this service serves, lists, or accepts lives in a single 419 <strong>Hugging Face Storage Bucket</strong>. A bucket is plain object storage, not a git repository, so there is no 420 commit history that grows as files pile up. This app is a FastAPI backend in front of that bucket:</p> 421 <ul style="color:var(--muted)"> 422 <li>File listings (folders, search, file-exists checks, stats) come from the bucket's tree API, with a short-lived in-memory cache on this server's side so repeated browsing doesn't re-hit that API constantly.</li> 423 <li>Serving an actual file's bytes (viewing, raw links, downloads, zips) streams from the bucket's public <code>resolve</code> URL — this server proxies that stream rather than storing a separate copy.</li> 424 <li>Uploads (<code>/api/put</code>, <code>/api/put-url</code>, <code>/api/gh-sync</code>) all ultimately call the Hugging Face bucket API to write objects into that same bucket, each scoped to whatever folder/path rules that endpoint enforces.</li> 425 <li>There is one HF write token, held only by this server - never handed to uploaders, contributors, or GitHub Actions runs. Everything that writes to storage goes through this backend's own validation first (folder allow-lists, OIDC identity checks, bans, rate limits) precisely because the write credential itself has no per-user restriction built in - the backend <em>is</em> the access control.</li> 426 <li><strong>No version history.</strong> Overwritten or deleted files can't be restored from history, and a GitHub sync that removes files from a folder removes them for good.</li> 427 <li>A few internal folders live in the same bucket (batch records, short links and the site's own moderation data). They are hidden from listings and search, and the moderation data is encrypted.</li> 428 </ul> 429 </section> 430 431 <p class="footnote">If something on this page doesn't match what the service actually does, trust the code 432 (<code>app/main.py</code>, <code>app/storage.py</code>, <code>app/gh_oidc.py</code> in 433 <a href="https://github.com/notamitgamer/cdnV2">notamitgamer/cdnV2</a>) over this page, and open an issue/PR 434 so this page gets corrected.</p> 435 {% endblock %}