docs

Log | Files | Refs | Activity

root / src / osma / index.md

index.md (2238B)


      1 ---
      2 label: OSMA
      3 icon: package
      4 order: 550
      5 ---
      6 
      7 # OSMA (Open Source Module Archive)
      8 
      9 **Registry lookups, without the wait.** OSMA is a lightning-fast, static search archive for querying open-source packages across NPM and PyPI, bypassing live registry bottlenecks.
     10 
     11 ## Why it exists
     12 
     13 Looking up package details on the official registries usually means dealing with:
     14 
     15 1. **Fastly security checks** — CAPTCHAs or verification screens just to view a text page
     16 2. **The multi-click tax** — clicking through 2-3 pages just to find a version tag or repo link
     17 
     18 **The fix:** complete static snapshots of both registries were taken — **NPM at 2026-04-29 09:42 IST**, **PyPI at 2026-04-29 08:45 IST** — and indexed into a custom backend. This lets you instantly search and verify packages without ever hitting a live registry rate limit. Clicking a package in the UI still fetches its extended live details securely via a side-panel.
     19 
     20 ## Live explorers & docs
     21 
     22 - **Home:** [data.amit.is-a.dev](https://data.amit.is-a.dev)
     23 - **NPM Explorer:** [data.amit.is-a.dev/npm](https://data.amit.is-a.dev/npm)
     24 - **PyPI Explorer:** [data.amit.is-a.dev/pypi](https://data.amit.is-a.dev/pypi)
     25 - **API Docs (live):** [data.amit.is-a.dev/docs](https://data.amit.is-a.dev/docs)
     26 
     27 ## How the backend works
     28 
     29 The backend runs on **FastAPI**, serving static `.csv` snapshots pulled securely from HuggingFace datasets.
     30 
     31 - **SQLite indexing** — on first boot, the server downloads the multi-gigabyte dataset and builds an optimized, indexed SQLite database (WAL mode)
     32 - **Speed** — because the dataset is localized, `/search` and `/browse` respond in milliseconds
     33 - **Rate limiting** — strict limits (10 requests/minute, 100/hour) protect the free-tier infrastructure, enforced via `slowapi`; developers can request an `X-Bypass-Token` via a GitHub issue
     34 - **Ranking logic** — search results are ranked `0` exact match → `1` starts with query → `2` contains query
     35 - **Limitation** — as a frozen archive, packages published or deleted after April 2026 aren't indexed
     36 
     37 ## Get started
     38 
     39 - [Quick Start](quickstart.md)
     40 - [NPM API Reference](npm-api.md)
     41 - [PyPI API Reference](pypi-api.md)
     42 - [Utility Endpoints](utility.md)
     43 - [Notes, Limits & FAQ](notes.md)
© notamitgamer • Site Built: 2026-09-05 01:53:16 UTC • git-mirror commit: c170d72 [view raw info]
Originally created with stagit • modified by notamitgamer
Forked from github.com/notamitgamer/git-mirror