Collector healthSources
Each source is an independent adapter returning the same NormalizedItem shape. Last collection: 05/11/2026, 03:23:01 UTC.
GitHubDEGRADED
Trending page for real 24h star deltas + Search API for discovery and enrichment.
7 of 23 search requests failed (HTTP 403: API rate limit exceeded for user ID 290997070. If you reach out to GitHub Support for help, please include the request ID E83E:D7253:323A022:A51D6ED:6AEBF623 and timestamp 2026-11-05 03:21:08 UTC. For more on scraping GitHub and how it may affect your rights, please review our Terms of Service (https://docs.github.com/en/site-policy/github-terms/github-terms-of-service)). The token is fine — this is the quota. Search allows 30 requests a minute per user, and these already go out one at a time with a retry, so hitting it means two collections overlapped: a scheduled run and a page revalidation, most likely.
Hugging FaceACTIVE
Public API: trending models and spaces, ranked by likes and downloads.
Hacker NewsACTIVE
Algolia API: front page, Show HN launches and domain-targeted stories.
PapersACTIVE
arXiv research via Hugging Face daily papers — upvotes, comments and linked code.
npmACTIVE
Registry search — weekly downloads, the one adoption metric here. Merges with the package’s repo.
TrustMRRACTIVE
Verified MRR and revenue, read from the site’s own page payload — there is no API, so their markup is the contract.
Product HuntDEGRADED
A week of launches, ranked by votes — the one source where the metric is a human verdict rather than a counter, and the only one with no anonymous read at all.
Stopped after 2 of 12 pages (HTTP 429: Too Many Requests). This is the quota, not the token: an application gets 6250 complexity points per 15 minutes, and one page of launches costs about 240. Two overlapping collections is the usual cause.
PyPIACTIVE
Enrichment, not discovery — PyPI has no search, so this only looks up package names another source already found, and attaches their downloads.
Two different situations, kept apart on purpose. The first group is waiting on us; the second is waiting on somebody else, and no amount of our work moves it.
Nothing in the way but the work
Replicate
Public API and a free token — it answers 401 unauthenticated, nothing worse. Trending and recently-updated models, with run counts as the adoption metric. Only the adapter is missing.
Reddit
Free OAuth app with limits a six-hourly run never approaches. r/LocalLLaMA and its neighbours carry launch chatter that never reaches GitHub.
YouTube
Data API v3 gives 10,000 free units a day and a search costs 100 — roughly a hundred searches, far more than we need. Wants a Google Cloud key.
Bluesky
Free and open, though its search refuses anonymous reads (403) — an app password is enough. The closest free stand-in for the signal X used to give us.
Google Trends
No official API, but the trending RSS feed answers without a key. It reports what a country is searching for, not interest in a direction we choose, so it would be a coarse background signal rather than a direction of its own.
Closed from the outside
X
Search sits on the paid tiers; the free one is built for posting, not reading. Anonymous scraping closed when guest tokens went away, so what remains needs a logged-in session — against their terms, and far too brittle to hang a scheduled collector on.
TikTok
No open discovery API. The Research API is approval-gated to academic institutions in the US and EU, and the Display API reaches only your own account's posts.
LinkedIn
No public search API at all — everything useful sits behind partner programmes you must apply to and be accepted for, and LinkedIn has taken scrapers to court. Closed to us in practice.
