Methodology

How sourc.dev tracks, verifies, and grades the AI ecosystem. Every claim on this platform is sourced, timestamped, and linked to its origin.

What we track

Entities are individual products in the AI ecosystem: models (LLMs), developer tools, SaaS products, APIs, and frameworks. Each entity has a canonical name, a provider (the company that makes it), a domain classification, and a set of verified attributes.

Companies (Providers) are the organisations that build and operate entities. A single provider can own multiple entities across multiple domains. Provider data includes headquarters, funding, compliance certifications, and key personnel.

Relations are verified integration links between entities. Every relation requires a signal URL (public proof), a signal date, a relation type, and an integration use case. A relation without evidence does not exist in our index.

Source tiers

Tier S

Vendor-direct, machine-verified

Data extracted directly from the vendor's official API or pricing page by our automated pipeline. Verified by hash comparison — if the page changes, we detect and re-verify.

Tier 1

Vendor-direct, human-verified

Data sourced from the vendor's official documentation, press releases, or trust portals. Verified by an operator or the Chrome-agent with screenshot evidence.

Tier 2

Aggregator, cross-referenced

Data from trusted aggregators (OpenRouter, Hugging Face, GitHub) cross-referenced against at least one additional source. Lower confidence, flagged for periodic re-verification.

Quality grades

Every entity receives a quality grade based on four dimensions: completeness, accuracy, freshness, and relation density.

AComplete data, all P1 attributes verified, multiple sources, recent verification. Institutional-grade.

BMostly complete, minor gaps in P2 attributes. All P1 attributes present. Verified within 90 days.

CPartial data. Some P1 attributes missing. Verification may be older than 90 days.

DSparse data. Fewer than half of P1 attributes present. Limited source verification.

FStub only. Entity exists in the index but has minimal verified data. Awaiting enrichment.

How prices are verified

Prices are extracted directly from vendor pricing pages using our automated pipeline and Chrome-agent. Every price data point includes:

The exact source URL where the price was observed
A timestamp of when it was verified
The raw price and unit as displayed by the vendor
Normalisation to per-1M tokens (for LLMs) for cross-comparison

Price history is append-only. When a price changes, the old value remains in the history with its original timestamp. This creates a complete audit trail of price movements over time.

Append-only data model

The attribute history table is append-only. No row is ever updated or deleted. Every data point ever collected remains in the history with its original source, timestamp, and confidence level.

This design means:

Complete audit trail — every change is traceable
No data loss — corrections are additive, not destructive
Version history — any entity can be reconstructed at any past point in time
Dispute resolution — conflicting values are both preserved with their sources

What we do not track (yet)

sourc.dev focuses on publicly verifiable data. We do not currently track:

Revenue — private companies do not disclose revenue
Headcount — employee counts are estimates unless officially disclosed
Internal roadmaps — we track what ships, not what is planned
Subjective quality — benchmarks are reported as-is, we do not editorialize
User sentiment — reviews and ratings are not verified data