Monitoring US federal data without building a scraper you have to babysit

US federal agencies publish a lot of structured data, and most of it is already available through clean JSON APIs. openFDA, the FCC's ECFS system, USAspending.gov, the Federal Register, ClinicalTrials.gov, and the DOJ FARA database all return real records over HTTP. The one-time pull is the easy part. You read the endpoint, parse the response, write the rows. An afternoon of work.

The upkeep is where the cost shows up. Agencies page their results, so you have to walk every page or you silently miss records. They reissue records, so you need dedupe keyed on a stable identity, not on a row hash that flips every time a field is normalized. You only want what is new since your last run, which means tracking state between runs. And when someone downstream asks where a number came from, you need a source URL for every row, not a vague note that says "from openFDA." None of that is hard. It is just steady maintenance that you own forever once you start.

The pattern

The approach that holds up is small. Read the official API and nothing else, so values are never guessed or interpolated. Return one flat, typed row per record, with a source URL on that row so any reader can verify it against the agency. Then offer a "what changed since last run" mode that keys on record identity, so a scheduled run returns only new or updated records and does not hand you the same record twice.

We packaged this pattern as a set of Apify actors against US public records. Each one reads only the official API, returns a typed schema, puts a verifiable source URL on every row, and supports the changes-since-last-run mode. They are pay-per-delivered-record with no start fee, and a miss or an error costs nothing.

FDA food recalls

FDA Food Recalls reads the openFDA Food Enforcement API. You can filter by recall class (I, II, or III), firm, state, and date. A realistic use is a daily run filtered to Class I recalls in your distribution states, so a quality team sees the serious ones the morning they post. In changes mode, a scheduled run only returns recalls that are new or updated since the last run.

FCC filings

FCC Filings reads the FCC ECFS API. You can search by docket or proceeding, by filer, or by full text, with exact docket matching. A common use is watching a single proceeding for new comments and replies. The changes mode lets a scheduled run pick up only the filings added since you last checked, instead of re-pulling the whole docket.

USAspending federal awards

USAspending Federal Awards reads the official USAspending.gov API and returns federal award records. You might track awards to a specific recipient or within an agency to follow contract activity over time. Run it on a schedule in changes mode and you get the newly posted or revised awards, not a fresh copy of records you already have.

Federal Register rules

Federal Register rules reads the Federal Register API and returns new rules and notices, filterable by agency and CFR reference. A compliance reader can watch the agencies and CFR parts that touch their work. The changes mode means a daily run surfaces only documents published since the previous run.

ClinicalTrials.gov

ClinicalTrials.gov reads the ClinicalTrials.gov v2 API. You can filter by condition, sponsor, and status. One use is following trials for a condition and getting notified when a status changes. The changes mode keys on the record so a scheduled run returns new and updated trials only.

DOJ FARA

DOJ FARA returns DOJ FARA records linking registrants to foreign principals, with country and dates. A researcher or journalist can track which registrants represent which foreign principals as filings come in. In changes mode, a run returns only the records added or updated since last time.

Why provenance matters

If an analyst or an automated agent acts on a row, the first question is where it came from. A source URL on every row answers that without a second tool. The reader can open the link and confirm the value against the agency that published it. Because each actor reads only the official API and never fills in a guessed value, what you get back is what the agency said, and you can prove it. The typed schema makes the same row safe to hand to an AI agent that needs predictable fields.

Close

These are not subscriptions. You pay per delivered record, with no start fee, and a miss or an error costs nothing. The changes-since-last-run mode keys on record identity, so a scheduled run does not re-bill you for records you already have. If you have been meaning to monitor a federal source and did not want to own a scraper, that is the gap these fill.