ERCOT API vs file downloads for market data pipelines

ERCOT offers more than one way to obtain market data, and the API isn't automatically the best source for every pipeline.

The right ingestion method depends on what matters most for a given dataset: latency, data volume, selective filtering, historical reconstruction, reproducibility, or plain operational simplicity. A sound architecture may use different mechanisms for different workflows — even for the same product.

Where this guide sits in the pipeline

1. Source
2. Fetch
3. Raw copy
4. Validate
5. Verify completeness
6. Normalize
7. Promote
8. Client system

The options

• Public API queries — filtered requests against report endpoints (for example, NP6-788-CD by SCED timestamp or settlement point), returned as paginated JSON.
• Published report files — the zipped report artifacts ERCOT posts for each product on ercot.com. The Public API also lists them per product through its archive endpoint, /archive/{emilId}, in ERCOT's API specification.
• EWS — ERCOT's SOAP-based web services, which ERCOT staff have pointed to for some time-sensitive needs. Check ERCOT's documentation for its access requirements.

How they compare

ERCOT products differ. Source selection should be evaluated per dataset, not decided once for the whole pipeline.

Selective filtering

This is the API's strength: one settlement point, one hour, one set of fields. A report file usually gives you everything the product publishes for its period, and you filter locally.

Bulk retrieval

Paging through everything a high-volume product publishes, one API query at a time, is the pattern that ran into ERCOT's bandwidth-based limits in March 2026 (Discussion #132). If you need the whole product anyway, a file is often the more natural unit of work: one artifact per publication, with no page boundaries to track. It is still data volume, though, so it still has to fit whatever limits apply to the channel you download it from.

Raw artifacts and reproducibility

A downloaded file is its own raw record: you can checksum it, store it and reprocess it later. API responses can serve the same purpose, but only if you save them explicitly before parsing (the Python guide shows how).

Latency

For NP6-788-CD in August 2025, ERCOT staff described a delay of approximately 5–10 minutes between posting on ercot.com and availability in the Public API, and suggested downloading the report files from ercot.com in the meantime (Discussion #104). ERCOT has since described releases planned to shrink that gap (Discussion #68). Both facts are specific to a product and a point in time — see why ERCOT API data can lag behind ercot.com — and neither means files are always faster.

Historical workflows

How far back each channel goes, and how older data is requested, varies by product. A June 2026 question on ERCOT's board about historical backfills of several Secure EWS products (Discussion #142) had no replies when we reviewed it, which is itself a reminder to verify retention and request windows per product before designing around them.

A hybrid strategy

Both paths should land in the same normalized schema, with the source recorded for every batch. Then you can move a product from API to files, or back, without changing anything downstream — which matters, because the best choice for a product can change when ERCOT changes its systems.

Record where every batch came from

Whichever channel you use, store which one produced each batch, along with the artifact or request behind it. When two sources disagree — a republished report, say, or an API query run before processing finished — you need to know which one each row came from, and when.

Practical takeaway

• ✓Choose the source per dataset, and revisit the choice when ERCOT changes its systems.
• ✓Use API queries for selective retrieval; consider files when you need the whole product.
• ✓Keep a raw artifact either way.
• ✓Land live and backfill paths in the same normalized schema.
• ✓Record the source channel for every batch.