Orchestrate Brazil CVM Filings with Airflow

Need Airflow CVM filings jobs that stay reproducible? Schedule apicvm calls in a DAG: resolve companies, list documents by ticker/type/year, download PDFs to object storage, and optionally enqueue text extraction. apicvm is the HTTP source; Airflow owns retries and schedules.

The problem

Nightly Brazil filing pulls usually fail for operational reasons:

Manual portal downloads do not retry or alert
Ad-hoc scripts ignore 429 and X-RateLimit-*
No audit trail of which document.id landed in the lake

Airflow + a stable filings API fixes the ops layer without scraping HTML.

Pattern

Variable: ticker watchlist
  → GET /v1/companies/resolve
  → GET /v1/documents (DFP/ITR/FRE, paginated)
  → GET /v1/documents/:id/file → S3/GCS
  → optional POST /v1/document-text-extractions

Store APICVM_KEY in an Airflow connection or secret backend — never in DAG code.

Minimal Python task

import os, requests

BASE = os.environ["APICVM_URL"]  # https://apicvm.dev
H = {"Authorization": f"Bearer {os.environ['APICVM_KEY']}"}

def list_dfp(ticker: str, year: int = 2024):
    page, rows = 1, []
    while True:
        r = requests.get(
            f"{BASE}/v1/documents",
            headers=H,
            params={
                "ticker": ticker,
                "type": "DFP",
                "year": year,
                "page": page,
                "perPage": 50,
            },
            timeout=60,
        )
        r.raise_for_status()
        payload = r.json()
        rows.extend(payload.get("data") or [])
        meta = payload.get("meta") or {}
        if page >= int(meta.get("lastPage") or 1):
            break
        page += 1
    return rows

Wire this into a @task or PythonOperator. Paginate as in Paginate and filter CVM documents.

Sketch DAG

from airflow import DAG
from airflow.decorators import task
from datetime import datetime

with DAG(
    "cvm_filings_nightly",
    start_date=datetime(2026, 1, 1),
    schedule="@daily",
    catchup=False,
) as dag:

    @task
    def pull_watchlist():
        tickers = ["PETR4", "VALE3", "HYPE3"]
        out = {}
        for t in tickers:
            out[t] = [d["id"] for d in list_dfp(t)]
        return out

    pull_watchlist()

Add a second task that downloads each id with GET /v1/documents/{id}/file and writes to your bucket. File download does not consume extraction credits.

Rate limits and retries

Prefer Airflow retries with exponential backoff on 429 / 5xx
Space ticker loops; check X-RateLimit-Remaining
Do not fan out hundreds of parallel tasks against one key

See Handle errors and rate limits.

Extraction (optional)

POST /v1/document-text-extractions returns 202 and delivers page markdown to your HTTPS callback. Airflow should treat extraction as fire-and-confirm-via-callback, not as a sync XCom value. Same pattern as GitHub Actions monitor for lighter schedules.

curl smoke test

export APICVM_KEY='apicvm_...'
export APICVM_URL='https://apicvm.dev'

curl -H "Authorization: Bearer $APICVM_KEY" \
  "$APICVM_URL/v1/documents?ticker=PETR4&type=ITR&year=2024&perPage=5"

Current limitations

No job-status polling endpoint for extractions — callbacks only.
Corpus ≠ live CVM portal; schedule lag is possible.
perPage max 50.

Next steps

Ready to integrate?

Get an API key and start querying Brazilian CVM filings programmatically.

Read the API docs Get API key