Orchestrate Brazil CVM Filings with Mage

The problem

Nightly research jobs that refresh DFP/ITR inventories need retries, secrets, and schedules — not ad-hoc curl. Mage pipelines fit; portal HTML does not.

Solution flow

schedule → resolve ticker → list documents → download PDFs → optional extract callback

Block sketch (Python)

import os, requests

BASE = os.environ["APICVM_URL"]
H = {"Authorization": f"Bearer {os.environ['APICVM_KEY']}"}

def list_dfp(ticker: str, year: int = 2024):
    r = requests.get(
        f"{BASE}/v1/documents",
        headers=H,
        params={"ticker": ticker, "type": "DFP", "year": year, "perPage": 20},
        timeout=60,
    )
    r.raise_for_status()
    return r.json()["data"]

Wire list_dfp as a Mage data loader; pass document id values to a downloader block.

Current limitations

• Text extraction is async via HTTPS callback (Pro) — do not poll a status URL that does not exist
• perPage max 50; paginate with page
• Empty lists can be corpus gaps

Next steps

• Read the API docs
• Get an API key
• Airflow pipeline
• Prefect pipeline

Related guides

• Getting started
• Async PDF extraction