Why "just ask the AI" doesn't work for research
- Errors add up. If each step is 90% right, five steps in a row are right only 59% of the time.
- Plausible isn't verified. A language model writes confident text. Without a hard rule, facts, company claims and guesses get mixed together.
- Nothing repeats. A chat answer can't be re-run next quarter with the same structure and quality bar.
The design
Split the work by what each part is good at. The AI decides and writes: which competitors matter, what the evidence means. Code executes: fetching sites, checking citations, merging data, drawing charts, laying out the PDF. A person approves at four fixed checkpoints.
What the validator checks
- Every field matches the JSON schema, and every claim carries at least one source id.
- Every cited id exists, with a URL and an access date.
- No placeholder text and no example URLs in a real run.
- No forbidden names.
- No dead links. Pages that block bots are flagged for a person to check.
The AI is told the rules. The code enforces them.
The first run
A competitor and market analysis for a company in the A2P SMS market. Nine research agents ran at once, one per competitor, with 30 more companies in an overview table. Before research started, about 50 claims from older internal files were checked against live sources, and 15 were corrected or dropped. Those corrections landed before a single new page was written.
What I'd do differently
Count sources and claims with one tool. The research files and the final report counted slightly differently (192 vs 189 sources), which is why this page rounds the numbers.