Fallacy Analyzer — paste an argument, see its fallacies.
A public AI product I built end to end on the Claude API. You submit a short argument and an 8-agent pipeline moderates the input, extracts the argumentative structure, classifies candidate fallacies, runs a verification plan against the web, evaluates each candidate against a typed catalogue of 10 fallacy types, scores the result, and writes a clean Markdown report.
Strict anti-hallucination policy on sources (tiers A → D). Public feed of analyses. Hourly window mechanic (5 slots, opens at a random minute between :00 and :05) to ration capacity and create a small ritual.
What it does, in one sentence
You submit a short text containing an argument. The product returns a structured analysis: what the argument is, which logical fallacies it commits (or doesn't), why, and which sources back the verification — with a confidence score per detection and a clean human-readable report.
8 agents, one analysis
Moderator
Input filter. Decides allow / block, marks grey-zone content, returns 1-3 topic tags. Haiku.
Argument parser
Extracts the argumentative structure from the raw text — premises, claim, implicit links.
Fallacy classifier
Proposes candidate fallacies against a typed catalogue of 10 named fallacies (skills/*.yaml).
Research planner
Designs verification queries: what to check on the web to confirm or refute each candidate.
Evidence retriever
Runs the queries with Claude's web_search tool (max 3 uses) and applies a strict source-quality policy: tiers A / B / C / D. Anti-hallucination rule on sources. Prompt caching on the system prompt.
Fallacy evaluator
Invoked once per candidate. Decides whether it is actually a fallacy, with a reasoned justification grounded in the evidence.
Score aggregator
Consolidates per-candidate evaluations into a single typed assessment with confidence scores.
Report writer Sonnet 4.6
Composes the final Markdown report — what the argument is, which fallacies were detected, why, with sources and confidence.
Each agent has its own CLAUDE.md with operating instructions and a typed JSON schema for output. The whole orchestration is in Python, with the Claude API as the only model layer.
10 named fallacies, typed
Each fallacy is a YAML skill loaded into the classifier and evaluator — definition, prototypical structure, examples and edge cases. The catalogue is extensible without touching agent code.
Hourly window with random open
5 slots per hour
Each natural hour opens a small window with exactly 5 analysis slots. Unused slots are lost; they don't accumulate.
Random open minute
The window doesn't open exactly at :00. It opens at a random minute between :00 and :05. Creates expectation and ritual.
1 submission per user per day
Cap prevents flooding and forces selection: which argument do you actually want analysed?
While the pipeline runs (2-8 minutes), the user sees a step-by-step progress indicator (1/7 → 7/7). Every analysis is public by default and lives in the feed alongside the others — so the product is also a small library of argument-quality examples.
Anti-hallucination by source policy
The verification stage doesn't trust any URL the model invents. A typed source policy classifies retrieved evidence into four tiers (A → D) and applies rules:
- Tier A — primary sources, peer-reviewed journals, government data.
- Tier B — established media with editorial process.
- Tier C — secondary blogs, opinion pieces — usable as context, not as proof.
- Tier D — discarded.
A claim verified only against tier C or D is reported as not verifiable rather than confirmed. The whole policy is a YAML file (config/source_policy.yaml) — editable without touching agent code.
What's under the hood
api Pipeline (Python)
- Claude API (Haiku 4.5 + Sonnet 4.6)
- Prompt caching on system prompts
- Server-side
web_searchtool - Typed JSON schemas between agents
- YAML-defined fallacy catalogue
language Web app
- Django + SQLite
- Gunicorn + Nginx on Lightsail
- Worker process (queue)
- Tag M2M + colour-by-hash for the feed
- HTTPS via bncert / Let's Encrypt
Want to try it on your own argument?
Free, public, no login needed to read the feed. Submitting an analysis only requires a Google sign-in.
arrow_outward Open falacias.cordcozy.com