How it works
The review pipeline, from diff to anchored findings.
ocra combines two proven designs: domain-specialized reviewers with a coordinating judge, and deterministic engineering for every step that must not fail.
Ingest → Select → Triage → Bundle → Matrix → Execute → Anchor → Filter → Verify → Judge → Report| Stage | What happens |
|---|---|
| Ingest | Read the change set and repository guidelines through a VCS adapter. |
| Select | Decide per file: review it, or exclude it with a reason (binary, secret, generated, too large…). |
| Triage | Assign a risk tier (trivial, lite, full) from size and sensitive paths such as auth/. |
| Bundle | Group related files. Small change sets form one bundle; larger ones are grouped by a light model that answers with file indices. |
| Matrix | Decide which reviewer runs on which bundle from the risk tier and each reviewer's scope (for example, no documentation for a code-only reviewer). Skipped pairs are listed in the report. |
| Execute | Run each planned (bundle, reviewer) pair as an isolated agent task with read-only tools. |
| Anchor | Resolve every finding's quoted code to exact lines. |
| Filter | Drop findings the repository's memory accepts or a reviewer dismissed, and compare with the previous review of the same pull request, before any model call is spent on them. |
| Verify | A standard-tier model fact-checks the findings of each file against its diff and the code around them. A finding is dropped only when that code proves it wrong; doubt keeps it, and so does a failed check. Dropped findings and the reason are listed in the report. |
| Judge | One top-tier call sees every finding: it merges reports of the same root cause from different reviewers, drops speculation and nitpicks, recalibrates severity, and writes a short summary. Each change carries a reason in the report. |
Verdict
The verdict is decided by a fixed rubric over the final findings, not by a model, so the same findings always give the same verdict:
| Findings | Verdict |
|---|---|
| none | approved |
| suggestions, or fewer than three warnings | approved_with_comments |
| three or more warnings, or critical findings the verifier did not confirm | minor_issues |
| any critical finding the verifier confirmed | significant_concerns |
Every finding shows whether Verify confirmed it, was unsure, or did not check it (verification skipped, failed, or out of budget). Unconfirmed critical findings are shown as critical, marked unverified, and cap the verdict at minor_issues, so a single model's unchecked claim cannot block a change. The judge can downgrade a confirmed critical finding with a reason but cannot drop it; an attempt is ignored and listed as a warning.
If the judge is disabled, has no top model or fails, findings are reported unjudged and the rubric still applies.
The verdict is advice, not a security gate. Every model in the pipeline reads the change, and text planted in it can steer a reviewer away from an issue or talk the judge out of one. Use ocra to help reviewers, not to replace required human review or security checks.
Modes
The default mode favors precision. --ultra trades cost for recall:
| Default | --ultra | |
|---|---|---|
| Reviewers | By risk tier and scope | Every reviewer at every tier (files outside a reviewer's scope are still skipped) |
| Samples | One run per reviewer and bundle | Two runs, merged by fingerprint |
| Judge | Drops speculation and nitpicks | Keeps them, marked low confidence; they do not count towards the verdict |
| Cost | Baseline | About twice or more |
Verification works the same in both modes. A planning phase and impact analysis of callers are not implemented.
Reviewers and tools
A reviewer is an agent with a focused prompt, its own rules, and a scope that tells the matrix where it is worth running:
| Reviewer | Looks for | Runs at | Skips |
|---|---|---|---|
correctness | Logic errors, broken contracts, error handling | every tier | nothing |
security | Exploitable issues reachable from untrusted input: injection, authorization, secrets, crypto, unsafe parsing, CI | lite and full | documentation, tests |
performance | Measurable regressions on paths that matter: complexity, N+1, blocking work, memory | lite and full | documentation, tests, config data, CI workflows |
Each one reviews only its own domain and states what not to flag. Every reviewer can only read: read_file, read_diff and code_search answer from the revision under review (a commit in range or commit mode, not your working tree), and findings are submitted through report_finding. It cannot edit files, run commands or browse the web.
Anchoring
Models are unreliable with line numbers, so they quote code instead. ocra resolves the quote:
- normalized match in the file's changed hunks;
- match in the whole file;
- match in another changed file (the model named the wrong file);
- (planned) a relocation request to a light model;
- otherwise the finding stays attached to the file, never dropped.
Robustness
- A per-task and a whole-run timeout, enforced even if a model stops responding.
- A failed task never fails the run; its files are reported as
failed. - Model failback chains per tier, with a circuit breaker per model (open after repeated failures, half-open probe after a cooldown).
- Rate limits are waited out: when a provider says to retry after a short wait (up to 90 seconds), every task pauses that model and retries it. A limit marked as daily, a longer or missing wait, or a fourth limit in a row takes the model out of the chain for the rest of the run, so ocra stops sending requests that would be refused.
- Every agent is capped at 20 steps, because each step resends the conversation.