Can DCI beat RAG? A field report on bash-agent retrieval for AML/CFT

On May 3, 2026, a team from Texas A&M, UCSD and Waterloo published « Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction » (arXiv 2605.05242). The thesis is provocative: for agents capable of reasoning, the semantic retriever (embeddings + top-K) has become a bottleneck. By replacing the vector index with direct bash access to the raw corpus, the authors gain +11 points of accuracy on BrowseComp-Plus while cutting cost by 29 %.

The pattern they name Direct Corpus Interaction (DCI) is disarmingly simple: the agent receives the question + a bash tool, and that's it. It runs grep, find, head, cat until it finds the answer. No embeddings, no index, no conventional retriever.

A French AML/CFT corpus fits the paper's thesis neatly: text structured by numbered articles (L.561-15 CMF, Art. 23 AMLR), dense vocabulary of exact references (Tracfin, ERMES, COSI, MiCA), moderate size. The DCI sweet spot. So we ran the test. Here's what we found.

First test: DCI 40 %, RAG 94 %

10 questions from the CASP/VASP pilot of AML-Bench-FR, comparison between our current RAG (Mistral Large + reranker) and a DCI agent on Haiku 4.5 — the equivalent of the paper's "DCI-Agent-Lite" which uses GPT-5.4-nano.

Apparent verdict: RAG crushes DCI. But looking at the DCI predictions in detail, we find four responses truncated to five characters and four unrecovered API errors. Something is off — and it's not DCI itself.

Three implementation biases, not a paradigm weakness

A close look at the predictions reveals three biases that explain most of the gap, regardless of DCI's intrinsic quality:

First, the generation budget is calibrated for a one-shot LLM, not for an agent. On an MCQ question, we allocate 50 tokens — enough for a single letter, lethal for an agent that needs to reason before calling a tool. The model starts its sentence "Let me look up article L.561…" and gets cut off before any tool call. Zero score on four questions.

Second, the bash sandbox is too strict. Several legitimate commands get rejected because the safety patterns match inside the arguments. On one particular question, 50 consecutive attempts are refused and the agent exhausts its budget without ever being able to search.

Third, the answer extractor is designed for terse LLMs, not for a verbose agent. The grader looks for the [A-E] letter in the answer; the agent writes "After analysis, the correct answer is C" and the grader picks up the A in "analysis". Fix: a mandatory output marker and an extractor that prioritizes it.

These three fixes bring the MCQ score back toward parity. The key insight: it isn't DCI that underperforms — it's the integration into our pipeline that assumed a direct LLM.

Second test: DCI on Mistral, and the rate-limit walls

Once the fixes were applied, we re-ran on Mistral (Magistral for its reasoning capabilities, better suited to multi-tool-use). New problem: DCI agents typically make 15-30 model calls per question. On Mistral's free tier (60 req/min), a single DCI agent saturates the quota in under a minute. 429 errors on 4 of 5 questions in the run.

The lesson is less technical than philosophical: DCI shifts the center of gravity of cost. RAG is one context-rich model call; DCI is many small context-poor calls that compose. The call quota becomes the bottleneck, not the token count.

Implemented solution: a rate limiter with exponential retry on 429 (3 s / 8 s / 15 s) and a conservative client-side cap. The DCI bench now runs cleanly on Mistral, just more slowly (45-90 s per question).

Where DCI wins, where it loses

On numeric short_answer questions ("What is the deadline to transmit an STR to Tracfin?", "What is the COSI threshold?"), DCI reaches parity with RAG (100 %). Consistent with the paper's thesis: grep on precise terms finds exact references (R.561-31, 8,000 €) without semantic noise.

On open_short, DCI is at -2 pp from RAG on this sample. No significant gap. The paper predicted parity or a slight RAG edge on synthesis questions spanning multiple articles — direction is consistent.

On MCQ, the apparent gap is due to the implementation biases described above, not to an intrinsic DCI weakness.

The product intuition we keep: DCI is likely interesting on exact-reference questions (specific article, threshold, date) and likely at parity on synthesis questions. The added latency and cost only justify themselves on the first segment.

The audit trail, where DCI surprises pleasantly

A question we hadn't anticipated: what do you show the auditor on a DCI response? On a RAG, the trace exposes the chunks (reference, snippet, reranker score). On DCI, there are no chunks in the retrieval sense — there's a command trajectory.

So we expose two levels: the detailed list of bash commands executed (each grep, cat, head with its timing) and the corpus files the agent actually touched (extracted by regex on the paths). It is, paradoxically, more transparent than a classical RAG where the top-K is silently filtered by the reranker. An MLRO auditing a DCI response sees exactly which searches the agent ran, in order, with results.

This is a non-trivial audit argument for a sector where ACPR-Tracfin oversight expects to be able to reconstruct the reasoning step by step.

The product decision: 5 analysis levels instead of one

Our conviction after these two weeks: DCI is not a RAG replacement — it's a complement for specific use cases. On factual questions with exact references and a comfortable time budget, DCI can deliver superior precision and traceability. On everyday questions, RAG remains unbeatable on cost and latency.

Hence the choice to expose it as a premium analysis tier, not the default stack. On the connected chat, the user can pick from five levels:

AML-1 and AML-2 are available on the guest chat without account creation. AML-3, AML-4 and AML-5 are reserved for paid accounts, in beta.

What we learned

DCI shifts the work. Instead of optimizing a retriever to feed the right chunks to the model, you give the model the right tools to search for itself. For an MLRO, that difference is visible in two concrete dimensions: traceability (DCI exposes the searches, RAG exposes the chunks) and the time-cost profile (DCI is slower and more expensive but potentially more precise on exact-fact questions).

The paper doesn't claim DCI is universally better; it says that in certain configurations, on certain question types, with certain models, DCI delivers a measurable improvement. Our preliminary tests on 10 questions confirm the direction without yet quantifying the advantage.

The full-scale test follows. The methodology is public, the export and bench scripts remain open, and every trajectory is observable post-hoc. The goal isn't to prove that DCI wins, but to measure where and at what cost it wins. That measurement is what lets an MLRO pick the right analysis level for the right question.

---

The original paper: Li et al., « Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction », arXiv 2605.05242, May 3, 2026.