AI reliability · 2026

NL Fiscaal — Deterministic Citation Control

A deterministic checker and override on top of an LLM that auto-corrects Dutch tax documents. It enforces the correct 'sub' notation on EU-directive citations, where the model alone was inconsistent, and surfaces doubtful cases as review cards instead of silently rewriting legal text.

snapshot

type: AI reliability
period: 2026
source: Private client project · Stichting NLFiscaal
build: Solo · in production, under senior review · hybrid AI + deterministic design

All work

problem

The correction tool lets an LLM rewrite tax documents to a house style guide. For EU-directive references the subdivision word must be 'sub', not the Dutch 'onderdeel' or 'onder'. Leaving that to the model made it non-deterministic: it could miss a case or correct inconsistently, and a wrong citation in a legal text is worse than a missed one.

outcomes

In production: the model stays the writer, the deterministic layer controls and surfaces doubt

Hardened with an adversarial ~25-agent review on real tax text: 0 false positives, 0 crashes, 0 text corruption

Turned a silent, inconsistent AI correction into a reviewable, deterministic one on exactly the error-prone spot

what i built

A deterministic regex checker that flags EU-directive citations with the wrong subdivision word as a reviewable DOUBT card, so the reviewer decides instead of trusting a silent auto-correct

A deterministic override that corrects the model's output where it used the wrong term, a safety net on the LLM

Covers named and numbered directives, including the /EG, /EEG and /EU suffix variants

Overlap-suppression so the deterministic cards never clash with what the model already corrected on the same span

Test-first: 43 unit tests plus Django integration tests, written before the implementation

tech stack

PythonDjangoDjango REST FrameworkCelerypytestregexAzure OpenAIDocker

Case study

This project is documented here as a case study; the source is not published as a public repository.

Discuss similar work