A golden dataset is a fixed set of real examples, each paired with an answer verified by a human expert, used as the reference to measure how accurately an AI system performs a given task.
In finance automation, a golden dataset for invoice processing contains actual invoices with their correct values: supplier, invoice number, amounts excluding and including tax, VAT rate, purchase order reference, and the expected decision (approve, flag or reject). The agent's outputs are compared field by field with these answers to compute accuracy per field and per decision.
A useful golden dataset follows four rules:
- it covers hard cases in proportion: credit notes, multi-page invoices, foreign currencies, partial deliveries, not only clean PDFs
- its answers are set by someone who knows the process and reviewed by a second person, the maker-checker logic applied to test data
- it stays separate from the examples used to configure the system, otherwise the score measures memory rather than accuracy
- it is versioned, so that results remain comparable from one test to the next
A golden dataset is used at three moments: before deployment, to decide whether accuracy is sufficient; at every change of model, prompt or rule, as a regression test; and during a proof of value, to show results on the company's own documents rather than on a demo set. Re-running it after a model update is the most direct way to catch model drift.
Phacet's native audit trail records every agent output and every correction made by the finance team, which gives teams the material to build a golden dataset from their own invoices, delivery notes and contracts. Combined with the AI confidence score, it turns the auditor's question, how do you know the AI is right, into a measured answer.
