Fine-tuning is a training technique that adapts a pre-trained AI model to a narrower task by continuing its training on a smaller set of examples. It changes the model's weights, not its instructions.
A large language model is first pre-trained on general text, then fine-tuned on labelled examples of the target behaviour, such as invoice lines paired with the right expense account. Instead of updating every parameter, LoRA, published in 2021, freezes the pre-trained weights and adds small trainable matrices; its authors report 10,000 times fewer trainable parameters and three times less GPU memory than full fine-tuning of GPT-3 175B.
Fine-tuning fits some finance needs and not others:
- Good fit: a fixed output format, a stable set of labels, a house style
- Poor fit: facts that change, such as negotiated prices or supplier bank details, frozen at the training date
- Hidden cost: each new base model means training and testing again against a golden dataset
- Traceability: knowledge stored in weights cannot point to its source document
A 2024 study by Gekhman and colleagues found that large language models "struggle to acquire new factual knowledge through fine-tuning", and that once such examples are learned, they linearly increase the model's tendency to hallucinate. A control must read the reference price from the current contract at the time of the check.
Fine-tuning differs from retrieval-augmented generation, which leaves the model unchanged and supplies documents with each request, and from embeddings, which represent meaning for search rather than generating answers.
Phacet's security page states that customer data is never used to train AI models. Company specifics come from configuration instead: business rules such as tolerance thresholds and validation rules, applied to 100% of flows, and workflows that blend LLM and deterministic rules.