How to stop AI getting numbers wrong

Why do AI chatbots get maths wrong?

A language model writes an answer one token at a time, choosing what is most likely to come next. A number in its answer is produced the same way as a word: as likely-looking characters, not as the result of a calculation. Numbers are also split into tokens in chunks that do not line up with place value, which is part of why long multiplications and long columns of figures go wrong.

Current models are far better at arithmetic than they were, and some assistants quietly run code for calculations. But you usually cannot see whether a figure was computed or predicted, and the answer is not guaranteed to be the same twice. For office work, "usually right" is not a standard: one wrong figure in a batch of 200 costs more time to find than the batch took to produce.

The mistakes that actually happen

• Percentage direction. Taking VAT out of a gross amount of 125 at 25% gives a net of 125 / 1.25 = 100. Taking 25% off gives 125 × 0.75 = 93.75. Both look plausible; only one is right.
• Rounding. Rounding VAT per line and rounding it on the invoice total give different totals. So do round-half-up and round-half-to-even. A model will pick one without telling you.
• Long sums. Adding 60 amounts, or multiplying two six-digit numbers, is where predicted digits drift.
• Silent omissions. A row is skipped, or a figure from an earlier message is reused, and the total still looks reasonable.
• Units and currencies. Hours against days, gross against net, euros against dollars.

The fix: split the work between a calculator and the model

Models are good at language: understanding what you asked, choosing which rule applies, and explaining an outcome in plain words. Calculators are good at arithmetic. Give each the part it is good at.

How to set it up, step by step

1. Write each rule as a formula with named inputs. For example net = gross / 1.25 and vat = gross - net. Put the rounding in the formula: round(x * 100) / 100 rounds to cents.
2. Keep formulas in one place, with a version. Not copied between spreadsheets, where they drift apart.
3. Evaluate with a deterministic engine and record the inputs, outputs, formula version and time for each run.
4. Give the model the result and ask it to explain — not to check or recompute it. Tell it so in the instructions.
5. Spot-check by re-running. The same inputs must give byte-identical outputs. If they do not, something other than the engine is doing arithmetic.
6. Keep a person on the decision. The engine and the model prepare; someone approves.

How OpenClerq does it

OpenClerq has a calculation engine written in Rust that runs on your machine. You send it a formula, or a set of formulas with named inputs, and it returns the values with a proof record: a calculation id, a timestamp, the operation, a proof hash and the engine version.

POST /calculate/eval
{
  "spec": {
    "formulas": { "net": "gross / 1.25", "vat": "gross - net" },
    "output_names": ["net", "vat"]
  },
  "inputs": { "gross": 125 }
}

→ {
    "values": { "net": 100, "vat": 25 },
    "proof": { "calculation_id": "…", "timestamp": "…",
               "proof_hash": "…", "is_audit_ready": true, … }
  }

In the desktop app's Ask panel, simple requests such as "add 25% to 100" or "calculate a * b where a=12 b=30" are recognised and sent to the engine first. The model receives the engine's result as context and explains it. OpenClerq's default system prompt tells the model never to present a number as its own calculation and not to recalculate figures it is given.

What about spreadsheets?

A spreadsheet formula is deterministic too, and if your team has clean, well-kept spreadsheets, keep them. Use AI to explain the results, draft the covering note, or spot a figure that looks out of line — not to produce the figures.

Spreadsheets go wrong in other ways: formulas copied between files drift, a range misses the new rows, and there is no record of which version of the formula produced last month's number. A single formula store with a proof record for each run fixes those. Either approach beats asking a chatbot to add up a column.

Frequently asked questions