AnnuityBench is a benchmark for how well today’s LLM tools handle genuinely hard actuarial work. Instead of hundreds of auto-scored questions, it works through a handful of difficult problems by hand.
As a profession, we need to understand how well this technology grasps the nuances of our craft. This is not an endorsement of handing off our work and blindly trusting the results, and it is not how I would use these tools in my own work.
The tasks
The problem set splits into three parts.
| Level | Task |
|---|---|
| Analyst | Calculate index values for a vol-control index given component values and a weighting method |
| ASA | Implement a daily delta hedge for an ATM call and calculate the annual performance |
| FSA | Prepare a report for the board with hedge performance attribution and suggestions for improvements |
The Task 1 and 2 prompts were deliberately vague to see how the models filled the gaps. Task 3 is an open-ended request for interpretation.
Scorecard
| Findings (by task) | GPT-5.5 | Opus 4.8 |
|---|---|---|
| Index construction (T1) | Correct | Correct |
| Delta implementation (T2) | Correct | Correct |
| Hedge financing (T2) | Omitted, but logged | Included |
| Equity dividends (T2) | Missed | Missed |
| Performance attribution (T3) | Textbook correct | Textbook correct |
| Board report (T3) | Weak interpretation | Mostly solid interpretation |
Note: “textbook correct” represents a solution that would be correct on an exam but lacks true domain insight.
Task 1: construct the index
Models were given a set of equity values, 10-year CMT yields, and a weighting algorithm, and were asked to produce historical values for a vol-control index. There was a small trap: only bond yields were given, not index prices.
The difficulty here was review, not accuracy. Opus took a simpler approach that was less exact but easier to validate.
Below is an example of a GPT formula. I have a strong preference for Excel implementations that are simple and easy to review; more detailed prompting or Excel-style “skills” should fix most of it.
Task 2: implement daily delta hedge and calculate profit/loss
Models were asked to take the prior results and develop a daily delta hedge for an ATM call, with hedge profit/loss over the last 10 years. They were not given futures returns or any suggestion that a solution built from raw components should include the cost of financing. Whether they would reach for it unprompted was the test.
Neither got it fully right:
- Opus implemented the financing but neglected to pay dividends on the equity shares held in the hedge.
- GPT chose not to implement financing (documented in the decision log) and also missed the dividends.
The decision logs earned their keep here. The GPT financing choice and several other small issues surfaced in seconds rather than through cell-by-cell review. Both logs are in the exhibits.
Task 3: develop profit/loss attribution and suggest hedge improvements
Models were given the prior work and asked to attribute the hedge results, then prepare a board report that summarized them and suggested hedge improvements.
This is the real step up: the bulk of the task is interpretation and communication, with no guidance beyond the audience being the board.
It is also where textbook training works against these models. The exam answer distracts from the useful one.
- Both models developed similar textbook attribution to individual Greeks.
- Opus distilled the attribution into simple language and produced the higher-quality report.
- The focus on textbook answers led both to more complicated, confusing solutions than an experienced actuary would have chosen.
Skim the Opus report. The conclusions are strong, with one stumble: rho and financing never resolve into a clean story.
Cost & runtime
These tasks ran on subscriptions rather than metered API access, so costs are rough estimates produced by the models. Tasks landed between $1 and $10 each, and the two models cost about the same in total (~$14 each). GPT-5.5 was cheaper on the index build and hedge backtest but pricier on attribution. Runtimes were 5-25 minutes per task.
Caveats
- Small additional prompts were made on the board reports before turning them into PDFs to get the sizing in better shape.
- In the interest of full disclosure, a family member works at Anthropic. I have made every effort to present these results impartially, and I would encourage everyone to try multiple tools to see what works best for them.
- I had 3-month UST rates on my machine when I started this task. These rates were used as a stand-in for the risk-free rate in any options or hedging math.
Select exhibits
Opus hedging report
Below is the Opus report on delta hedge performance over the prior ten calendar years. I was impressed by the results achieved from a single prompt with minimal guidance.
Positives:
- Technical concepts distilled into simple language.
- A lot of the commentary is correct.
Negatives:
- There are a couple of silly parts included (e.g., suggestion to keep receiving dividends on equity held).
- You can see the influence of textbook answers not quite translating to reality in the rho vs carry discussion. This could easily all have been bucketed into just talking about the cost of financing.
GPT-5.5 hedging report
The GPT numbers match the Opus report, but the interpretation is not as strong: it is stuck in textbook language and does not explain as clearly.
Positives:
- Attribution performed correctly if it was an exam problem.
Negatives:
- Difficult to understand what the actual drivers are.
- Suggested improvements are weak.
Opus Task 2 decision log (before fixes)
One minor issue this log surfaced: the model held the interest rate for financing flat over the term rather than floating it.
GPT-5.5 Task 2 decision log (before fixes)
The GPT decision log made it easy to identify that the hedge implementation had ignored the cost of financing.
If you have feedback or ideas for tests you would like to see, send me a note.