RESEARCH
Building an equity risk model for AI agents
An agent that manages portfolios is only as good as the risk model behind it. Most risk-model failures are evaluation failures: the model looks calibrated on the portfolios people test and fails on the portfolios an optimizer produces.
This is the risk model inside Abacus Labs: why it exists, what can go wrong, what we built, how we scored it on 406 month-ends from 1993 to 2026, which bugs the scoring found, and whether a language model could have done any of it.
1. Why a risk model
2. What to watch out for
3. What we built
4. How we evaluated
5. What went wrong and how we fixed it
6. The decision
7. Can an LLM do this
8. What it means for agents
1. Why a risk model
Every risk number Abacus Labs returns comes from one object: a covariance matrix over every US-listed stock and ETF, estimated as of the decision date. Tracking error against a benchmark is a quadratic form in that matrix. So is the volatility of an account before and after a trade list, and so is a volatility target. The exposure analytics an agent shows a person, how much of the account is a bet on one industry or on small caps, are its factor structure.
The optimizer uses the same matrix as its objective. When an agent asks Abacus Labs to cut a concentrated position while keeping tracking error under two percent, the exact solver minimizes risk in that matrix subject to taxes, lots, whole shares and wash sales. If the matrix under-states risk by thirty percent for the portfolios the solver produces, every option is wrong by thirty percent, and the certificate certifies the wrong thing.
A language model cannot estimate this object. For six thousand names it has about eighteen million entries, and the only way to know them is to regress thirty years of daily returns with the names that later failed still in the panel. We did test whether a frontier model could tune or edit the finished estimator. The answer, in section 7, is: a little, and not reliably.
2. What to watch out for
These are the failure modes the literature documents and that we designed against.
Survivorship and delisting returns
A universe built from today's listings omits every name that failed. In our panel, six to fifteen percent of listed names leave the market within twelve months, every year since 1990. A delisted name has to earn its terminal return, what a holder actually received, or small-name risk is understated by construction. Where no price exists, Shumway (1997) measured performance delistings at about minus thirty percent on NYSE and minus fifty-five percent on Nasdaq.
Look-ahead in fundamentals and index membership
Restated fundamentals are not what the market knew. Lyle, Siano and Yohn (2024) find that when a quarterly earnings value changes in a commercial database, it changes by about one hundred percent of the first-reported value. Timing matters at the day level too: nearly half of 10-K and 10-Q filings are accepted between 16:00 and 17:30 Eastern, after the close. Today's index constituents scaled back by price are not the index as it stood.
Too many names, too few days
Six thousand names and 250 trading days make the sample covariance singular, and the in-sample risk of a Markowitz portfolio understates realized risk by a factor of (1 minus N/T) squared (Bun, Bouchaud and Potters 2017; Ledoit and Wolf 2017). The fix is structure, a factor model with far fewer factors than names, plus shrinkage of what remains (Ledoit and Wolf 2004).
Optimization bias
A portfolio chosen to minimize a risk estimate is correlated with the error in that estimate, so its forecast is biased low even when the estimator is unbiased. Shepard (2009) puts the size at (1 minus K/T) to the minus two for a K-factor model, about thirty percent for fifty factors and two hundred days. In the Barra USE4 notes, all one hundred random-alpha optimized portfolios failed the confidence band before the eigenfactor adjustment (Menchero, Orr and Wang 2011). Random portfolios carry no such bias, so optimized minus random is the diagnostic.
Short histories and IPOs
A name listed sixty days ago has sixty days of history. Filling its missing returns with zero before a factor fit shrinks its loadings toward zero, so the optimizer sees a low-risk name and buys it. A prototype of ours did this and predicted an equal-weighted basket of 865 short-history names at 9.3 percent volatility against 12.4 percent realized. Missing must stay missing, and a structural prior from seasoned names of the same size and sector fills the gap.
ETFs and dual share classes
An ETF is a portfolio, so its risk belongs to its constituents. When exposures come from regression instead, its residual is tracking noise and its specific variance is tiny, so a swap from a stock into an ETF looks nearly free. That is the tax-loss-harvesting use case, so it has to be right. Dual share classes of one issuer have residuals that move together, which a diagonal specific-risk matrix cannot see.
Calibrating the index versus what the optimizer builds
Every model with correlations is calibrated on the S&P 500. The question that matters is the fifty-name tracker with wash-sale locks. Chan, Karceski and Lakonishok (1999) show why: tracking error against the S&P 500 falls from 5.12 to 4.01 percent going from one factor to nine, while minimum-variance risk does not move. Factors earn their place through active risk.
Thirty years of a changing universe
Point-in-time market capitalization exists in our data from 1994 and point-in-time fundamentals from 2009, when XBRL filing began. A model spanning 1993 to 2026 is several models stitched together, with factors switching on as their data arrives. The evaluation has to stratify by era.
3. What we built
The statistical model came first
Our first point-in-time model was statistical: twenty factors from an expectation-maximization principal-component fit over a 1,008-day window, exposures shrunk toward a sector-and-size prior for young names, and a structural blend for specific risk. A second version added three style factors and eleven sector factors on the residuals. It fixed the zero-fill and survivorship defects of the prototype.
Its remaining defect was the one the literature predicts. A fifty-name S&P 500 tracker was under-predicted by about 1.25 times at a one-month horizon, and a simulation that re-estimated the model on its own draws attributed only 1.04 of that to estimation error. The rest was misspecification: risk the statistical factors do not span. That is the case for named factors.
The fundamental family
The second model is a Barra and Axioma-class characteristic model, built from the public parts of the USE4 methodology notes and the Axioma US factsheets. We do not claim it beats those models. It is the same class of model, evaluated the way the literature recommends, with every parameter written down before the first full build so that no factor was chosen or dropped after seeing results.
- Market. Exposure one for every name. With the industry constraint it equals the cap-weighted return of the estimation universe.
- Industries. Up to fifty-two groups from SEC-filed industry codes as they stood on each date. A group needs twenty names or it rolls up to its sector. Cap-weighted industry returns sum to zero.
- Price styles, from 1991. Beta, momentum, residual volatility, size, non-linear size, liquidity and dividend yield. The short-horizon variant adds short-term reversal.
- Fundamental styles, from 2009. Book-to-price, earnings yield, leverage, profitability and investment, with growth from 2012. A style is active only when the names with a value carry half of estimation-universe capitalization. Before 2009 the model is price-only, and says so.
The estimation universe is operating companies with enough valid returns, the top 3,000 by size, which leave only below rank 3,500. Regressions use square-root-of-cap weights with Huber reweighting. Missing exposures of active styles are imputed by regression on size and industry, and flagged.
The factor covariance is an exponentially weighted estimate with Newey-West lags of five for volatility and two for correlation, Ledoit-Wolf shrinkage of the correlation toward the identity, a volatility regime adjustment, and the Menchero, Wang and Orr eigenfactor adjustment from one hundred Monte Carlo draws. Three half-life variants share exposures and factor returns:
| variant | volatility half-life | correlation half-life | regime half-life | specific half-life |
|---|---|---|---|---|
| fundamental (medium) | 84 days | 504 days | 42 days | 84 days |
| fundamental (short) | 42 days | 252 days | 21 days | 42 days |
| fundamental (long) | 252 days | 504 days | 168 days | 252 days |
Specific risk is the exponentially weighted variance of each name's residuals, blended with a structural model that regresses log specific volatility on the exposures over seasoned names. The blend weight grows with history, so a name with twenty valid days runs almost entirely on the structural prior. The result is shrunk toward its size decile with the Bayesian weight the USE4 notes publish. ETFs get exposures from a ridge regression of their returns on the factor returns, pulled toward a market-only prior. A hybrid variant adds five transient statistical factors on the residuals.
The point-in-time pipeline
Every input is dated by when it became knowable. Daily total returns are built forward from unadjusted bars in a commercial price history, with a terminal return for about 27,000 delisted securities. Zero-volume days and gaps are missing, never zero. The universe on each date is every equity listed on a major exchange that day, including the ones that later failed, and benchmark membership is effective-dated. A filing is usable from the first close after it was accepted before 16:00 Eastern. Every model becomes usable on the next business day after its data ends. Delisting returns never enter an estimate; they enter realized returns in evaluation only. Coverage is one hundred percent of the daily equity and ETF universe at every year end from 1995 to 2025.
4. How we evaluated
All model versions and five naive baselines were scored on the same 406 month-end dates from January 1993 to October 2026, at horizons of 21, 63, 126 and 252 trading days.
- Bias statistic
- Realized return over the horizon divided by predicted volatility over the horizon, then root-mean-square across dates. 1.0 is calibrated. Above 1.0 the model under-predicts risk. The 95 percent band under normality is 1 plus or minus the square root of 2/T, about 0.07 at 406 dates.
- MRAD
- Mean absolute deviation of the rolling twelve-month bias statistic from 1.0, because a full-sample 1.0 can hide years that cancel. The floor is 0.17 under normality.
- QLIKE
- A loss on predicted variance that ranks forecasts consistently when the realized proxy is noisy (Patton 2011).
- Mincer-Zarnowitz
- The slope of realized variance on predicted variance. 1.0 means the forecast moves one for one with realized risk.
Portfolio families: random equal-weighted portfolios of 20, 50 and 100 names; the equal-weighted top 1,000; a basket of short-history names; random long-short pairs; the S&P 500, Nasdaq 100 and Russell 3000; minimum-tracking-error trackers of each with 25, 50, 100 and 250 names, at fixed weights, buy-and-hold and wash-sale-locked; long-only and unconstrained minimum variance; one hundred random-alpha optimized portfolios per date; single names by size decile; single ETFs; replacement-basket swaps. Baselines: a 252-day sample covariance, Ledoit-Wolf linear shrinkage, constant correlation, a single-factor market model, and a diagonal EWMA with no correlations.
Integrity tests ran alongside: a truncation test that deleted every row after the model date from the raw inputs, rebuilt, and checked the model was bit-identical; an availability audit over all 1,764 weekly models; and a survivorship count. Before any of it ran, we wrote down the switch rule in section 6.
Random portfolios do not discriminate
statistical v2 ########## 0.98 fundamental (medium) ########## 1.03 fundamental (long) ########## 1.04 fundamental (short) ########### 1.06 hybrid ########## 1.03 sample covariance ########## 1.05 Ledoit-Wolf ########## 1.04 constant correlation ########## 1.03 single factor ########### 1.11 diagonal, no correlation ############################################## 4.61
RMS bias statistic, random 100-name equal-weighted portfolios from the top 1,000 names, 21-day horizon, 406 month-ends 1993-01 to 2026-10. 1.0 is calibrated. One # is 0.1.
Every estimator that has correlations at all sits within a few hundredths of 1.0 on random portfolios. The one that ignores correlations under-predicts by 4.6 times. This is the test most risk models are shown passing, and it cannot tell a sample covariance from a seventy-factor model.
Optimized portfolios do
statistical v2 ############ 1.20 fundamental (medium) ########## 0.98 fundamental (long) ########## 0.98 fundamental (short) ########## 0.99 hybrid ########## 0.97 sample covariance ################### 1.93 Ledoit-Wolf ############### 1.52 constant correlation ################ 1.58 single factor ################################# 3.34
RMS bias statistic, long-only minimum-variance portfolio of the top 1,000 names, 21-day horizon, 406 month-ends 1993-01 to 2026-10. One # is 0.1. The diagonal baseline reads 15.8 and is off the scale.
The same estimators spread from 0.97 to 3.34 once an optimizer picks the portfolio. The sample covariance under-predicts by nearly two times, Ledoit-Wolf shrinkage by half, the statistical model by a fifth. The fundamental family is within the band. On unconstrained minimum variance the statistical model reads 2.13 and the fundamental (medium) model 0.99 at the same horizon.
Trackers, the portfolios agents actually hold
25 names statistical v2 ######################### 1.26
fundamental (medium) ##################### 1.03
50 names statistical v2 ######################## 1.21
fundamental (medium) ###################### 1.12
100 names statistical v2 ####################### 1.14
fundamental (medium) ####################### 1.15
250 names statistical v2 ########################## 1.31
fundamental (medium) ########################## 1.30
RMS bias statistic of active return, long-only minimum-tracking-error trackers of the S&P 500 at fixed weights, 21-day horizon, 406 month-ends 1993-01 to 2026-10. One # is 0.05.
This is the honest chart. The 25-name tracker went from under-predicting by a quarter to within the band. Trackers of 100 and 250 names did not move: their active risk is small, so whatever risk the factors do not span is a large share of it. This is the open problem we work on. The literature's candidates are nonlinear shrinkage of the factor covariance and an alignment factor built from the optimizer's own holdings (Saxena and Stubbs 2010).
How many names a tracker needs
For the fundamental (medium) model, median predicted and realized annualized tracking error of minimum-tracking-error trackers at a 63-day horizon:
| benchmark | names | predicted TE | realized TE |
|---|---|---|---|
| S&P 500 | 25 | 2.9% | 3.2% |
| S&P 500 | 50 | 1.8% | 2.0% |
| S&P 500 | 100 | 1.0% | 1.2% |
| S&P 500 | 250 | 0.4% | 0.5% |
| Nasdaq 100 | 25 | 1.7% | 1.9% |
| Nasdaq 100 | 50 | 0.7% | 0.9% |
| Russell 3000 | 100 | 1.5% | 1.9% |
| Russell 3000 | 250 | 0.8% | 1.2% |
Realized tracking error runs ten to forty percent above predicted. Abacus Labs reports the predicted number with that correction, as an interval rather than a point. A one-percent target on the S&P 500 needs about 250 names by realized tracking error, which is also what the model itself claims. On the Russell 3000 no name count in the grid reaches one percent.
Crisis onsets
No model in the set is calibrated in the first weeks of a crisis. On random 100-name portfolios at the 21-day horizon in the 2008 to 2009 regime, the fundamental (medium) model reads 2.20, the statistical model 1.99 and constant correlation 2.02. In the 2020 regime the medium model reads 1.97 and the short variant 1.41. The regime adjustment shortens the lag; it does not remove it. Agents get the shorter-half-life variant for stress scenarios, and these numbers ship with every model version.
5. What went wrong and how we fixed it
The evaluation suite found more bugs in itself than in the models. Each one is now a test.
- A bias statistic of 44 is a bug, not a result. The realized-return panel was built from one model version's universe. That version had no ETF rows, so an index tracker holding an ETF lost the position silently, which showed up as a fake minus six percent buy-and-hold day and an S&P 500 tracker bias statistic of 44. The panel is now built from the union of every version's universe, and no table is written if any family exceeds a sanity ceiling.
- Tracker candidates must be benchmark constituents. The first tracker optimizer could pick from the whole universe, so a Nasdaq 100 tracker held names outside the Nasdaq 100. Candidates are now restricted to the benchmark's effective-dated constituents.
- Check that every family actually ran. One test family produced no rows for a whole run and nothing complained. Every family now reports a row count, and a missing family fails the run.
- A windowed product aggregate rounded differently by partition length. Total-return indices built with a database's windowed product were not bit-identical after truncation. The truncation test caught it. Running products are now computed sequentially, one multiplication per bar.
- Filings accepted after the close leaked into that day's decision. The original rule dated a filing to its acceptance day. It is now the first close after acceptance before 16:00 Eastern.
- Zero is not a return. The prototype filled missing returns with zero. Missing stays missing throughout, and a unit test asserts that deleting or randomizing every row after the model date leaves the model unchanged.
6. The decision
The switch rule, written before the fundamental model was built: the candidate becomes the default only if the mean absolute bias deviation of the tracker families and of the optimized families both improve at the 21- and 63-day horizons, and no random or index family gets worse by more than 0.05.
| test | 21 days | 63 days | rule |
|---|---|---|---|
| tracker families, improvement in mean |bias − 1| | +0.065 | +0.096 | pass |
| optimized families, improvement in mean |bias − 1| | +0.444 | +0.627 | pass |
| worst random or index family change | −0.051 | fail by 0.001 |
The family that missed was the equal-weighted top 1,000 at the 63-day horizon, from 1.019 under the statistical model to 1.070 under the fundamental (medium) model. The rule said no. We adopted the fundamental (medium) model as the default anyway, and wrote down why.
First, the tolerance of 0.05 sits inside the sampling band of about 0.07 for 406 dates, so a miss of 0.001 is noise in either direction; we had flagged this before the run. Second, the gains on optimized portfolios are nine to twelve times the tolerance, and optimized portfolios are what the engine produces. Third, the family that moved is the one least like anything an agent holds. The cost is a seven percent under-prediction of that family at one horizon, which we watch on every rebuild. The rule stands, without override, for the next version.
7. Can an LLM do this
We ran two experiments with frontier language models against the fundamental (medium) model, both scored on a held-out period from 2016 to 2026 that the models never saw.
Tuning the estimator's settings. Two frontier models and a random search each got five rounds of three candidates over fifteen estimation settings such as half-lives, Newey-West lags and shrinkage. They saw only training diagnostics from 1993 to 2014 with calendar time hidden. The objective was the mean absolute bias deviation over trackers, optimized and random families at 21 and 63 days, lower is better.
| arm | train | test | vs base, 95% interval | switch rule |
|---|---|---|---|---|
| base, fundamental (medium) | 0.097 | 0.1086 | ||
| model A, best of 15 | 0.068 | 0.1043 | −0.004 (−0.032, +0.008) | fail: optimized worse |
| model B, best of 15 | 0.074 | 0.0926 | −0.016 (−0.025, +0.005) | fail by 0.002 |
| random search, best of 15 | 0.081 | 0.1107 | +0.002 (−0.021, +0.017) | fail |
| one untuned random draw | 0.101 | 0.1045 | −0.004 (−0.009, +0.002) | pass |
The analysis was good: model A found the dominant defect, trackers under-predicting in every block, and refused a blanket risk increase that would have broken index calibration. The improvement did not generalize. Thirty percent gains in training shrank to between zero and fifteen percent on test, no interval excluded zero, and an untuned random draw tied the tuned picks.
Editing the matrices directly. In the second experiment one model read each month's actual output, sixty-one factor volatilities, the full correlation matrix, specific risk by sector and size and a twelve-month track record, and wrote multipliers, for 130 months walk-forward with the date hidden.
| arm | test | trackers | optimized | random and index | P(better than base) |
|---|---|---|---|---|---|
| base, fundamental (medium) | 0.1086 | 0.193 | 0.062 | 0.071 | |
| LLM monthly edits | 0.0880 | 0.153 | 0.040 | 0.071 | 0.97 |
| one-line mechanical rule | 0.0918 | 0.151 | 0.038 | 0.087 | 0.81 |
The edits cut the out-of-sample objective by nineteen percent. But a one-line rule fed the same trailing bias numbers got fifteen percent, and head to head the model against the rule was a coin flip at 0.52. The model's specific-risk multipliers correlated 0.91 with the rule's at about half the strength. Where the rule broke random portfolios, the model did not; that stability was its one real edge. Both fail the switch rule, and in the first quarter of 2020 the edits were clearly worse, so there was no hindsight at work.
Our conclusion: a language model is a useful analyst of a risk model's diagnostics and a poor estimator of its numbers. The estimator stays mechanical, the evaluation stays pre-registered, and the language model's job in Abacus Labs is the conversation, not the covariance.
8. What it means for agents
An agent never sees the matrix. It sees what the matrix implies for one account.
analyze returns the account's risk as annualized volatility, its tracking error against the benchmark the person cares about, its exposures to styles and industries in plain words, and its concentration. propose returns options that each carry risk before and after, against the benchmark, and recomputes the same numbers when the agent sends an option id with a change. commit returns the final trades with the same risk block, from the same model, as of the same date.
"risk": { "before": 0.31, "after": 0.22, "benchmark": 0.17 }
Behind those three numbers are the model above, the delisting returns, the point-in-time filing dates, and the trackers that read 1.03 and 1.30 in the chart. We publish the evaluation with each model version, including the families where it is not yet calibrated.