Exposure Orthogonalization (net_of)#

A style exposure usually carries some of the market, industry and country blocks inside it. A size factor is partly a “big industries” bet, a value factor is partly a “cheap industries” bet. If you want to read the style block on its own, you have to strip that out first.

The net_of field on an exposure group does this: the group’s exposure is replaced by its residual after a weighted projection onto the named sibling groups. The resulting exposure measures its effect net of them.

In this tutorial we will:

Build two models that are identical except that one takes the style block net of the market and industry blocks.
State the invariants the orthogonalized model guarantees, and sketch why they hold.
Check it: see what moves in the served exposures, the factor returns, the same-date fit quantities and the factor covariance, and what does not.
Reconstruct predicted asset returns from exposures() and fret() in both models and show the reconstruction is invariant, provided the exposures are lagged correctly.

Throughout this notebook we work with a randomly generated dataset. The results should generalize to real data, but we do not show any real data on our public API. Bayesline clients can run this notebook on real data.

Imports & Setup#

For this tutorial notebook, you will need to import the following packages.

import numpy as np
import pandas as pd

from bayesline.api.equity import (
    CategoricalExposureGroupSettings,
    CategoricalFilterSettings,
    ContinuousExposureGroupSettings,
    ExposureSettings,
    FactorCovarianceReportSettings,
    FactorRiskModelSettings,
    IdioReportSettings,
    McapExposureReportSettings,
    ModelConstructionSettings,
    UniverseSettings,
)
from bayesline.apiclient import BayeslineApiClient

We will also need to have a Bayesline API client configured.

bln = BayeslineApiClient.new_client(
    endpoint="https://[ENDPOINT]",
    api_key="[API-KEY]",
)

We will work with the US 500 dataset throughout.

DATASET = "bayesline/Bayesline-US-500-1y"

dataset = bln.equity.riskdatasets.load(DATASET)

The net_of field names sibling factor groups of the same ExposureSettings, so the first thing to establish is which hierarchies this dataset offers. Group names default to the hierarchy name, which is what we will use below.

menu = dataset.describe().exposure_settings_menu

print("continuous :", sorted(menu.continuous_hierarchies))
print("categorical:", sorted(menu.categorical_hierarchies))

continuous : ['market', 'other', 'style']
categorical: ['continent', 'estimation_universe', 'trbc']

1. Two models, one setting apart#

Both models have a market intercept, a categorical trbc industry block and a continuous style block. In the second model the style block is declared net_of=["market", "trbc"].

Two of the settings need explaining before we build them.

orthogonalization_weights is owned by the exposures, not by the model. The projection that produces the net-of exposure is defined once, on the ExposureSettings, and is taken over the consuming context’s estimation universe. That makes the orthogonalized exposure one well-defined derived exposure: whatever consumes it (a model, an exposure report, an attribution) sees the same numbers. A consuming model always estimates in this projection basis, whatever its own regression weights happen to be.

orthogonalization_weights defaults to "SqrtCap". Here we set it and the model’s regression weights to "InvIdioVar", for two reasons.

The first is that when the two schemes coincide, the source factors’ returns are exactly the returns of the small model that contains only the source groups: the sources absorb everything the target used to carry on the shared directions, and nothing else. Section 3b builds that small model and checks this.

The second is practical: InvIdioVar is the scheme whose per-asset weights the client will hand back, so we can verify the projection’s defining property directly in section 3a. weights() serves the idiosyncratic volatility \(iv\), and the regression weight is \(1/iv^2\), the same convention the factor returns validation tutorial uses.

A model whose regression weights differ from orthogonalization_weights is still perfectly valid and fits identically, but its source factor returns then only approximate that small model, and a warning is logged when the model is built.

Ordering is by declaration. When several groups declare net_of, the projections apply in the declaration order of the target groups. Each later projection sees the already-orthogonalized earlier targets, so a source may itself be a target that was declared earlier. That is how you chain, for example, an uploaded factor group net of the market, and then the styles net of the market and that factor. Only dense groups (continuous, tsbeta, uploaded) can be targets, but any group can be a source. The chain must be acyclic; a cycle is rejected when the settings are parsed.

base_settings = FactorRiskModelSettings(
    universe=UniverseSettings(),
    exposures=ExposureSettings(
        exposures=[
            ContinuousExposureGroupSettings(hierarchy="market"),
            CategoricalExposureGroupSettings(hierarchy="trbc"),
            ContinuousExposureGroupSettings(
                hierarchy="style",
                standardize_method="equal_weighted",
            ),
        ],
        orthogonalization_weights="InvIdioVar",  # inactive here
    ),
    modelconstruction=ModelConstructionSettings(
        weights="InvIdioVar",
        estimation_universe=UniverseSettings(
            categorical_filters=[
                CategoricalFilterSettings(hierarchy="estimation_universe")
            ],
        ),
        return_clip_bounds=(None, None),
        zero_sum_constraints={"trbc": "mcap_weighted"},
    ),
)

netof_settings = FactorRiskModelSettings(
    universe=UniverseSettings(),
    exposures=ExposureSettings(
        exposures=[
            ContinuousExposureGroupSettings(hierarchy="market"),
            CategoricalExposureGroupSettings(hierarchy="trbc"),
            ContinuousExposureGroupSettings(
                hierarchy="style",
                standardize_method="equal_weighted",
                net_of=["market", "trbc"],
            ),
        ],
        orthogonalization_weights="InvIdioVar",
    ),
    modelconstruction=ModelConstructionSettings(
        weights="InvIdioVar",
        estimation_universe=UniverseSettings(
            categorical_filters=[
                CategoricalFilterSettings(hierarchy="estimation_universe")
            ],
        ),
        return_clip_bounds=(None, None),
        zero_sum_constraints={"trbc": "mcap_weighted"},
    ),
)

sources_settings = FactorRiskModelSettings(
    universe=UniverseSettings(),
    exposures=ExposureSettings(
        exposures=[
            ContinuousExposureGroupSettings(hierarchy="market"),
            CategoricalExposureGroupSettings(hierarchy="trbc"),
        ],
        orthogonalization_weights="InvIdioVar",  # inactive here
    ),
    modelconstruction=ModelConstructionSettings(
        weights="InvIdioVar",
        estimation_universe=UniverseSettings(
            categorical_filters=[
                CategoricalFilterSettings(hierarchy="estimation_universe")
            ],
        ),
        return_clip_bounds=(None, None),
        zero_sum_constraints={"trbc": "mcap_weighted"},
    ),
)

The three settings objects are identical field for field except for their exposure list. netof_settings adds net_of=["market", "trbc"] to the style group; sources_settings drops the style group altogether, and section 3b comes back to it. The one field that carries the whole feature:

print(netof_settings.exposures[0].exposures[-1].model_dump_json(indent=2))

{
  "exposure_type": "continuous",
  "hierarchy": {
    "hierarchy_type": "level",
    "name": "style",
    "level": 1
  },
  "factor_group": "style",
  "include": "All",
  "exclude": [],
  "standardize_method": "equal_weighted",
  "net_of": [
    "market",
    "trbc"
  ]
}

Now we build both models.

base_model = bln.equity.riskmodels.load(
    base_settings.with_dataset(DATASET)
).get_model()

netof_model = bln.equity.riskmodels.load(
    netof_settings.with_dataset(DATASET)
).get_model()

2. The invariants#

Per date, the model is a weighted cross-sectional regression of asset returns on exposures, estimated under zero-sum constraints,

\[\begin{align*} r = Xf + \varepsilon , \qquad Cf = 0 , \end{align*}\]

with \(r\) the asset returns, \(X\) the exposure matrix, \(f\) the factor returns, \(\varepsilon\) the residuals and \(C\) the constraint matrix. In this tutorial’s models \(C\) has one row: the capitalization-weighted industry returns sum to zero. Partition the exposure columns into the source block \(X_S\) (the groups named in net_of) and the target block \(X_T\) (the groups that carry it), and let \(W\) be the diagonal matrix of orthogonalization_weights over the estimation universe.

Orthogonalization changes the exposures and nothing else about the model: the target block is replaced by its residual from a \(W\)-weighted cross-sectional regression on the sources,

\[\begin{align*} X_T \;\to\; \tilde X_T = X_T - X_S\Gamma , \end{align*}\]

with \(\Gamma\) the projection coefficients, while the source block stays exactly as it is. Column by column, \(\tilde X_T\) is the Frisch-Waugh-Lovell residual. Everything the orthogonalized model produces wears a tilde: \(\tilde f\) for its factor returns, \(\tilde\varepsilon\) for its residuals, \(\tilde\Sigma\) for its factor covariance. One more model appears below: the sources-only model, the same regression with the target block removed, whose factor returns we write \(f_S^{\text{only}}\). Its definition carries an assumption we keep throughout: the regression weights are the projection weights \(W\), and the tutorial’s models are configured that way.

The orthogonalized model satisfies the following invariants. How the engine carries the zero-sum constraints, the ridge penalty and the rank deficiency of the sources through the projection is out of scope here; the invariants hold exactly in the constrained, penalized regression the engine actually runs, and sections 3 and 4 verify each invariant by number.

	Invariant
I1	The served source exposures are unchanged	\(\tilde X_S = X_S\)
I2	The targets are weighted-orthogonal to the sources	\(X_S^\top W\tilde X_T = 0\)
I3	The same-date fit is unchanged: fitted values, residuals, idiosyncratic returns, \(R^2\)	\(\tilde\varepsilon = \varepsilon\)
I4	The target factor returns and t-statistics are unchanged	\(\tilde f_T = f_T\)
I5*	The source factor returns equal the sources-only model’s	\(\tilde f_S = f_S^{\text{only}}\)
I6	Predicted asset returns rebuilt from lagged exposures and factor returns are unchanged	\(\tilde X\tilde f = Xf\)
I7	The reported factor returns still satisfy the constraints	\(C\tilde f = 0\)

The asterisk marks the one invariant that does not hold in general: I5 requires the sources to carry no L2 penalty. The other invariants are unconditional.

What is not invariant:

N1 — the served target exposures: they are replaced by \(\tilde X_T\), which differs materially from \(X_T\).
N2 — the source factor returns and their t-statistics: \(\tilde f_S = f_S + \Gamma f_T\), the sources absorb what the targets carried on the shared directions.
N3 — the factor covariance and predicted risk: \(\tilde\Sigma\) is not a rotation of \(\Sigma\), because it is estimated across dates whose bases differ.

The invariants speak only of sources and targets. A group that is neither, present in the model but not named in any net_of and not carrying one, is a bystander: the projection neither modifies its columns nor uses them, so its served exposures, factor returns and t-statistics are all unchanged, and of the covariance blocks involving it only the ones that pair it with a source move. This tutorial has no bystanders: market and industry are the sources, style is the target.

Why the invariants hold#

I2 is the normal equation of the projection: the residual of a weighted regression is orthogonal to the regressors in that weighting. The source block here is rank-deficient by construction (the industry one-hots sum to the market column); directions the sources cannot resolve are dropped from the projection rather than regularized. Equally important is what I2 is not:

Not unweighted orthogonality.
Not orthogonality over the full reporting universe: assets outside the estimation universe receive the same linear combination \(X_T - X_S\Gamma\), but they never entered \(W\), so nothing is zero for them.
Not orthogonality across dates: \(\Gamma_t\) is re-estimated every day, and nothing constrains the target’s time series against the sources’.
Not a re-standardization: the projection is the last step of exposure construction, so the net-of column keeps whatever dispersion the residual has.

The rest follows from a change of basis. The orthogonalized design spans the same column space as the raw one: \(\tilde X = XA\) for an invertible, per-date matrix \(A\). Whatever the model’s own regression weights are, the fit in the new basis is a reparametrization of the raw one: same fitted values, same residuals, and therefore the same idiosyncratic returns, \(R^2\) and \(\sigma^2\) (I3). The coefficients relabel as

\[\begin{align*} \tilde f_T = f_T, \qquad \tilde f_S = f_S + \Gamma f_T . \end{align*}\]

The target returns and their t-statistics are untouched (I4); the source returns absorb the component the targets carried on the shared directions, and their t-statistics move with them (N2). I4 can look surprising, since the projection removes the collinearity between targets and sources, but a multiple-regression coefficient is already the partial one: the target rows of \(A^{-1}\) are the identity, so the target coefficient and its sampling variance are both the raw model’s. What drops is the target’s variance inflation factor, not its t-statistic. The engine carries the constraints and the L2 (ridge) penalty through the reparametrization exactly, so all of the above holds with or without regularization.

I7 needs one more ingredient. The relabeling gives \(C\tilde f = C_S\Gamma f_T\), with \(C_S\) the source columns of \(C\), and that is not zero for a generic \(\Gamma\). But the source block is rank-deficient, so \(\Gamma\) is not unique, and the engine picks the representative with \(C_S\Gamma = 0\): the reported returns then satisfy the constraints, while the served exposures do not depend on that choice.

I5 is the invariant with two cases, and the one thin-industry corrections violate. Without L2 shrinkage on the source factors, Frisch-Waugh-Lovell gives it exactly: \(\tilde f_S\) is what the sources-only model produces, with the same weights and the same constraints. With L2 shrinkage on the sources, which in practice means the thin-industry correction since that is the penalty that lands on categorical groups, the relabeling N2 still holds but I5 breaks, at first order in the penalty: the full model shrinks the source factor’s partial coefficient, net of the orthogonalized target, while the small model shrinks its total coefficient, and the two differ. Shrinkage on the targets alone is harmless, since the coupling term in the projection basis is proportional to the source-side penalty. Relaxing the weights assumption behind \(f_S^{\text{only}}\) breaks I5 the same way: with regression weights that differ from the projection weights the equivalence is only approximate, and the engine logs a warning at build time. I5 is a statement about returns only: even in the exact case the source t-statistics do not match the sources-only model’s, because the residual variances of the two models differ. This tutorial runs the exact case: both weighting schemes are "InvIdioVar", and no shrinkage is configured.

Time#

\(A_t\) is re-estimated every date. The factor return dated \(t\) was fitted on \(X_{t-1}\), so it carries \(A_{t-1}\), while exposures() serves the same-date \(\tilde X_t = X_tA_t\). Paired with the correct lag the bases cancel and \(\tilde X_{t-1}\tilde f_t = X_{t-1}f_t\) exactly (I6); paired without the lag the product drifts by the day-to-day change \(A_t - A_{t-1}\), so always lag the exposures when multiplying against factor returns. Section 4 checks the lagged product.

The factor covariance is estimated across a window of dates, each carrying its own \(A_t\), so no single rotation relates the two matrices: risk decompositions and predicted total risk change (N3), even though every same-date quantity is invariant. The size of the gap depends on how stable the projection is from day to day.

3. What moves, and what does not#

This section measures I1 to I5, I7 and N1 to N3 on the two models; section 4 measures I6. The checks are illustrations, so where one date tells the story we check a single date rather than sweep the history.

First we pull the served exposures of both models into pandas frames indexed by date and asset. Every factor name has the form "{factor_group}.{factor}", which is how we tell the source and target blocks apart below.

X_base = base_model.exposures().to_pandas()
X_base["date"] = pd.to_datetime(X_base["date"])
X_base = X_base.set_index(["date", "bayesid"]).sort_index()
X_base.head()

		market.Market	style.Dividend	style.Growth	style.Leverage	style.Momentum	style.Size	style.Value	style.Volatility	trbc.Academic & Educational Services	trbc.Basic Materials	...	trbc.Consumer Non-Cyclicals	trbc.Energy	trbc.Financials	trbc.Government Activity	trbc.Healthcare	trbc.Industrials	trbc.Institutions, Associations & Organizations	trbc.Real Estate	trbc.Technology	trbc.Utilities
date	bayesid
2025-03-31	IC006CA2E0	1.0	0.172157	0.174729	-0.200713	0.547304	1.535040	0.139031	-0.233793	0.0	0.0	...	0.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
	IC0121F541	1.0	-0.765423	0.595332	2.426294	0.543658	-1.213441	-1.192885	-1.051606	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	1.0	0.0
	IC012D373B	1.0	-0.521896	-0.075329	0.874999	-0.999617	-0.114643	-0.360329	0.819630	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	0.0
	IC01430182	1.0	-1.311038	0.304943	-1.406144	-0.300638	-0.741255	-0.838054	0.958004	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
	IC015A481B	1.0	-1.194783	0.641426	-0.256342	-1.010033	-0.485858	0.055083	0.000344	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	0.0

5 rows × 21 columns

X_netof = netof_model.exposures().to_pandas()
X_netof["date"] = pd.to_datetime(X_netof["date"])
X_netof = X_netof.set_index(["date", "bayesid"]).sort_index()
X_netof.head()

		market.Market	style.Dividend	style.Growth	style.Leverage	style.Momentum	style.Size	style.Value	style.Volatility	trbc.Academic & Educational Services	trbc.Basic Materials	...	trbc.Consumer Non-Cyclicals	trbc.Energy	trbc.Financials	trbc.Government Activity	trbc.Healthcare	trbc.Industrials	trbc.Institutions, Associations & Organizations	trbc.Real Estate	trbc.Technology	trbc.Utilities
date	bayesid
2025-03-31	IC006CA2E0	1.0	-0.248521	1.000184	1.193171	0.007691	1.353103	-1.581763	0.860925	0.0	0.0	...	0.0	0.0	1.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
	IC0121F541	1.0	-0.313065	0.379718	2.585052	0.467686	-1.179875	-0.726704	-1.200021	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	1.0	0.0
	IC012D373B	1.0	-0.194245	-0.386526	0.823835	-1.115739	0.070040	0.016580	1.112704	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	0.0
	IC01430182	1.0	-0.946769	0.066424	-1.144182	-0.390112	-0.789330	-0.167750	1.027516	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
	IC015A481B	1.0	-0.867133	0.330229	-0.307506	-1.126156	-0.301176	0.431992	0.293418	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0	0.0

5 rows × 21 columns

factors = list(X_base.columns)
assert factors == list(X_netof.columns), "both models must serve the same factors"

source_cols = [c for c in factors if c.split(".")[0] in ("market", "trbc")]
target_cols = [c for c in factors if c.split(".")[0] == "style"]

print(f"source factors (market + trbc): {len(source_cols)}")
print(f"target factors (style)        : {len(target_cols)}")

source factors (market + trbc): 14
target factors (style)        : 7

estu = base_model.estimation_universe().to_pandas()
estu["date"] = pd.to_datetime(estu["date"])
estu = (
    estu.set_index("date").sort_index().stack(dropna=False).fillna(0.0).astype(bool)
)
estu.head()

date                  
2025-03-31  IC006CA2E0    True
            IC0121F541    True
            IC012D373B    True
            IC01430182    True
            IC015A481B    True
dtype: bool

check_date = X_base.index.get_level_values("date").max()
print(f"check date: {check_date.date().isoformat()}")

check date: 2026-03-31

3a. The served exposures (I1, I2, N1)#

The exposure report serves the net-of exposures; the orthogonalization is baked into the derived exposure rather than applied downstream.

Theory (I1, N1). The projection only ever replaces the target block, so the served market and industry exposures must be identical between the two models. The style block is replaced by its residual, so it must differ materially.

X0_base = X_base.loc[check_date]
X0_netof = X_netof.loc[check_date]

source_gap = float((X0_base[source_cols] - X0_netof[source_cols]).abs().max().max())
target_gap = float((X0_base[target_cols] - X0_netof[target_cols]).abs().max().max())

print(f"max |dX| on market + trbc (~= 0) : {source_gap:.2e}")
print(f"max |dX| on style         (!= 0) : {target_gap:.4f}")

max |dX| on market + trbc (~= 0) : 0.00e+00
max |dX| on style         (!= 0) : 1.8232

assert source_gap < 1e-7
assert target_gap > 1e-2

The defining property of the projection (I2) is a weighted orthogonality: on each date, over the estimation universe,

\[\begin{align*} X_S^\top W\tilde X_T = 0 , \end{align*}\]

with \(W\) = orthogonalization_weights. Since we chose InvIdioVar for both schemes, those weights are readable: weights() serves the idiosyncratic volatility and the weight is its inverse square. Assets with no usable volatility fall outside the fit and drop out, and so do industries with no members in the estimation universe on the check date.

iv = netof_model.weights().to_pandas()
iv["date"] = pd.to_datetime(iv["date"])
iv = iv.set_index("date").sort_index().stack(dropna=False)

w_proj = 1.0 / iv**2
w_proj.head()

date                  
2025-03-31  IC006CA2E0    7588.635742
            IC0121F541    7874.625977
            IC012D373B    3746.918213
            IC01430182    1287.552734
            IC015A481B    6340.185547
dtype: float32

def weighted_overlap(X):
    """Max normalized w-weighted inner product of target against source columns."""
    df = X.loc[check_date].join(w_proj.loc[check_date].rename("w"))
    df = df[estu.loc[check_date].reindex(df.index, fill_value=False)]
    df = df[np.isfinite(df).all(axis=1)].astype(np.float64)
    A, B, w = df[target_cols].to_numpy(), df[source_cols].to_numpy(), df["w"].to_numpy()
    B = B[:, (B != 0).any(axis=0)]  # industries with no estu members drop out
    num = np.abs(A.T @ (w[:, None] * B))
    norms = np.outer(np.sqrt(w @ A**2), np.sqrt(w @ B**2))
    return float((num / norms).max())


ortho_base_max = weighted_overlap(X_base)
ortho_netof_max = weighted_overlap(X_netof)

Theory (I2). \(X_S^\top W\tilde X_T = 0\) is the defining property of the projection, so for the net-of model this number is zero. Nothing forces it to be small in the base model.

print(f"max normalized <style, source>_w, base  (!= 0) : {ortho_base_max:.4f}")
print(f"max normalized <style, source>_w, net-of (~= 0) : {ortho_netof_max:.2e}")

max normalized <style, source>_w, base  (!= 0) : 0.9335
max normalized <style, source>_w, net-of (~= 0) : 1.12e-06

assert ortho_netof_max < 1e-5
assert ortho_base_max > 1e-2

The base model’s style columns are materially correlated with the market and industry blocks in the regression metric; the net-of model’s are orthogonal. One readable special case: the market column is all ones, so the \(w\)-weighted mean of every net-of style column is zero; a different orthogonalization_weights would zero a different average.

Takeaway: the sources are served unchanged (I1); the target block is a residual (N1) orthogonal to the sources in the regression metric (I2), and reads as “style, net of the market and the industry mix”.

3b. The factor returns (I4, N2, I5, I7)#

Removing the sources’ overlap from the target column leaves the target’s own coefficient alone; what changes is the exposure it multiplies. The source coefficients pick up the piece the target used to carry on the shared directions. That reattribution is the point of the setting.

f_base = base_model.fret().to_pandas()
f_base["date"] = pd.to_datetime(f_base["date"])
f_base = f_base.set_index("date").sort_index()
f_base.head()

	market.Market	style.Dividend	style.Growth	style.Leverage	style.Momentum	style.Size	style.Value	style.Volatility	trbc.Academic & Educational Services	trbc.Basic Materials	...	trbc.Consumer Non-Cyclicals	trbc.Energy	trbc.Financials	trbc.Government Activity	trbc.Healthcare	trbc.Industrials	trbc.Institutions, Associations & Organizations	trbc.Real Estate	trbc.Technology	trbc.Utilities
date
2025-03-31	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.0	0.000000	...	0.000000	0.000000	0.000000	0.0	0.000000	0.000000	0.0	0.000000	0.000000	0.000000
2025-04-01	0.002349	-0.000828	0.000181	-0.001776	0.001438	-0.000864	-0.001598	0.002693	0.0	0.003887	...	0.003093	0.007449	0.003196	0.0	-0.014328	0.002002	0.0	-0.000121	0.000316	0.004612
2025-04-02	0.010565	0.000391	-0.000395	0.000822	-0.000005	-0.001726	0.000266	0.007732	0.0	0.000522	...	0.001291	-0.005086	0.005169	0.0	0.001768	0.001434	0.0	-0.001885	-0.003345	-0.001410
2025-04-03	-0.058157	-0.004164	0.003631	-0.001744	0.002370	0.002108	-0.006327	-0.034921	0.0	0.000392	...	-0.005432	-0.016804	-0.008991	0.0	0.021937	-0.007915	0.0	0.009922	0.000248	0.032806
2025-04-04	-0.059380	0.005454	0.001089	-0.001759	-0.004905	-0.005297	-0.000690	-0.016186	0.0	-0.009005	...	0.007506	-0.037403	-0.006039	0.0	-0.005199	-0.005068	0.0	-0.005398	0.000025	-0.006741

5 rows × 21 columns

f_netof = netof_model.fret().to_pandas()
f_netof["date"] = pd.to_datetime(f_netof["date"])
f_netof = f_netof.set_index("date").sort_index()
f_netof.head()

	market.Market	style.Dividend	style.Growth	style.Leverage	style.Momentum	style.Size	style.Value	style.Volatility	trbc.Academic & Educational Services	trbc.Basic Materials	...	trbc.Consumer Non-Cyclicals	trbc.Energy	trbc.Financials	trbc.Government Activity	trbc.Healthcare	trbc.Industrials	trbc.Institutions, Associations & Organizations	trbc.Real Estate	trbc.Technology	trbc.Utilities
date
2025-03-31	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.0	0.000000	...	0.000000	0.000000	0.000000	0.0	0.000000	0.000000	0.0	0.000000	0.000000	0.000000
2025-04-01	0.002001	-0.000828	0.000181	-0.001776	0.001438	-0.000864	-0.001598	0.002693	0.0	0.002009	...	-0.001560	0.004767	0.000441	0.0	-0.015734	0.002726	0.0	-0.002793	0.002643	-0.000622
2025-04-02	0.007321	0.000391	-0.000395	0.000822	-0.000005	-0.001726	0.000266	0.007732	0.0	0.000981	...	-0.008621	-0.003939	-0.000566	0.0	-0.000323	0.002448	0.0	-0.002452	0.000601	-0.005314
2025-04-03	-0.043249	-0.004164	0.003631	-0.001744	0.002370	0.002108	-0.006327	-0.034921	0.0	-0.004115	...	0.045016	-0.030257	0.002565	0.0	0.030958	-0.008080	0.0	0.012033	-0.014216	0.047002
2025-04-04	-0.053936	0.005454	0.001089	-0.001759	-0.004905	-0.005297	-0.000690	-0.016186	0.0	-0.005331	...	0.033765	-0.037891	0.004840	0.0	-0.000823	-0.006211	0.0	0.004144	-0.009887	0.004461

5 rows × 21 columns

Theory (I4, N2). \(\tilde f_T = f_T\): the style returns are invariant. \(\tilde f_S = f_S + \Gamma f_T\): the market and industry returns absorb the reattributed piece, so they must move materially.

fret_target_gap = float((f_netof[target_cols] - f_base[target_cols]).abs().max().max())
fret_source_gap = float((f_netof[source_cols] - f_base[source_cols]).abs().max().max())

print(f"max |d fret| on style         (~= 0) : {fret_target_gap:.2e}")
print(f"max |d fret| on market + trbc (!= 0) : {fret_source_gap:.2e}")

max |d fret| on style         (~= 0) : 0.00e+00
max |d fret| on market + trbc (!= 0) : 6.83e-02

assert fret_target_gap < 1e-4
assert fret_source_gap > 1e-6

The small-model invariant (I5)#

I5 pins down where the move in N2 lands. Because the regression weights coincide with orthogonalization_weights, and no shrinkage is configured on the source factors (thin_category_shrinkage is empty by default, so the sources carry no ridge), the net-of model’s market and industry returns are the returns of the model that has only market and industries in it.

So we build that model and look.

sources_model = bln.equity.riskmodels.load(
    sources_settings.with_dataset(DATASET)
).get_model()

f_sources = sources_model.fret().to_pandas()
f_sources["date"] = pd.to_datetime(f_sources["date"])
f_sources = f_sources.set_index("date").sort_index()
f_sources.head()

	market.Market	trbc.Academic & Educational Services	trbc.Basic Materials	trbc.Consumer Cyclicals	trbc.Consumer Non-Cyclicals	trbc.Energy	trbc.Financials	trbc.Government Activity	trbc.Healthcare	trbc.Industrials	trbc.Institutions, Associations & Organizations	trbc.Real Estate	trbc.Technology	trbc.Utilities
date
2025-03-31	0.000000	0.0	0.000000	0.000000	0.000000	0.000000	0.000000	0.0	0.000000	0.000000	0.0	0.000000	0.000000	0.000000
2025-04-01	0.002001	0.0	0.002009	0.001984	-0.001560	0.004767	0.000441	0.0	-0.015734	0.002726	0.0	-0.002793	0.002643	-0.000622
2025-04-02	0.007321	0.0	0.000981	0.005751	-0.008621	-0.003939	-0.000566	0.0	-0.000323	0.002448	0.0	-0.002452	0.000601	-0.005314
2025-04-03	-0.043249	0.0	-0.004115	-0.006436	0.045016	-0.030257	0.002565	0.0	0.030958	-0.008080	0.0	0.012033	-0.014216	0.047002
2025-04-04	-0.053936	0.0	-0.005331	0.020711	0.033765	-0.037891	0.004840	0.0	-0.000823	-0.006211	0.0	0.004144	-0.009887	0.004461

Theory (I5). With the two weighting schemes coinciding and no ridge on the sources, the net-of model’s source returns are the sources-only model’s returns. The base model’s are not.

small_model_gap = float((f_netof[source_cols] - f_sources[source_cols]).abs().max().max())
base_model_gap = float((f_base[source_cols] - f_sources[source_cols]).abs().max().max())

print(f"max |net-of fret - sources-only fret| (~= 0) : {small_model_gap:.2e}")
print(f"max |base fret - sources-only fret|   (!= 0) : {base_model_gap:.2e}")

max |net-of fret - sources-only fret| (~= 0) : 6.71e-08
max |base fret - sources-only fret|   (!= 0) : 6.83e-02

assert small_model_gap < 1e-6
assert base_model_gap > 1e-4

Taking the style block net of the sources hands the sources back the returns they would have had if the style block had never been in the regression.

Takeaway: target factor returns are invariant (I4); source factor returns are reattributed (N2) and land on the sources-only model (I5).

The constraints on the reported returns (I7)#

The models estimate under the capitalization-weighted zero-sum constraint on the industries, and I7 says the reported net-of returns still satisfy it. The constraint weights are the industry capitalizations over the estimation universe of the previous serving date, and the mcap-exposure report serves exactly those sums when it is run on the estimation universe.

mcap_settings = netof_settings.model_copy(
    update={"universe": [netof_settings.modelconstruction[0].estimation_universe]}
)
mcap_report = bln.equity.reports.load(
    McapExposureReportSettings(
        factor_model_settings=mcap_settings
    ).with_dataset(DATASET)
).calculate(start_date=None, end_date=None)

f_dates = f_netof.index
x_dates = X_netof.index.get_level_values("date").unique().sort_values()
constraint_date = x_dates[x_dates < f_dates.max()].max()
return_date = f_dates[f_dates > constraint_date].min()

c_row = (
    mcap_report.accessor.get_data(
        [("date", constraint_date.date().isoformat())],
        expand=("factor_group", "factor"),
        value_cols=("mcap_exposure",),
    )
    .to_pandas()
    .rename(columns=lambda c: c.split("^")[0])
)
c_row = c_row[c_row["factor_group"] == "trbc"].set_index("factor")["mcap_exposure"]
c_row.head()

factor
Academic & Educational Services    0.000000e+00
Basic Materials                    1.091547e+12
Consumer Cyclicals                 6.714494e+12
Consumer Non-Cyclicals             4.269818e+12
Energy                             2.592001e+12
Name: mcap_exposure, dtype: float32

Theory (I7). The engine picks the projection representative under which the reported returns keep satisfying the constraints, so the capitalization-weighted sum of the net-of industry returns is zero. An equal-weighted sum has no reason to vanish, which shows the zero is the constraint and not a triviality.

f_ind = f_netof.loc[return_date, [f"trbc.{leaf}" for leaf in c_row.index]]
f_ind.index = c_row.index

constraint_gap = float(abs((c_row * f_ind).sum()) / (c_row.abs() * f_ind.abs()).sum())
equal_weight_gap = float(abs(f_ind.sum()) / f_ind.abs().sum())

print(f"constraint date -> return date       : {constraint_date.date()} -> {return_date.date()}")
print(f"|mcap-weighted sum| / gross  (~= 0)  : {constraint_gap:.2e}")
print(f"|equal-weighted sum| / gross (!= 0)  : {equal_weight_gap:.2e}")

constraint date -> return date       : 2026-03-30 -> 2026-03-31
|mcap-weighted sum| / gross  (~= 0)  : 1.05e-07
|equal-weighted sum| / gross (!= 0)  : 5.31e-01

assert constraint_gap < 1e-3
assert equal_weight_gap > 1e-3

3c. Same-date fit quantities (I3, I4)#

The idiosyncratic return report is the direct read on I3: the residuals must be identical asset by asset and date by date.

def load_idio_returns(settings):
    report = bln.equity.reports.load(
        IdioReportSettings(factor_model_settings=settings).with_dataset(DATASET)
    ).calculate(start_date=None, end_date=None)
    df = report.accessor.get_data(
        [], expand=("date", "asset_id"), value_cols=("idio_return",)
    ).to_pandas()
    df = df.rename(columns=lambda c: c.split("^")[0])
    df["date"] = pd.to_datetime(df["date"])
    return df.set_index(["date", "asset_id"]).sort_index()


idio_base = load_idio_returns(base_settings)
idio_netof = load_idio_returns(netof_settings)
idio_base.head()

		idio_return
date	asset_id
2025-03-31	IC006CA2E0	NaN
	IC0121F541	NaN
	IC012D373B	NaN
	IC01430182	NaN
	IC015A481B	NaN

Theory (I3). The two designs span the same column space, so the fit is the same and the residuals are identical asset by asset and date by date.

idio_date = idio_base.index.get_level_values("date").max()
idio_gap = float(
    (idio_netof.loc[idio_date, "idio_return"] - idio_base.loc[idio_date, "idio_return"])
    .abs()
    .max()
)

print(f"date                                 : {idio_date.date().isoformat()}")
print(f"max |idio net-of - idio base| (~= 0) : {idio_gap:.2e}")

date                                 : 2026-03-31
max |idio net-of - idio base| (~= 0) : 1.12e-08

assert idio_gap < 1e-6

The t-statistics follow the coefficients: the target’s are unchanged (I4), the sources’ move with their returns (N2).

t_base = base_model.t_stats().to_pandas()
t_base["date"] = pd.to_datetime(t_base["date"])
t_base = t_base.set_index("date").sort_index()

t_netof = netof_model.t_stats().to_pandas()
t_netof["date"] = pd.to_datetime(t_netof["date"])
t_netof = t_netof.set_index("date").sort_index()

t_target_gap = float((t_netof[target_cols] - t_base[target_cols]).abs().max().max())
t_source_gap = float((t_netof[source_cols] - t_base[source_cols]).abs().max().max())

print(f"max |d t-stat| on style         (~= 0) : {t_target_gap:.2e}")
print(f"max |d t-stat| on market + trbc (!= 0) : {t_source_gap:.2e}")

max |d t-stat| on style         (~= 0) : 9.54e-07
max |d t-stat| on market + trbc (!= 0) : 2.31e+01

assert t_target_gap < 1e-4
assert t_source_gap > 1e-2

Takeaway: the model fits the data identically (I3): orthogonalization is a change of basis, not a different regression.

3d. The factor covariance, and predicted risk (N3)#

Risk forecasts are where the invariance ends (N3): the covariance is estimated across a window of dates, each in its own basis, and an EWMA over a rotated return series is not the rotation of the EWMA.

# the covariance report labels factors by their leaf name; map them back onto the
# fully qualified "{factor_group}.{factor}" names the exposures use
qualified = {f: f"{g}.{f}" for g, fs in base_model.factors().items() for f in fs}
assert len(qualified) == sum(len(fs) for fs in base_model.factors().values()), (
    "leaf factor names are not unique across groups"
)

def load_factor_covariance(settings):
    report = bln.equity.reports.load(
        FactorCovarianceReportSettings(
            factor_model_settings=settings
        ).with_dataset(DATASET)
    ).calculate(start_date=None, end_date=None)
    df = report.get_covariance().to_pandas()
    df = df.rename(columns=lambda c: c.split("^")[0])
    df["date"] = pd.to_datetime(df["date"])
    return (
        df.set_index(["date", "factor"])
        .rename(index=qualified, columns=qualified)
        .sort_index()
    )


fcov_base = load_factor_covariance(base_settings)
fcov_netof = load_factor_covariance(netof_settings)
fcov_base.head()

		trbc.Academic & Educational Services	trbc.Basic Materials	trbc.Consumer Cyclicals	trbc.Consumer Non-Cyclicals	style.Dividend	trbc.Energy	trbc.Financials	trbc.Government Activity	style.Growth	trbc.Healthcare	...	trbc.Institutions, Associations & Organizations	style.Leverage	market.Market	style.Momentum	trbc.Real Estate	style.Size	trbc.Technology	trbc.Utilities	style.Value	style.Volatility
date	factor
2025-03-31	market.Market	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
	style.Dividend	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
	style.Growth	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
	style.Leverage	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
	style.Momentum	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

5 rows × 21 columns

We compare on the last date both the model and the covariance report cover.

Theory (I4, N3). The style/style block is built from the style returns, which are invariant (I4), so it is unchanged. The blocks that touch a source move (N3); industries with no members carry zero returns in both models, so their rows sit out.

last_date = (
    fcov_base.index.get_level_values("date")
    .unique()
    .intersection(X_base.index.get_level_values("date").unique())
    .max()
)

F_base = fcov_base.loc[last_date]
F_netof = fcov_netof.loc[last_date].reindex(
    index=F_base.index, columns=F_base.columns
)

d_fcov = (F_netof - F_base).abs()

fcov_style_gap = float(np.nanmax(d_fcov.loc[target_cols, target_cols].to_numpy()))
fcov_source_gap = float(np.nanmax(d_fcov.loc[source_cols].to_numpy()))

print(f"fcov style/style block        (~= 0) : {fcov_style_gap:.2e}")
print(f"fcov source-touching entries  (!= 0) : {fcov_source_gap:.2e}")

fcov style/style block        (~= 0) : 0.00e+00
fcov source-touching entries  (!= 0) : 1.23e-02

assert fcov_style_gap < 1e-5
assert fcov_source_gap > 1e-4

Now the risk of an actual portfolio. We take the equally weighted estimation universe on the last date, compute its factor exposure \(b=w^\top X\) in each model’s own basis, and read off the predicted factor volatility \(\sqrt{b^\top Fb}\). This is factor risk only; the idiosyncratic part is a same-date quantity and does not move.

Theory (N3). Predicted risk is the one quantity the theory does not pin down: the covariance is estimated across a window of dates, each with its own rotation, so the number must move. How far it moves is not predicted, only that it is non-zero.

held = estu.loc[last_date]
held = held[held].index
w = np.full(len(held), 1.0 / len(held))

# A factor with no assets in the estimation universe has an undefined return, and so
# an undefined variance; drop those rows and columns before forming the quadratic form.
usable = F_base.index[
    np.isfinite(F_base.to_numpy()).all(axis=1)
    & np.isfinite(F_netof.to_numpy()).all(axis=1)
]

b_base = X_base.loc[last_date].loc[held, usable].fillna(0.0).to_numpy().T @ w
b_netof = X_netof.loc[last_date].loc[held, usable].fillna(0.0).to_numpy().T @ w

vol_base = float(np.sqrt(b_base @ F_base.loc[usable, usable].to_numpy() @ b_base))
vol_netof = float(np.sqrt(b_netof @ F_netof.loc[usable, usable].to_numpy() @ b_netof))
vol_change = vol_netof / vol_base - 1.0

print(f"date                                 : {last_date.date().isoformat()}")
print(f"assets held                          : {len(held):,}")
print(f"factors with an undefined variance   : {len(F_base.index) - len(usable)}")
print(f"predicted factor vol, base           : {vol_base:.4f}")
print(f"predicted factor vol, net-of         : {vol_netof:.4f}")
print(f"relative change              (!= 0)  : {vol_change:+.2%}")

date                                 : 2026-03-31
assets held                          : 492
factors with an undefined variance   : 0
predicted factor vol, base           : 0.1502
predicted factor vol, net-of         : 0.1609
relative change              (!= 0)  : +7.15%

assert abs(vol_change) > 1e-3

Takeaway: expect risk forecasts to move; how far depends on how stable the daily projection is, and the randomly generated data used here makes it unstable. If you are tying out risk numbers against a reference, the reference has to use the same orthogonalization convention; there is no post-hoc rotation that recovers one from the other.

4. Reconstructing predicted returns from exposures and factor returns (I6)#

This is the check that matters most in practice, and the one that is easiest to get wrong.

Two facts combine here:

Factor returns at date \(t\) pair with exposures at date \(t-1\). The cross-sectional regression run on date \(t-1\) explains the next day’s asset return, so the coefficient it produces is stamped with date \(t\) while the design matrix it multiplies is the one from \(t-1\).
exposures() serves the same-date orthogonalized exposures. There is no lag baked into them.

Put those together and the reconstruction is

\[\begin{align*} \hat r_{i,t} = \sum_j \tilde X_{i,j,t-1}\,\tilde f_{j,t}, \end{align*}\]

which by the cancellation in section 2 must give the same answer in both models: both sides carry the same \(A_{t-1}\), the one belonging to the date the regression was run on.

def reconstruct(X, f):
    """Sum lagged exposures times factor returns per asset and date."""
    ft = f.shift(-1).dropna(how="all")
    Xp = X[f.columns].loc[X.index.get_level_values("date").isin(ft.index)]
    return pd.Series(
        np.nansum(
            Xp.to_numpy() * ft.reindex(Xp.index.get_level_values("date")).to_numpy(),
            axis=1,
        ),
        index=Xp.index,
        name="reconstruction",
    )


recon_base = reconstruct(X_base, f_base)
recon_netof = reconstruct(X_netof, f_netof)
recon_base.head()

date        bayesid   
2025-03-31  IC006CA2E0    0.004399
            IC0121F541    0.000004
            IC012D373B    0.004661
            IC01430182    0.009969
            IC015A481B    0.004792
Name: reconstruction, dtype: float32

We compare on the estimation universe, which is where the regression was actually fit.

Theory (I6). \(\tilde X_{t-1}\tilde f_t = X_{t-1}f_t\) exactly, so the two reconstructions agree.

in_estu = estu.reindex(recon_base.index).fillna(False).astype(bool).to_numpy()

lhs = recon_netof.to_numpy()[in_estu]
rhs = recon_base.to_numpy()[in_estu]
scale = float(np.sqrt(np.mean(rhs**2)))
recon_gap = float(np.abs(lhs - rhs).max() / scale)

print(f"assets x dates compared                 : {lhs.size:,}")
print(f"rms of reconstruction                   : {scale:.4f}")
print(f"max |difference|                        : {np.abs(lhs - rhs).max():.2e}")
print(f"max |difference| / rms          (~= 0)  : {recon_gap:.2e}")

assets x dates compared                 : 123,853
rms of reconstruction                   : 0.0151
max |difference|                        : 4.47e-08
max |difference| / rms          (~= 0)  : 2.95e-06

The reconstructions are predicted asset returns, so we compare them on a scale-normalized basis rather than in absolute terms.

np.testing.assert_allclose(lhs / scale, rhs / scale, rtol=0.0, atol=2e-5)

The two models reconstruct the same predicted asset returns.

Checks at a glance#

Every check in sections 3 and 4, keyed to the invariant or non-invariant it verifies.

checks = pd.DataFrame(
    [
        ("I1", "served source exposures (~= 0)", "zero", f"{source_gap:.1e}"),
        ("N1", "served target exposures (!= 0)", "non-zero", f"{target_gap:.1e}"),
        ("I2", "weighted <style, source>, net-of (~= 0)", "zero", f"{ortho_netof_max:.1e}"),
        ("I2", "weighted <style, source>, base (!= 0)", "non-zero", f"{ortho_base_max:.1e}"),
        ("I4", "target factor returns (~= 0)", "zero", f"{fret_target_gap:.1e}"),
        ("N2", "source factor returns (!= 0)", "non-zero", f"{fret_source_gap:.1e}"),
        ("I5", "source fret vs sources-only model (~= 0)", "zero", f"{small_model_gap:.1e}"),
        ("I5", "base fret vs sources-only model (!= 0)", "non-zero", f"{base_model_gap:.1e}"),
        ("I7", "mcap-weighted industry return sum (~= 0)", "zero", f"{constraint_gap:.1e}"),
        ("I7", "equal-weighted industry return sum (!= 0)", "non-zero", f"{equal_weight_gap:.1e}"),
        ("I3", "idiosyncratic returns (~= 0)", "zero", f"{idio_gap:.1e}"),
        ("I4", "target t-stats (~= 0)", "zero", f"{t_target_gap:.1e}"),
        ("N2", "source t-stats (!= 0)", "non-zero", f"{t_source_gap:.1e}"),
        ("I4", "fcov style/style block (~= 0)", "zero", f"{fcov_style_gap:.1e}"),
        ("N3", "fcov source-touching blocks (!= 0)", "non-zero", f"{fcov_source_gap:.1e}"),
        ("N3", "predicted factor vol change (!= 0)", "non-zero", f"{vol_change:.1e}"),
        ("I6", "lagged reconstruction (~= 0)", "zero", f"{recon_gap:.1e}"),
    ],
    columns=["id", "quantity", "theory", "measured"],
)
checks

	id	quantity	theory	measured
0	I1	served source exposures (~= 0)	zero	0.0e+00
1	N1	served target exposures (!= 0)	non-zero	1.8e+00
2	I2	weighted <style, source>, net-of (~= 0)	zero	1.1e-06
3	I2	weighted <style, source>, base (!= 0)	non-zero	9.3e-01
4	I4	target factor returns (~= 0)	zero	0.0e+00
5	N2	source factor returns (!= 0)	non-zero	6.8e-02
6	I5	source fret vs sources-only model (~= 0)	zero	6.7e-08
7	I5	base fret vs sources-only model (!= 0)	non-zero	6.8e-02
8	I7	mcap-weighted industry return sum (~= 0)	zero	1.1e-07
9	I7	equal-weighted industry return sum (!= 0)	non-zero	5.3e-01
10	I3	idiosyncratic returns (~= 0)	zero	1.1e-08
11	I4	target t-stats (~= 0)	zero	9.5e-07
12	N2	source t-stats (!= 0)	non-zero	2.3e+01
13	I4	fcov style/style block (~= 0)	zero	0.0e+00
14	N3	fcov source-touching blocks (!= 0)	non-zero	1.2e-02
15	N3	predicted factor vol change (!= 0)	non-zero	7.2e-02
16	I6	lagged reconstruction (~= 0)	zero	3.0e-06

Exposure Orthogonalization (net_of)

Contents

Exposure Orthogonalization (net_of)#

Imports & Setup#

1. Two models, one setting apart#

2. The invariants#

Why the invariants hold#

Time#

3. What moves, and what does not#

3a. The served exposures (I1, I2, N1)#

3b. The factor returns (I4, N2, I5, I7)#

The small-model invariant (I5)#

The constraints on the reported returns (I7)#

3c. Same-date fit quantities (I3, I4)#

3d. The factor covariance, and predicted risk (N3)#

4. Reconstructing predicted returns from exposures and factor returns (I6)#

Checks at a glance#