{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "28743367",
      "metadata": {},
      "source": [
        "# Factor Covariance Matrix Forecasts\n",
        "\n",
        "In this tutorial we are going to create several covariance matrices of factor returns. This covariance matrix follows from a standard factor model. We will also tie out the numbers against a pandas/numpy-based reimplementation. More specifically, we will:\n",
        "- Create a basic risk model.\n",
        "- Extract the factor returns.\n",
        "- Use the covariance matrix forecast report to compute the covariance matrix.\n",
        "- Use pandas to extract the factor volatility and correlation matrix time-series.\n",
        "- Replicate the volatility forecast.\n",
        "- Replicate the correlation forecast.\n",
        "\n",
        "Throughout this notebook we work with a randomly generated dataset. The results should generalize to real data, but for legal reasons we do not show any real data on our public API. Bayesline clients can run this notebook on real data."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5f63f2ee",
      "metadata": {},
      "source": [
        "## Imports & Setup\n",
        "\n",
        "For this tutorial notebook, you will need to import the following packages."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "84d7731b",
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import polars as pl\n",
        "import numpy as np\n",
        "\n",
        "from bayesline.api.equity import (\n",
        "    FactorCovarianceReportSettings,\n",
        "    ExposureSettings,\n",
        "    CategoricalExposureGroupSettings,\n",
        "    ContinuousExposureGroupSettings,\n",
        "    FactorRiskModelSettings,\n",
        "    ModelConstructionSettings,\n",
        "    UniverseSettings,\n",
        ")\n",
        "from bayesline.apiclient import BayeslineApiClient"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8499d1d2",
      "metadata": {},
      "source": [
        "We will also need to have a Bayesline API client configured."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d91890f6",
      "metadata": {
        "tags": [
          "skip-execution"
        ]
      },
      "outputs": [],
      "source": [
        "bln = BayeslineApiClient.new_client(\n",
        "    endpoint=\"https://[ENDPOINT]\",\n",
        "    api_key=\"[API-KEY]\",\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2218e6ea",
      "metadata": {},
      "source": [
        "## Creating the covariance matrix forecasts\n",
        "\n",
        "Let's first set up a basic risk model and use it to generate the forecasts. We choose to run with mostly default settings. The steps involved are:\n",
        "1. Creating the settings of the risk model.\n",
        "2. Loading the report model engine.\n",
        "3. Running the engine to generate the covariance report.\n",
        "\n",
        "The first step is creating the risk model settings."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "id": "3ec54c2c",
      "metadata": {},
      "outputs": [],
      "source": [
        "factorriskmodel_settings = FactorRiskModelSettings(\n",
        "    universe=UniverseSettings(),\n",
        "    exposures=ExposureSettings(\n",
        "        exposures=[\n",
        "            ContinuousExposureGroupSettings(hierarchy=\"market\"),\n",
        "            CategoricalExposureGroupSettings(hierarchy=\"trbc\"),\n",
        "            ContinuousExposureGroupSettings(hierarchy=\"style\"),\n",
        "        ]\n",
        "    ),\n",
        "    modelconstruction=ModelConstructionSettings(\n",
        "        estimation_universe=None,\n",
        "        zero_sum_constraints={\"trbc\": \"mcap_weighted\"},\n",
        "    ),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9bdfcbac",
      "metadata": {},
      "source": [
        "Next, we create the report engine from the report settings. We run with the defaults here."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "id": "80b68156",
      "metadata": {},
      "outputs": [],
      "source": [
        "report_settings = FactorCovarianceReportSettings(\n",
        "    factor_model_settings=factorriskmodel_settings\n",
        ")\n",
        "report_engine = bln.equity.reports.load(\n",
        "    report_settings.with_dataset(\"bayesline/Bayesline-US-All-1y\")\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8bceda67",
      "metadata": {},
      "source": [
        "Let's see what these settings really are by printing them out."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "id": "474692a6",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"halflife_vol\": 60.0,\n",
            "  \"halflife_cor\": 120.0,\n",
            "  \"halflife_vra\": null,\n",
            "  \"nw_lags_vol\": 0,\n",
            "  \"nw_lags_vol_halflife_override\": null,\n",
            "  \"nw_lags_cor\": 0,\n",
            "  \"shrink_cor_method\": null,\n",
            "  \"shrink_cor_length\": null,\n",
            "  \"combine_standardized\": false\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "print(report_settings.factor_cov_settings.model_dump_json(indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "31401d40",
      "metadata": {},
      "source": [
        "The different settings that jointly make up the covariance matrix are:\n",
        "1. **halflife_factor_vol** The halflife of the factor volatility. The default is a 60-day halflife.\n",
        "2. **halflife_factor_vra** The halflife of the cross-sectional factor volatility adjustment. The default is to not do any adjustment.\n",
        "3. **halflife_factor_cor** The halflife of the factor correlation. The default is a 120-day halflife.\n",
        "4. **nw_lags_factor_vol** The overlap or Newey-West lags to incluce on the factor volatility forecast. The default is zero, meaning no autocorrelation correction is performed.\n",
        "5. **nw_lags_factor_cor** The overlap or Newey-West lags to incluce on the factor correlation forecast. The default is zero, meaning no autocorrelation correction is performed.\n",
        "\n",
        "Now we get the actual time-series of the covariance matrices."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "id": "ea65cfe8",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2025-04-01</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <td>0.028429</td>\n",
              "      <td>-0.001902</td>\n",
              "      <td>0.007420</td>\n",
              "      <td>0.003352</td>\n",
              "      <td>-0.003835</td>\n",
              "      <td>0.013457</td>\n",
              "      <td>0.003418</td>\n",
              "      <td>-0.008200</td>\n",
              "      <td>0.004291</td>\n",
              "      <td>-0.060944</td>\n",
              "      <td>...</td>\n",
              "      <td>0.018205</td>\n",
              "      <td>-0.000299</td>\n",
              "      <td>0.001203</td>\n",
              "      <td>0.004113</td>\n",
              "      <td>-0.004071</td>\n",
              "      <td>0.001764</td>\n",
              "      <td>0.008914</td>\n",
              "      <td>0.006076</td>\n",
              "      <td>-0.003778</td>\n",
              "      <td>0.006381</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Basic Materials</th>\n",
              "      <td>-0.001902</td>\n",
              "      <td>0.000127</td>\n",
              "      <td>-0.000496</td>\n",
              "      <td>-0.000224</td>\n",
              "      <td>0.000257</td>\n",
              "      <td>-0.000900</td>\n",
              "      <td>-0.000229</td>\n",
              "      <td>0.000549</td>\n",
              "      <td>-0.000287</td>\n",
              "      <td>0.004077</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.001218</td>\n",
              "      <td>0.000020</td>\n",
              "      <td>-0.000080</td>\n",
              "      <td>-0.000275</td>\n",
              "      <td>0.000272</td>\n",
              "      <td>-0.000118</td>\n",
              "      <td>-0.000596</td>\n",
              "      <td>-0.000406</td>\n",
              "      <td>0.000253</td>\n",
              "      <td>-0.000427</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <td>0.007420</td>\n",
              "      <td>-0.000496</td>\n",
              "      <td>0.001936</td>\n",
              "      <td>0.000875</td>\n",
              "      <td>-0.001001</td>\n",
              "      <td>0.003512</td>\n",
              "      <td>0.000892</td>\n",
              "      <td>-0.002140</td>\n",
              "      <td>0.001120</td>\n",
              "      <td>-0.015905</td>\n",
              "      <td>...</td>\n",
              "      <td>0.004751</td>\n",
              "      <td>-0.000078</td>\n",
              "      <td>0.000314</td>\n",
              "      <td>0.001073</td>\n",
              "      <td>-0.001063</td>\n",
              "      <td>0.000460</td>\n",
              "      <td>0.002326</td>\n",
              "      <td>0.001586</td>\n",
              "      <td>-0.000986</td>\n",
              "      <td>0.001665</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <td>0.003352</td>\n",
              "      <td>-0.000224</td>\n",
              "      <td>0.000875</td>\n",
              "      <td>0.000395</td>\n",
              "      <td>-0.000452</td>\n",
              "      <td>0.001587</td>\n",
              "      <td>0.000403</td>\n",
              "      <td>-0.000967</td>\n",
              "      <td>0.000506</td>\n",
              "      <td>-0.007185</td>\n",
              "      <td>...</td>\n",
              "      <td>0.002146</td>\n",
              "      <td>-0.000035</td>\n",
              "      <td>0.000142</td>\n",
              "      <td>0.000485</td>\n",
              "      <td>-0.000480</td>\n",
              "      <td>0.000208</td>\n",
              "      <td>0.001051</td>\n",
              "      <td>0.000716</td>\n",
              "      <td>-0.000445</td>\n",
              "      <td>0.000752</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Dividend</th>\n",
              "      <td>-0.003835</td>\n",
              "      <td>0.000257</td>\n",
              "      <td>-0.001001</td>\n",
              "      <td>-0.000452</td>\n",
              "      <td>0.000517</td>\n",
              "      <td>-0.001815</td>\n",
              "      <td>-0.000461</td>\n",
              "      <td>0.001106</td>\n",
              "      <td>-0.000579</td>\n",
              "      <td>0.008220</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.002456</td>\n",
              "      <td>0.000040</td>\n",
              "      <td>-0.000162</td>\n",
              "      <td>-0.000555</td>\n",
              "      <td>0.000549</td>\n",
              "      <td>-0.000238</td>\n",
              "      <td>-0.001202</td>\n",
              "      <td>-0.000819</td>\n",
              "      <td>0.000510</td>\n",
              "      <td>-0.000861</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2026-03-31</th>\n",
              "      <th>Size</th>\n",
              "      <td>-0.000405</td>\n",
              "      <td>-0.000013</td>\n",
              "      <td>0.000116</td>\n",
              "      <td>-0.001077</td>\n",
              "      <td>-0.000072</td>\n",
              "      <td>-0.000336</td>\n",
              "      <td>0.000726</td>\n",
              "      <td>-0.002446</td>\n",
              "      <td>0.000066</td>\n",
              "      <td>-0.000563</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003470</td>\n",
              "      <td>-0.000025</td>\n",
              "      <td>0.001394</td>\n",
              "      <td>0.000010</td>\n",
              "      <td>-0.000314</td>\n",
              "      <td>0.000658</td>\n",
              "      <td>-0.000034</td>\n",
              "      <td>-0.001104</td>\n",
              "      <td>0.000190</td>\n",
              "      <td>0.001719</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Technology</th>\n",
              "      <td>-0.001527</td>\n",
              "      <td>-0.004031</td>\n",
              "      <td>-0.002431</td>\n",
              "      <td>-0.003394</td>\n",
              "      <td>-0.000199</td>\n",
              "      <td>-0.002704</td>\n",
              "      <td>-0.000786</td>\n",
              "      <td>-0.002595</td>\n",
              "      <td>0.000120</td>\n",
              "      <td>-0.002557</td>\n",
              "      <td>...</td>\n",
              "      <td>0.001805</td>\n",
              "      <td>-0.000812</td>\n",
              "      <td>0.001121</td>\n",
              "      <td>0.000666</td>\n",
              "      <td>-0.002040</td>\n",
              "      <td>-0.000034</td>\n",
              "      <td>0.003285</td>\n",
              "      <td>-0.001418</td>\n",
              "      <td>-0.000566</td>\n",
              "      <td>0.001943</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Utilities</th>\n",
              "      <td>-0.002906</td>\n",
              "      <td>0.003537</td>\n",
              "      <td>-0.002813</td>\n",
              "      <td>0.005933</td>\n",
              "      <td>0.000259</td>\n",
              "      <td>0.005272</td>\n",
              "      <td>-0.001585</td>\n",
              "      <td>0.010294</td>\n",
              "      <td>-0.000297</td>\n",
              "      <td>0.000660</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.001325</td>\n",
              "      <td>-0.000956</td>\n",
              "      <td>-0.004809</td>\n",
              "      <td>0.003463</td>\n",
              "      <td>0.005295</td>\n",
              "      <td>-0.001104</td>\n",
              "      <td>-0.001418</td>\n",
              "      <td>0.016741</td>\n",
              "      <td>-0.001023</td>\n",
              "      <td>-0.002881</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Value</th>\n",
              "      <td>0.000100</td>\n",
              "      <td>0.001360</td>\n",
              "      <td>0.001193</td>\n",
              "      <td>0.000096</td>\n",
              "      <td>0.000155</td>\n",
              "      <td>0.001086</td>\n",
              "      <td>-0.000130</td>\n",
              "      <td>0.000400</td>\n",
              "      <td>0.000088</td>\n",
              "      <td>0.000146</td>\n",
              "      <td>...</td>\n",
              "      <td>0.002249</td>\n",
              "      <td>0.000358</td>\n",
              "      <td>0.001284</td>\n",
              "      <td>-0.000321</td>\n",
              "      <td>-0.000345</td>\n",
              "      <td>0.000190</td>\n",
              "      <td>-0.000566</td>\n",
              "      <td>-0.001023</td>\n",
              "      <td>0.001101</td>\n",
              "      <td>0.000709</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Volatility</th>\n",
              "      <td>-0.008378</td>\n",
              "      <td>-0.002410</td>\n",
              "      <td>-0.002730</td>\n",
              "      <td>-0.006672</td>\n",
              "      <td>-0.000017</td>\n",
              "      <td>-0.005933</td>\n",
              "      <td>0.004099</td>\n",
              "      <td>-0.036540</td>\n",
              "      <td>-0.000107</td>\n",
              "      <td>-0.007134</td>\n",
              "      <td>...</td>\n",
              "      <td>0.027899</td>\n",
              "      <td>-0.001629</td>\n",
              "      <td>0.014689</td>\n",
              "      <td>0.003581</td>\n",
              "      <td>-0.002040</td>\n",
              "      <td>0.001719</td>\n",
              "      <td>0.001943</td>\n",
              "      <td>-0.002881</td>\n",
              "      <td>0.000709</td>\n",
              "      <td>0.022883</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5271 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                            Academic & Educational Services  \\\n",
              "date       factor                                                             \n",
              "2025-04-01 Academic & Educational Services                         0.028429   \n",
              "           Basic Materials                                        -0.001902   \n",
              "           Consumer Cyclicals                                      0.007420   \n",
              "           Consumer Non-Cyclicals                                  0.003352   \n",
              "           Dividend                                               -0.003835   \n",
              "...                                                                     ...   \n",
              "2026-03-31 Size                                                   -0.000405   \n",
              "           Technology                                             -0.001527   \n",
              "           Utilities                                              -0.002906   \n",
              "           Value                                                   0.000100   \n",
              "           Volatility                                             -0.008378   \n",
              "\n",
              "                                            Basic Materials  \\\n",
              "date       factor                                             \n",
              "2025-04-01 Academic & Educational Services        -0.001902   \n",
              "           Basic Materials                         0.000127   \n",
              "           Consumer Cyclicals                     -0.000496   \n",
              "           Consumer Non-Cyclicals                 -0.000224   \n",
              "           Dividend                                0.000257   \n",
              "...                                                     ...   \n",
              "2026-03-31 Size                                   -0.000013   \n",
              "           Technology                             -0.004031   \n",
              "           Utilities                               0.003537   \n",
              "           Value                                   0.001360   \n",
              "           Volatility                             -0.002410   \n",
              "\n",
              "                                            Consumer Cyclicals  \\\n",
              "date       factor                                                \n",
              "2025-04-01 Academic & Educational Services            0.007420   \n",
              "           Basic Materials                           -0.000496   \n",
              "           Consumer Cyclicals                         0.001936   \n",
              "           Consumer Non-Cyclicals                     0.000875   \n",
              "           Dividend                                  -0.001001   \n",
              "...                                                        ...   \n",
              "2026-03-31 Size                                       0.000116   \n",
              "           Technology                                -0.002431   \n",
              "           Utilities                                 -0.002813   \n",
              "           Value                                      0.001193   \n",
              "           Volatility                                -0.002730   \n",
              "\n",
              "                                            Consumer Non-Cyclicals  Dividend  \\\n",
              "date       factor                                                              \n",
              "2025-04-01 Academic & Educational Services                0.003352 -0.003835   \n",
              "           Basic Materials                               -0.000224  0.000257   \n",
              "           Consumer Cyclicals                             0.000875 -0.001001   \n",
              "           Consumer Non-Cyclicals                         0.000395 -0.000452   \n",
              "           Dividend                                      -0.000452  0.000517   \n",
              "...                                                            ...       ...   \n",
              "2026-03-31 Size                                          -0.001077 -0.000072   \n",
              "           Technology                                    -0.003394 -0.000199   \n",
              "           Utilities                                      0.005933  0.000259   \n",
              "           Value                                          0.000096  0.000155   \n",
              "           Volatility                                    -0.006672 -0.000017   \n",
              "\n",
              "                                              Energy  Financials  \\\n",
              "date       factor                                                  \n",
              "2025-04-01 Academic & Educational Services  0.013457    0.003418   \n",
              "           Basic Materials                 -0.000900   -0.000229   \n",
              "           Consumer Cyclicals               0.003512    0.000892   \n",
              "           Consumer Non-Cyclicals           0.001587    0.000403   \n",
              "           Dividend                        -0.001815   -0.000461   \n",
              "...                                              ...         ...   \n",
              "2026-03-31 Size                            -0.000336    0.000726   \n",
              "           Technology                      -0.002704   -0.000786   \n",
              "           Utilities                        0.005272   -0.001585   \n",
              "           Value                            0.001086   -0.000130   \n",
              "           Volatility                      -0.005933    0.004099   \n",
              "\n",
              "                                            Government Activity    Growth  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Academic & Educational Services            -0.008200  0.004291   \n",
              "           Basic Materials                             0.000549 -0.000287   \n",
              "           Consumer Cyclicals                         -0.002140  0.001120   \n",
              "           Consumer Non-Cyclicals                     -0.000967  0.000506   \n",
              "           Dividend                                    0.001106 -0.000579   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Size                                       -0.002446  0.000066   \n",
              "           Technology                                 -0.002595  0.000120   \n",
              "           Utilities                                   0.010294 -0.000297   \n",
              "           Value                                       0.000400  0.000088   \n",
              "           Volatility                                 -0.036540 -0.000107   \n",
              "\n",
              "                                            Healthcare  ...  \\\n",
              "date       factor                                       ...   \n",
              "2025-04-01 Academic & Educational Services   -0.060944  ...   \n",
              "           Basic Materials                    0.004077  ...   \n",
              "           Consumer Cyclicals                -0.015905  ...   \n",
              "           Consumer Non-Cyclicals            -0.007185  ...   \n",
              "           Dividend                           0.008220  ...   \n",
              "...                                                ...  ...   \n",
              "2026-03-31 Size                              -0.000563  ...   \n",
              "           Technology                        -0.002557  ...   \n",
              "           Utilities                          0.000660  ...   \n",
              "           Value                              0.000146  ...   \n",
              "           Volatility                        -0.007134  ...   \n",
              "\n",
              "                                            Institutions, Associations & Organizations  \\\n",
              "date       factor                                                                        \n",
              "2025-04-01 Academic & Educational Services                                    0.018205   \n",
              "           Basic Materials                                                   -0.001218   \n",
              "           Consumer Cyclicals                                                 0.004751   \n",
              "           Consumer Non-Cyclicals                                             0.002146   \n",
              "           Dividend                                                          -0.002456   \n",
              "...                                                                                ...   \n",
              "2026-03-31 Size                                                               0.003470   \n",
              "           Technology                                                         0.001805   \n",
              "           Utilities                                                         -0.001325   \n",
              "           Value                                                              0.002249   \n",
              "           Volatility                                                         0.027899   \n",
              "\n",
              "                                            Leverage    Market  Momentum  \\\n",
              "date       factor                                                          \n",
              "2025-04-01 Academic & Educational Services -0.000299  0.001203  0.004113   \n",
              "           Basic Materials                  0.000020 -0.000080 -0.000275   \n",
              "           Consumer Cyclicals              -0.000078  0.000314  0.001073   \n",
              "           Consumer Non-Cyclicals          -0.000035  0.000142  0.000485   \n",
              "           Dividend                         0.000040 -0.000162 -0.000555   \n",
              "...                                              ...       ...       ...   \n",
              "2026-03-31 Size                            -0.000025  0.001394  0.000010   \n",
              "           Technology                      -0.000812  0.001121  0.000666   \n",
              "           Utilities                       -0.000956 -0.004809  0.003463   \n",
              "           Value                            0.000358  0.001284 -0.000321   \n",
              "           Volatility                      -0.001629  0.014689  0.003581   \n",
              "\n",
              "                                            Real Estate      Size  Technology  \\\n",
              "date       factor                                                               \n",
              "2025-04-01 Academic & Educational Services    -0.004071  0.001764    0.008914   \n",
              "           Basic Materials                     0.000272 -0.000118   -0.000596   \n",
              "           Consumer Cyclicals                 -0.001063  0.000460    0.002326   \n",
              "           Consumer Non-Cyclicals             -0.000480  0.000208    0.001051   \n",
              "           Dividend                            0.000549 -0.000238   -0.001202   \n",
              "...                                                 ...       ...         ...   \n",
              "2026-03-31 Size                               -0.000314  0.000658   -0.000034   \n",
              "           Technology                         -0.002040 -0.000034    0.003285   \n",
              "           Utilities                           0.005295 -0.001104   -0.001418   \n",
              "           Value                              -0.000345  0.000190   -0.000566   \n",
              "           Volatility                         -0.002040  0.001719    0.001943   \n",
              "\n",
              "                                            Utilities     Value  Volatility  \n",
              "date       factor                                                            \n",
              "2025-04-01 Academic & Educational Services   0.006076 -0.003778    0.006381  \n",
              "           Basic Materials                  -0.000406  0.000253   -0.000427  \n",
              "           Consumer Cyclicals                0.001586 -0.000986    0.001665  \n",
              "           Consumer Non-Cyclicals            0.000716 -0.000445    0.000752  \n",
              "           Dividend                         -0.000819  0.000510   -0.000861  \n",
              "...                                               ...       ...         ...  \n",
              "2026-03-31 Size                             -0.001104  0.000190    0.001719  \n",
              "           Technology                       -0.001418 -0.000566    0.001943  \n",
              "           Utilities                         0.016741 -0.001023   -0.002881  \n",
              "           Value                            -0.001023  0.001101    0.000709  \n",
              "           Volatility                       -0.002881  0.000709    0.022883  \n",
              "\n",
              "[5271 rows x 21 columns]"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# generate the report data\n",
        "report = report_engine.calculate(start_date=None, end_date=None)\n",
        "\n",
        "# get the covariance matrix (and skip the first date)\n",
        "df_report = report.get_covariance().filter(pl.col(\"date\") > pl.col(\"date\").min())\n",
        "\n",
        "# convert to pandas\n",
        "df_vcov = df_report.to_pandas().set_index([\"date\", \"factor\"]).rename(columns=lambda c: c.split(\"^\")[0])\n",
        "df_vcov"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "dc151aa5",
      "metadata": {},
      "source": [
        "For downstream comparisons, we split these into factor volatilities and correlations."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "id": "6c9e6bcb",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.187616</td>\n",
              "      <td>0.135531</td>\n",
              "      <td>0.084327</td>\n",
              "      <td>0.101573</td>\n",
              "      <td>0.026105</td>\n",
              "      <td>0.170614</td>\n",
              "      <td>0.059500</td>\n",
              "      <td>0.558629</td>\n",
              "      <td>0.016535</td>\n",
              "      <td>0.111353</td>\n",
              "      <td>...</td>\n",
              "      <td>0.219064</td>\n",
              "      <td>0.029519</td>\n",
              "      <td>0.119038</td>\n",
              "      <td>0.067170</td>\n",
              "      <td>0.088456</td>\n",
              "      <td>0.023991</td>\n",
              "      <td>0.057002</td>\n",
              "      <td>0.128496</td>\n",
              "      <td>0.033882</td>\n",
              "      <td>0.141479</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>0.186606</td>\n",
              "      <td>0.135337</td>\n",
              "      <td>0.083818</td>\n",
              "      <td>0.100988</td>\n",
              "      <td>0.025976</td>\n",
              "      <td>0.175105</td>\n",
              "      <td>0.059289</td>\n",
              "      <td>0.562564</td>\n",
              "      <td>0.016750</td>\n",
              "      <td>0.113434</td>\n",
              "      <td>...</td>\n",
              "      <td>0.227100</td>\n",
              "      <td>0.029780</td>\n",
              "      <td>0.119238</td>\n",
              "      <td>0.068937</td>\n",
              "      <td>0.088074</td>\n",
              "      <td>0.024135</td>\n",
              "      <td>0.057452</td>\n",
              "      <td>0.128235</td>\n",
              "      <td>0.033692</td>\n",
              "      <td>0.143974</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>0.185685</td>\n",
              "      <td>0.137119</td>\n",
              "      <td>0.083329</td>\n",
              "      <td>0.103572</td>\n",
              "      <td>0.025822</td>\n",
              "      <td>0.178022</td>\n",
              "      <td>0.059141</td>\n",
              "      <td>0.560597</td>\n",
              "      <td>0.016813</td>\n",
              "      <td>0.113787</td>\n",
              "      <td>...</td>\n",
              "      <td>0.225841</td>\n",
              "      <td>0.029600</td>\n",
              "      <td>0.120772</td>\n",
              "      <td>0.069185</td>\n",
              "      <td>0.087590</td>\n",
              "      <td>0.024301</td>\n",
              "      <td>0.057317</td>\n",
              "      <td>0.129381</td>\n",
              "      <td>0.033491</td>\n",
              "      <td>0.143906</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>0.184682</td>\n",
              "      <td>0.137250</td>\n",
              "      <td>0.083053</td>\n",
              "      <td>0.102941</td>\n",
              "      <td>0.025749</td>\n",
              "      <td>0.177712</td>\n",
              "      <td>0.059539</td>\n",
              "      <td>0.563603</td>\n",
              "      <td>0.016737</td>\n",
              "      <td>0.113786</td>\n",
              "      <td>...</td>\n",
              "      <td>0.228698</td>\n",
              "      <td>0.029524</td>\n",
              "      <td>0.120785</td>\n",
              "      <td>0.070996</td>\n",
              "      <td>0.087427</td>\n",
              "      <td>0.024305</td>\n",
              "      <td>0.057499</td>\n",
              "      <td>0.128630</td>\n",
              "      <td>0.033333</td>\n",
              "      <td>0.145707</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.184419</td>\n",
              "      <td>0.136663</td>\n",
              "      <td>0.082587</td>\n",
              "      <td>0.104044</td>\n",
              "      <td>0.025750</td>\n",
              "      <td>0.185598</td>\n",
              "      <td>0.059971</td>\n",
              "      <td>0.572888</td>\n",
              "      <td>0.016677</td>\n",
              "      <td>0.113163</td>\n",
              "      <td>...</td>\n",
              "      <td>0.236239</td>\n",
              "      <td>0.029880</td>\n",
              "      <td>0.123521</td>\n",
              "      <td>0.071575</td>\n",
              "      <td>0.086945</td>\n",
              "      <td>0.025659</td>\n",
              "      <td>0.057318</td>\n",
              "      <td>0.129389</td>\n",
              "      <td>0.033188</td>\n",
              "      <td>0.151272</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "            Academic & Educational Services  Basic Materials  \\\n",
              "date                                                           \n",
              "2026-03-25                         0.187616         0.135531   \n",
              "2026-03-26                         0.186606         0.135337   \n",
              "2026-03-27                         0.185685         0.137119   \n",
              "2026-03-30                         0.184682         0.137250   \n",
              "2026-03-31                         0.184419         0.136663   \n",
              "\n",
              "            Consumer Cyclicals  Consumer Non-Cyclicals  Dividend    Energy  \\\n",
              "date                                                                         \n",
              "2026-03-25            0.084327                0.101573  0.026105  0.170614   \n",
              "2026-03-26            0.083818                0.100988  0.025976  0.175105   \n",
              "2026-03-27            0.083329                0.103572  0.025822  0.178022   \n",
              "2026-03-30            0.083053                0.102941  0.025749  0.177712   \n",
              "2026-03-31            0.082587                0.104044  0.025750  0.185598   \n",
              "\n",
              "            Financials  Government Activity    Growth  Healthcare  ...  \\\n",
              "date                                                               ...   \n",
              "2026-03-25    0.059500             0.558629  0.016535    0.111353  ...   \n",
              "2026-03-26    0.059289             0.562564  0.016750    0.113434  ...   \n",
              "2026-03-27    0.059141             0.560597  0.016813    0.113787  ...   \n",
              "2026-03-30    0.059539             0.563603  0.016737    0.113786  ...   \n",
              "2026-03-31    0.059971             0.572888  0.016677    0.113163  ...   \n",
              "\n",
              "            Institutions, Associations & Organizations  Leverage    Market  \\\n",
              "date                                                                         \n",
              "2026-03-25                                    0.219064  0.029519  0.119038   \n",
              "2026-03-26                                    0.227100  0.029780  0.119238   \n",
              "2026-03-27                                    0.225841  0.029600  0.120772   \n",
              "2026-03-30                                    0.228698  0.029524  0.120785   \n",
              "2026-03-31                                    0.236239  0.029880  0.123521   \n",
              "\n",
              "            Momentum  Real Estate      Size  Technology  Utilities     Value  \\\n",
              "date                                                                           \n",
              "2026-03-25  0.067170     0.088456  0.023991    0.057002   0.128496  0.033882   \n",
              "2026-03-26  0.068937     0.088074  0.024135    0.057452   0.128235  0.033692   \n",
              "2026-03-27  0.069185     0.087590  0.024301    0.057317   0.129381  0.033491   \n",
              "2026-03-30  0.070996     0.087427  0.024305    0.057499   0.128630  0.033333   \n",
              "2026-03-31  0.071575     0.086945  0.025659    0.057318   0.129389  0.033188   \n",
              "\n",
              "            Volatility  \n",
              "date                    \n",
              "2026-03-25    0.141479  \n",
              "2026-03-26    0.143974  \n",
              "2026-03-27    0.143906  \n",
              "2026-03-30    0.145707  \n",
              "2026-03-31    0.151272  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate actual factor volatilities from vcovs\n",
        "df_vol = df_vcov.groupby(level=\"date\").apply(\n",
        "    lambda df: pd.Series(np.diag(df) ** 0.5, df.index.droplevel(\"date\"))\n",
        ")\n",
        "df_vol.columns.name = None\n",
        "df_vol.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "id": "8cc7b7d6",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.187616</td>\n",
              "      <td>0.135531</td>\n",
              "      <td>0.084327</td>\n",
              "      <td>0.101573</td>\n",
              "      <td>0.026105</td>\n",
              "      <td>0.170614</td>\n",
              "      <td>0.059500</td>\n",
              "      <td>0.558629</td>\n",
              "      <td>0.016535</td>\n",
              "      <td>0.111353</td>\n",
              "      <td>...</td>\n",
              "      <td>0.219064</td>\n",
              "      <td>0.029519</td>\n",
              "      <td>0.119038</td>\n",
              "      <td>0.067170</td>\n",
              "      <td>0.088456</td>\n",
              "      <td>0.023991</td>\n",
              "      <td>0.057002</td>\n",
              "      <td>0.128496</td>\n",
              "      <td>0.033882</td>\n",
              "      <td>0.141479</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>0.186606</td>\n",
              "      <td>0.135337</td>\n",
              "      <td>0.083818</td>\n",
              "      <td>0.100988</td>\n",
              "      <td>0.025976</td>\n",
              "      <td>0.175105</td>\n",
              "      <td>0.059289</td>\n",
              "      <td>0.562564</td>\n",
              "      <td>0.016750</td>\n",
              "      <td>0.113434</td>\n",
              "      <td>...</td>\n",
              "      <td>0.227100</td>\n",
              "      <td>0.029780</td>\n",
              "      <td>0.119238</td>\n",
              "      <td>0.068937</td>\n",
              "      <td>0.088074</td>\n",
              "      <td>0.024135</td>\n",
              "      <td>0.057452</td>\n",
              "      <td>0.128235</td>\n",
              "      <td>0.033692</td>\n",
              "      <td>0.143974</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>0.185685</td>\n",
              "      <td>0.137119</td>\n",
              "      <td>0.083329</td>\n",
              "      <td>0.103572</td>\n",
              "      <td>0.025822</td>\n",
              "      <td>0.178022</td>\n",
              "      <td>0.059141</td>\n",
              "      <td>0.560597</td>\n",
              "      <td>0.016813</td>\n",
              "      <td>0.113787</td>\n",
              "      <td>...</td>\n",
              "      <td>0.225841</td>\n",
              "      <td>0.029600</td>\n",
              "      <td>0.120772</td>\n",
              "      <td>0.069185</td>\n",
              "      <td>0.087590</td>\n",
              "      <td>0.024301</td>\n",
              "      <td>0.057317</td>\n",
              "      <td>0.129381</td>\n",
              "      <td>0.033491</td>\n",
              "      <td>0.143906</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>0.184682</td>\n",
              "      <td>0.137250</td>\n",
              "      <td>0.083053</td>\n",
              "      <td>0.102941</td>\n",
              "      <td>0.025749</td>\n",
              "      <td>0.177712</td>\n",
              "      <td>0.059539</td>\n",
              "      <td>0.563603</td>\n",
              "      <td>0.016737</td>\n",
              "      <td>0.113786</td>\n",
              "      <td>...</td>\n",
              "      <td>0.228698</td>\n",
              "      <td>0.029524</td>\n",
              "      <td>0.120785</td>\n",
              "      <td>0.070996</td>\n",
              "      <td>0.087427</td>\n",
              "      <td>0.024305</td>\n",
              "      <td>0.057499</td>\n",
              "      <td>0.128630</td>\n",
              "      <td>0.033333</td>\n",
              "      <td>0.145707</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.184419</td>\n",
              "      <td>0.136663</td>\n",
              "      <td>0.082587</td>\n",
              "      <td>0.104044</td>\n",
              "      <td>0.025750</td>\n",
              "      <td>0.185598</td>\n",
              "      <td>0.059971</td>\n",
              "      <td>0.572888</td>\n",
              "      <td>0.016677</td>\n",
              "      <td>0.113163</td>\n",
              "      <td>...</td>\n",
              "      <td>0.236239</td>\n",
              "      <td>0.029880</td>\n",
              "      <td>0.123521</td>\n",
              "      <td>0.071575</td>\n",
              "      <td>0.086945</td>\n",
              "      <td>0.025659</td>\n",
              "      <td>0.057318</td>\n",
              "      <td>0.129389</td>\n",
              "      <td>0.033188</td>\n",
              "      <td>0.151272</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "            Academic & Educational Services  Basic Materials  \\\n",
              "date                                                           \n",
              "2026-03-25                         0.187616         0.135531   \n",
              "2026-03-26                         0.186606         0.135337   \n",
              "2026-03-27                         0.185685         0.137119   \n",
              "2026-03-30                         0.184682         0.137250   \n",
              "2026-03-31                         0.184419         0.136663   \n",
              "\n",
              "            Consumer Cyclicals  Consumer Non-Cyclicals  Dividend    Energy  \\\n",
              "date                                                                         \n",
              "2026-03-25            0.084327                0.101573  0.026105  0.170614   \n",
              "2026-03-26            0.083818                0.100988  0.025976  0.175105   \n",
              "2026-03-27            0.083329                0.103572  0.025822  0.178022   \n",
              "2026-03-30            0.083053                0.102941  0.025749  0.177712   \n",
              "2026-03-31            0.082587                0.104044  0.025750  0.185598   \n",
              "\n",
              "            Financials  Government Activity    Growth  Healthcare  ...  \\\n",
              "date                                                               ...   \n",
              "2026-03-25    0.059500             0.558629  0.016535    0.111353  ...   \n",
              "2026-03-26    0.059289             0.562564  0.016750    0.113434  ...   \n",
              "2026-03-27    0.059141             0.560597  0.016813    0.113787  ...   \n",
              "2026-03-30    0.059539             0.563603  0.016737    0.113786  ...   \n",
              "2026-03-31    0.059971             0.572888  0.016677    0.113163  ...   \n",
              "\n",
              "            Institutions, Associations & Organizations  Leverage    Market  \\\n",
              "date                                                                         \n",
              "2026-03-25                                    0.219064  0.029519  0.119038   \n",
              "2026-03-26                                    0.227100  0.029780  0.119238   \n",
              "2026-03-27                                    0.225841  0.029600  0.120772   \n",
              "2026-03-30                                    0.228698  0.029524  0.120785   \n",
              "2026-03-31                                    0.236239  0.029880  0.123521   \n",
              "\n",
              "            Momentum  Real Estate      Size  Technology  Utilities     Value  \\\n",
              "date                                                                           \n",
              "2026-03-25  0.067170     0.088456  0.023991    0.057002   0.128496  0.033882   \n",
              "2026-03-26  0.068937     0.088074  0.024135    0.057452   0.128235  0.033692   \n",
              "2026-03-27  0.069185     0.087590  0.024301    0.057317   0.129381  0.033491   \n",
              "2026-03-30  0.070996     0.087427  0.024305    0.057499   0.128630  0.033333   \n",
              "2026-03-31  0.071575     0.086945  0.025659    0.057318   0.129389  0.033188   \n",
              "\n",
              "            Volatility  \n",
              "date                    \n",
              "2026-03-25    0.141479  \n",
              "2026-03-26    0.143974  \n",
              "2026-03-27    0.143906  \n",
              "2026-03-30    0.145707  \n",
              "2026-03-31    0.151272  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate actual factor volatilities from vcovs\n",
        "df_vol = df_vcov.groupby(level=\"date\").apply(\n",
        "    lambda df: pd.Series(np.diag(df) ** 0.5, df.index.droplevel(\"date\"))\n",
        ")\n",
        "df_vol.columns.name = None\n",
        "df_vol.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "id": "c307c44c",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2026-03-31</th>\n",
              "      <th>Size</th>\n",
              "      <td>-0.085507</td>\n",
              "      <td>-0.003677</td>\n",
              "      <td>0.054784</td>\n",
              "      <td>-0.403332</td>\n",
              "      <td>-0.109727</td>\n",
              "      <td>-0.070638</td>\n",
              "      <td>0.471539</td>\n",
              "      <td>-0.166400</td>\n",
              "      <td>0.153275</td>\n",
              "      <td>-0.193859</td>\n",
              "      <td>...</td>\n",
              "      <td>0.572390</td>\n",
              "      <td>-0.033232</td>\n",
              "      <td>0.439762</td>\n",
              "      <td>0.005628</td>\n",
              "      <td>-0.140956</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.023050</td>\n",
              "      <td>-0.332382</td>\n",
              "      <td>0.223332</td>\n",
              "      <td>0.442835</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Technology</th>\n",
              "      <td>-0.144443</td>\n",
              "      <td>-0.514597</td>\n",
              "      <td>-0.513656</td>\n",
              "      <td>-0.569050</td>\n",
              "      <td>-0.134638</td>\n",
              "      <td>-0.254184</td>\n",
              "      <td>-0.228676</td>\n",
              "      <td>-0.079027</td>\n",
              "      <td>0.125612</td>\n",
              "      <td>-0.394287</td>\n",
              "      <td>...</td>\n",
              "      <td>0.133313</td>\n",
              "      <td>-0.474139</td>\n",
              "      <td>0.158343</td>\n",
              "      <td>0.162342</td>\n",
              "      <td>-0.409340</td>\n",
              "      <td>-0.023050</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.191221</td>\n",
              "      <td>-0.297795</td>\n",
              "      <td>0.224097</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Utilities</th>\n",
              "      <td>-0.121789</td>\n",
              "      <td>0.200024</td>\n",
              "      <td>-0.263288</td>\n",
              "      <td>0.440704</td>\n",
              "      <td>0.077829</td>\n",
              "      <td>0.219548</td>\n",
              "      <td>-0.204212</td>\n",
              "      <td>0.138880</td>\n",
              "      <td>-0.137641</td>\n",
              "      <td>0.045077</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.043349</td>\n",
              "      <td>-0.247217</td>\n",
              "      <td>-0.300915</td>\n",
              "      <td>0.373880</td>\n",
              "      <td>0.470684</td>\n",
              "      <td>-0.332382</td>\n",
              "      <td>-0.191221</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.238161</td>\n",
              "      <td>-0.147191</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Value</th>\n",
              "      <td>0.016314</td>\n",
              "      <td>0.299783</td>\n",
              "      <td>0.435408</td>\n",
              "      <td>0.027769</td>\n",
              "      <td>0.181625</td>\n",
              "      <td>0.176295</td>\n",
              "      <td>-0.065339</td>\n",
              "      <td>0.021029</td>\n",
              "      <td>0.158217</td>\n",
              "      <td>0.038926</td>\n",
              "      <td>...</td>\n",
              "      <td>0.286905</td>\n",
              "      <td>0.361267</td>\n",
              "      <td>0.313244</td>\n",
              "      <td>-0.134942</td>\n",
              "      <td>-0.119496</td>\n",
              "      <td>0.223332</td>\n",
              "      <td>-0.297795</td>\n",
              "      <td>-0.238161</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.141309</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Volatility</th>\n",
              "      <td>-0.300310</td>\n",
              "      <td>-0.116572</td>\n",
              "      <td>-0.218530</td>\n",
              "      <td>-0.423916</td>\n",
              "      <td>-0.004297</td>\n",
              "      <td>-0.211313</td>\n",
              "      <td>0.451786</td>\n",
              "      <td>-0.421643</td>\n",
              "      <td>-0.042291</td>\n",
              "      <td>-0.416726</td>\n",
              "      <td>...</td>\n",
              "      <td>0.780696</td>\n",
              "      <td>-0.360300</td>\n",
              "      <td>0.786145</td>\n",
              "      <td>0.330703</td>\n",
              "      <td>-0.155102</td>\n",
              "      <td>0.442835</td>\n",
              "      <td>0.224097</td>\n",
              "      <td>-0.147191</td>\n",
              "      <td>0.141309</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                       Academic & Educational Services  Basic Materials  \\\n",
              "date       factor                                                         \n",
              "2026-03-31 Size                              -0.085507        -0.003677   \n",
              "           Technology                        -0.144443        -0.514597   \n",
              "           Utilities                         -0.121789         0.200024   \n",
              "           Value                              0.016314         0.299783   \n",
              "           Volatility                        -0.300310        -0.116572   \n",
              "\n",
              "                       Consumer Cyclicals  Consumer Non-Cyclicals  Dividend  \\\n",
              "date       factor                                                             \n",
              "2026-03-31 Size                  0.054784               -0.403332 -0.109727   \n",
              "           Technology           -0.513656               -0.569050 -0.134638   \n",
              "           Utilities            -0.263288                0.440704  0.077829   \n",
              "           Value                 0.435408                0.027769  0.181625   \n",
              "           Volatility           -0.218530               -0.423916 -0.004297   \n",
              "\n",
              "                         Energy  Financials  Government Activity    Growth  \\\n",
              "date       factor                                                            \n",
              "2026-03-31 Size       -0.070638    0.471539            -0.166400  0.153275   \n",
              "           Technology -0.254184   -0.228676            -0.079027  0.125612   \n",
              "           Utilities   0.219548   -0.204212             0.138880 -0.137641   \n",
              "           Value       0.176295   -0.065339             0.021029  0.158217   \n",
              "           Volatility -0.211313    0.451786            -0.421643 -0.042291   \n",
              "\n",
              "                       Healthcare  ...  \\\n",
              "date       factor                  ...   \n",
              "2026-03-31 Size         -0.193859  ...   \n",
              "           Technology   -0.394287  ...   \n",
              "           Utilities     0.045077  ...   \n",
              "           Value         0.038926  ...   \n",
              "           Volatility   -0.416726  ...   \n",
              "\n",
              "                       Institutions, Associations & Organizations  Leverage  \\\n",
              "date       factor                                                             \n",
              "2026-03-31 Size                                          0.572390 -0.033232   \n",
              "           Technology                                    0.133313 -0.474139   \n",
              "           Utilities                                    -0.043349 -0.247217   \n",
              "           Value                                         0.286905  0.361267   \n",
              "           Volatility                                    0.780696 -0.360300   \n",
              "\n",
              "                         Market  Momentum  Real Estate      Size  Technology  \\\n",
              "date       factor                                                              \n",
              "2026-03-31 Size        0.439762  0.005628    -0.140956  1.000000   -0.023050   \n",
              "           Technology  0.158343  0.162342    -0.409340 -0.023050    1.000000   \n",
              "           Utilities  -0.300915  0.373880     0.470684 -0.332382   -0.191221   \n",
              "           Value       0.313244 -0.134942    -0.119496  0.223332   -0.297795   \n",
              "           Volatility  0.786145  0.330703    -0.155102  0.442835    0.224097   \n",
              "\n",
              "                       Utilities     Value  Volatility  \n",
              "date       factor                                       \n",
              "2026-03-31 Size        -0.332382  0.223332    0.442835  \n",
              "           Technology  -0.191221 -0.297795    0.224097  \n",
              "           Utilities    1.000000 -0.238161   -0.147191  \n",
              "           Value       -0.238161  1.000000    0.141309  \n",
              "           Volatility  -0.147191  0.141309    1.000000  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate actual factor correlations from vcovs\n",
        "df_cor = df_vcov.groupby(level=\"date\").apply(\n",
        "    lambda df: df.droplevel(\"date\")\n",
        "    / np.outer(np.diag(df) ** 0.5, np.diag(df) ** 0.5)\n",
        ")\n",
        "df_cor.tail()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b2f5be04",
      "metadata": {},
      "source": [
        "## Manually replicating the covariance forecasts\n",
        "\n",
        "We can also estimated the risk model and get the returns directly. From these returns we can construct the covariance forecasts. Bayesline returns dataframes in `polars`, but they can be easily converted to `pandas` dataframes. We also remove the factor group (market, style, industry, etc.) for convenience."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "id": "8d00139f",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>...</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.005473</td>\n",
              "      <td>0.000339</td>\n",
              "      <td>-0.002247</td>\n",
              "      <td>-0.001601</td>\n",
              "      <td>0.002613</td>\n",
              "      <td>0.001224</td>\n",
              "      <td>0.000568</td>\n",
              "      <td>0.005741</td>\n",
              "      <td>0.013586</td>\n",
              "      <td>0.010203</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003327</td>\n",
              "      <td>-0.007366</td>\n",
              "      <td>-0.000172</td>\n",
              "      <td>-0.070391</td>\n",
              "      <td>0.007425</td>\n",
              "      <td>0.000251</td>\n",
              "      <td>0.004958</td>\n",
              "      <td>-0.003215</td>\n",
              "      <td>-0.001444</td>\n",
              "      <td>-0.001395</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>-0.008469</td>\n",
              "      <td>0.000710</td>\n",
              "      <td>-0.001850</td>\n",
              "      <td>0.002916</td>\n",
              "      <td>-0.009811</td>\n",
              "      <td>-0.002130</td>\n",
              "      <td>0.000614</td>\n",
              "      <td>-0.017646</td>\n",
              "      <td>0.004058</td>\n",
              "      <td>0.007467</td>\n",
              "      <td>...</td>\n",
              "      <td>0.001529</td>\n",
              "      <td>0.024926</td>\n",
              "      <td>-0.002430</td>\n",
              "      <td>0.051728</td>\n",
              "      <td>0.014200</td>\n",
              "      <td>-0.002864</td>\n",
              "      <td>-0.036858</td>\n",
              "      <td>0.003011</td>\n",
              "      <td>-0.005447</td>\n",
              "      <td>0.006608</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>-0.013285</td>\n",
              "      <td>-0.000276</td>\n",
              "      <td>0.001343</td>\n",
              "      <td>-0.000156</td>\n",
              "      <td>0.005478</td>\n",
              "      <td>-0.002220</td>\n",
              "      <td>0.000315</td>\n",
              "      <td>-0.008710</td>\n",
              "      <td>0.005140</td>\n",
              "      <td>0.015198</td>\n",
              "      <td>...</td>\n",
              "      <td>0.014589</td>\n",
              "      <td>0.021390</td>\n",
              "      <td>-0.002866</td>\n",
              "      <td>0.023144</td>\n",
              "      <td>-0.008786</td>\n",
              "      <td>0.000920</td>\n",
              "      <td>0.004319</td>\n",
              "      <td>0.001757</td>\n",
              "      <td>-0.002835</td>\n",
              "      <td>0.012709</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>-0.007674</td>\n",
              "      <td>0.001191</td>\n",
              "      <td>-0.000534</td>\n",
              "      <td>0.001419</td>\n",
              "      <td>-0.010091</td>\n",
              "      <td>0.001552</td>\n",
              "      <td>-0.000997</td>\n",
              "      <td>-0.015886</td>\n",
              "      <td>0.003952</td>\n",
              "      <td>0.009292</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000040</td>\n",
              "      <td>-0.009472</td>\n",
              "      <td>0.005412</td>\n",
              "      <td>0.048471</td>\n",
              "      <td>0.007164</td>\n",
              "      <td>-0.004804</td>\n",
              "      <td>-0.025022</td>\n",
              "      <td>0.004594</td>\n",
              "      <td>-0.004454</td>\n",
              "      <td>0.001800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.016616</td>\n",
              "      <td>0.001627</td>\n",
              "      <td>-0.000674</td>\n",
              "      <td>-0.003220</td>\n",
              "      <td>0.006854</td>\n",
              "      <td>0.004943</td>\n",
              "      <td>-0.001126</td>\n",
              "      <td>0.024977</td>\n",
              "      <td>-0.010185</td>\n",
              "      <td>-0.004721</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.010800</td>\n",
              "      <td>-0.032568</td>\n",
              "      <td>0.005563</td>\n",
              "      <td>-0.068596</td>\n",
              "      <td>0.002293</td>\n",
              "      <td>0.000754</td>\n",
              "      <td>0.036771</td>\n",
              "      <td>-0.001710</td>\n",
              "      <td>0.002518</td>\n",
              "      <td>-0.011382</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "              Market  Dividend    Growth  Leverage  Momentum      Size  \\\n",
              "date                                                                     \n",
              "2026-03-25  0.005473  0.000339 -0.002247 -0.001601  0.002613  0.001224   \n",
              "2026-03-26 -0.008469  0.000710 -0.001850  0.002916 -0.009811 -0.002130   \n",
              "2026-03-27 -0.013285 -0.000276  0.001343 -0.000156  0.005478 -0.002220   \n",
              "2026-03-30 -0.007674  0.001191 -0.000534  0.001419 -0.010091  0.001552   \n",
              "2026-03-31  0.016616  0.001627 -0.000674 -0.003220  0.006854  0.004943   \n",
              "\n",
              "               Value  Volatility  Academic & Educational Services  \\\n",
              "date                                                                \n",
              "2026-03-25  0.000568    0.005741                         0.013586   \n",
              "2026-03-26  0.000614   -0.017646                         0.004058   \n",
              "2026-03-27  0.000315   -0.008710                         0.005140   \n",
              "2026-03-30 -0.000997   -0.015886                         0.003952   \n",
              "2026-03-31 -0.001126    0.024977                        -0.010185   \n",
              "\n",
              "            Basic Materials  ...  Consumer Non-Cyclicals    Energy  \\\n",
              "date                         ...                                     \n",
              "2026-03-25         0.010203  ...                0.003327 -0.007366   \n",
              "2026-03-26         0.007467  ...                0.001529  0.024926   \n",
              "2026-03-27         0.015198  ...                0.014589  0.021390   \n",
              "2026-03-30         0.009292  ...                0.000040 -0.009472   \n",
              "2026-03-31        -0.004721  ...               -0.010800 -0.032568   \n",
              "\n",
              "            Financials  Government Activity  Healthcare  Industrials  \\\n",
              "date                                                                   \n",
              "2026-03-25   -0.000172            -0.070391    0.007425     0.000251   \n",
              "2026-03-26   -0.002430             0.051728    0.014200    -0.002864   \n",
              "2026-03-27   -0.002866             0.023144   -0.008786     0.000920   \n",
              "2026-03-30    0.005412             0.048471    0.007164    -0.004804   \n",
              "2026-03-31    0.005563            -0.068596    0.002293     0.000754   \n",
              "\n",
              "            Institutions, Associations & Organizations  Real Estate  \\\n",
              "date                                                                  \n",
              "2026-03-25                                    0.004958    -0.003215   \n",
              "2026-03-26                                   -0.036858     0.003011   \n",
              "2026-03-27                                    0.004319     0.001757   \n",
              "2026-03-30                                   -0.025022     0.004594   \n",
              "2026-03-31                                    0.036771    -0.001710   \n",
              "\n",
              "            Technology  Utilities  \n",
              "date                               \n",
              "2026-03-25   -0.001444  -0.001395  \n",
              "2026-03-26   -0.005447   0.006608  \n",
              "2026-03-27   -0.002835   0.012709  \n",
              "2026-03-30   -0.004454   0.001800  \n",
              "2026-03-31    0.002518  -0.011382  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "risk_model = bln.equity.riskmodels.load(\n",
        "    factorriskmodel_settings.with_dataset(\"bayesline/Bayesline-US-All-1y\")\n",
        ").get_model()\n",
        "df_factor_returns = risk_model.fret().tail(-1).to_pandas().set_index(\"date\").rename(columns=lambda c: c.split(\".\")[1])\n",
        "df_factor_returns.tail()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "72a258ab",
      "metadata": {},
      "source": [
        "From these returns we can run standard pandas functions to get the EWMAs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "id": "01bb544a",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>...</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.119038</td>\n",
              "      <td>0.026105</td>\n",
              "      <td>0.016535</td>\n",
              "      <td>0.029519</td>\n",
              "      <td>0.067170</td>\n",
              "      <td>0.023991</td>\n",
              "      <td>0.033882</td>\n",
              "      <td>0.141479</td>\n",
              "      <td>0.187616</td>\n",
              "      <td>0.135531</td>\n",
              "      <td>...</td>\n",
              "      <td>0.101573</td>\n",
              "      <td>0.170614</td>\n",
              "      <td>0.059500</td>\n",
              "      <td>0.558629</td>\n",
              "      <td>0.111353</td>\n",
              "      <td>0.067498</td>\n",
              "      <td>0.219064</td>\n",
              "      <td>0.088456</td>\n",
              "      <td>0.057002</td>\n",
              "      <td>0.128496</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>0.119238</td>\n",
              "      <td>0.025976</td>\n",
              "      <td>0.016750</td>\n",
              "      <td>0.029780</td>\n",
              "      <td>0.068937</td>\n",
              "      <td>0.024135</td>\n",
              "      <td>0.033692</td>\n",
              "      <td>0.143974</td>\n",
              "      <td>0.186605</td>\n",
              "      <td>0.135337</td>\n",
              "      <td>...</td>\n",
              "      <td>0.100988</td>\n",
              "      <td>0.175105</td>\n",
              "      <td>0.059289</td>\n",
              "      <td>0.562564</td>\n",
              "      <td>0.113434</td>\n",
              "      <td>0.067273</td>\n",
              "      <td>0.227100</td>\n",
              "      <td>0.088074</td>\n",
              "      <td>0.057452</td>\n",
              "      <td>0.128235</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>0.120772</td>\n",
              "      <td>0.025822</td>\n",
              "      <td>0.016813</td>\n",
              "      <td>0.029600</td>\n",
              "      <td>0.069185</td>\n",
              "      <td>0.024301</td>\n",
              "      <td>0.033491</td>\n",
              "      <td>0.143906</td>\n",
              "      <td>0.185685</td>\n",
              "      <td>0.137119</td>\n",
              "      <td>...</td>\n",
              "      <td>0.103572</td>\n",
              "      <td>0.178022</td>\n",
              "      <td>0.059141</td>\n",
              "      <td>0.560598</td>\n",
              "      <td>0.113787</td>\n",
              "      <td>0.066882</td>\n",
              "      <td>0.225841</td>\n",
              "      <td>0.087590</td>\n",
              "      <td>0.057317</td>\n",
              "      <td>0.129381</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>0.120785</td>\n",
              "      <td>0.025749</td>\n",
              "      <td>0.016737</td>\n",
              "      <td>0.029524</td>\n",
              "      <td>0.070996</td>\n",
              "      <td>0.024305</td>\n",
              "      <td>0.033333</td>\n",
              "      <td>0.145707</td>\n",
              "      <td>0.184682</td>\n",
              "      <td>0.137250</td>\n",
              "      <td>...</td>\n",
              "      <td>0.102941</td>\n",
              "      <td>0.177712</td>\n",
              "      <td>0.059539</td>\n",
              "      <td>0.563603</td>\n",
              "      <td>0.113786</td>\n",
              "      <td>0.067004</td>\n",
              "      <td>0.228698</td>\n",
              "      <td>0.087426</td>\n",
              "      <td>0.057499</td>\n",
              "      <td>0.128630</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.123521</td>\n",
              "      <td>0.025750</td>\n",
              "      <td>0.016677</td>\n",
              "      <td>0.029880</td>\n",
              "      <td>0.071575</td>\n",
              "      <td>0.025659</td>\n",
              "      <td>0.033188</td>\n",
              "      <td>0.151272</td>\n",
              "      <td>0.184419</td>\n",
              "      <td>0.136663</td>\n",
              "      <td>...</td>\n",
              "      <td>0.104044</td>\n",
              "      <td>0.185598</td>\n",
              "      <td>0.059971</td>\n",
              "      <td>0.572888</td>\n",
              "      <td>0.113163</td>\n",
              "      <td>0.066608</td>\n",
              "      <td>0.236239</td>\n",
              "      <td>0.086945</td>\n",
              "      <td>0.057318</td>\n",
              "      <td>0.129389</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "              Market  Dividend    Growth  Leverage  Momentum      Size  \\\n",
              "date                                                                     \n",
              "2026-03-25  0.119038  0.026105  0.016535  0.029519  0.067170  0.023991   \n",
              "2026-03-26  0.119238  0.025976  0.016750  0.029780  0.068937  0.024135   \n",
              "2026-03-27  0.120772  0.025822  0.016813  0.029600  0.069185  0.024301   \n",
              "2026-03-30  0.120785  0.025749  0.016737  0.029524  0.070996  0.024305   \n",
              "2026-03-31  0.123521  0.025750  0.016677  0.029880  0.071575  0.025659   \n",
              "\n",
              "               Value  Volatility  Academic & Educational Services  \\\n",
              "date                                                                \n",
              "2026-03-25  0.033882    0.141479                         0.187616   \n",
              "2026-03-26  0.033692    0.143974                         0.186605   \n",
              "2026-03-27  0.033491    0.143906                         0.185685   \n",
              "2026-03-30  0.033333    0.145707                         0.184682   \n",
              "2026-03-31  0.033188    0.151272                         0.184419   \n",
              "\n",
              "            Basic Materials  ...  Consumer Non-Cyclicals    Energy  \\\n",
              "date                         ...                                     \n",
              "2026-03-25         0.135531  ...                0.101573  0.170614   \n",
              "2026-03-26         0.135337  ...                0.100988  0.175105   \n",
              "2026-03-27         0.137119  ...                0.103572  0.178022   \n",
              "2026-03-30         0.137250  ...                0.102941  0.177712   \n",
              "2026-03-31         0.136663  ...                0.104044  0.185598   \n",
              "\n",
              "            Financials  Government Activity  Healthcare  Industrials  \\\n",
              "date                                                                   \n",
              "2026-03-25    0.059500             0.558629    0.111353     0.067498   \n",
              "2026-03-26    0.059289             0.562564    0.113434     0.067273   \n",
              "2026-03-27    0.059141             0.560598    0.113787     0.066882   \n",
              "2026-03-30    0.059539             0.563603    0.113786     0.067004   \n",
              "2026-03-31    0.059971             0.572888    0.113163     0.066608   \n",
              "\n",
              "            Institutions, Associations & Organizations  Real Estate  \\\n",
              "date                                                                  \n",
              "2026-03-25                                    0.219064     0.088456   \n",
              "2026-03-26                                    0.227100     0.088074   \n",
              "2026-03-27                                    0.225841     0.087590   \n",
              "2026-03-30                                    0.228698     0.087426   \n",
              "2026-03-31                                    0.236239     0.086945   \n",
              "\n",
              "            Technology  Utilities  \n",
              "date                               \n",
              "2026-03-25    0.057002   0.128496  \n",
              "2026-03-26    0.057452   0.128235  \n",
              "2026-03-27    0.057317   0.129381  \n",
              "2026-03-30    0.057499   0.128630  \n",
              "2026-03-31    0.057318   0.129389  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate expected factor volatilities using the ewma\n",
        "df_vol_tieout = (\n",
        "    pd.DataFrame(df_factor_returns**2)\n",
        "    .ewm(halflife=report_settings.factor_cov_settings.halflife_vol)\n",
        "    .mean()\n",
        "    .astype(np.float32)\n",
        "    ** 0.5\n",
        "    * 252**0.5\n",
        ")\n",
        "df_vol_tieout.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "id": "ddfea5c7",
      "metadata": {},
      "outputs": [],
      "source": [
        "pd.testing.assert_frame_equal(df_vol, df_vol_tieout, check_column_type=False, check_categorical=False, check_like=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fd3670a2",
      "metadata": {},
      "source": [
        "The correlation are a bit more involved. We need to create the outer products and then run EWMAs on each cell in the outer product matrix."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "id": "7063f223",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>...</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2025-04-01</th>\n",
              "      <th>Market</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Dividend</th>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Growth</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Leverage</th>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Momentum</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2026-03-31</th>\n",
              "      <th>Industrials</th>\n",
              "      <td>0.226230</td>\n",
              "      <td>-0.025260</td>\n",
              "      <td>0.044600</td>\n",
              "      <td>0.208107</td>\n",
              "      <td>0.121678</td>\n",
              "      <td>0.166434</td>\n",
              "      <td>0.317685</td>\n",
              "      <td>0.188553</td>\n",
              "      <td>0.045103</td>\n",
              "      <td>0.377426</td>\n",
              "      <td>...</td>\n",
              "      <td>0.116825</td>\n",
              "      <td>0.066352</td>\n",
              "      <td>0.117271</td>\n",
              "      <td>-0.059579</td>\n",
              "      <td>-0.078656</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.212436</td>\n",
              "      <td>0.056580</td>\n",
              "      <td>-0.484509</td>\n",
              "      <td>0.035689</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <td>0.409774</td>\n",
              "      <td>-0.008875</td>\n",
              "      <td>0.259716</td>\n",
              "      <td>-0.342170</td>\n",
              "      <td>0.606199</td>\n",
              "      <td>0.572391</td>\n",
              "      <td>0.286905</td>\n",
              "      <td>0.780696</td>\n",
              "      <td>-0.292175</td>\n",
              "      <td>-0.005230</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.253746</td>\n",
              "      <td>-0.003886</td>\n",
              "      <td>0.343091</td>\n",
              "      <td>-0.243576</td>\n",
              "      <td>-0.391429</td>\n",
              "      <td>0.212436</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.174275</td>\n",
              "      <td>0.133313</td>\n",
              "      <td>-0.043349</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Real Estate</th>\n",
              "      <td>-0.112195</td>\n",
              "      <td>0.105823</td>\n",
              "      <td>-0.077864</td>\n",
              "      <td>0.055883</td>\n",
              "      <td>0.010700</td>\n",
              "      <td>-0.140957</td>\n",
              "      <td>-0.119496</td>\n",
              "      <td>-0.155102</td>\n",
              "      <td>-0.018747</td>\n",
              "      <td>0.170929</td>\n",
              "      <td>...</td>\n",
              "      <td>0.298852</td>\n",
              "      <td>0.077360</td>\n",
              "      <td>0.034780</td>\n",
              "      <td>0.046858</td>\n",
              "      <td>0.138198</td>\n",
              "      <td>0.056580</td>\n",
              "      <td>-0.174275</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.409341</td>\n",
              "      <td>0.470684</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Technology</th>\n",
              "      <td>0.158343</td>\n",
              "      <td>-0.134638</td>\n",
              "      <td>0.125612</td>\n",
              "      <td>-0.474139</td>\n",
              "      <td>0.162342</td>\n",
              "      <td>-0.023049</td>\n",
              "      <td>-0.297794</td>\n",
              "      <td>0.224097</td>\n",
              "      <td>-0.144443</td>\n",
              "      <td>-0.514597</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.569050</td>\n",
              "      <td>-0.254184</td>\n",
              "      <td>-0.228676</td>\n",
              "      <td>-0.079027</td>\n",
              "      <td>-0.394288</td>\n",
              "      <td>-0.484509</td>\n",
              "      <td>0.133313</td>\n",
              "      <td>-0.409341</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.191221</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Utilities</th>\n",
              "      <td>-0.300915</td>\n",
              "      <td>0.077828</td>\n",
              "      <td>-0.137641</td>\n",
              "      <td>-0.247217</td>\n",
              "      <td>0.373880</td>\n",
              "      <td>-0.332381</td>\n",
              "      <td>-0.238161</td>\n",
              "      <td>-0.147191</td>\n",
              "      <td>-0.121789</td>\n",
              "      <td>0.200024</td>\n",
              "      <td>...</td>\n",
              "      <td>0.440704</td>\n",
              "      <td>0.219548</td>\n",
              "      <td>-0.204212</td>\n",
              "      <td>0.138880</td>\n",
              "      <td>0.045077</td>\n",
              "      <td>0.035689</td>\n",
              "      <td>-0.043349</td>\n",
              "      <td>0.470684</td>\n",
              "      <td>-0.191221</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5271 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                         Market  Dividend  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Market                                      1.000000 -1.000000   \n",
              "           Dividend                                   -1.000000  1.000000   \n",
              "           Growth                                      1.000000 -1.000000   \n",
              "           Leverage                                   -1.000000  1.000000   \n",
              "           Momentum                                    1.000000 -1.000000   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Industrials                                 0.226230 -0.025260   \n",
              "           Institutions, Associations & Organizations  0.409774 -0.008875   \n",
              "           Real Estate                                -0.112195  0.105823   \n",
              "           Technology                                  0.158343 -0.134638   \n",
              "           Utilities                                  -0.300915  0.077828   \n",
              "\n",
              "                                                         Growth  Leverage  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Market                                      1.000000 -1.000000   \n",
              "           Dividend                                   -1.000000  1.000000   \n",
              "           Growth                                      1.000000 -1.000000   \n",
              "           Leverage                                   -1.000000  1.000000   \n",
              "           Momentum                                    1.000000 -1.000000   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Industrials                                 0.044600  0.208107   \n",
              "           Institutions, Associations & Organizations  0.259716 -0.342170   \n",
              "           Real Estate                                -0.077864  0.055883   \n",
              "           Technology                                  0.125612 -0.474139   \n",
              "           Utilities                                  -0.137641 -0.247217   \n",
              "\n",
              "                                                       Momentum      Size  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Market                                      1.000000  1.000000   \n",
              "           Dividend                                   -1.000000 -1.000000   \n",
              "           Growth                                      1.000000  1.000000   \n",
              "           Leverage                                   -1.000000 -1.000000   \n",
              "           Momentum                                    1.000000  1.000000   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Industrials                                 0.121678  0.166434   \n",
              "           Institutions, Associations & Organizations  0.606199  0.572391   \n",
              "           Real Estate                                 0.010700 -0.140957   \n",
              "           Technology                                  0.162342 -0.023049   \n",
              "           Utilities                                   0.373880 -0.332381   \n",
              "\n",
              "                                                          Value  Volatility  \\\n",
              "date       factor                                                             \n",
              "2025-04-01 Market                                     -1.000000    1.000000   \n",
              "           Dividend                                    1.000000   -1.000000   \n",
              "           Growth                                     -1.000000    1.000000   \n",
              "           Leverage                                    1.000000   -1.000000   \n",
              "           Momentum                                   -1.000000    1.000000   \n",
              "...                                                         ...         ...   \n",
              "2026-03-31 Industrials                                 0.317685    0.188553   \n",
              "           Institutions, Associations & Organizations  0.286905    0.780696   \n",
              "           Real Estate                                -0.119496   -0.155102   \n",
              "           Technology                                 -0.297794    0.224097   \n",
              "           Utilities                                  -0.238161   -0.147191   \n",
              "\n",
              "                                                       Academic & Educational Services  \\\n",
              "date       factor                                                                        \n",
              "2025-04-01 Market                                                             1.000000   \n",
              "           Dividend                                                          -1.000000   \n",
              "           Growth                                                             1.000000   \n",
              "           Leverage                                                          -1.000000   \n",
              "           Momentum                                                           1.000000   \n",
              "...                                                                                ...   \n",
              "2026-03-31 Industrials                                                        0.045103   \n",
              "           Institutions, Associations & Organizations                        -0.292175   \n",
              "           Real Estate                                                       -0.018747   \n",
              "           Technology                                                        -0.144443   \n",
              "           Utilities                                                         -0.121789   \n",
              "\n",
              "                                                       Basic Materials  ...  \\\n",
              "date       factor                                                       ...   \n",
              "2025-04-01 Market                                            -1.000000  ...   \n",
              "           Dividend                                           1.000000  ...   \n",
              "           Growth                                            -1.000000  ...   \n",
              "           Leverage                                           1.000000  ...   \n",
              "           Momentum                                          -1.000000  ...   \n",
              "...                                                                ...  ...   \n",
              "2026-03-31 Industrials                                        0.377426  ...   \n",
              "           Institutions, Associations & Organizations        -0.005230  ...   \n",
              "           Real Estate                                        0.170929  ...   \n",
              "           Technology                                        -0.514597  ...   \n",
              "           Utilities                                          0.200024  ...   \n",
              "\n",
              "                                                       Consumer Non-Cyclicals  \\\n",
              "date       factor                                                               \n",
              "2025-04-01 Market                                                    1.000000   \n",
              "           Dividend                                                 -1.000000   \n",
              "           Growth                                                    1.000000   \n",
              "           Leverage                                                 -1.000000   \n",
              "           Momentum                                                  1.000000   \n",
              "...                                                                       ...   \n",
              "2026-03-31 Industrials                                               0.116825   \n",
              "           Institutions, Associations & Organizations               -0.253746   \n",
              "           Real Estate                                               0.298852   \n",
              "           Technology                                               -0.569050   \n",
              "           Utilities                                                 0.440704   \n",
              "\n",
              "                                                         Energy  Financials  \\\n",
              "date       factor                                                             \n",
              "2025-04-01 Market                                      1.000000    1.000000   \n",
              "           Dividend                                   -1.000000   -1.000000   \n",
              "           Growth                                      1.000000    1.000000   \n",
              "           Leverage                                   -1.000000   -1.000000   \n",
              "           Momentum                                    1.000000    1.000000   \n",
              "...                                                         ...         ...   \n",
              "2026-03-31 Industrials                                 0.066352    0.117271   \n",
              "           Institutions, Associations & Organizations -0.003886    0.343091   \n",
              "           Real Estate                                 0.077360    0.034780   \n",
              "           Technology                                 -0.254184   -0.228676   \n",
              "           Utilities                                   0.219548   -0.204212   \n",
              "\n",
              "                                                       Government Activity  \\\n",
              "date       factor                                                            \n",
              "2025-04-01 Market                                                -1.000000   \n",
              "           Dividend                                               1.000000   \n",
              "           Growth                                                -1.000000   \n",
              "           Leverage                                               1.000000   \n",
              "           Momentum                                              -1.000000   \n",
              "...                                                                    ...   \n",
              "2026-03-31 Industrials                                           -0.059579   \n",
              "           Institutions, Associations & Organizations            -0.243576   \n",
              "           Real Estate                                            0.046858   \n",
              "           Technology                                            -0.079027   \n",
              "           Utilities                                              0.138880   \n",
              "\n",
              "                                                       Healthcare  \\\n",
              "date       factor                                                   \n",
              "2025-04-01 Market                                       -1.000000   \n",
              "           Dividend                                      1.000000   \n",
              "           Growth                                       -1.000000   \n",
              "           Leverage                                      1.000000   \n",
              "           Momentum                                     -1.000000   \n",
              "...                                                           ...   \n",
              "2026-03-31 Industrials                                  -0.078656   \n",
              "           Institutions, Associations & Organizations   -0.391429   \n",
              "           Real Estate                                   0.138198   \n",
              "           Technology                                   -0.394288   \n",
              "           Utilities                                     0.045077   \n",
              "\n",
              "                                                       Industrials  \\\n",
              "date       factor                                                    \n",
              "2025-04-01 Market                                         1.000000   \n",
              "           Dividend                                      -1.000000   \n",
              "           Growth                                         1.000000   \n",
              "           Leverage                                      -1.000000   \n",
              "           Momentum                                       1.000000   \n",
              "...                                                            ...   \n",
              "2026-03-31 Industrials                                    1.000000   \n",
              "           Institutions, Associations & Organizations     0.212436   \n",
              "           Real Estate                                    0.056580   \n",
              "           Technology                                    -0.484509   \n",
              "           Utilities                                      0.035689   \n",
              "\n",
              "                                                       Institutions, Associations & Organizations  \\\n",
              "date       factor                                                                                   \n",
              "2025-04-01 Market                                                                        1.000000   \n",
              "           Dividend                                                                     -1.000000   \n",
              "           Growth                                                                        1.000000   \n",
              "           Leverage                                                                     -1.000000   \n",
              "           Momentum                                                                      1.000000   \n",
              "...                                                                                           ...   \n",
              "2026-03-31 Industrials                                                                   0.212436   \n",
              "           Institutions, Associations & Organizations                                    1.000000   \n",
              "           Real Estate                                                                  -0.174275   \n",
              "           Technology                                                                    0.133313   \n",
              "           Utilities                                                                    -0.043349   \n",
              "\n",
              "                                                       Real Estate  \\\n",
              "date       factor                                                    \n",
              "2025-04-01 Market                                        -1.000000   \n",
              "           Dividend                                       1.000000   \n",
              "           Growth                                        -1.000000   \n",
              "           Leverage                                       1.000000   \n",
              "           Momentum                                      -1.000000   \n",
              "...                                                            ...   \n",
              "2026-03-31 Industrials                                    0.056580   \n",
              "           Institutions, Associations & Organizations    -0.174275   \n",
              "           Real Estate                                    1.000000   \n",
              "           Technology                                    -0.409341   \n",
              "           Utilities                                      0.470684   \n",
              "\n",
              "                                                       Technology  Utilities  \n",
              "date       factor                                                             \n",
              "2025-04-01 Market                                        1.000000   1.000000  \n",
              "           Dividend                                     -1.000000  -1.000000  \n",
              "           Growth                                        1.000000   1.000000  \n",
              "           Leverage                                     -1.000000  -1.000000  \n",
              "           Momentum                                      1.000000   1.000000  \n",
              "...                                                           ...        ...  \n",
              "2026-03-31 Industrials                                  -0.484509   0.035689  \n",
              "           Institutions, Associations & Organizations    0.133313  -0.043349  \n",
              "           Real Estate                                  -0.409341   0.470684  \n",
              "           Technology                                    1.000000  -0.191221  \n",
              "           Utilities                                    -0.191221   1.000000  \n",
              "\n",
              "[5271 rows x 21 columns]"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate the ewma on the outer product (vcov with mean zero)\n",
        "df_factor_returns_outer = df_factor_returns.groupby(\"date\").apply(\n",
        "    lambda df: pd.DataFrame(np.outer(df, df), df.columns, df.columns)\n",
        ")\n",
        "df_factor_returns_outer.index.names = [\"date\", \"factor\"]\n",
        "df_cor_tieout = (\n",
        "    pd.DataFrame(df_factor_returns_outer)\n",
        "    .unstack()\n",
        "    .ewm(halflife=report_settings.factor_cov_settings.halflife_cor)\n",
        "    .mean()\n",
        "    .stack(future_stack=True)\n",
        "    .reindex(df_factor_returns_outer.columns, axis=1)\n",
        "    .groupby(\"date\")\n",
        "    .apply(\n",
        "        lambda df: df.droplevel(\"date\")\n",
        "        / np.outer(np.diag(df) ** 0.5, np.diag(df) ** 0.5)\n",
        "    )\n",
        "    .astype(np.float32)\n",
        ")\n",
        "\n",
        "df_cor_tieout"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "id": "a74a1d33",
      "metadata": {},
      "outputs": [],
      "source": [
        "pd.testing.assert_frame_equal(df_cor, df_cor_tieout, check_index_type=False, check_categorical=False, check_like=True, atol=1e-5)"
      ]
    }
  ],
  "metadata": {
    "jupytext": {
      "text_representation": {
        "extension": ".py",
        "format_name": "percent",
        "format_version": "1.3",
        "jupytext_version": "1.19.3"
      }
    },
    "kernelspec": {
      "display_name": ".venv",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.15"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}