{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "e7578988",
      "metadata": {},
      "source": [
        "# Factor Covariance Matrix Forecasts\n",
        "\n",
        "In this tutorial we are going to create several covariance matrices of factor returns. This covariance matrix follows from a standard factor model. We will also tie out the numbers against a pandas/numpy-based reimplementation. More specifically, we will:\n",
        "- Create a basic risk model.\n",
        "- Extract the factor returns.\n",
        "- Use the covariance matrix forecast report to compute the covariance matrix.\n",
        "- Use pandas to extract the factor volatility and correlation matrix time-series.\n",
        "- Replicate the volatility forecast.\n",
        "- Replicate the correlation forecast.\n",
        "\n",
        "Throughout this notebook we work with a randomly generated dataset. The results should generalize to real data, but for legal reasons we do not show any real data on our public API. Bayesline clients can run this notebook on real data."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b6c3069e",
      "metadata": {},
      "source": [
        "## Imports & Setup\n",
        "\n",
        "For this tutorial notebook, you will need to import the following packages."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "eb865731",
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import polars as pl\n",
        "import numpy as np\n",
        "\n",
        "from bayesline.api.equity import (\n",
        "    FactorCovarianceReportSettings,\n",
        "    ExposureSettings,\n",
        "    CategoricalExposureGroupSettings,\n",
        "    ContinuousExposureGroupSettings,\n",
        "    FactorRiskModelSettings,\n",
        "    ModelConstructionSettings,\n",
        "    UniverseSettings,\n",
        ")\n",
        "from bayesline.apiclient import BayeslineApiClient"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "03edb433",
      "metadata": {},
      "source": [
        "We will also need to have a Bayesline API client configured."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "605d2052",
      "metadata": {
        "tags": [
          "skip-execution"
        ]
      },
      "outputs": [],
      "source": [
        "bln = BayeslineApiClient.new_client(\n",
        "    endpoint=\"https://[ENDPOINT]\",\n",
        "    api_key=\"[API-KEY]\",\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "da39ba01",
      "metadata": {},
      "source": [
        "## Creating the covariance matrix forecasts\n",
        "\n",
        "Let's first set up a basic risk model and use it to generate the forecasts. We choose to run with mostly default settings. The steps involved are:\n",
        "1. Creating the settings of the risk model.\n",
        "2. Loading the report model engine.\n",
        "3. Running the engine to generate the covariance report.\n",
        "\n",
        "The first step is creating the risk model settings."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "id": "da63b4c5",
      "metadata": {},
      "outputs": [],
      "source": [
        "factorriskmodel_settings = FactorRiskModelSettings(\n",
        "    universe=UniverseSettings(),\n",
        "    exposures=ExposureSettings(\n",
        "        exposures=[\n",
        "            ContinuousExposureGroupSettings(hierarchy=\"market\"),\n",
        "            CategoricalExposureGroupSettings(hierarchy=\"trbc\"),\n",
        "            ContinuousExposureGroupSettings(hierarchy=\"style\"),\n",
        "        ]\n",
        "    ),\n",
        "    modelconstruction=ModelConstructionSettings(\n",
        "        estimation_universe=None,\n",
        "        zero_sum_constraints={\"trbc\": \"mcap_weighted\"},\n",
        "    ),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7eabb104",
      "metadata": {},
      "source": [
        "Next, we create the report engine from the report settings. We run with the defaults here."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "id": "503ebf3a",
      "metadata": {},
      "outputs": [],
      "source": [
        "report_settings = FactorCovarianceReportSettings(\n",
        "    factor_model_settings=factorriskmodel_settings\n",
        ")\n",
        "report_engine = bln.equity.reports.load(\n",
        "    report_settings.with_dataset(\"bayesline/Bayesline-US-All-1y\")\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f18fb859",
      "metadata": {},
      "source": [
        "Let's see what these settings really are by printing them out."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "id": "86ab9622",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"halflife_vol\": 60.0,\n",
            "  \"halflife_cor\": 120.0,\n",
            "  \"halflife_vra\": null,\n",
            "  \"nw_lags_vol\": 0,\n",
            "  \"nw_lags_vol_halflife_override\": null,\n",
            "  \"nw_lags_cor\": 0,\n",
            "  \"shrink_cor_method\": null,\n",
            "  \"shrink_cor_length\": null,\n",
            "  \"combine_standardized\": false\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "print(report_settings.factor_cov_settings.model_dump_json(indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "da8334b9",
      "metadata": {},
      "source": [
        "The different settings that jointly make up the covariance matrix are:\n",
        "1. **halflife_factor_vol** The halflife of the factor volatility. The default is a 60-day halflife.\n",
        "2. **halflife_factor_vra** The halflife of the cross-sectional factor volatility adjustment. The default is to not do any adjustment.\n",
        "3. **halflife_factor_cor** The halflife of the factor correlation. The default is a 120-day halflife.\n",
        "4. **nw_lags_factor_vol** The overlap or Newey-West lags to incluce on the factor volatility forecast. The default is zero, meaning no autocorrelation correction is performed.\n",
        "5. **nw_lags_factor_cor** The overlap or Newey-West lags to incluce on the factor correlation forecast. The default is zero, meaning no autocorrelation correction is performed.\n",
        "\n",
        "Now we get the actual time-series of the covariance matrices."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "id": "f429ee87",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2025-04-01</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <td>0.029966</td>\n",
              "      <td>-0.002397</td>\n",
              "      <td>0.007511</td>\n",
              "      <td>0.003567</td>\n",
              "      <td>-0.003908</td>\n",
              "      <td>0.013510</td>\n",
              "      <td>0.003488</td>\n",
              "      <td>-0.008661</td>\n",
              "      <td>0.004479</td>\n",
              "      <td>-0.062701</td>\n",
              "      <td>...</td>\n",
              "      <td>0.018827</td>\n",
              "      <td>-0.000213</td>\n",
              "      <td>0.001271</td>\n",
              "      <td>0.004183</td>\n",
              "      <td>-0.004209</td>\n",
              "      <td>0.001777</td>\n",
              "      <td>0.009248</td>\n",
              "      <td>0.006246</td>\n",
              "      <td>-0.003862</td>\n",
              "      <td>0.006650</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Basic Materials</th>\n",
              "      <td>-0.002397</td>\n",
              "      <td>0.000192</td>\n",
              "      <td>-0.000601</td>\n",
              "      <td>-0.000285</td>\n",
              "      <td>0.000313</td>\n",
              "      <td>-0.001081</td>\n",
              "      <td>-0.000279</td>\n",
              "      <td>0.000693</td>\n",
              "      <td>-0.000358</td>\n",
              "      <td>0.005015</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.001506</td>\n",
              "      <td>0.000017</td>\n",
              "      <td>-0.000102</td>\n",
              "      <td>-0.000335</td>\n",
              "      <td>0.000337</td>\n",
              "      <td>-0.000142</td>\n",
              "      <td>-0.000740</td>\n",
              "      <td>-0.000500</td>\n",
              "      <td>0.000309</td>\n",
              "      <td>-0.000532</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <td>0.007511</td>\n",
              "      <td>-0.000601</td>\n",
              "      <td>0.001883</td>\n",
              "      <td>0.000894</td>\n",
              "      <td>-0.000980</td>\n",
              "      <td>0.003386</td>\n",
              "      <td>0.000874</td>\n",
              "      <td>-0.002171</td>\n",
              "      <td>0.001123</td>\n",
              "      <td>-0.015716</td>\n",
              "      <td>...</td>\n",
              "      <td>0.004719</td>\n",
              "      <td>-0.000053</td>\n",
              "      <td>0.000319</td>\n",
              "      <td>0.001048</td>\n",
              "      <td>-0.001055</td>\n",
              "      <td>0.000446</td>\n",
              "      <td>0.002318</td>\n",
              "      <td>0.001566</td>\n",
              "      <td>-0.000968</td>\n",
              "      <td>0.001667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <td>0.003567</td>\n",
              "      <td>-0.000285</td>\n",
              "      <td>0.000894</td>\n",
              "      <td>0.000425</td>\n",
              "      <td>-0.000465</td>\n",
              "      <td>0.001608</td>\n",
              "      <td>0.000415</td>\n",
              "      <td>-0.001031</td>\n",
              "      <td>0.000533</td>\n",
              "      <td>-0.007464</td>\n",
              "      <td>...</td>\n",
              "      <td>0.002241</td>\n",
              "      <td>-0.000025</td>\n",
              "      <td>0.000151</td>\n",
              "      <td>0.000498</td>\n",
              "      <td>-0.000501</td>\n",
              "      <td>0.000212</td>\n",
              "      <td>0.001101</td>\n",
              "      <td>0.000744</td>\n",
              "      <td>-0.000460</td>\n",
              "      <td>0.000792</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Dividend</th>\n",
              "      <td>-0.003908</td>\n",
              "      <td>0.000313</td>\n",
              "      <td>-0.000980</td>\n",
              "      <td>-0.000465</td>\n",
              "      <td>0.000510</td>\n",
              "      <td>-0.001762</td>\n",
              "      <td>-0.000455</td>\n",
              "      <td>0.001130</td>\n",
              "      <td>-0.000584</td>\n",
              "      <td>0.008177</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.002455</td>\n",
              "      <td>0.000028</td>\n",
              "      <td>-0.000166</td>\n",
              "      <td>-0.000545</td>\n",
              "      <td>0.000549</td>\n",
              "      <td>-0.000232</td>\n",
              "      <td>-0.001206</td>\n",
              "      <td>-0.000815</td>\n",
              "      <td>0.000504</td>\n",
              "      <td>-0.000867</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2026-03-31</th>\n",
              "      <th>Size</th>\n",
              "      <td>-0.000416</td>\n",
              "      <td>-0.000027</td>\n",
              "      <td>0.000110</td>\n",
              "      <td>-0.001081</td>\n",
              "      <td>-0.000074</td>\n",
              "      <td>-0.000338</td>\n",
              "      <td>0.000728</td>\n",
              "      <td>-0.002423</td>\n",
              "      <td>0.000064</td>\n",
              "      <td>-0.000572</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003468</td>\n",
              "      <td>-0.000020</td>\n",
              "      <td>0.001408</td>\n",
              "      <td>0.000008</td>\n",
              "      <td>-0.000319</td>\n",
              "      <td>0.000658</td>\n",
              "      <td>-0.000029</td>\n",
              "      <td>-0.001117</td>\n",
              "      <td>0.000194</td>\n",
              "      <td>0.001727</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Technology</th>\n",
              "      <td>-0.001592</td>\n",
              "      <td>-0.003989</td>\n",
              "      <td>-0.002406</td>\n",
              "      <td>-0.003404</td>\n",
              "      <td>-0.000201</td>\n",
              "      <td>-0.002714</td>\n",
              "      <td>-0.000762</td>\n",
              "      <td>-0.002654</td>\n",
              "      <td>0.000126</td>\n",
              "      <td>-0.002552</td>\n",
              "      <td>...</td>\n",
              "      <td>0.001815</td>\n",
              "      <td>-0.000801</td>\n",
              "      <td>0.001161</td>\n",
              "      <td>0.000645</td>\n",
              "      <td>-0.002039</td>\n",
              "      <td>-0.000029</td>\n",
              "      <td>0.003253</td>\n",
              "      <td>-0.001454</td>\n",
              "      <td>-0.000563</td>\n",
              "      <td>0.001975</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Utilities</th>\n",
              "      <td>-0.002772</td>\n",
              "      <td>0.003643</td>\n",
              "      <td>-0.002818</td>\n",
              "      <td>0.006006</td>\n",
              "      <td>0.000266</td>\n",
              "      <td>0.005321</td>\n",
              "      <td>-0.001576</td>\n",
              "      <td>0.010350</td>\n",
              "      <td>-0.000291</td>\n",
              "      <td>0.000662</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.001393</td>\n",
              "      <td>-0.000992</td>\n",
              "      <td>-0.004899</td>\n",
              "      <td>0.003477</td>\n",
              "      <td>0.005372</td>\n",
              "      <td>-0.001117</td>\n",
              "      <td>-0.001454</td>\n",
              "      <td>0.016975</td>\n",
              "      <td>-0.001069</td>\n",
              "      <td>-0.002930</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Value</th>\n",
              "      <td>0.000130</td>\n",
              "      <td>0.001376</td>\n",
              "      <td>0.001224</td>\n",
              "      <td>0.000094</td>\n",
              "      <td>0.000155</td>\n",
              "      <td>0.001080</td>\n",
              "      <td>-0.000145</td>\n",
              "      <td>0.000409</td>\n",
              "      <td>0.000085</td>\n",
              "      <td>0.000148</td>\n",
              "      <td>...</td>\n",
              "      <td>0.002241</td>\n",
              "      <td>0.000370</td>\n",
              "      <td>0.001333</td>\n",
              "      <td>-0.000333</td>\n",
              "      <td>-0.000357</td>\n",
              "      <td>0.000194</td>\n",
              "      <td>-0.000563</td>\n",
              "      <td>-0.001069</td>\n",
              "      <td>0.001127</td>\n",
              "      <td>0.000708</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Volatility</th>\n",
              "      <td>-0.008483</td>\n",
              "      <td>-0.002486</td>\n",
              "      <td>-0.002836</td>\n",
              "      <td>-0.006738</td>\n",
              "      <td>-0.000010</td>\n",
              "      <td>-0.005903</td>\n",
              "      <td>0.004154</td>\n",
              "      <td>-0.036847</td>\n",
              "      <td>-0.000093</td>\n",
              "      <td>-0.007229</td>\n",
              "      <td>...</td>\n",
              "      <td>0.028196</td>\n",
              "      <td>-0.001636</td>\n",
              "      <td>0.014895</td>\n",
              "      <td>0.003633</td>\n",
              "      <td>-0.002074</td>\n",
              "      <td>0.001727</td>\n",
              "      <td>0.001975</td>\n",
              "      <td>-0.002930</td>\n",
              "      <td>0.000708</td>\n",
              "      <td>0.023168</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5271 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                            Academic & Educational Services  \\\n",
              "date       factor                                                             \n",
              "2025-04-01 Academic & Educational Services                         0.029966   \n",
              "           Basic Materials                                        -0.002397   \n",
              "           Consumer Cyclicals                                      0.007511   \n",
              "           Consumer Non-Cyclicals                                  0.003567   \n",
              "           Dividend                                               -0.003908   \n",
              "...                                                                     ...   \n",
              "2026-03-31 Size                                                   -0.000416   \n",
              "           Technology                                             -0.001592   \n",
              "           Utilities                                              -0.002772   \n",
              "           Value                                                   0.000130   \n",
              "           Volatility                                             -0.008483   \n",
              "\n",
              "                                            Basic Materials  \\\n",
              "date       factor                                             \n",
              "2025-04-01 Academic & Educational Services        -0.002397   \n",
              "           Basic Materials                         0.000192   \n",
              "           Consumer Cyclicals                     -0.000601   \n",
              "           Consumer Non-Cyclicals                 -0.000285   \n",
              "           Dividend                                0.000313   \n",
              "...                                                     ...   \n",
              "2026-03-31 Size                                   -0.000027   \n",
              "           Technology                             -0.003989   \n",
              "           Utilities                               0.003643   \n",
              "           Value                                   0.001376   \n",
              "           Volatility                             -0.002486   \n",
              "\n",
              "                                            Consumer Cyclicals  \\\n",
              "date       factor                                                \n",
              "2025-04-01 Academic & Educational Services            0.007511   \n",
              "           Basic Materials                           -0.000601   \n",
              "           Consumer Cyclicals                         0.001883   \n",
              "           Consumer Non-Cyclicals                     0.000894   \n",
              "           Dividend                                  -0.000980   \n",
              "...                                                        ...   \n",
              "2026-03-31 Size                                       0.000110   \n",
              "           Technology                                -0.002406   \n",
              "           Utilities                                 -0.002818   \n",
              "           Value                                      0.001224   \n",
              "           Volatility                                -0.002836   \n",
              "\n",
              "                                            Consumer Non-Cyclicals  Dividend  \\\n",
              "date       factor                                                              \n",
              "2025-04-01 Academic & Educational Services                0.003567 -0.003908   \n",
              "           Basic Materials                               -0.000285  0.000313   \n",
              "           Consumer Cyclicals                             0.000894 -0.000980   \n",
              "           Consumer Non-Cyclicals                         0.000425 -0.000465   \n",
              "           Dividend                                      -0.000465  0.000510   \n",
              "...                                                            ...       ...   \n",
              "2026-03-31 Size                                          -0.001081 -0.000074   \n",
              "           Technology                                    -0.003404 -0.000201   \n",
              "           Utilities                                      0.006006  0.000266   \n",
              "           Value                                          0.000094  0.000155   \n",
              "           Volatility                                    -0.006738 -0.000010   \n",
              "\n",
              "                                              Energy  Financials  \\\n",
              "date       factor                                                  \n",
              "2025-04-01 Academic & Educational Services  0.013510    0.003488   \n",
              "           Basic Materials                 -0.001081   -0.000279   \n",
              "           Consumer Cyclicals               0.003386    0.000874   \n",
              "           Consumer Non-Cyclicals           0.001608    0.000415   \n",
              "           Dividend                        -0.001762   -0.000455   \n",
              "...                                              ...         ...   \n",
              "2026-03-31 Size                            -0.000338    0.000728   \n",
              "           Technology                      -0.002714   -0.000762   \n",
              "           Utilities                        0.005321   -0.001576   \n",
              "           Value                            0.001080   -0.000145   \n",
              "           Volatility                      -0.005903    0.004154   \n",
              "\n",
              "                                            Government Activity    Growth  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Academic & Educational Services            -0.008661  0.004479   \n",
              "           Basic Materials                             0.000693 -0.000358   \n",
              "           Consumer Cyclicals                         -0.002171  0.001123   \n",
              "           Consumer Non-Cyclicals                     -0.001031  0.000533   \n",
              "           Dividend                                    0.001130 -0.000584   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Size                                       -0.002423  0.000064   \n",
              "           Technology                                 -0.002654  0.000126   \n",
              "           Utilities                                   0.010350 -0.000291   \n",
              "           Value                                       0.000409  0.000085   \n",
              "           Volatility                                 -0.036847 -0.000093   \n",
              "\n",
              "                                            Healthcare  ...  \\\n",
              "date       factor                                       ...   \n",
              "2025-04-01 Academic & Educational Services   -0.062701  ...   \n",
              "           Basic Materials                    0.005015  ...   \n",
              "           Consumer Cyclicals                -0.015716  ...   \n",
              "           Consumer Non-Cyclicals            -0.007464  ...   \n",
              "           Dividend                           0.008177  ...   \n",
              "...                                                ...  ...   \n",
              "2026-03-31 Size                              -0.000572  ...   \n",
              "           Technology                        -0.002552  ...   \n",
              "           Utilities                          0.000662  ...   \n",
              "           Value                              0.000148  ...   \n",
              "           Volatility                        -0.007229  ...   \n",
              "\n",
              "                                            Institutions, Associations & Organizations  \\\n",
              "date       factor                                                                        \n",
              "2025-04-01 Academic & Educational Services                                    0.018827   \n",
              "           Basic Materials                                                   -0.001506   \n",
              "           Consumer Cyclicals                                                 0.004719   \n",
              "           Consumer Non-Cyclicals                                             0.002241   \n",
              "           Dividend                                                          -0.002455   \n",
              "...                                                                                ...   \n",
              "2026-03-31 Size                                                               0.003468   \n",
              "           Technology                                                         0.001815   \n",
              "           Utilities                                                         -0.001393   \n",
              "           Value                                                              0.002241   \n",
              "           Volatility                                                         0.028196   \n",
              "\n",
              "                                            Leverage    Market  Momentum  \\\n",
              "date       factor                                                          \n",
              "2025-04-01 Academic & Educational Services -0.000213  0.001271  0.004183   \n",
              "           Basic Materials                  0.000017 -0.000102 -0.000335   \n",
              "           Consumer Cyclicals              -0.000053  0.000319  0.001048   \n",
              "           Consumer Non-Cyclicals          -0.000025  0.000151  0.000498   \n",
              "           Dividend                         0.000028 -0.000166 -0.000545   \n",
              "...                                              ...       ...       ...   \n",
              "2026-03-31 Size                            -0.000020  0.001408  0.000008   \n",
              "           Technology                      -0.000801  0.001161  0.000645   \n",
              "           Utilities                       -0.000992 -0.004899  0.003477   \n",
              "           Value                            0.000370  0.001333 -0.000333   \n",
              "           Volatility                      -0.001636  0.014895  0.003633   \n",
              "\n",
              "                                            Real Estate      Size  Technology  \\\n",
              "date       factor                                                               \n",
              "2025-04-01 Academic & Educational Services    -0.004209  0.001777    0.009248   \n",
              "           Basic Materials                     0.000337 -0.000142   -0.000740   \n",
              "           Consumer Cyclicals                 -0.001055  0.000446    0.002318   \n",
              "           Consumer Non-Cyclicals             -0.000501  0.000212    0.001101   \n",
              "           Dividend                            0.000549 -0.000232   -0.001206   \n",
              "...                                                 ...       ...         ...   \n",
              "2026-03-31 Size                               -0.000319  0.000658   -0.000029   \n",
              "           Technology                         -0.002039 -0.000029    0.003253   \n",
              "           Utilities                           0.005372 -0.001117   -0.001454   \n",
              "           Value                              -0.000357  0.000194   -0.000563   \n",
              "           Volatility                         -0.002074  0.001727    0.001975   \n",
              "\n",
              "                                            Utilities     Value  Volatility  \n",
              "date       factor                                                            \n",
              "2025-04-01 Academic & Educational Services   0.006246 -0.003862    0.006650  \n",
              "           Basic Materials                  -0.000500  0.000309   -0.000532  \n",
              "           Consumer Cyclicals                0.001566 -0.000968    0.001667  \n",
              "           Consumer Non-Cyclicals            0.000744 -0.000460    0.000792  \n",
              "           Dividend                         -0.000815  0.000504   -0.000867  \n",
              "...                                               ...       ...         ...  \n",
              "2026-03-31 Size                             -0.001117  0.000194    0.001727  \n",
              "           Technology                       -0.001454 -0.000563    0.001975  \n",
              "           Utilities                         0.016975 -0.001069   -0.002930  \n",
              "           Value                            -0.001069  0.001127    0.000708  \n",
              "           Volatility                       -0.002930  0.000708    0.023168  \n",
              "\n",
              "[5271 rows x 21 columns]"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# generate the report data\n",
        "report = report_engine.calculate(start_date=None, end_date=None)\n",
        "\n",
        "# get the covariance matrix (and skip the first date)\n",
        "df_report = report.get_covariance().filter(pl.col(\"date\") > pl.col(\"date\").min())\n",
        "\n",
        "# convert to pandas\n",
        "df_vcov = df_report.to_pandas().set_index([\"date\", \"factor\"]).rename(columns=lambda c: c.split(\"^\")[0])\n",
        "df_vcov"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "79e1c5d5",
      "metadata": {},
      "source": [
        "For downstream comparisons, we split these into factor volatilities and correlations."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "id": "44d74006",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.187725</td>\n",
              "      <td>0.135723</td>\n",
              "      <td>0.084518</td>\n",
              "      <td>0.101834</td>\n",
              "      <td>0.026181</td>\n",
              "      <td>0.171109</td>\n",
              "      <td>0.059769</td>\n",
              "      <td>0.558746</td>\n",
              "      <td>0.016477</td>\n",
              "      <td>0.111581</td>\n",
              "      <td>...</td>\n",
              "      <td>0.219731</td>\n",
              "      <td>0.029652</td>\n",
              "      <td>0.119914</td>\n",
              "      <td>0.067208</td>\n",
              "      <td>0.088902</td>\n",
              "      <td>0.023988</td>\n",
              "      <td>0.056709</td>\n",
              "      <td>0.129350</td>\n",
              "      <td>0.034274</td>\n",
              "      <td>0.142350</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>0.186729</td>\n",
              "      <td>0.135498</td>\n",
              "      <td>0.084006</td>\n",
              "      <td>0.101249</td>\n",
              "      <td>0.026044</td>\n",
              "      <td>0.175761</td>\n",
              "      <td>0.059556</td>\n",
              "      <td>0.562685</td>\n",
              "      <td>0.016709</td>\n",
              "      <td>0.113658</td>\n",
              "      <td>...</td>\n",
              "      <td>0.227805</td>\n",
              "      <td>0.029929</td>\n",
              "      <td>0.120131</td>\n",
              "      <td>0.068976</td>\n",
              "      <td>0.088513</td>\n",
              "      <td>0.024126</td>\n",
              "      <td>0.057143</td>\n",
              "      <td>0.129082</td>\n",
              "      <td>0.034084</td>\n",
              "      <td>0.144876</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>0.185819</td>\n",
              "      <td>0.137364</td>\n",
              "      <td>0.083513</td>\n",
              "      <td>0.103850</td>\n",
              "      <td>0.025891</td>\n",
              "      <td>0.178639</td>\n",
              "      <td>0.059380</td>\n",
              "      <td>0.560700</td>\n",
              "      <td>0.016760</td>\n",
              "      <td>0.114019</td>\n",
              "      <td>...</td>\n",
              "      <td>0.226535</td>\n",
              "      <td>0.029748</td>\n",
              "      <td>0.121707</td>\n",
              "      <td>0.069221</td>\n",
              "      <td>0.088033</td>\n",
              "      <td>0.024283</td>\n",
              "      <td>0.057027</td>\n",
              "      <td>0.130254</td>\n",
              "      <td>0.033878</td>\n",
              "      <td>0.144801</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>0.184813</td>\n",
              "      <td>0.137546</td>\n",
              "      <td>0.083225</td>\n",
              "      <td>0.103217</td>\n",
              "      <td>0.025810</td>\n",
              "      <td>0.178309</td>\n",
              "      <td>0.059767</td>\n",
              "      <td>0.563712</td>\n",
              "      <td>0.016689</td>\n",
              "      <td>0.114020</td>\n",
              "      <td>...</td>\n",
              "      <td>0.229350</td>\n",
              "      <td>0.029669</td>\n",
              "      <td>0.121738</td>\n",
              "      <td>0.071040</td>\n",
              "      <td>0.087866</td>\n",
              "      <td>0.024296</td>\n",
              "      <td>0.057213</td>\n",
              "      <td>0.129501</td>\n",
              "      <td>0.033712</td>\n",
              "      <td>0.146612</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.184630</td>\n",
              "      <td>0.136953</td>\n",
              "      <td>0.082755</td>\n",
              "      <td>0.104334</td>\n",
              "      <td>0.025810</td>\n",
              "      <td>0.186179</td>\n",
              "      <td>0.060200</td>\n",
              "      <td>0.572984</td>\n",
              "      <td>0.016630</td>\n",
              "      <td>0.113394</td>\n",
              "      <td>...</td>\n",
              "      <td>0.236829</td>\n",
              "      <td>0.030019</td>\n",
              "      <td>0.124546</td>\n",
              "      <td>0.071623</td>\n",
              "      <td>0.087384</td>\n",
              "      <td>0.025655</td>\n",
              "      <td>0.057034</td>\n",
              "      <td>0.130287</td>\n",
              "      <td>0.033566</td>\n",
              "      <td>0.152210</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "            Academic & Educational Services  Basic Materials  \\\n",
              "date                                                           \n",
              "2026-03-25                         0.187725         0.135723   \n",
              "2026-03-26                         0.186729         0.135498   \n",
              "2026-03-27                         0.185819         0.137364   \n",
              "2026-03-30                         0.184813         0.137546   \n",
              "2026-03-31                         0.184630         0.136953   \n",
              "\n",
              "            Consumer Cyclicals  Consumer Non-Cyclicals  Dividend    Energy  \\\n",
              "date                                                                         \n",
              "2026-03-25            0.084518                0.101834  0.026181  0.171109   \n",
              "2026-03-26            0.084006                0.101249  0.026044  0.175761   \n",
              "2026-03-27            0.083513                0.103850  0.025891  0.178639   \n",
              "2026-03-30            0.083225                0.103217  0.025810  0.178309   \n",
              "2026-03-31            0.082755                0.104334  0.025810  0.186179   \n",
              "\n",
              "            Financials  Government Activity    Growth  Healthcare  ...  \\\n",
              "date                                                               ...   \n",
              "2026-03-25    0.059769             0.558746  0.016477    0.111581  ...   \n",
              "2026-03-26    0.059556             0.562685  0.016709    0.113658  ...   \n",
              "2026-03-27    0.059380             0.560700  0.016760    0.114019  ...   \n",
              "2026-03-30    0.059767             0.563712  0.016689    0.114020  ...   \n",
              "2026-03-31    0.060200             0.572984  0.016630    0.113394  ...   \n",
              "\n",
              "            Institutions, Associations & Organizations  Leverage    Market  \\\n",
              "date                                                                         \n",
              "2026-03-25                                    0.219731  0.029652  0.119914   \n",
              "2026-03-26                                    0.227805  0.029929  0.120131   \n",
              "2026-03-27                                    0.226535  0.029748  0.121707   \n",
              "2026-03-30                                    0.229350  0.029669  0.121738   \n",
              "2026-03-31                                    0.236829  0.030019  0.124546   \n",
              "\n",
              "            Momentum  Real Estate      Size  Technology  Utilities     Value  \\\n",
              "date                                                                           \n",
              "2026-03-25  0.067208     0.088902  0.023988    0.056709   0.129350  0.034274   \n",
              "2026-03-26  0.068976     0.088513  0.024126    0.057143   0.129082  0.034084   \n",
              "2026-03-27  0.069221     0.088033  0.024283    0.057027   0.130254  0.033878   \n",
              "2026-03-30  0.071040     0.087866  0.024296    0.057213   0.129501  0.033712   \n",
              "2026-03-31  0.071623     0.087384  0.025655    0.057034   0.130287  0.033566   \n",
              "\n",
              "            Volatility  \n",
              "date                    \n",
              "2026-03-25    0.142350  \n",
              "2026-03-26    0.144876  \n",
              "2026-03-27    0.144801  \n",
              "2026-03-30    0.146612  \n",
              "2026-03-31    0.152210  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate actual factor volatilities from vcovs\n",
        "df_vol = df_vcov.groupby(level=\"date\").apply(\n",
        "    lambda df: pd.Series(np.diag(df) ** 0.5, df.index.droplevel(\"date\"))\n",
        ")\n",
        "df_vol.columns.name = None\n",
        "df_vol.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "id": "b672bd38",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.187725</td>\n",
              "      <td>0.135723</td>\n",
              "      <td>0.084518</td>\n",
              "      <td>0.101834</td>\n",
              "      <td>0.026181</td>\n",
              "      <td>0.171109</td>\n",
              "      <td>0.059769</td>\n",
              "      <td>0.558746</td>\n",
              "      <td>0.016477</td>\n",
              "      <td>0.111581</td>\n",
              "      <td>...</td>\n",
              "      <td>0.219731</td>\n",
              "      <td>0.029652</td>\n",
              "      <td>0.119914</td>\n",
              "      <td>0.067208</td>\n",
              "      <td>0.088902</td>\n",
              "      <td>0.023988</td>\n",
              "      <td>0.056709</td>\n",
              "      <td>0.129350</td>\n",
              "      <td>0.034274</td>\n",
              "      <td>0.142350</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>0.186729</td>\n",
              "      <td>0.135498</td>\n",
              "      <td>0.084006</td>\n",
              "      <td>0.101249</td>\n",
              "      <td>0.026044</td>\n",
              "      <td>0.175761</td>\n",
              "      <td>0.059556</td>\n",
              "      <td>0.562685</td>\n",
              "      <td>0.016709</td>\n",
              "      <td>0.113658</td>\n",
              "      <td>...</td>\n",
              "      <td>0.227805</td>\n",
              "      <td>0.029929</td>\n",
              "      <td>0.120131</td>\n",
              "      <td>0.068976</td>\n",
              "      <td>0.088513</td>\n",
              "      <td>0.024126</td>\n",
              "      <td>0.057143</td>\n",
              "      <td>0.129082</td>\n",
              "      <td>0.034084</td>\n",
              "      <td>0.144876</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>0.185819</td>\n",
              "      <td>0.137364</td>\n",
              "      <td>0.083513</td>\n",
              "      <td>0.103850</td>\n",
              "      <td>0.025891</td>\n",
              "      <td>0.178639</td>\n",
              "      <td>0.059380</td>\n",
              "      <td>0.560700</td>\n",
              "      <td>0.016760</td>\n",
              "      <td>0.114019</td>\n",
              "      <td>...</td>\n",
              "      <td>0.226535</td>\n",
              "      <td>0.029748</td>\n",
              "      <td>0.121707</td>\n",
              "      <td>0.069221</td>\n",
              "      <td>0.088033</td>\n",
              "      <td>0.024283</td>\n",
              "      <td>0.057027</td>\n",
              "      <td>0.130254</td>\n",
              "      <td>0.033878</td>\n",
              "      <td>0.144801</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>0.184813</td>\n",
              "      <td>0.137546</td>\n",
              "      <td>0.083225</td>\n",
              "      <td>0.103217</td>\n",
              "      <td>0.025810</td>\n",
              "      <td>0.178309</td>\n",
              "      <td>0.059767</td>\n",
              "      <td>0.563712</td>\n",
              "      <td>0.016689</td>\n",
              "      <td>0.114020</td>\n",
              "      <td>...</td>\n",
              "      <td>0.229350</td>\n",
              "      <td>0.029669</td>\n",
              "      <td>0.121738</td>\n",
              "      <td>0.071040</td>\n",
              "      <td>0.087866</td>\n",
              "      <td>0.024296</td>\n",
              "      <td>0.057213</td>\n",
              "      <td>0.129501</td>\n",
              "      <td>0.033712</td>\n",
              "      <td>0.146612</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.184630</td>\n",
              "      <td>0.136953</td>\n",
              "      <td>0.082755</td>\n",
              "      <td>0.104334</td>\n",
              "      <td>0.025810</td>\n",
              "      <td>0.186179</td>\n",
              "      <td>0.060200</td>\n",
              "      <td>0.572984</td>\n",
              "      <td>0.016630</td>\n",
              "      <td>0.113394</td>\n",
              "      <td>...</td>\n",
              "      <td>0.236829</td>\n",
              "      <td>0.030019</td>\n",
              "      <td>0.124546</td>\n",
              "      <td>0.071623</td>\n",
              "      <td>0.087384</td>\n",
              "      <td>0.025655</td>\n",
              "      <td>0.057034</td>\n",
              "      <td>0.130287</td>\n",
              "      <td>0.033566</td>\n",
              "      <td>0.152210</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "            Academic & Educational Services  Basic Materials  \\\n",
              "date                                                           \n",
              "2026-03-25                         0.187725         0.135723   \n",
              "2026-03-26                         0.186729         0.135498   \n",
              "2026-03-27                         0.185819         0.137364   \n",
              "2026-03-30                         0.184813         0.137546   \n",
              "2026-03-31                         0.184630         0.136953   \n",
              "\n",
              "            Consumer Cyclicals  Consumer Non-Cyclicals  Dividend    Energy  \\\n",
              "date                                                                         \n",
              "2026-03-25            0.084518                0.101834  0.026181  0.171109   \n",
              "2026-03-26            0.084006                0.101249  0.026044  0.175761   \n",
              "2026-03-27            0.083513                0.103850  0.025891  0.178639   \n",
              "2026-03-30            0.083225                0.103217  0.025810  0.178309   \n",
              "2026-03-31            0.082755                0.104334  0.025810  0.186179   \n",
              "\n",
              "            Financials  Government Activity    Growth  Healthcare  ...  \\\n",
              "date                                                               ...   \n",
              "2026-03-25    0.059769             0.558746  0.016477    0.111581  ...   \n",
              "2026-03-26    0.059556             0.562685  0.016709    0.113658  ...   \n",
              "2026-03-27    0.059380             0.560700  0.016760    0.114019  ...   \n",
              "2026-03-30    0.059767             0.563712  0.016689    0.114020  ...   \n",
              "2026-03-31    0.060200             0.572984  0.016630    0.113394  ...   \n",
              "\n",
              "            Institutions, Associations & Organizations  Leverage    Market  \\\n",
              "date                                                                         \n",
              "2026-03-25                                    0.219731  0.029652  0.119914   \n",
              "2026-03-26                                    0.227805  0.029929  0.120131   \n",
              "2026-03-27                                    0.226535  0.029748  0.121707   \n",
              "2026-03-30                                    0.229350  0.029669  0.121738   \n",
              "2026-03-31                                    0.236829  0.030019  0.124546   \n",
              "\n",
              "            Momentum  Real Estate      Size  Technology  Utilities     Value  \\\n",
              "date                                                                           \n",
              "2026-03-25  0.067208     0.088902  0.023988    0.056709   0.129350  0.034274   \n",
              "2026-03-26  0.068976     0.088513  0.024126    0.057143   0.129082  0.034084   \n",
              "2026-03-27  0.069221     0.088033  0.024283    0.057027   0.130254  0.033878   \n",
              "2026-03-30  0.071040     0.087866  0.024296    0.057213   0.129501  0.033712   \n",
              "2026-03-31  0.071623     0.087384  0.025655    0.057034   0.130287  0.033566   \n",
              "\n",
              "            Volatility  \n",
              "date                    \n",
              "2026-03-25    0.142350  \n",
              "2026-03-26    0.144876  \n",
              "2026-03-27    0.144801  \n",
              "2026-03-30    0.146612  \n",
              "2026-03-31    0.152210  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate actual factor volatilities from vcovs\n",
        "df_vol = df_vcov.groupby(level=\"date\").apply(\n",
        "    lambda df: pd.Series(np.diag(df) ** 0.5, df.index.droplevel(\"date\"))\n",
        ")\n",
        "df_vol.columns.name = None\n",
        "df_vol.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "id": "3891875c",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2026-03-31</th>\n",
              "      <th>Size</th>\n",
              "      <td>-0.087767</td>\n",
              "      <td>-0.007812</td>\n",
              "      <td>0.051913</td>\n",
              "      <td>-0.403710</td>\n",
              "      <td>-0.111696</td>\n",
              "      <td>-0.070753</td>\n",
              "      <td>0.471439</td>\n",
              "      <td>-0.164838</td>\n",
              "      <td>0.150275</td>\n",
              "      <td>-0.196580</td>\n",
              "      <td>...</td>\n",
              "      <td>0.570746</td>\n",
              "      <td>-0.026066</td>\n",
              "      <td>0.440511</td>\n",
              "      <td>0.004331</td>\n",
              "      <td>-0.142113</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.020136</td>\n",
              "      <td>-0.334070</td>\n",
              "      <td>0.225717</td>\n",
              "      <td>0.442306</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Technology</th>\n",
              "      <td>-0.151158</td>\n",
              "      <td>-0.510701</td>\n",
              "      <td>-0.509806</td>\n",
              "      <td>-0.571969</td>\n",
              "      <td>-0.136784</td>\n",
              "      <td>-0.255635</td>\n",
              "      <td>-0.221979</td>\n",
              "      <td>-0.081213</td>\n",
              "      <td>0.132799</td>\n",
              "      <td>-0.394590</td>\n",
              "      <td>...</td>\n",
              "      <td>0.134354</td>\n",
              "      <td>-0.467682</td>\n",
              "      <td>0.163508</td>\n",
              "      <td>0.158012</td>\n",
              "      <td>-0.409212</td>\n",
              "      <td>-0.020136</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.195636</td>\n",
              "      <td>-0.294188</td>\n",
              "      <td>0.227499</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Utilities</th>\n",
              "      <td>-0.115229</td>\n",
              "      <td>0.204157</td>\n",
              "      <td>-0.261373</td>\n",
              "      <td>0.441839</td>\n",
              "      <td>0.079028</td>\n",
              "      <td>0.219362</td>\n",
              "      <td>-0.200980</td>\n",
              "      <td>0.138650</td>\n",
              "      <td>-0.134463</td>\n",
              "      <td>0.044824</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.045137</td>\n",
              "      <td>-0.253754</td>\n",
              "      <td>-0.301886</td>\n",
              "      <td>0.372597</td>\n",
              "      <td>0.471882</td>\n",
              "      <td>-0.334070</td>\n",
              "      <td>-0.195636</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.244381</td>\n",
              "      <td>-0.147736</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Value</th>\n",
              "      <td>0.020946</td>\n",
              "      <td>0.299401</td>\n",
              "      <td>0.440687</td>\n",
              "      <td>0.026834</td>\n",
              "      <td>0.179429</td>\n",
              "      <td>0.172757</td>\n",
              "      <td>-0.071787</td>\n",
              "      <td>0.021291</td>\n",
              "      <td>0.152658</td>\n",
              "      <td>0.038806</td>\n",
              "      <td>...</td>\n",
              "      <td>0.281907</td>\n",
              "      <td>0.367435</td>\n",
              "      <td>0.318979</td>\n",
              "      <td>-0.138385</td>\n",
              "      <td>-0.121592</td>\n",
              "      <td>0.225717</td>\n",
              "      <td>-0.294188</td>\n",
              "      <td>-0.244381</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.138647</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Volatility</th>\n",
              "      <td>-0.301855</td>\n",
              "      <td>-0.119242</td>\n",
              "      <td>-0.225170</td>\n",
              "      <td>-0.424275</td>\n",
              "      <td>-0.002648</td>\n",
              "      <td>-0.208306</td>\n",
              "      <td>0.453329</td>\n",
              "      <td>-0.422491</td>\n",
              "      <td>-0.036698</td>\n",
              "      <td>-0.418838</td>\n",
              "      <td>...</td>\n",
              "      <td>0.782194</td>\n",
              "      <td>-0.358040</td>\n",
              "      <td>0.785705</td>\n",
              "      <td>0.333288</td>\n",
              "      <td>-0.155940</td>\n",
              "      <td>0.442306</td>\n",
              "      <td>0.227499</td>\n",
              "      <td>-0.147736</td>\n",
              "      <td>0.138647</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                       Academic & Educational Services  Basic Materials  \\\n",
              "date       factor                                                         \n",
              "2026-03-31 Size                              -0.087767        -0.007812   \n",
              "           Technology                        -0.151158        -0.510701   \n",
              "           Utilities                         -0.115229         0.204157   \n",
              "           Value                              0.020946         0.299401   \n",
              "           Volatility                        -0.301855        -0.119242   \n",
              "\n",
              "                       Consumer Cyclicals  Consumer Non-Cyclicals  Dividend  \\\n",
              "date       factor                                                             \n",
              "2026-03-31 Size                  0.051913               -0.403710 -0.111696   \n",
              "           Technology           -0.509806               -0.571969 -0.136784   \n",
              "           Utilities            -0.261373                0.441839  0.079028   \n",
              "           Value                 0.440687                0.026834  0.179429   \n",
              "           Volatility           -0.225170               -0.424275 -0.002648   \n",
              "\n",
              "                         Energy  Financials  Government Activity    Growth  \\\n",
              "date       factor                                                            \n",
              "2026-03-31 Size       -0.070753    0.471439            -0.164838  0.150275   \n",
              "           Technology -0.255635   -0.221979            -0.081213  0.132799   \n",
              "           Utilities   0.219362   -0.200980             0.138650 -0.134463   \n",
              "           Value       0.172757   -0.071787             0.021291  0.152658   \n",
              "           Volatility -0.208306    0.453329            -0.422491 -0.036698   \n",
              "\n",
              "                       Healthcare  ...  \\\n",
              "date       factor                  ...   \n",
              "2026-03-31 Size         -0.196580  ...   \n",
              "           Technology   -0.394590  ...   \n",
              "           Utilities     0.044824  ...   \n",
              "           Value         0.038806  ...   \n",
              "           Volatility   -0.418838  ...   \n",
              "\n",
              "                       Institutions, Associations & Organizations  Leverage  \\\n",
              "date       factor                                                             \n",
              "2026-03-31 Size                                          0.570746 -0.026066   \n",
              "           Technology                                    0.134354 -0.467682   \n",
              "           Utilities                                    -0.045137 -0.253754   \n",
              "           Value                                         0.281907  0.367435   \n",
              "           Volatility                                    0.782194 -0.358040   \n",
              "\n",
              "                         Market  Momentum  Real Estate      Size  Technology  \\\n",
              "date       factor                                                              \n",
              "2026-03-31 Size        0.440511  0.004331    -0.142113  1.000000   -0.020136   \n",
              "           Technology  0.163508  0.158012    -0.409212 -0.020136    1.000000   \n",
              "           Utilities  -0.301886  0.372597     0.471882 -0.334070   -0.195636   \n",
              "           Value       0.318979 -0.138385    -0.121592  0.225717   -0.294188   \n",
              "           Volatility  0.785705  0.333288    -0.155940  0.442306    0.227499   \n",
              "\n",
              "                       Utilities     Value  Volatility  \n",
              "date       factor                                       \n",
              "2026-03-31 Size        -0.334070  0.225717    0.442306  \n",
              "           Technology  -0.195636 -0.294188    0.227499  \n",
              "           Utilities    1.000000 -0.244381   -0.147736  \n",
              "           Value       -0.244381  1.000000    0.138647  \n",
              "           Volatility  -0.147736  0.138647    1.000000  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate actual factor correlations from vcovs\n",
        "df_cor = df_vcov.groupby(level=\"date\").apply(\n",
        "    lambda df: df.droplevel(\"date\")\n",
        "    / np.outer(np.diag(df) ** 0.5, np.diag(df) ** 0.5)\n",
        ")\n",
        "df_cor.tail()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f657ef9e",
      "metadata": {},
      "source": [
        "## Manually replicating the covariance forecasts\n",
        "\n",
        "We can also estimated the risk model and get the returns directly. From these returns we can construct the covariance forecasts. Bayesline returns dataframes in `polars`, but they can be easily converted to `pandas` dataframes. We also remove the factor group (market, style, industry, etc.) for convenience."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "id": "c2ef118f",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>...</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.005580</td>\n",
              "      <td>0.000363</td>\n",
              "      <td>-0.002220</td>\n",
              "      <td>-0.001572</td>\n",
              "      <td>0.002646</td>\n",
              "      <td>0.001181</td>\n",
              "      <td>0.000537</td>\n",
              "      <td>0.005788</td>\n",
              "      <td>0.013241</td>\n",
              "      <td>0.010212</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003293</td>\n",
              "      <td>-0.007817</td>\n",
              "      <td>-0.000165</td>\n",
              "      <td>-0.070588</td>\n",
              "      <td>0.007417</td>\n",
              "      <td>0.000180</td>\n",
              "      <td>0.004814</td>\n",
              "      <td>-0.003258</td>\n",
              "      <td>-0.001323</td>\n",
              "      <td>-0.001459</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>-0.008602</td>\n",
              "      <td>0.000615</td>\n",
              "      <td>-0.001893</td>\n",
              "      <td>0.002980</td>\n",
              "      <td>-0.009817</td>\n",
              "      <td>-0.002108</td>\n",
              "      <td>0.000657</td>\n",
              "      <td>-0.017798</td>\n",
              "      <td>0.004285</td>\n",
              "      <td>0.007291</td>\n",
              "      <td>...</td>\n",
              "      <td>0.001567</td>\n",
              "      <td>0.025334</td>\n",
              "      <td>-0.002422</td>\n",
              "      <td>0.051750</td>\n",
              "      <td>0.014207</td>\n",
              "      <td>-0.003142</td>\n",
              "      <td>-0.036999</td>\n",
              "      <td>0.002984</td>\n",
              "      <td>-0.005370</td>\n",
              "      <td>0.006618</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>-0.013475</td>\n",
              "      <td>-0.000330</td>\n",
              "      <td>0.001293</td>\n",
              "      <td>-0.000141</td>\n",
              "      <td>0.005472</td>\n",
              "      <td>-0.002188</td>\n",
              "      <td>0.000218</td>\n",
              "      <td>-0.008729</td>\n",
              "      <td>0.005280</td>\n",
              "      <td>0.015455</td>\n",
              "      <td>...</td>\n",
              "      <td>0.014650</td>\n",
              "      <td>0.021335</td>\n",
              "      <td>-0.002698</td>\n",
              "      <td>0.023007</td>\n",
              "      <td>-0.008833</td>\n",
              "      <td>0.000926</td>\n",
              "      <td>0.004219</td>\n",
              "      <td>0.001851</td>\n",
              "      <td>-0.002942</td>\n",
              "      <td>0.012854</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>-0.007827</td>\n",
              "      <td>0.001134</td>\n",
              "      <td>-0.000577</td>\n",
              "      <td>0.001407</td>\n",
              "      <td>-0.010111</td>\n",
              "      <td>0.001593</td>\n",
              "      <td>-0.000949</td>\n",
              "      <td>-0.015981</td>\n",
              "      <td>0.003921</td>\n",
              "      <td>0.009553</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000090</td>\n",
              "      <td>-0.009390</td>\n",
              "      <td>0.005387</td>\n",
              "      <td>0.048501</td>\n",
              "      <td>0.007191</td>\n",
              "      <td>-0.004815</td>\n",
              "      <td>-0.024947</td>\n",
              "      <td>0.004605</td>\n",
              "      <td>-0.004455</td>\n",
              "      <td>0.001864</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.016869</td>\n",
              "      <td>0.001627</td>\n",
              "      <td>-0.000683</td>\n",
              "      <td>-0.003213</td>\n",
              "      <td>0.006868</td>\n",
              "      <td>0.004951</td>\n",
              "      <td>-0.001138</td>\n",
              "      <td>0.025127</td>\n",
              "      <td>-0.010651</td>\n",
              "      <td>-0.004681</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.010863</td>\n",
              "      <td>-0.032599</td>\n",
              "      <td>0.005578</td>\n",
              "      <td>-0.068569</td>\n",
              "      <td>0.002256</td>\n",
              "      <td>0.000796</td>\n",
              "      <td>0.036700</td>\n",
              "      <td>-0.001743</td>\n",
              "      <td>0.002512</td>\n",
              "      <td>-0.011541</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "              Market  Dividend    Growth  Leverage  Momentum      Size  \\\n",
              "date                                                                     \n",
              "2026-03-25  0.005580  0.000363 -0.002220 -0.001572  0.002646  0.001181   \n",
              "2026-03-26 -0.008602  0.000615 -0.001893  0.002980 -0.009817 -0.002108   \n",
              "2026-03-27 -0.013475 -0.000330  0.001293 -0.000141  0.005472 -0.002188   \n",
              "2026-03-30 -0.007827  0.001134 -0.000577  0.001407 -0.010111  0.001593   \n",
              "2026-03-31  0.016869  0.001627 -0.000683 -0.003213  0.006868  0.004951   \n",
              "\n",
              "               Value  Volatility  Academic & Educational Services  \\\n",
              "date                                                                \n",
              "2026-03-25  0.000537    0.005788                         0.013241   \n",
              "2026-03-26  0.000657   -0.017798                         0.004285   \n",
              "2026-03-27  0.000218   -0.008729                         0.005280   \n",
              "2026-03-30 -0.000949   -0.015981                         0.003921   \n",
              "2026-03-31 -0.001138    0.025127                        -0.010651   \n",
              "\n",
              "            Basic Materials  ...  Consumer Non-Cyclicals    Energy  \\\n",
              "date                         ...                                     \n",
              "2026-03-25         0.010212  ...                0.003293 -0.007817   \n",
              "2026-03-26         0.007291  ...                0.001567  0.025334   \n",
              "2026-03-27         0.015455  ...                0.014650  0.021335   \n",
              "2026-03-30         0.009553  ...                0.000090 -0.009390   \n",
              "2026-03-31        -0.004681  ...               -0.010863 -0.032599   \n",
              "\n",
              "            Financials  Government Activity  Healthcare  Industrials  \\\n",
              "date                                                                   \n",
              "2026-03-25   -0.000165            -0.070588    0.007417     0.000180   \n",
              "2026-03-26   -0.002422             0.051750    0.014207    -0.003142   \n",
              "2026-03-27   -0.002698             0.023007   -0.008833     0.000926   \n",
              "2026-03-30    0.005387             0.048501    0.007191    -0.004815   \n",
              "2026-03-31    0.005578            -0.068569    0.002256     0.000796   \n",
              "\n",
              "            Institutions, Associations & Organizations  Real Estate  \\\n",
              "date                                                                  \n",
              "2026-03-25                                    0.004814    -0.003258   \n",
              "2026-03-26                                   -0.036999     0.002984   \n",
              "2026-03-27                                    0.004219     0.001851   \n",
              "2026-03-30                                   -0.024947     0.004605   \n",
              "2026-03-31                                    0.036700    -0.001743   \n",
              "\n",
              "            Technology  Utilities  \n",
              "date                               \n",
              "2026-03-25   -0.001323  -0.001459  \n",
              "2026-03-26   -0.005370   0.006618  \n",
              "2026-03-27   -0.002942   0.012854  \n",
              "2026-03-30   -0.004455   0.001864  \n",
              "2026-03-31    0.002512  -0.011541  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "risk_model = bln.equity.riskmodels.load(\n",
        "    factorriskmodel_settings.with_dataset(\"bayesline/Bayesline-US-All-1y\")\n",
        ").get_model()\n",
        "df_factor_returns = risk_model.fret().tail(-1).to_pandas().set_index(\"date\").rename(columns=lambda c: c.split(\".\")[1])\n",
        "df_factor_returns.tail()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3a83f572",
      "metadata": {},
      "source": [
        "From these returns we can run standard pandas functions to get the EWMAs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "id": "7099bed5",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>...</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.119914</td>\n",
              "      <td>0.026181</td>\n",
              "      <td>0.016477</td>\n",
              "      <td>0.029652</td>\n",
              "      <td>0.067208</td>\n",
              "      <td>0.023988</td>\n",
              "      <td>0.034274</td>\n",
              "      <td>0.142350</td>\n",
              "      <td>0.187725</td>\n",
              "      <td>0.135723</td>\n",
              "      <td>...</td>\n",
              "      <td>0.101834</td>\n",
              "      <td>0.171109</td>\n",
              "      <td>0.059769</td>\n",
              "      <td>0.558746</td>\n",
              "      <td>0.111581</td>\n",
              "      <td>0.067564</td>\n",
              "      <td>0.219731</td>\n",
              "      <td>0.088902</td>\n",
              "      <td>0.056709</td>\n",
              "      <td>0.129350</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>0.120131</td>\n",
              "      <td>0.026044</td>\n",
              "      <td>0.016709</td>\n",
              "      <td>0.029929</td>\n",
              "      <td>0.068976</td>\n",
              "      <td>0.024126</td>\n",
              "      <td>0.034084</td>\n",
              "      <td>0.144876</td>\n",
              "      <td>0.186729</td>\n",
              "      <td>0.135498</td>\n",
              "      <td>...</td>\n",
              "      <td>0.101249</td>\n",
              "      <td>0.175761</td>\n",
              "      <td>0.059556</td>\n",
              "      <td>0.562685</td>\n",
              "      <td>0.113658</td>\n",
              "      <td>0.067376</td>\n",
              "      <td>0.227805</td>\n",
              "      <td>0.088513</td>\n",
              "      <td>0.057143</td>\n",
              "      <td>0.129082</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>0.121707</td>\n",
              "      <td>0.025891</td>\n",
              "      <td>0.016760</td>\n",
              "      <td>0.029748</td>\n",
              "      <td>0.069221</td>\n",
              "      <td>0.024283</td>\n",
              "      <td>0.033878</td>\n",
              "      <td>0.144801</td>\n",
              "      <td>0.185819</td>\n",
              "      <td>0.137364</td>\n",
              "      <td>...</td>\n",
              "      <td>0.103850</td>\n",
              "      <td>0.178639</td>\n",
              "      <td>0.059380</td>\n",
              "      <td>0.560700</td>\n",
              "      <td>0.114019</td>\n",
              "      <td>0.066984</td>\n",
              "      <td>0.226535</td>\n",
              "      <td>0.088033</td>\n",
              "      <td>0.057027</td>\n",
              "      <td>0.130254</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>0.121738</td>\n",
              "      <td>0.025810</td>\n",
              "      <td>0.016689</td>\n",
              "      <td>0.029669</td>\n",
              "      <td>0.071040</td>\n",
              "      <td>0.024296</td>\n",
              "      <td>0.033712</td>\n",
              "      <td>0.146612</td>\n",
              "      <td>0.184813</td>\n",
              "      <td>0.137546</td>\n",
              "      <td>...</td>\n",
              "      <td>0.103217</td>\n",
              "      <td>0.178309</td>\n",
              "      <td>0.059767</td>\n",
              "      <td>0.563712</td>\n",
              "      <td>0.114020</td>\n",
              "      <td>0.067107</td>\n",
              "      <td>0.229350</td>\n",
              "      <td>0.087866</td>\n",
              "      <td>0.057213</td>\n",
              "      <td>0.129501</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.124546</td>\n",
              "      <td>0.025810</td>\n",
              "      <td>0.016630</td>\n",
              "      <td>0.030019</td>\n",
              "      <td>0.071623</td>\n",
              "      <td>0.025655</td>\n",
              "      <td>0.033566</td>\n",
              "      <td>0.152210</td>\n",
              "      <td>0.184630</td>\n",
              "      <td>0.136953</td>\n",
              "      <td>...</td>\n",
              "      <td>0.104334</td>\n",
              "      <td>0.186179</td>\n",
              "      <td>0.060200</td>\n",
              "      <td>0.572983</td>\n",
              "      <td>0.113394</td>\n",
              "      <td>0.066713</td>\n",
              "      <td>0.236829</td>\n",
              "      <td>0.087384</td>\n",
              "      <td>0.057034</td>\n",
              "      <td>0.130287</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "              Market  Dividend    Growth  Leverage  Momentum      Size  \\\n",
              "date                                                                     \n",
              "2026-03-25  0.119914  0.026181  0.016477  0.029652  0.067208  0.023988   \n",
              "2026-03-26  0.120131  0.026044  0.016709  0.029929  0.068976  0.024126   \n",
              "2026-03-27  0.121707  0.025891  0.016760  0.029748  0.069221  0.024283   \n",
              "2026-03-30  0.121738  0.025810  0.016689  0.029669  0.071040  0.024296   \n",
              "2026-03-31  0.124546  0.025810  0.016630  0.030019  0.071623  0.025655   \n",
              "\n",
              "               Value  Volatility  Academic & Educational Services  \\\n",
              "date                                                                \n",
              "2026-03-25  0.034274    0.142350                         0.187725   \n",
              "2026-03-26  0.034084    0.144876                         0.186729   \n",
              "2026-03-27  0.033878    0.144801                         0.185819   \n",
              "2026-03-30  0.033712    0.146612                         0.184813   \n",
              "2026-03-31  0.033566    0.152210                         0.184630   \n",
              "\n",
              "            Basic Materials  ...  Consumer Non-Cyclicals    Energy  \\\n",
              "date                         ...                                     \n",
              "2026-03-25         0.135723  ...                0.101834  0.171109   \n",
              "2026-03-26         0.135498  ...                0.101249  0.175761   \n",
              "2026-03-27         0.137364  ...                0.103850  0.178639   \n",
              "2026-03-30         0.137546  ...                0.103217  0.178309   \n",
              "2026-03-31         0.136953  ...                0.104334  0.186179   \n",
              "\n",
              "            Financials  Government Activity  Healthcare  Industrials  \\\n",
              "date                                                                   \n",
              "2026-03-25    0.059769             0.558746    0.111581     0.067564   \n",
              "2026-03-26    0.059556             0.562685    0.113658     0.067376   \n",
              "2026-03-27    0.059380             0.560700    0.114019     0.066984   \n",
              "2026-03-30    0.059767             0.563712    0.114020     0.067107   \n",
              "2026-03-31    0.060200             0.572983    0.113394     0.066713   \n",
              "\n",
              "            Institutions, Associations & Organizations  Real Estate  \\\n",
              "date                                                                  \n",
              "2026-03-25                                    0.219731     0.088902   \n",
              "2026-03-26                                    0.227805     0.088513   \n",
              "2026-03-27                                    0.226535     0.088033   \n",
              "2026-03-30                                    0.229350     0.087866   \n",
              "2026-03-31                                    0.236829     0.087384   \n",
              "\n",
              "            Technology  Utilities  \n",
              "date                               \n",
              "2026-03-25    0.056709   0.129350  \n",
              "2026-03-26    0.057143   0.129082  \n",
              "2026-03-27    0.057027   0.130254  \n",
              "2026-03-30    0.057213   0.129501  \n",
              "2026-03-31    0.057034   0.130287  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate expected factor volatilities using the ewma\n",
        "df_vol_tieout = (\n",
        "    pd.DataFrame(df_factor_returns**2)\n",
        "    .ewm(halflife=report_settings.factor_cov_settings.halflife_vol)\n",
        "    .mean()\n",
        "    .astype(np.float32)\n",
        "    ** 0.5\n",
        "    * 252**0.5\n",
        ")\n",
        "df_vol_tieout.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "id": "605fc54c",
      "metadata": {},
      "outputs": [],
      "source": [
        "pd.testing.assert_frame_equal(df_vol, df_vol_tieout, check_column_type=False, check_categorical=False, check_like=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9e0d029a",
      "metadata": {},
      "source": [
        "The correlation are a bit more involved. We need to create the outer products and then run EWMAs on each cell in the outer product matrix."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "id": "82f03756",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>...</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2025-04-01</th>\n",
              "      <th>Market</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Dividend</th>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Growth</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Leverage</th>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Momentum</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2026-03-31</th>\n",
              "      <th>Industrials</th>\n",
              "      <td>0.228479</td>\n",
              "      <td>-0.025782</td>\n",
              "      <td>0.039055</td>\n",
              "      <td>0.202773</td>\n",
              "      <td>0.124681</td>\n",
              "      <td>0.167971</td>\n",
              "      <td>0.316804</td>\n",
              "      <td>0.188615</td>\n",
              "      <td>0.044724</td>\n",
              "      <td>0.373740</td>\n",
              "      <td>...</td>\n",
              "      <td>0.112988</td>\n",
              "      <td>0.068361</td>\n",
              "      <td>0.111934</td>\n",
              "      <td>-0.059869</td>\n",
              "      <td>-0.082290</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.212385</td>\n",
              "      <td>0.054179</td>\n",
              "      <td>-0.479444</td>\n",
              "      <td>0.035091</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <td>0.412659</td>\n",
              "      <td>-0.006597</td>\n",
              "      <td>0.264721</td>\n",
              "      <td>-0.339985</td>\n",
              "      <td>0.607500</td>\n",
              "      <td>0.570746</td>\n",
              "      <td>0.281907</td>\n",
              "      <td>0.782194</td>\n",
              "      <td>-0.298033</td>\n",
              "      <td>-0.006043</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.252321</td>\n",
              "      <td>-0.003293</td>\n",
              "      <td>0.346594</td>\n",
              "      <td>-0.244793</td>\n",
              "      <td>-0.394259</td>\n",
              "      <td>0.212385</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.174273</td>\n",
              "      <td>0.134354</td>\n",
              "      <td>-0.045137</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Real Estate</th>\n",
              "      <td>-0.113487</td>\n",
              "      <td>0.107564</td>\n",
              "      <td>-0.076570</td>\n",
              "      <td>0.047704</td>\n",
              "      <td>0.011607</td>\n",
              "      <td>-0.142113</td>\n",
              "      <td>-0.121591</td>\n",
              "      <td>-0.155940</td>\n",
              "      <td>-0.009027</td>\n",
              "      <td>0.169025</td>\n",
              "      <td>...</td>\n",
              "      <td>0.298918</td>\n",
              "      <td>0.080334</td>\n",
              "      <td>0.030914</td>\n",
              "      <td>0.048033</td>\n",
              "      <td>0.138668</td>\n",
              "      <td>0.054179</td>\n",
              "      <td>-0.174273</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.409212</td>\n",
              "      <td>0.471882</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Technology</th>\n",
              "      <td>0.163508</td>\n",
              "      <td>-0.136784</td>\n",
              "      <td>0.132799</td>\n",
              "      <td>-0.467683</td>\n",
              "      <td>0.158012</td>\n",
              "      <td>-0.020136</td>\n",
              "      <td>-0.294188</td>\n",
              "      <td>0.227499</td>\n",
              "      <td>-0.151158</td>\n",
              "      <td>-0.510701</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.571968</td>\n",
              "      <td>-0.255635</td>\n",
              "      <td>-0.221979</td>\n",
              "      <td>-0.081213</td>\n",
              "      <td>-0.394590</td>\n",
              "      <td>-0.479444</td>\n",
              "      <td>0.134354</td>\n",
              "      <td>-0.409212</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.195636</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Utilities</th>\n",
              "      <td>-0.301886</td>\n",
              "      <td>0.079028</td>\n",
              "      <td>-0.134463</td>\n",
              "      <td>-0.253754</td>\n",
              "      <td>0.372597</td>\n",
              "      <td>-0.334071</td>\n",
              "      <td>-0.244381</td>\n",
              "      <td>-0.147736</td>\n",
              "      <td>-0.115229</td>\n",
              "      <td>0.204157</td>\n",
              "      <td>...</td>\n",
              "      <td>0.441839</td>\n",
              "      <td>0.219362</td>\n",
              "      <td>-0.200980</td>\n",
              "      <td>0.138650</td>\n",
              "      <td>0.044825</td>\n",
              "      <td>0.035091</td>\n",
              "      <td>-0.045137</td>\n",
              "      <td>0.471882</td>\n",
              "      <td>-0.195636</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5271 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                         Market  Dividend  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Market                                      1.000000 -1.000000   \n",
              "           Dividend                                   -1.000000  1.000000   \n",
              "           Growth                                      1.000000 -1.000000   \n",
              "           Leverage                                   -1.000000  1.000000   \n",
              "           Momentum                                    1.000000 -1.000000   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Industrials                                 0.228479 -0.025782   \n",
              "           Institutions, Associations & Organizations  0.412659 -0.006597   \n",
              "           Real Estate                                -0.113487  0.107564   \n",
              "           Technology                                  0.163508 -0.136784   \n",
              "           Utilities                                  -0.301886  0.079028   \n",
              "\n",
              "                                                         Growth  Leverage  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Market                                      1.000000 -1.000000   \n",
              "           Dividend                                   -1.000000  1.000000   \n",
              "           Growth                                      1.000000 -1.000000   \n",
              "           Leverage                                   -1.000000  1.000000   \n",
              "           Momentum                                    1.000000 -1.000000   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Industrials                                 0.039055  0.202773   \n",
              "           Institutions, Associations & Organizations  0.264721 -0.339985   \n",
              "           Real Estate                                -0.076570  0.047704   \n",
              "           Technology                                  0.132799 -0.467683   \n",
              "           Utilities                                  -0.134463 -0.253754   \n",
              "\n",
              "                                                       Momentum      Size  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Market                                      1.000000  1.000000   \n",
              "           Dividend                                   -1.000000 -1.000000   \n",
              "           Growth                                      1.000000  1.000000   \n",
              "           Leverage                                   -1.000000 -1.000000   \n",
              "           Momentum                                    1.000000  1.000000   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Industrials                                 0.124681  0.167971   \n",
              "           Institutions, Associations & Organizations  0.607500  0.570746   \n",
              "           Real Estate                                 0.011607 -0.142113   \n",
              "           Technology                                  0.158012 -0.020136   \n",
              "           Utilities                                   0.372597 -0.334071   \n",
              "\n",
              "                                                          Value  Volatility  \\\n",
              "date       factor                                                             \n",
              "2025-04-01 Market                                     -1.000000    1.000000   \n",
              "           Dividend                                    1.000000   -1.000000   \n",
              "           Growth                                     -1.000000    1.000000   \n",
              "           Leverage                                    1.000000   -1.000000   \n",
              "           Momentum                                   -1.000000    1.000000   \n",
              "...                                                         ...         ...   \n",
              "2026-03-31 Industrials                                 0.316804    0.188615   \n",
              "           Institutions, Associations & Organizations  0.281907    0.782194   \n",
              "           Real Estate                                -0.121591   -0.155940   \n",
              "           Technology                                 -0.294188    0.227499   \n",
              "           Utilities                                  -0.244381   -0.147736   \n",
              "\n",
              "                                                       Academic & Educational Services  \\\n",
              "date       factor                                                                        \n",
              "2025-04-01 Market                                                             1.000000   \n",
              "           Dividend                                                          -1.000000   \n",
              "           Growth                                                             1.000000   \n",
              "           Leverage                                                          -1.000000   \n",
              "           Momentum                                                           1.000000   \n",
              "...                                                                                ...   \n",
              "2026-03-31 Industrials                                                        0.044724   \n",
              "           Institutions, Associations & Organizations                        -0.298033   \n",
              "           Real Estate                                                       -0.009027   \n",
              "           Technology                                                        -0.151158   \n",
              "           Utilities                                                         -0.115229   \n",
              "\n",
              "                                                       Basic Materials  ...  \\\n",
              "date       factor                                                       ...   \n",
              "2025-04-01 Market                                            -1.000000  ...   \n",
              "           Dividend                                           1.000000  ...   \n",
              "           Growth                                            -1.000000  ...   \n",
              "           Leverage                                           1.000000  ...   \n",
              "           Momentum                                          -1.000000  ...   \n",
              "...                                                                ...  ...   \n",
              "2026-03-31 Industrials                                        0.373740  ...   \n",
              "           Institutions, Associations & Organizations        -0.006043  ...   \n",
              "           Real Estate                                        0.169025  ...   \n",
              "           Technology                                        -0.510701  ...   \n",
              "           Utilities                                          0.204157  ...   \n",
              "\n",
              "                                                       Consumer Non-Cyclicals  \\\n",
              "date       factor                                                               \n",
              "2025-04-01 Market                                                    1.000000   \n",
              "           Dividend                                                 -1.000000   \n",
              "           Growth                                                    1.000000   \n",
              "           Leverage                                                 -1.000000   \n",
              "           Momentum                                                  1.000000   \n",
              "...                                                                       ...   \n",
              "2026-03-31 Industrials                                               0.112988   \n",
              "           Institutions, Associations & Organizations               -0.252321   \n",
              "           Real Estate                                               0.298918   \n",
              "           Technology                                               -0.571968   \n",
              "           Utilities                                                 0.441839   \n",
              "\n",
              "                                                         Energy  Financials  \\\n",
              "date       factor                                                             \n",
              "2025-04-01 Market                                      1.000000    1.000000   \n",
              "           Dividend                                   -1.000000   -1.000000   \n",
              "           Growth                                      1.000000    1.000000   \n",
              "           Leverage                                   -1.000000   -1.000000   \n",
              "           Momentum                                    1.000000    1.000000   \n",
              "...                                                         ...         ...   \n",
              "2026-03-31 Industrials                                 0.068361    0.111934   \n",
              "           Institutions, Associations & Organizations -0.003293    0.346594   \n",
              "           Real Estate                                 0.080334    0.030914   \n",
              "           Technology                                 -0.255635   -0.221979   \n",
              "           Utilities                                   0.219362   -0.200980   \n",
              "\n",
              "                                                       Government Activity  \\\n",
              "date       factor                                                            \n",
              "2025-04-01 Market                                                -1.000000   \n",
              "           Dividend                                               1.000000   \n",
              "           Growth                                                -1.000000   \n",
              "           Leverage                                               1.000000   \n",
              "           Momentum                                              -1.000000   \n",
              "...                                                                    ...   \n",
              "2026-03-31 Industrials                                           -0.059869   \n",
              "           Institutions, Associations & Organizations            -0.244793   \n",
              "           Real Estate                                            0.048033   \n",
              "           Technology                                            -0.081213   \n",
              "           Utilities                                              0.138650   \n",
              "\n",
              "                                                       Healthcare  \\\n",
              "date       factor                                                   \n",
              "2025-04-01 Market                                       -1.000000   \n",
              "           Dividend                                      1.000000   \n",
              "           Growth                                       -1.000000   \n",
              "           Leverage                                      1.000000   \n",
              "           Momentum                                     -1.000000   \n",
              "...                                                           ...   \n",
              "2026-03-31 Industrials                                  -0.082290   \n",
              "           Institutions, Associations & Organizations   -0.394259   \n",
              "           Real Estate                                   0.138668   \n",
              "           Technology                                   -0.394590   \n",
              "           Utilities                                     0.044825   \n",
              "\n",
              "                                                       Industrials  \\\n",
              "date       factor                                                    \n",
              "2025-04-01 Market                                         1.000000   \n",
              "           Dividend                                      -1.000000   \n",
              "           Growth                                         1.000000   \n",
              "           Leverage                                      -1.000000   \n",
              "           Momentum                                       1.000000   \n",
              "...                                                            ...   \n",
              "2026-03-31 Industrials                                    1.000000   \n",
              "           Institutions, Associations & Organizations     0.212385   \n",
              "           Real Estate                                    0.054179   \n",
              "           Technology                                    -0.479444   \n",
              "           Utilities                                      0.035091   \n",
              "\n",
              "                                                       Institutions, Associations & Organizations  \\\n",
              "date       factor                                                                                   \n",
              "2025-04-01 Market                                                                        1.000000   \n",
              "           Dividend                                                                     -1.000000   \n",
              "           Growth                                                                        1.000000   \n",
              "           Leverage                                                                     -1.000000   \n",
              "           Momentum                                                                      1.000000   \n",
              "...                                                                                           ...   \n",
              "2026-03-31 Industrials                                                                   0.212385   \n",
              "           Institutions, Associations & Organizations                                    1.000000   \n",
              "           Real Estate                                                                  -0.174273   \n",
              "           Technology                                                                    0.134354   \n",
              "           Utilities                                                                    -0.045137   \n",
              "\n",
              "                                                       Real Estate  \\\n",
              "date       factor                                                    \n",
              "2025-04-01 Market                                        -1.000000   \n",
              "           Dividend                                       1.000000   \n",
              "           Growth                                        -1.000000   \n",
              "           Leverage                                       1.000000   \n",
              "           Momentum                                      -1.000000   \n",
              "...                                                            ...   \n",
              "2026-03-31 Industrials                                    0.054179   \n",
              "           Institutions, Associations & Organizations    -0.174273   \n",
              "           Real Estate                                    1.000000   \n",
              "           Technology                                    -0.409212   \n",
              "           Utilities                                      0.471882   \n",
              "\n",
              "                                                       Technology  Utilities  \n",
              "date       factor                                                             \n",
              "2025-04-01 Market                                        1.000000   1.000000  \n",
              "           Dividend                                     -1.000000  -1.000000  \n",
              "           Growth                                        1.000000   1.000000  \n",
              "           Leverage                                     -1.000000  -1.000000  \n",
              "           Momentum                                      1.000000   1.000000  \n",
              "...                                                           ...        ...  \n",
              "2026-03-31 Industrials                                  -0.479444   0.035091  \n",
              "           Institutions, Associations & Organizations    0.134354  -0.045137  \n",
              "           Real Estate                                  -0.409212   0.471882  \n",
              "           Technology                                    1.000000  -0.195636  \n",
              "           Utilities                                    -0.195636   1.000000  \n",
              "\n",
              "[5271 rows x 21 columns]"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate the ewma on the outer product (vcov with mean zero)\n",
        "df_factor_returns_outer = df_factor_returns.groupby(\"date\").apply(\n",
        "    lambda df: pd.DataFrame(np.outer(df, df), df.columns, df.columns)\n",
        ")\n",
        "df_factor_returns_outer.index.names = [\"date\", \"factor\"]\n",
        "df_cor_tieout = (\n",
        "    pd.DataFrame(df_factor_returns_outer)\n",
        "    .unstack()\n",
        "    .ewm(halflife=report_settings.factor_cov_settings.halflife_cor)\n",
        "    .mean()\n",
        "    .stack(future_stack=True)\n",
        "    .reindex(df_factor_returns_outer.columns, axis=1)\n",
        "    .groupby(\"date\")\n",
        "    .apply(\n",
        "        lambda df: df.droplevel(\"date\")\n",
        "        / np.outer(np.diag(df) ** 0.5, np.diag(df) ** 0.5)\n",
        "    )\n",
        "    .astype(np.float32)\n",
        ")\n",
        "\n",
        "df_cor_tieout"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "id": "04151f79",
      "metadata": {},
      "outputs": [],
      "source": [
        "pd.testing.assert_frame_equal(df_cor, df_cor_tieout, check_index_type=False, check_categorical=False, check_like=True, atol=1e-5)"
      ]
    }
  ],
  "metadata": {
    "jupytext": {
      "text_representation": {
        "extension": ".py",
        "format_name": "percent",
        "format_version": "1.3",
        "jupytext_version": "1.19.3"
      }
    },
    "kernelspec": {
      "display_name": ".venv",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.15"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}