{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "b2d49509",
      "metadata": {},
      "source": [
        "# Factor Covariance Matrix Forecasts\n",
        "\n",
        "In this tutorial we are going to create several covariance matrices of factor returns. This covariance matrix follows from a standard factor model. We will also tie out the numbers against a pandas/numpy-based reimplementation. More specifically, we will:\n",
        "- Create a basic risk model.\n",
        "- Extract the factor returns.\n",
        "- Use the covariance matrix forecast report to compute the covariance matrix.\n",
        "- Use pandas to extract the factor volatility and correlation matrix time-series.\n",
        "- Replicate the volatility forecast.\n",
        "- Replicate the correlation forecast.\n",
        "\n",
        "Throughout this notebook we work with a randomly generated dataset. The results should generalize to real data, but for legal reasons we do not show any real data on our public API. Bayesline clients can run this notebook on real data."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2a11063d",
      "metadata": {},
      "source": [
        "## Imports & Setup\n",
        "\n",
        "For this tutorial notebook, you will need to import the following packages."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "37b4d219",
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import polars as pl\n",
        "import numpy as np\n",
        "\n",
        "from bayesline.api.equity import (\n",
        "    FactorCovarianceReportSettings,\n",
        "    ExposureSettings,\n",
        "    CategoricalExposureGroupSettings,\n",
        "    ContinuousExposureGroupSettings,\n",
        "    FactorRiskModelSettings,\n",
        "    ModelConstructionSettings,\n",
        "    UniverseSettings,\n",
        ")\n",
        "from bayesline.apiclient import BayeslineApiClient"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f2815716",
      "metadata": {},
      "source": [
        "We will also need to have a Bayesline API client configured."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ddc3c72c",
      "metadata": {
        "tags": [
          "skip-execution"
        ]
      },
      "outputs": [],
      "source": [
        "bln = BayeslineApiClient.new_client(\n",
        "    endpoint=\"https://[ENDPOINT]\",\n",
        "    api_key=\"[API-KEY]\",\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "961bb375",
      "metadata": {},
      "source": [
        "## Creating the covariance matrix forecasts\n",
        "\n",
        "Let's first set up a basic risk model and use it to generate the forecasts. We choose to run with mostly default settings. The steps involved are:\n",
        "1. Creating the settings of the risk model.\n",
        "2. Loading the report model engine.\n",
        "3. Running the engine to generate the covariance report.\n",
        "\n",
        "The first step is creating the risk model settings."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "id": "cf1eaebe",
      "metadata": {},
      "outputs": [],
      "source": [
        "factorriskmodel_settings = FactorRiskModelSettings(\n",
        "    universe=UniverseSettings(),\n",
        "    exposures=ExposureSettings(\n",
        "        exposures=[\n",
        "            ContinuousExposureGroupSettings(hierarchy=\"market\"),\n",
        "            CategoricalExposureGroupSettings(hierarchy=\"trbc\"),\n",
        "            ContinuousExposureGroupSettings(hierarchy=\"style\"),\n",
        "        ]\n",
        "    ),\n",
        "    modelconstruction=ModelConstructionSettings(\n",
        "        estimation_universe=None,\n",
        "        zero_sum_constraints={\"trbc\": \"mcap_weighted\"},\n",
        "    ),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7df84329",
      "metadata": {},
      "source": [
        "Next, we create the report engine from the report settings. We run with the defaults here."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "id": "8bbd68f2",
      "metadata": {},
      "outputs": [],
      "source": [
        "report_settings = FactorCovarianceReportSettings(\n",
        "    factor_model_settings=factorriskmodel_settings\n",
        ")\n",
        "report_engine = bln.equity.reports.load(\n",
        "    report_settings.with_dataset(\"bayesline/Bayesline-US-All-1y\")\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a1676e4c",
      "metadata": {},
      "source": [
        "Let's see what these settings really are by printing them out."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "id": "a710cbbc",
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"halflife_vol\": 60.0,\n",
            "  \"halflife_cor\": 120.0,\n",
            "  \"halflife_vra\": null,\n",
            "  \"nw_lags_vol\": 0,\n",
            "  \"nw_lags_vol_halflife_override\": null,\n",
            "  \"nw_lags_cor\": 0,\n",
            "  \"shrink_cor_method\": null,\n",
            "  \"shrink_cor_length\": null,\n",
            "  \"combine_standardized\": false\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "print(report_settings.factor_cov_settings.model_dump_json(indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d40b646e",
      "metadata": {},
      "source": [
        "The different settings that jointly make up the covariance matrix are:\n",
        "1. **halflife_factor_vol** The halflife of the factor volatility. The default is a 60-day halflife.\n",
        "2. **halflife_factor_vra** The halflife of the cross-sectional factor volatility adjustment. The default is to not do any adjustment.\n",
        "3. **halflife_factor_cor** The halflife of the factor correlation. The default is a 120-day halflife.\n",
        "4. **nw_lags_factor_vol** The overlap or Newey-West lags to incluce on the factor volatility forecast. The default is zero, meaning no autocorrelation correction is performed.\n",
        "5. **nw_lags_factor_cor** The overlap or Newey-West lags to incluce on the factor correlation forecast. The default is zero, meaning no autocorrelation correction is performed.\n",
        "\n",
        "Now we get the actual time-series of the covariance matrices."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "id": "5613e365",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2025-04-01</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <td>0.028461</td>\n",
              "      <td>-0.001975</td>\n",
              "      <td>0.007310</td>\n",
              "      <td>0.003482</td>\n",
              "      <td>-0.003814</td>\n",
              "      <td>0.013186</td>\n",
              "      <td>0.003410</td>\n",
              "      <td>-0.008398</td>\n",
              "      <td>0.004290</td>\n",
              "      <td>-0.061013</td>\n",
              "      <td>...</td>\n",
              "      <td>0.018185</td>\n",
              "      <td>-0.000225</td>\n",
              "      <td>0.001207</td>\n",
              "      <td>0.004089</td>\n",
              "      <td>-0.003952</td>\n",
              "      <td>0.001755</td>\n",
              "      <td>0.008957</td>\n",
              "      <td>0.006069</td>\n",
              "      <td>-0.003784</td>\n",
              "      <td>0.006459</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Basic Materials</th>\n",
              "      <td>-0.001975</td>\n",
              "      <td>0.000137</td>\n",
              "      <td>-0.000507</td>\n",
              "      <td>-0.000242</td>\n",
              "      <td>0.000265</td>\n",
              "      <td>-0.000915</td>\n",
              "      <td>-0.000237</td>\n",
              "      <td>0.000583</td>\n",
              "      <td>-0.000298</td>\n",
              "      <td>0.004234</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.001262</td>\n",
              "      <td>0.000016</td>\n",
              "      <td>-0.000084</td>\n",
              "      <td>-0.000284</td>\n",
              "      <td>0.000274</td>\n",
              "      <td>-0.000122</td>\n",
              "      <td>-0.000622</td>\n",
              "      <td>-0.000421</td>\n",
              "      <td>0.000263</td>\n",
              "      <td>-0.000448</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <td>0.007310</td>\n",
              "      <td>-0.000507</td>\n",
              "      <td>0.001878</td>\n",
              "      <td>0.000894</td>\n",
              "      <td>-0.000980</td>\n",
              "      <td>0.003387</td>\n",
              "      <td>0.000876</td>\n",
              "      <td>-0.002157</td>\n",
              "      <td>0.001102</td>\n",
              "      <td>-0.015672</td>\n",
              "      <td>...</td>\n",
              "      <td>0.004671</td>\n",
              "      <td>-0.000058</td>\n",
              "      <td>0.000310</td>\n",
              "      <td>0.001050</td>\n",
              "      <td>-0.001015</td>\n",
              "      <td>0.000451</td>\n",
              "      <td>0.002301</td>\n",
              "      <td>0.001559</td>\n",
              "      <td>-0.000972</td>\n",
              "      <td>0.001659</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <td>0.003482</td>\n",
              "      <td>-0.000242</td>\n",
              "      <td>0.000894</td>\n",
              "      <td>0.000426</td>\n",
              "      <td>-0.000467</td>\n",
              "      <td>0.001613</td>\n",
              "      <td>0.000417</td>\n",
              "      <td>-0.001027</td>\n",
              "      <td>0.000525</td>\n",
              "      <td>-0.007464</td>\n",
              "      <td>...</td>\n",
              "      <td>0.002225</td>\n",
              "      <td>-0.000028</td>\n",
              "      <td>0.000148</td>\n",
              "      <td>0.000500</td>\n",
              "      <td>-0.000484</td>\n",
              "      <td>0.000215</td>\n",
              "      <td>0.001096</td>\n",
              "      <td>0.000742</td>\n",
              "      <td>-0.000463</td>\n",
              "      <td>0.000790</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Dividend</th>\n",
              "      <td>-0.003814</td>\n",
              "      <td>0.000265</td>\n",
              "      <td>-0.000980</td>\n",
              "      <td>-0.000467</td>\n",
              "      <td>0.000511</td>\n",
              "      <td>-0.001767</td>\n",
              "      <td>-0.000457</td>\n",
              "      <td>0.001126</td>\n",
              "      <td>-0.000575</td>\n",
              "      <td>0.008177</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.002437</td>\n",
              "      <td>0.000030</td>\n",
              "      <td>-0.000162</td>\n",
              "      <td>-0.000548</td>\n",
              "      <td>0.000530</td>\n",
              "      <td>-0.000235</td>\n",
              "      <td>-0.001200</td>\n",
              "      <td>-0.000813</td>\n",
              "      <td>0.000507</td>\n",
              "      <td>-0.000866</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2026-03-31</th>\n",
              "      <th>Size</th>\n",
              "      <td>-0.000408</td>\n",
              "      <td>-0.000009</td>\n",
              "      <td>0.000112</td>\n",
              "      <td>-0.001085</td>\n",
              "      <td>-0.000074</td>\n",
              "      <td>-0.000341</td>\n",
              "      <td>0.000731</td>\n",
              "      <td>-0.002446</td>\n",
              "      <td>0.000065</td>\n",
              "      <td>-0.000573</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003500</td>\n",
              "      <td>-0.000023</td>\n",
              "      <td>0.001415</td>\n",
              "      <td>0.000011</td>\n",
              "      <td>-0.000320</td>\n",
              "      <td>0.000663</td>\n",
              "      <td>-0.000032</td>\n",
              "      <td>-0.001118</td>\n",
              "      <td>0.000196</td>\n",
              "      <td>0.001740</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Technology</th>\n",
              "      <td>-0.001537</td>\n",
              "      <td>-0.004048</td>\n",
              "      <td>-0.002443</td>\n",
              "      <td>-0.003422</td>\n",
              "      <td>-0.000196</td>\n",
              "      <td>-0.002747</td>\n",
              "      <td>-0.000773</td>\n",
              "      <td>-0.002704</td>\n",
              "      <td>0.000123</td>\n",
              "      <td>-0.002570</td>\n",
              "      <td>...</td>\n",
              "      <td>0.001855</td>\n",
              "      <td>-0.000808</td>\n",
              "      <td>0.001157</td>\n",
              "      <td>0.000667</td>\n",
              "      <td>-0.002062</td>\n",
              "      <td>-0.000032</td>\n",
              "      <td>0.003295</td>\n",
              "      <td>-0.001451</td>\n",
              "      <td>-0.000566</td>\n",
              "      <td>0.001996</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Utilities</th>\n",
              "      <td>-0.002903</td>\n",
              "      <td>0.003570</td>\n",
              "      <td>-0.002790</td>\n",
              "      <td>0.006001</td>\n",
              "      <td>0.000260</td>\n",
              "      <td>0.005314</td>\n",
              "      <td>-0.001584</td>\n",
              "      <td>0.010334</td>\n",
              "      <td>-0.000296</td>\n",
              "      <td>0.000663</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.001439</td>\n",
              "      <td>-0.000973</td>\n",
              "      <td>-0.004867</td>\n",
              "      <td>0.003461</td>\n",
              "      <td>0.005367</td>\n",
              "      <td>-0.001118</td>\n",
              "      <td>-0.001451</td>\n",
              "      <td>0.016901</td>\n",
              "      <td>-0.001056</td>\n",
              "      <td>-0.002930</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Value</th>\n",
              "      <td>0.000100</td>\n",
              "      <td>0.001364</td>\n",
              "      <td>0.001202</td>\n",
              "      <td>0.000088</td>\n",
              "      <td>0.000154</td>\n",
              "      <td>0.001082</td>\n",
              "      <td>-0.000131</td>\n",
              "      <td>0.000415</td>\n",
              "      <td>0.000086</td>\n",
              "      <td>0.000152</td>\n",
              "      <td>...</td>\n",
              "      <td>0.002268</td>\n",
              "      <td>0.000365</td>\n",
              "      <td>0.001313</td>\n",
              "      <td>-0.000328</td>\n",
              "      <td>-0.000358</td>\n",
              "      <td>0.000196</td>\n",
              "      <td>-0.000566</td>\n",
              "      <td>-0.001056</td>\n",
              "      <td>0.001118</td>\n",
              "      <td>0.000716</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Volatility</th>\n",
              "      <td>-0.008443</td>\n",
              "      <td>-0.002458</td>\n",
              "      <td>-0.002833</td>\n",
              "      <td>-0.006751</td>\n",
              "      <td>-0.000023</td>\n",
              "      <td>-0.005901</td>\n",
              "      <td>0.004123</td>\n",
              "      <td>-0.036713</td>\n",
              "      <td>-0.000102</td>\n",
              "      <td>-0.007210</td>\n",
              "      <td>...</td>\n",
              "      <td>0.028142</td>\n",
              "      <td>-0.001643</td>\n",
              "      <td>0.014878</td>\n",
              "      <td>0.003621</td>\n",
              "      <td>-0.002065</td>\n",
              "      <td>0.001740</td>\n",
              "      <td>0.001996</td>\n",
              "      <td>-0.002930</td>\n",
              "      <td>0.000716</td>\n",
              "      <td>0.023099</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5271 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                            Academic & Educational Services  \\\n",
              "date       factor                                                             \n",
              "2025-04-01 Academic & Educational Services                         0.028461   \n",
              "           Basic Materials                                        -0.001975   \n",
              "           Consumer Cyclicals                                      0.007310   \n",
              "           Consumer Non-Cyclicals                                  0.003482   \n",
              "           Dividend                                               -0.003814   \n",
              "...                                                                     ...   \n",
              "2026-03-31 Size                                                   -0.000408   \n",
              "           Technology                                             -0.001537   \n",
              "           Utilities                                              -0.002903   \n",
              "           Value                                                   0.000100   \n",
              "           Volatility                                             -0.008443   \n",
              "\n",
              "                                            Basic Materials  \\\n",
              "date       factor                                             \n",
              "2025-04-01 Academic & Educational Services        -0.001975   \n",
              "           Basic Materials                         0.000137   \n",
              "           Consumer Cyclicals                     -0.000507   \n",
              "           Consumer Non-Cyclicals                 -0.000242   \n",
              "           Dividend                                0.000265   \n",
              "...                                                     ...   \n",
              "2026-03-31 Size                                   -0.000009   \n",
              "           Technology                             -0.004048   \n",
              "           Utilities                               0.003570   \n",
              "           Value                                   0.001364   \n",
              "           Volatility                             -0.002458   \n",
              "\n",
              "                                            Consumer Cyclicals  \\\n",
              "date       factor                                                \n",
              "2025-04-01 Academic & Educational Services            0.007310   \n",
              "           Basic Materials                           -0.000507   \n",
              "           Consumer Cyclicals                         0.001878   \n",
              "           Consumer Non-Cyclicals                     0.000894   \n",
              "           Dividend                                  -0.000980   \n",
              "...                                                        ...   \n",
              "2026-03-31 Size                                       0.000112   \n",
              "           Technology                                -0.002443   \n",
              "           Utilities                                 -0.002790   \n",
              "           Value                                      0.001202   \n",
              "           Volatility                                -0.002833   \n",
              "\n",
              "                                            Consumer Non-Cyclicals  Dividend  \\\n",
              "date       factor                                                              \n",
              "2025-04-01 Academic & Educational Services                0.003482 -0.003814   \n",
              "           Basic Materials                               -0.000242  0.000265   \n",
              "           Consumer Cyclicals                             0.000894 -0.000980   \n",
              "           Consumer Non-Cyclicals                         0.000426 -0.000467   \n",
              "           Dividend                                      -0.000467  0.000511   \n",
              "...                                                            ...       ...   \n",
              "2026-03-31 Size                                          -0.001085 -0.000074   \n",
              "           Technology                                    -0.003422 -0.000196   \n",
              "           Utilities                                      0.006001  0.000260   \n",
              "           Value                                          0.000088  0.000154   \n",
              "           Volatility                                    -0.006751 -0.000023   \n",
              "\n",
              "                                              Energy  Financials  \\\n",
              "date       factor                                                  \n",
              "2025-04-01 Academic & Educational Services  0.013186    0.003410   \n",
              "           Basic Materials                 -0.000915   -0.000237   \n",
              "           Consumer Cyclicals               0.003387    0.000876   \n",
              "           Consumer Non-Cyclicals           0.001613    0.000417   \n",
              "           Dividend                        -0.001767   -0.000457   \n",
              "...                                              ...         ...   \n",
              "2026-03-31 Size                            -0.000341    0.000731   \n",
              "           Technology                      -0.002747   -0.000773   \n",
              "           Utilities                        0.005314   -0.001584   \n",
              "           Value                            0.001082   -0.000131   \n",
              "           Volatility                      -0.005901    0.004123   \n",
              "\n",
              "                                            Government Activity    Growth  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Academic & Educational Services            -0.008398  0.004290   \n",
              "           Basic Materials                             0.000583 -0.000298   \n",
              "           Consumer Cyclicals                         -0.002157  0.001102   \n",
              "           Consumer Non-Cyclicals                     -0.001027  0.000525   \n",
              "           Dividend                                    0.001126 -0.000575   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Size                                       -0.002446  0.000065   \n",
              "           Technology                                 -0.002704  0.000123   \n",
              "           Utilities                                   0.010334 -0.000296   \n",
              "           Value                                       0.000415  0.000086   \n",
              "           Volatility                                 -0.036713 -0.000102   \n",
              "\n",
              "                                            Healthcare  ...  \\\n",
              "date       factor                                       ...   \n",
              "2025-04-01 Academic & Educational Services   -0.061013  ...   \n",
              "           Basic Materials                    0.004234  ...   \n",
              "           Consumer Cyclicals                -0.015672  ...   \n",
              "           Consumer Non-Cyclicals            -0.007464  ...   \n",
              "           Dividend                           0.008177  ...   \n",
              "...                                                ...  ...   \n",
              "2026-03-31 Size                              -0.000573  ...   \n",
              "           Technology                        -0.002570  ...   \n",
              "           Utilities                          0.000663  ...   \n",
              "           Value                              0.000152  ...   \n",
              "           Volatility                        -0.007210  ...   \n",
              "\n",
              "                                            Institutions, Associations & Organizations  \\\n",
              "date       factor                                                                        \n",
              "2025-04-01 Academic & Educational Services                                    0.018185   \n",
              "           Basic Materials                                                   -0.001262   \n",
              "           Consumer Cyclicals                                                 0.004671   \n",
              "           Consumer Non-Cyclicals                                             0.002225   \n",
              "           Dividend                                                          -0.002437   \n",
              "...                                                                                ...   \n",
              "2026-03-31 Size                                                               0.003500   \n",
              "           Technology                                                         0.001855   \n",
              "           Utilities                                                         -0.001439   \n",
              "           Value                                                              0.002268   \n",
              "           Volatility                                                         0.028142   \n",
              "\n",
              "                                            Leverage    Market  Momentum  \\\n",
              "date       factor                                                          \n",
              "2025-04-01 Academic & Educational Services -0.000225  0.001207  0.004089   \n",
              "           Basic Materials                  0.000016 -0.000084 -0.000284   \n",
              "           Consumer Cyclicals              -0.000058  0.000310  0.001050   \n",
              "           Consumer Non-Cyclicals          -0.000028  0.000148  0.000500   \n",
              "           Dividend                         0.000030 -0.000162 -0.000548   \n",
              "...                                              ...       ...       ...   \n",
              "2026-03-31 Size                            -0.000023  0.001415  0.000011   \n",
              "           Technology                      -0.000808  0.001157  0.000667   \n",
              "           Utilities                       -0.000973 -0.004867  0.003461   \n",
              "           Value                            0.000365  0.001313 -0.000328   \n",
              "           Volatility                      -0.001643  0.014878  0.003621   \n",
              "\n",
              "                                            Real Estate      Size  Technology  \\\n",
              "date       factor                                                               \n",
              "2025-04-01 Academic & Educational Services    -0.003952  0.001755    0.008957   \n",
              "           Basic Materials                     0.000274 -0.000122   -0.000622   \n",
              "           Consumer Cyclicals                 -0.001015  0.000451    0.002301   \n",
              "           Consumer Non-Cyclicals             -0.000484  0.000215    0.001096   \n",
              "           Dividend                            0.000530 -0.000235   -0.001200   \n",
              "...                                                 ...       ...         ...   \n",
              "2026-03-31 Size                               -0.000320  0.000663   -0.000032   \n",
              "           Technology                         -0.002062 -0.000032    0.003295   \n",
              "           Utilities                           0.005367 -0.001118   -0.001451   \n",
              "           Value                              -0.000358  0.000196   -0.000566   \n",
              "           Volatility                         -0.002065  0.001740    0.001996   \n",
              "\n",
              "                                            Utilities     Value  Volatility  \n",
              "date       factor                                                            \n",
              "2025-04-01 Academic & Educational Services   0.006069 -0.003784    0.006459  \n",
              "           Basic Materials                  -0.000421  0.000263   -0.000448  \n",
              "           Consumer Cyclicals                0.001559 -0.000972    0.001659  \n",
              "           Consumer Non-Cyclicals            0.000742 -0.000463    0.000790  \n",
              "           Dividend                         -0.000813  0.000507   -0.000866  \n",
              "...                                               ...       ...         ...  \n",
              "2026-03-31 Size                             -0.001118  0.000196    0.001740  \n",
              "           Technology                       -0.001451 -0.000566    0.001996  \n",
              "           Utilities                         0.016901 -0.001056   -0.002930  \n",
              "           Value                            -0.001056  0.001118    0.000716  \n",
              "           Volatility                       -0.002930  0.000716    0.023099  \n",
              "\n",
              "[5271 rows x 21 columns]"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# generate the report data\n",
        "report = report_engine.calculate(start_date=None, end_date=None)\n",
        "\n",
        "# get the covariance matrix (and skip the first date)\n",
        "df_report = report.get_covariance().filter(pl.col(\"date\") > pl.col(\"date\").min())\n",
        "\n",
        "# convert to pandas\n",
        "df_vcov = df_report.to_pandas().set_index([\"date\", \"factor\"]).rename(columns=lambda c: c.split(\"^\")[0])\n",
        "df_vcov"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "58201166",
      "metadata": {},
      "source": [
        "For downstream comparisons, we split these into factor volatilities and correlations."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "id": "1789e656",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.187684</td>\n",
              "      <td>0.135533</td>\n",
              "      <td>0.084404</td>\n",
              "      <td>0.101847</td>\n",
              "      <td>0.026099</td>\n",
              "      <td>0.171107</td>\n",
              "      <td>0.059623</td>\n",
              "      <td>0.558476</td>\n",
              "      <td>0.016555</td>\n",
              "      <td>0.111548</td>\n",
              "      <td>...</td>\n",
              "      <td>0.219636</td>\n",
              "      <td>0.029571</td>\n",
              "      <td>0.119714</td>\n",
              "      <td>0.067182</td>\n",
              "      <td>0.088935</td>\n",
              "      <td>0.024066</td>\n",
              "      <td>0.057046</td>\n",
              "      <td>0.129037</td>\n",
              "      <td>0.034140</td>\n",
              "      <td>0.142146</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>0.186673</td>\n",
              "      <td>0.135342</td>\n",
              "      <td>0.083893</td>\n",
              "      <td>0.101263</td>\n",
              "      <td>0.025962</td>\n",
              "      <td>0.175770</td>\n",
              "      <td>0.059410</td>\n",
              "      <td>0.562395</td>\n",
              "      <td>0.016781</td>\n",
              "      <td>0.113629</td>\n",
              "      <td>...</td>\n",
              "      <td>0.227715</td>\n",
              "      <td>0.029849</td>\n",
              "      <td>0.119927</td>\n",
              "      <td>0.068960</td>\n",
              "      <td>0.088547</td>\n",
              "      <td>0.024209</td>\n",
              "      <td>0.057490</td>\n",
              "      <td>0.128775</td>\n",
              "      <td>0.033951</td>\n",
              "      <td>0.144664</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>0.185756</td>\n",
              "      <td>0.137121</td>\n",
              "      <td>0.083399</td>\n",
              "      <td>0.103873</td>\n",
              "      <td>0.025808</td>\n",
              "      <td>0.178650</td>\n",
              "      <td>0.059235</td>\n",
              "      <td>0.560397</td>\n",
              "      <td>0.016834</td>\n",
              "      <td>0.113983</td>\n",
              "      <td>...</td>\n",
              "      <td>0.226444</td>\n",
              "      <td>0.029669</td>\n",
              "      <td>0.121502</td>\n",
              "      <td>0.069202</td>\n",
              "      <td>0.088067</td>\n",
              "      <td>0.024372</td>\n",
              "      <td>0.057378</td>\n",
              "      <td>0.129958</td>\n",
              "      <td>0.033746</td>\n",
              "      <td>0.144584</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>0.184752</td>\n",
              "      <td>0.137265</td>\n",
              "      <td>0.083120</td>\n",
              "      <td>0.103240</td>\n",
              "      <td>0.025731</td>\n",
              "      <td>0.178317</td>\n",
              "      <td>0.059636</td>\n",
              "      <td>0.563394</td>\n",
              "      <td>0.016762</td>\n",
              "      <td>0.113979</td>\n",
              "      <td>...</td>\n",
              "      <td>0.229290</td>\n",
              "      <td>0.029593</td>\n",
              "      <td>0.121534</td>\n",
              "      <td>0.071026</td>\n",
              "      <td>0.087889</td>\n",
              "      <td>0.024379</td>\n",
              "      <td>0.057571</td>\n",
              "      <td>0.129206</td>\n",
              "      <td>0.033583</td>\n",
              "      <td>0.146394</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.184494</td>\n",
              "      <td>0.136686</td>\n",
              "      <td>0.082649</td>\n",
              "      <td>0.104374</td>\n",
              "      <td>0.025733</td>\n",
              "      <td>0.186215</td>\n",
              "      <td>0.060048</td>\n",
              "      <td>0.572618</td>\n",
              "      <td>0.016700</td>\n",
              "      <td>0.113353</td>\n",
              "      <td>...</td>\n",
              "      <td>0.236841</td>\n",
              "      <td>0.029946</td>\n",
              "      <td>0.124351</td>\n",
              "      <td>0.071614</td>\n",
              "      <td>0.087416</td>\n",
              "      <td>0.025741</td>\n",
              "      <td>0.057403</td>\n",
              "      <td>0.130003</td>\n",
              "      <td>0.033433</td>\n",
              "      <td>0.151984</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "            Academic & Educational Services  Basic Materials  \\\n",
              "date                                                           \n",
              "2026-03-25                         0.187684         0.135533   \n",
              "2026-03-26                         0.186673         0.135342   \n",
              "2026-03-27                         0.185756         0.137121   \n",
              "2026-03-30                         0.184752         0.137265   \n",
              "2026-03-31                         0.184494         0.136686   \n",
              "\n",
              "            Consumer Cyclicals  Consumer Non-Cyclicals  Dividend    Energy  \\\n",
              "date                                                                         \n",
              "2026-03-25            0.084404                0.101847  0.026099  0.171107   \n",
              "2026-03-26            0.083893                0.101263  0.025962  0.175770   \n",
              "2026-03-27            0.083399                0.103873  0.025808  0.178650   \n",
              "2026-03-30            0.083120                0.103240  0.025731  0.178317   \n",
              "2026-03-31            0.082649                0.104374  0.025733  0.186215   \n",
              "\n",
              "            Financials  Government Activity    Growth  Healthcare  ...  \\\n",
              "date                                                               ...   \n",
              "2026-03-25    0.059623             0.558476  0.016555    0.111548  ...   \n",
              "2026-03-26    0.059410             0.562395  0.016781    0.113629  ...   \n",
              "2026-03-27    0.059235             0.560397  0.016834    0.113983  ...   \n",
              "2026-03-30    0.059636             0.563394  0.016762    0.113979  ...   \n",
              "2026-03-31    0.060048             0.572618  0.016700    0.113353  ...   \n",
              "\n",
              "            Institutions, Associations & Organizations  Leverage    Market  \\\n",
              "date                                                                         \n",
              "2026-03-25                                    0.219636  0.029571  0.119714   \n",
              "2026-03-26                                    0.227715  0.029849  0.119927   \n",
              "2026-03-27                                    0.226444  0.029669  0.121502   \n",
              "2026-03-30                                    0.229290  0.029593  0.121534   \n",
              "2026-03-31                                    0.236841  0.029946  0.124351   \n",
              "\n",
              "            Momentum  Real Estate      Size  Technology  Utilities     Value  \\\n",
              "date                                                                           \n",
              "2026-03-25  0.067182     0.088935  0.024066    0.057046   0.129037  0.034140   \n",
              "2026-03-26  0.068960     0.088547  0.024209    0.057490   0.128775  0.033951   \n",
              "2026-03-27  0.069202     0.088067  0.024372    0.057378   0.129958  0.033746   \n",
              "2026-03-30  0.071026     0.087889  0.024379    0.057571   0.129206  0.033583   \n",
              "2026-03-31  0.071614     0.087416  0.025741    0.057403   0.130003  0.033433   \n",
              "\n",
              "            Volatility  \n",
              "date                    \n",
              "2026-03-25    0.142146  \n",
              "2026-03-26    0.144664  \n",
              "2026-03-27    0.144584  \n",
              "2026-03-30    0.146394  \n",
              "2026-03-31    0.151984  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate actual factor volatilities from vcovs\n",
        "df_vol = df_vcov.groupby(level=\"date\").apply(\n",
        "    lambda df: pd.Series(np.diag(df) ** 0.5, df.index.droplevel(\"date\"))\n",
        ")\n",
        "df_vol.columns.name = None\n",
        "df_vol.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "id": "5ec74b6e",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.187684</td>\n",
              "      <td>0.135533</td>\n",
              "      <td>0.084404</td>\n",
              "      <td>0.101847</td>\n",
              "      <td>0.026099</td>\n",
              "      <td>0.171107</td>\n",
              "      <td>0.059623</td>\n",
              "      <td>0.558476</td>\n",
              "      <td>0.016555</td>\n",
              "      <td>0.111548</td>\n",
              "      <td>...</td>\n",
              "      <td>0.219636</td>\n",
              "      <td>0.029571</td>\n",
              "      <td>0.119714</td>\n",
              "      <td>0.067182</td>\n",
              "      <td>0.088935</td>\n",
              "      <td>0.024066</td>\n",
              "      <td>0.057046</td>\n",
              "      <td>0.129037</td>\n",
              "      <td>0.034140</td>\n",
              "      <td>0.142146</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>0.186673</td>\n",
              "      <td>0.135342</td>\n",
              "      <td>0.083893</td>\n",
              "      <td>0.101263</td>\n",
              "      <td>0.025962</td>\n",
              "      <td>0.175770</td>\n",
              "      <td>0.059410</td>\n",
              "      <td>0.562395</td>\n",
              "      <td>0.016781</td>\n",
              "      <td>0.113629</td>\n",
              "      <td>...</td>\n",
              "      <td>0.227715</td>\n",
              "      <td>0.029849</td>\n",
              "      <td>0.119927</td>\n",
              "      <td>0.068960</td>\n",
              "      <td>0.088547</td>\n",
              "      <td>0.024209</td>\n",
              "      <td>0.057490</td>\n",
              "      <td>0.128775</td>\n",
              "      <td>0.033951</td>\n",
              "      <td>0.144664</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>0.185756</td>\n",
              "      <td>0.137121</td>\n",
              "      <td>0.083399</td>\n",
              "      <td>0.103873</td>\n",
              "      <td>0.025808</td>\n",
              "      <td>0.178650</td>\n",
              "      <td>0.059235</td>\n",
              "      <td>0.560397</td>\n",
              "      <td>0.016834</td>\n",
              "      <td>0.113983</td>\n",
              "      <td>...</td>\n",
              "      <td>0.226444</td>\n",
              "      <td>0.029669</td>\n",
              "      <td>0.121502</td>\n",
              "      <td>0.069202</td>\n",
              "      <td>0.088067</td>\n",
              "      <td>0.024372</td>\n",
              "      <td>0.057378</td>\n",
              "      <td>0.129958</td>\n",
              "      <td>0.033746</td>\n",
              "      <td>0.144584</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>0.184752</td>\n",
              "      <td>0.137265</td>\n",
              "      <td>0.083120</td>\n",
              "      <td>0.103240</td>\n",
              "      <td>0.025731</td>\n",
              "      <td>0.178317</td>\n",
              "      <td>0.059636</td>\n",
              "      <td>0.563394</td>\n",
              "      <td>0.016762</td>\n",
              "      <td>0.113979</td>\n",
              "      <td>...</td>\n",
              "      <td>0.229290</td>\n",
              "      <td>0.029593</td>\n",
              "      <td>0.121534</td>\n",
              "      <td>0.071026</td>\n",
              "      <td>0.087889</td>\n",
              "      <td>0.024379</td>\n",
              "      <td>0.057571</td>\n",
              "      <td>0.129206</td>\n",
              "      <td>0.033583</td>\n",
              "      <td>0.146394</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.184494</td>\n",
              "      <td>0.136686</td>\n",
              "      <td>0.082649</td>\n",
              "      <td>0.104374</td>\n",
              "      <td>0.025733</td>\n",
              "      <td>0.186215</td>\n",
              "      <td>0.060048</td>\n",
              "      <td>0.572618</td>\n",
              "      <td>0.016700</td>\n",
              "      <td>0.113353</td>\n",
              "      <td>...</td>\n",
              "      <td>0.236841</td>\n",
              "      <td>0.029946</td>\n",
              "      <td>0.124351</td>\n",
              "      <td>0.071614</td>\n",
              "      <td>0.087416</td>\n",
              "      <td>0.025741</td>\n",
              "      <td>0.057403</td>\n",
              "      <td>0.130003</td>\n",
              "      <td>0.033433</td>\n",
              "      <td>0.151984</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "            Academic & Educational Services  Basic Materials  \\\n",
              "date                                                           \n",
              "2026-03-25                         0.187684         0.135533   \n",
              "2026-03-26                         0.186673         0.135342   \n",
              "2026-03-27                         0.185756         0.137121   \n",
              "2026-03-30                         0.184752         0.137265   \n",
              "2026-03-31                         0.184494         0.136686   \n",
              "\n",
              "            Consumer Cyclicals  Consumer Non-Cyclicals  Dividend    Energy  \\\n",
              "date                                                                         \n",
              "2026-03-25            0.084404                0.101847  0.026099  0.171107   \n",
              "2026-03-26            0.083893                0.101263  0.025962  0.175770   \n",
              "2026-03-27            0.083399                0.103873  0.025808  0.178650   \n",
              "2026-03-30            0.083120                0.103240  0.025731  0.178317   \n",
              "2026-03-31            0.082649                0.104374  0.025733  0.186215   \n",
              "\n",
              "            Financials  Government Activity    Growth  Healthcare  ...  \\\n",
              "date                                                               ...   \n",
              "2026-03-25    0.059623             0.558476  0.016555    0.111548  ...   \n",
              "2026-03-26    0.059410             0.562395  0.016781    0.113629  ...   \n",
              "2026-03-27    0.059235             0.560397  0.016834    0.113983  ...   \n",
              "2026-03-30    0.059636             0.563394  0.016762    0.113979  ...   \n",
              "2026-03-31    0.060048             0.572618  0.016700    0.113353  ...   \n",
              "\n",
              "            Institutions, Associations & Organizations  Leverage    Market  \\\n",
              "date                                                                         \n",
              "2026-03-25                                    0.219636  0.029571  0.119714   \n",
              "2026-03-26                                    0.227715  0.029849  0.119927   \n",
              "2026-03-27                                    0.226444  0.029669  0.121502   \n",
              "2026-03-30                                    0.229290  0.029593  0.121534   \n",
              "2026-03-31                                    0.236841  0.029946  0.124351   \n",
              "\n",
              "            Momentum  Real Estate      Size  Technology  Utilities     Value  \\\n",
              "date                                                                           \n",
              "2026-03-25  0.067182     0.088935  0.024066    0.057046   0.129037  0.034140   \n",
              "2026-03-26  0.068960     0.088547  0.024209    0.057490   0.128775  0.033951   \n",
              "2026-03-27  0.069202     0.088067  0.024372    0.057378   0.129958  0.033746   \n",
              "2026-03-30  0.071026     0.087889  0.024379    0.057571   0.129206  0.033583   \n",
              "2026-03-31  0.071614     0.087416  0.025741    0.057403   0.130003  0.033433   \n",
              "\n",
              "            Volatility  \n",
              "date                    \n",
              "2026-03-25    0.142146  \n",
              "2026-03-26    0.144664  \n",
              "2026-03-27    0.144584  \n",
              "2026-03-30    0.146394  \n",
              "2026-03-31    0.151984  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate actual factor volatilities from vcovs\n",
        "df_vol = df_vcov.groupby(level=\"date\").apply(\n",
        "    lambda df: pd.Series(np.diag(df) ** 0.5, df.index.droplevel(\"date\"))\n",
        ")\n",
        "df_vol.columns.name = None\n",
        "df_vol.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "id": "68d3f412",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>...</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Market</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Size</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2026-03-31</th>\n",
              "      <th>Size</th>\n",
              "      <td>-0.085841</td>\n",
              "      <td>-0.002684</td>\n",
              "      <td>0.052847</td>\n",
              "      <td>-0.403941</td>\n",
              "      <td>-0.111629</td>\n",
              "      <td>-0.071182</td>\n",
              "      <td>0.472602</td>\n",
              "      <td>-0.165917</td>\n",
              "      <td>0.151280</td>\n",
              "      <td>-0.196416</td>\n",
              "      <td>...</td>\n",
              "      <td>0.574063</td>\n",
              "      <td>-0.030011</td>\n",
              "      <td>0.442030</td>\n",
              "      <td>0.006216</td>\n",
              "      <td>-0.142327</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.021525</td>\n",
              "      <td>-0.334196</td>\n",
              "      <td>0.227721</td>\n",
              "      <td>0.444806</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Technology</th>\n",
              "      <td>-0.145151</td>\n",
              "      <td>-0.515929</td>\n",
              "      <td>-0.514876</td>\n",
              "      <td>-0.571166</td>\n",
              "      <td>-0.132626</td>\n",
              "      <td>-0.256960</td>\n",
              "      <td>-0.224223</td>\n",
              "      <td>-0.082265</td>\n",
              "      <td>0.128251</td>\n",
              "      <td>-0.395049</td>\n",
              "      <td>...</td>\n",
              "      <td>0.136438</td>\n",
              "      <td>-0.470045</td>\n",
              "      <td>0.162046</td>\n",
              "      <td>0.162310</td>\n",
              "      <td>-0.411016</td>\n",
              "      <td>-0.021525</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.194446</td>\n",
              "      <td>-0.294997</td>\n",
              "      <td>0.228783</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Utilities</th>\n",
              "      <td>-0.121029</td>\n",
              "      <td>0.200927</td>\n",
              "      <td>-0.259636</td>\n",
              "      <td>0.442244</td>\n",
              "      <td>0.077573</td>\n",
              "      <td>0.219515</td>\n",
              "      <td>-0.202845</td>\n",
              "      <td>0.138821</td>\n",
              "      <td>-0.136157</td>\n",
              "      <td>0.044973</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.046746</td>\n",
              "      <td>-0.249991</td>\n",
              "      <td>-0.301050</td>\n",
              "      <td>0.371753</td>\n",
              "      <td>0.472253</td>\n",
              "      <td>-0.334196</td>\n",
              "      <td>-0.194446</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.242867</td>\n",
              "      <td>-0.148294</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Value</th>\n",
              "      <td>0.016262</td>\n",
              "      <td>0.298450</td>\n",
              "      <td>0.434840</td>\n",
              "      <td>0.025358</td>\n",
              "      <td>0.179505</td>\n",
              "      <td>0.173794</td>\n",
              "      <td>-0.065180</td>\n",
              "      <td>0.021673</td>\n",
              "      <td>0.154796</td>\n",
              "      <td>0.040146</td>\n",
              "      <td>...</td>\n",
              "      <td>0.286446</td>\n",
              "      <td>0.364942</td>\n",
              "      <td>0.315728</td>\n",
              "      <td>-0.137116</td>\n",
              "      <td>-0.122327</td>\n",
              "      <td>0.227721</td>\n",
              "      <td>-0.294997</td>\n",
              "      <td>-0.242867</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.140896</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Volatility</th>\n",
              "      <td>-0.301098</td>\n",
              "      <td>-0.118324</td>\n",
              "      <td>-0.225568</td>\n",
              "      <td>-0.425596</td>\n",
              "      <td>-0.005789</td>\n",
              "      <td>-0.208490</td>\n",
              "      <td>0.451712</td>\n",
              "      <td>-0.421851</td>\n",
              "      <td>-0.040011</td>\n",
              "      <td>-0.418486</td>\n",
              "      <td>...</td>\n",
              "      <td>0.781808</td>\n",
              "      <td>-0.361023</td>\n",
              "      <td>0.787197</td>\n",
              "      <td>0.332704</td>\n",
              "      <td>-0.155426</td>\n",
              "      <td>0.444806</td>\n",
              "      <td>0.228783</td>\n",
              "      <td>-0.148294</td>\n",
              "      <td>0.140896</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                       Academic & Educational Services  Basic Materials  \\\n",
              "date       factor                                                         \n",
              "2026-03-31 Size                              -0.085841        -0.002684   \n",
              "           Technology                        -0.145151        -0.515929   \n",
              "           Utilities                         -0.121029         0.200927   \n",
              "           Value                              0.016262         0.298450   \n",
              "           Volatility                        -0.301098        -0.118324   \n",
              "\n",
              "                       Consumer Cyclicals  Consumer Non-Cyclicals  Dividend  \\\n",
              "date       factor                                                             \n",
              "2026-03-31 Size                  0.052847               -0.403941 -0.111629   \n",
              "           Technology           -0.514876               -0.571166 -0.132626   \n",
              "           Utilities            -0.259636                0.442244  0.077573   \n",
              "           Value                 0.434840                0.025358  0.179505   \n",
              "           Volatility           -0.225568               -0.425596 -0.005789   \n",
              "\n",
              "                         Energy  Financials  Government Activity    Growth  \\\n",
              "date       factor                                                            \n",
              "2026-03-31 Size       -0.071182    0.472602            -0.165917  0.151280   \n",
              "           Technology -0.256960   -0.224223            -0.082265  0.128251   \n",
              "           Utilities   0.219515   -0.202845             0.138821 -0.136157   \n",
              "           Value       0.173794   -0.065180             0.021673  0.154796   \n",
              "           Volatility -0.208490    0.451712            -0.421851 -0.040011   \n",
              "\n",
              "                       Healthcare  ...  \\\n",
              "date       factor                  ...   \n",
              "2026-03-31 Size         -0.196416  ...   \n",
              "           Technology   -0.395049  ...   \n",
              "           Utilities     0.044973  ...   \n",
              "           Value         0.040146  ...   \n",
              "           Volatility   -0.418486  ...   \n",
              "\n",
              "                       Institutions, Associations & Organizations  Leverage  \\\n",
              "date       factor                                                             \n",
              "2026-03-31 Size                                          0.574063 -0.030011   \n",
              "           Technology                                    0.136438 -0.470045   \n",
              "           Utilities                                    -0.046746 -0.249991   \n",
              "           Value                                         0.286446  0.364942   \n",
              "           Volatility                                    0.781808 -0.361023   \n",
              "\n",
              "                         Market  Momentum  Real Estate      Size  Technology  \\\n",
              "date       factor                                                              \n",
              "2026-03-31 Size        0.442030  0.006216    -0.142327  1.000000   -0.021525   \n",
              "           Technology  0.162046  0.162310    -0.411016 -0.021525    1.000000   \n",
              "           Utilities  -0.301050  0.371753     0.472253 -0.334196   -0.194446   \n",
              "           Value       0.315728 -0.137116    -0.122327  0.227721   -0.294997   \n",
              "           Volatility  0.787197  0.332704    -0.155426  0.444806    0.228783   \n",
              "\n",
              "                       Utilities     Value  Volatility  \n",
              "date       factor                                       \n",
              "2026-03-31 Size        -0.334196  0.227721    0.444806  \n",
              "           Technology  -0.194446 -0.294997    0.228783  \n",
              "           Utilities    1.000000 -0.242867   -0.148294  \n",
              "           Value       -0.242867  1.000000    0.140896  \n",
              "           Volatility  -0.148294  0.140896    1.000000  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate actual factor correlations from vcovs\n",
        "df_cor = df_vcov.groupby(level=\"date\").apply(\n",
        "    lambda df: df.droplevel(\"date\")\n",
        "    / np.outer(np.diag(df) ** 0.5, np.diag(df) ** 0.5)\n",
        ")\n",
        "df_cor.tail()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "df0e2392",
      "metadata": {},
      "source": [
        "## Manually replicating the covariance forecasts\n",
        "\n",
        "We can also estimated the risk model and get the returns directly. From these returns we can construct the covariance forecasts. Bayesline returns dataframes in `polars`, but they can be easily converted to `pandas` dataframes. We also remove the factor group (market, style, industry, etc.) for convenience."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "id": "773df388",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>...</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.005545</td>\n",
              "      <td>0.000347</td>\n",
              "      <td>-0.002265</td>\n",
              "      <td>-0.001562</td>\n",
              "      <td>0.002639</td>\n",
              "      <td>0.001209</td>\n",
              "      <td>0.000531</td>\n",
              "      <td>0.005782</td>\n",
              "      <td>0.013582</td>\n",
              "      <td>0.010119</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003309</td>\n",
              "      <td>-0.007811</td>\n",
              "      <td>-0.000147</td>\n",
              "      <td>-0.070527</td>\n",
              "      <td>0.007364</td>\n",
              "      <td>0.000178</td>\n",
              "      <td>0.004781</td>\n",
              "      <td>-0.003222</td>\n",
              "      <td>-0.001333</td>\n",
              "      <td>-0.001449</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>-0.008572</td>\n",
              "      <td>0.000614</td>\n",
              "      <td>-0.001881</td>\n",
              "      <td>0.002976</td>\n",
              "      <td>-0.009838</td>\n",
              "      <td>-0.002133</td>\n",
              "      <td>0.000665</td>\n",
              "      <td>-0.017761</td>\n",
              "      <td>0.004051</td>\n",
              "      <td>0.007486</td>\n",
              "      <td>...</td>\n",
              "      <td>0.001583</td>\n",
              "      <td>0.025360</td>\n",
              "      <td>-0.002418</td>\n",
              "      <td>0.051660</td>\n",
              "      <td>0.014214</td>\n",
              "      <td>-0.003017</td>\n",
              "      <td>-0.036999</td>\n",
              "      <td>0.002994</td>\n",
              "      <td>-0.005429</td>\n",
              "      <td>0.006636</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>-0.013455</td>\n",
              "      <td>-0.000276</td>\n",
              "      <td>0.001305</td>\n",
              "      <td>-0.000166</td>\n",
              "      <td>0.005455</td>\n",
              "      <td>-0.002215</td>\n",
              "      <td>0.000204</td>\n",
              "      <td>-0.008686</td>\n",
              "      <td>0.005188</td>\n",
              "      <td>0.015191</td>\n",
              "      <td>...</td>\n",
              "      <td>0.014673</td>\n",
              "      <td>0.021339</td>\n",
              "      <td>-0.002691</td>\n",
              "      <td>0.022885</td>\n",
              "      <td>-0.008802</td>\n",
              "      <td>0.000996</td>\n",
              "      <td>0.004197</td>\n",
              "      <td>0.001861</td>\n",
              "      <td>-0.002984</td>\n",
              "      <td>0.012870</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>-0.007819</td>\n",
              "      <td>0.001160</td>\n",
              "      <td>-0.000572</td>\n",
              "      <td>0.001426</td>\n",
              "      <td>-0.010121</td>\n",
              "      <td>0.001568</td>\n",
              "      <td>-0.000970</td>\n",
              "      <td>-0.015963</td>\n",
              "      <td>0.003955</td>\n",
              "      <td>0.009354</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000108</td>\n",
              "      <td>-0.009378</td>\n",
              "      <td>0.005429</td>\n",
              "      <td>0.048425</td>\n",
              "      <td>0.007160</td>\n",
              "      <td>-0.004746</td>\n",
              "      <td>-0.025031</td>\n",
              "      <td>0.004538</td>\n",
              "      <td>-0.004506</td>\n",
              "      <td>0.001859</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.016875</td>\n",
              "      <td>0.001635</td>\n",
              "      <td>-0.000666</td>\n",
              "      <td>-0.003213</td>\n",
              "      <td>0.006885</td>\n",
              "      <td>0.004965</td>\n",
              "      <td>-0.001093</td>\n",
              "      <td>0.025092</td>\n",
              "      <td>-0.010212</td>\n",
              "      <td>-0.004790</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.010916</td>\n",
              "      <td>-0.032651</td>\n",
              "      <td>0.005498</td>\n",
              "      <td>-0.068412</td>\n",
              "      <td>0.002255</td>\n",
              "      <td>0.000748</td>\n",
              "      <td>0.036847</td>\n",
              "      <td>-0.001887</td>\n",
              "      <td>0.002617</td>\n",
              "      <td>-0.011560</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "              Market  Dividend    Growth  Leverage  Momentum      Size  \\\n",
              "date                                                                     \n",
              "2026-03-25  0.005545  0.000347 -0.002265 -0.001562  0.002639  0.001209   \n",
              "2026-03-26 -0.008572  0.000614 -0.001881  0.002976 -0.009838 -0.002133   \n",
              "2026-03-27 -0.013455 -0.000276  0.001305 -0.000166  0.005455 -0.002215   \n",
              "2026-03-30 -0.007819  0.001160 -0.000572  0.001426 -0.010121  0.001568   \n",
              "2026-03-31  0.016875  0.001635 -0.000666 -0.003213  0.006885  0.004965   \n",
              "\n",
              "               Value  Volatility  Academic & Educational Services  \\\n",
              "date                                                                \n",
              "2026-03-25  0.000531    0.005782                         0.013582   \n",
              "2026-03-26  0.000665   -0.017761                         0.004051   \n",
              "2026-03-27  0.000204   -0.008686                         0.005188   \n",
              "2026-03-30 -0.000970   -0.015963                         0.003955   \n",
              "2026-03-31 -0.001093    0.025092                        -0.010212   \n",
              "\n",
              "            Basic Materials  ...  Consumer Non-Cyclicals    Energy  \\\n",
              "date                         ...                                     \n",
              "2026-03-25         0.010119  ...                0.003309 -0.007811   \n",
              "2026-03-26         0.007486  ...                0.001583  0.025360   \n",
              "2026-03-27         0.015191  ...                0.014673  0.021339   \n",
              "2026-03-30         0.009354  ...                0.000108 -0.009378   \n",
              "2026-03-31        -0.004790  ...               -0.010916 -0.032651   \n",
              "\n",
              "            Financials  Government Activity  Healthcare  Industrials  \\\n",
              "date                                                                   \n",
              "2026-03-25   -0.000147            -0.070527    0.007364     0.000178   \n",
              "2026-03-26   -0.002418             0.051660    0.014214    -0.003017   \n",
              "2026-03-27   -0.002691             0.022885   -0.008802     0.000996   \n",
              "2026-03-30    0.005429             0.048425    0.007160    -0.004746   \n",
              "2026-03-31    0.005498            -0.068412    0.002255     0.000748   \n",
              "\n",
              "            Institutions, Associations & Organizations  Real Estate  \\\n",
              "date                                                                  \n",
              "2026-03-25                                    0.004781    -0.003222   \n",
              "2026-03-26                                   -0.036999     0.002994   \n",
              "2026-03-27                                    0.004197     0.001861   \n",
              "2026-03-30                                   -0.025031     0.004538   \n",
              "2026-03-31                                    0.036847    -0.001887   \n",
              "\n",
              "            Technology  Utilities  \n",
              "date                               \n",
              "2026-03-25   -0.001333  -0.001449  \n",
              "2026-03-26   -0.005429   0.006636  \n",
              "2026-03-27   -0.002984   0.012870  \n",
              "2026-03-30   -0.004506   0.001859  \n",
              "2026-03-31    0.002617  -0.011560  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "risk_model = bln.equity.riskmodels.load(\n",
        "    factorriskmodel_settings.with_dataset(\"bayesline/Bayesline-US-All-1y\")\n",
        ").get_model()\n",
        "df_factor_returns = risk_model.fret().tail(-1).to_pandas().set_index(\"date\").rename(columns=lambda c: c.split(\".\")[1])\n",
        "df_factor_returns.tail()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "aefad075",
      "metadata": {},
      "source": [
        "From these returns we can run standard pandas functions to get the EWMAs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "id": "b6d9de92",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>...</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2026-03-25</th>\n",
              "      <td>0.119714</td>\n",
              "      <td>0.026099</td>\n",
              "      <td>0.016555</td>\n",
              "      <td>0.029571</td>\n",
              "      <td>0.067182</td>\n",
              "      <td>0.024066</td>\n",
              "      <td>0.034140</td>\n",
              "      <td>0.142146</td>\n",
              "      <td>0.187684</td>\n",
              "      <td>0.135533</td>\n",
              "      <td>...</td>\n",
              "      <td>0.101847</td>\n",
              "      <td>0.171107</td>\n",
              "      <td>0.059623</td>\n",
              "      <td>0.558476</td>\n",
              "      <td>0.111548</td>\n",
              "      <td>0.067597</td>\n",
              "      <td>0.219636</td>\n",
              "      <td>0.088935</td>\n",
              "      <td>0.057046</td>\n",
              "      <td>0.129037</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-26</th>\n",
              "      <td>0.119927</td>\n",
              "      <td>0.025962</td>\n",
              "      <td>0.016781</td>\n",
              "      <td>0.029849</td>\n",
              "      <td>0.068960</td>\n",
              "      <td>0.024209</td>\n",
              "      <td>0.033951</td>\n",
              "      <td>0.144664</td>\n",
              "      <td>0.186673</td>\n",
              "      <td>0.135342</td>\n",
              "      <td>...</td>\n",
              "      <td>0.101263</td>\n",
              "      <td>0.175770</td>\n",
              "      <td>0.059410</td>\n",
              "      <td>0.562395</td>\n",
              "      <td>0.113629</td>\n",
              "      <td>0.067391</td>\n",
              "      <td>0.227715</td>\n",
              "      <td>0.088547</td>\n",
              "      <td>0.057490</td>\n",
              "      <td>0.128775</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-27</th>\n",
              "      <td>0.121502</td>\n",
              "      <td>0.025808</td>\n",
              "      <td>0.016834</td>\n",
              "      <td>0.029669</td>\n",
              "      <td>0.069202</td>\n",
              "      <td>0.024372</td>\n",
              "      <td>0.033746</td>\n",
              "      <td>0.144584</td>\n",
              "      <td>0.185756</td>\n",
              "      <td>0.137121</td>\n",
              "      <td>...</td>\n",
              "      <td>0.103873</td>\n",
              "      <td>0.178649</td>\n",
              "      <td>0.059235</td>\n",
              "      <td>0.560397</td>\n",
              "      <td>0.113983</td>\n",
              "      <td>0.067003</td>\n",
              "      <td>0.226444</td>\n",
              "      <td>0.088067</td>\n",
              "      <td>0.057378</td>\n",
              "      <td>0.129958</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-30</th>\n",
              "      <td>0.121534</td>\n",
              "      <td>0.025731</td>\n",
              "      <td>0.016762</td>\n",
              "      <td>0.029593</td>\n",
              "      <td>0.071026</td>\n",
              "      <td>0.024379</td>\n",
              "      <td>0.033583</td>\n",
              "      <td>0.146394</td>\n",
              "      <td>0.184752</td>\n",
              "      <td>0.137265</td>\n",
              "      <td>...</td>\n",
              "      <td>0.103240</td>\n",
              "      <td>0.178317</td>\n",
              "      <td>0.059636</td>\n",
              "      <td>0.563394</td>\n",
              "      <td>0.113979</td>\n",
              "      <td>0.067110</td>\n",
              "      <td>0.229290</td>\n",
              "      <td>0.087889</td>\n",
              "      <td>0.057571</td>\n",
              "      <td>0.129206</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2026-03-31</th>\n",
              "      <td>0.124351</td>\n",
              "      <td>0.025733</td>\n",
              "      <td>0.016700</td>\n",
              "      <td>0.029946</td>\n",
              "      <td>0.071614</td>\n",
              "      <td>0.025741</td>\n",
              "      <td>0.033433</td>\n",
              "      <td>0.151984</td>\n",
              "      <td>0.184494</td>\n",
              "      <td>0.136686</td>\n",
              "      <td>...</td>\n",
              "      <td>0.104374</td>\n",
              "      <td>0.186215</td>\n",
              "      <td>0.060048</td>\n",
              "      <td>0.572618</td>\n",
              "      <td>0.113353</td>\n",
              "      <td>0.066714</td>\n",
              "      <td>0.236841</td>\n",
              "      <td>0.087416</td>\n",
              "      <td>0.057403</td>\n",
              "      <td>0.130003</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "              Market  Dividend    Growth  Leverage  Momentum      Size  \\\n",
              "date                                                                     \n",
              "2026-03-25  0.119714  0.026099  0.016555  0.029571  0.067182  0.024066   \n",
              "2026-03-26  0.119927  0.025962  0.016781  0.029849  0.068960  0.024209   \n",
              "2026-03-27  0.121502  0.025808  0.016834  0.029669  0.069202  0.024372   \n",
              "2026-03-30  0.121534  0.025731  0.016762  0.029593  0.071026  0.024379   \n",
              "2026-03-31  0.124351  0.025733  0.016700  0.029946  0.071614  0.025741   \n",
              "\n",
              "               Value  Volatility  Academic & Educational Services  \\\n",
              "date                                                                \n",
              "2026-03-25  0.034140    0.142146                         0.187684   \n",
              "2026-03-26  0.033951    0.144664                         0.186673   \n",
              "2026-03-27  0.033746    0.144584                         0.185756   \n",
              "2026-03-30  0.033583    0.146394                         0.184752   \n",
              "2026-03-31  0.033433    0.151984                         0.184494   \n",
              "\n",
              "            Basic Materials  ...  Consumer Non-Cyclicals    Energy  \\\n",
              "date                         ...                                     \n",
              "2026-03-25         0.135533  ...                0.101847  0.171107   \n",
              "2026-03-26         0.135342  ...                0.101263  0.175770   \n",
              "2026-03-27         0.137121  ...                0.103873  0.178649   \n",
              "2026-03-30         0.137265  ...                0.103240  0.178317   \n",
              "2026-03-31         0.136686  ...                0.104374  0.186215   \n",
              "\n",
              "            Financials  Government Activity  Healthcare  Industrials  \\\n",
              "date                                                                   \n",
              "2026-03-25    0.059623             0.558476    0.111548     0.067597   \n",
              "2026-03-26    0.059410             0.562395    0.113629     0.067391   \n",
              "2026-03-27    0.059235             0.560397    0.113983     0.067003   \n",
              "2026-03-30    0.059636             0.563394    0.113979     0.067110   \n",
              "2026-03-31    0.060048             0.572618    0.113353     0.066714   \n",
              "\n",
              "            Institutions, Associations & Organizations  Real Estate  \\\n",
              "date                                                                  \n",
              "2026-03-25                                    0.219636     0.088935   \n",
              "2026-03-26                                    0.227715     0.088547   \n",
              "2026-03-27                                    0.226444     0.088067   \n",
              "2026-03-30                                    0.229290     0.087889   \n",
              "2026-03-31                                    0.236841     0.087416   \n",
              "\n",
              "            Technology  Utilities  \n",
              "date                               \n",
              "2026-03-25    0.057046   0.129037  \n",
              "2026-03-26    0.057490   0.128775  \n",
              "2026-03-27    0.057378   0.129958  \n",
              "2026-03-30    0.057571   0.129206  \n",
              "2026-03-31    0.057403   0.130003  \n",
              "\n",
              "[5 rows x 21 columns]"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate expected factor volatilities using the ewma\n",
        "df_vol_tieout = (\n",
        "    pd.DataFrame(df_factor_returns**2)\n",
        "    .ewm(halflife=report_settings.factor_cov_settings.halflife_vol)\n",
        "    .mean()\n",
        "    .astype(np.float32)\n",
        "    ** 0.5\n",
        "    * 252**0.5\n",
        ")\n",
        "df_vol_tieout.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "id": "098e5e07",
      "metadata": {},
      "outputs": [],
      "source": [
        "pd.testing.assert_frame_equal(df_vol, df_vol_tieout, check_column_type=False, check_categorical=False, check_like=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a4e34c46",
      "metadata": {},
      "source": [
        "The correlation are a bit more involved. We need to create the outer products and then run EWMAs on each cell in the outer product matrix."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "id": "e692650b",
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Leverage</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Academic &amp; Educational Services</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>...</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Government Activity</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <th>Real Estate</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2025-04-01</th>\n",
              "      <th>Market</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Dividend</th>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Growth</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Leverage</th>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Momentum</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2026-03-31</th>\n",
              "      <th>Industrials</th>\n",
              "      <td>0.226267</td>\n",
              "      <td>-0.025949</td>\n",
              "      <td>0.042911</td>\n",
              "      <td>0.205076</td>\n",
              "      <td>0.119979</td>\n",
              "      <td>0.169108</td>\n",
              "      <td>0.317757</td>\n",
              "      <td>0.185423</td>\n",
              "      <td>0.044676</td>\n",
              "      <td>0.375895</td>\n",
              "      <td>...</td>\n",
              "      <td>0.114218</td>\n",
              "      <td>0.069578</td>\n",
              "      <td>0.114124</td>\n",
              "      <td>-0.058272</td>\n",
              "      <td>-0.078380</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.210333</td>\n",
              "      <td>0.056403</td>\n",
              "      <td>-0.482192</td>\n",
              "      <td>0.034718</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Institutions, Associations &amp; Organizations</th>\n",
              "      <td>0.413368</td>\n",
              "      <td>-0.009360</td>\n",
              "      <td>0.261116</td>\n",
              "      <td>-0.340772</td>\n",
              "      <td>0.606009</td>\n",
              "      <td>0.574063</td>\n",
              "      <td>0.286446</td>\n",
              "      <td>0.781808</td>\n",
              "      <td>-0.292448</td>\n",
              "      <td>-0.006832</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.255683</td>\n",
              "      <td>-0.004063</td>\n",
              "      <td>0.344633</td>\n",
              "      <td>-0.243807</td>\n",
              "      <td>-0.393242</td>\n",
              "      <td>0.210333</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.175821</td>\n",
              "      <td>0.136438</td>\n",
              "      <td>-0.046746</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Real Estate</th>\n",
              "      <td>-0.111979</td>\n",
              "      <td>0.101780</td>\n",
              "      <td>-0.076714</td>\n",
              "      <td>0.050771</td>\n",
              "      <td>0.009966</td>\n",
              "      <td>-0.142327</td>\n",
              "      <td>-0.122327</td>\n",
              "      <td>-0.155426</td>\n",
              "      <td>-0.015587</td>\n",
              "      <td>0.171827</td>\n",
              "      <td>...</td>\n",
              "      <td>0.300349</td>\n",
              "      <td>0.081663</td>\n",
              "      <td>0.033409</td>\n",
              "      <td>0.047162</td>\n",
              "      <td>0.137231</td>\n",
              "      <td>0.056403</td>\n",
              "      <td>-0.175821</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.411016</td>\n",
              "      <td>0.472253</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Technology</th>\n",
              "      <td>0.162046</td>\n",
              "      <td>-0.132626</td>\n",
              "      <td>0.128251</td>\n",
              "      <td>-0.470045</td>\n",
              "      <td>0.162310</td>\n",
              "      <td>-0.021525</td>\n",
              "      <td>-0.294996</td>\n",
              "      <td>0.228783</td>\n",
              "      <td>-0.145151</td>\n",
              "      <td>-0.515929</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.571166</td>\n",
              "      <td>-0.256960</td>\n",
              "      <td>-0.224223</td>\n",
              "      <td>-0.082265</td>\n",
              "      <td>-0.395049</td>\n",
              "      <td>-0.482192</td>\n",
              "      <td>0.136438</td>\n",
              "      <td>-0.411016</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.194446</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Utilities</th>\n",
              "      <td>-0.301050</td>\n",
              "      <td>0.077573</td>\n",
              "      <td>-0.136157</td>\n",
              "      <td>-0.249991</td>\n",
              "      <td>0.371753</td>\n",
              "      <td>-0.334196</td>\n",
              "      <td>-0.242868</td>\n",
              "      <td>-0.148294</td>\n",
              "      <td>-0.121028</td>\n",
              "      <td>0.200927</td>\n",
              "      <td>...</td>\n",
              "      <td>0.442244</td>\n",
              "      <td>0.219515</td>\n",
              "      <td>-0.202845</td>\n",
              "      <td>0.138822</td>\n",
              "      <td>0.044974</td>\n",
              "      <td>0.034718</td>\n",
              "      <td>-0.046746</td>\n",
              "      <td>0.472253</td>\n",
              "      <td>-0.194446</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5271 rows × 21 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                         Market  Dividend  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Market                                      1.000000 -1.000000   \n",
              "           Dividend                                   -1.000000  1.000000   \n",
              "           Growth                                      1.000000 -1.000000   \n",
              "           Leverage                                   -1.000000  1.000000   \n",
              "           Momentum                                    1.000000 -1.000000   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Industrials                                 0.226267 -0.025949   \n",
              "           Institutions, Associations & Organizations  0.413368 -0.009360   \n",
              "           Real Estate                                -0.111979  0.101780   \n",
              "           Technology                                  0.162046 -0.132626   \n",
              "           Utilities                                  -0.301050  0.077573   \n",
              "\n",
              "                                                         Growth  Leverage  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Market                                      1.000000 -1.000000   \n",
              "           Dividend                                   -1.000000  1.000000   \n",
              "           Growth                                      1.000000 -1.000000   \n",
              "           Leverage                                   -1.000000  1.000000   \n",
              "           Momentum                                    1.000000 -1.000000   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Industrials                                 0.042911  0.205076   \n",
              "           Institutions, Associations & Organizations  0.261116 -0.340772   \n",
              "           Real Estate                                -0.076714  0.050771   \n",
              "           Technology                                  0.128251 -0.470045   \n",
              "           Utilities                                  -0.136157 -0.249991   \n",
              "\n",
              "                                                       Momentum      Size  \\\n",
              "date       factor                                                           \n",
              "2025-04-01 Market                                      1.000000  1.000000   \n",
              "           Dividend                                   -1.000000 -1.000000   \n",
              "           Growth                                      1.000000  1.000000   \n",
              "           Leverage                                   -1.000000 -1.000000   \n",
              "           Momentum                                    1.000000  1.000000   \n",
              "...                                                         ...       ...   \n",
              "2026-03-31 Industrials                                 0.119979  0.169108   \n",
              "           Institutions, Associations & Organizations  0.606009  0.574063   \n",
              "           Real Estate                                 0.009966 -0.142327   \n",
              "           Technology                                  0.162310 -0.021525   \n",
              "           Utilities                                   0.371753 -0.334196   \n",
              "\n",
              "                                                          Value  Volatility  \\\n",
              "date       factor                                                             \n",
              "2025-04-01 Market                                     -1.000000    1.000000   \n",
              "           Dividend                                    1.000000   -1.000000   \n",
              "           Growth                                     -1.000000    1.000000   \n",
              "           Leverage                                    1.000000   -1.000000   \n",
              "           Momentum                                   -1.000000    1.000000   \n",
              "...                                                         ...         ...   \n",
              "2026-03-31 Industrials                                 0.317757    0.185423   \n",
              "           Institutions, Associations & Organizations  0.286446    0.781808   \n",
              "           Real Estate                                -0.122327   -0.155426   \n",
              "           Technology                                 -0.294996    0.228783   \n",
              "           Utilities                                  -0.242868   -0.148294   \n",
              "\n",
              "                                                       Academic & Educational Services  \\\n",
              "date       factor                                                                        \n",
              "2025-04-01 Market                                                             1.000000   \n",
              "           Dividend                                                          -1.000000   \n",
              "           Growth                                                             1.000000   \n",
              "           Leverage                                                          -1.000000   \n",
              "           Momentum                                                           1.000000   \n",
              "...                                                                                ...   \n",
              "2026-03-31 Industrials                                                        0.044676   \n",
              "           Institutions, Associations & Organizations                        -0.292448   \n",
              "           Real Estate                                                       -0.015587   \n",
              "           Technology                                                        -0.145151   \n",
              "           Utilities                                                         -0.121028   \n",
              "\n",
              "                                                       Basic Materials  ...  \\\n",
              "date       factor                                                       ...   \n",
              "2025-04-01 Market                                            -1.000000  ...   \n",
              "           Dividend                                           1.000000  ...   \n",
              "           Growth                                            -1.000000  ...   \n",
              "           Leverage                                           1.000000  ...   \n",
              "           Momentum                                          -1.000000  ...   \n",
              "...                                                                ...  ...   \n",
              "2026-03-31 Industrials                                        0.375895  ...   \n",
              "           Institutions, Associations & Organizations        -0.006832  ...   \n",
              "           Real Estate                                        0.171827  ...   \n",
              "           Technology                                        -0.515929  ...   \n",
              "           Utilities                                          0.200927  ...   \n",
              "\n",
              "                                                       Consumer Non-Cyclicals  \\\n",
              "date       factor                                                               \n",
              "2025-04-01 Market                                                    1.000000   \n",
              "           Dividend                                                 -1.000000   \n",
              "           Growth                                                    1.000000   \n",
              "           Leverage                                                 -1.000000   \n",
              "           Momentum                                                  1.000000   \n",
              "...                                                                       ...   \n",
              "2026-03-31 Industrials                                               0.114218   \n",
              "           Institutions, Associations & Organizations               -0.255683   \n",
              "           Real Estate                                               0.300349   \n",
              "           Technology                                               -0.571166   \n",
              "           Utilities                                                 0.442244   \n",
              "\n",
              "                                                         Energy  Financials  \\\n",
              "date       factor                                                             \n",
              "2025-04-01 Market                                      1.000000    1.000000   \n",
              "           Dividend                                   -1.000000   -1.000000   \n",
              "           Growth                                      1.000000    1.000000   \n",
              "           Leverage                                   -1.000000   -1.000000   \n",
              "           Momentum                                    1.000000    1.000000   \n",
              "...                                                         ...         ...   \n",
              "2026-03-31 Industrials                                 0.069578    0.114124   \n",
              "           Institutions, Associations & Organizations -0.004063    0.344633   \n",
              "           Real Estate                                 0.081663    0.033409   \n",
              "           Technology                                 -0.256960   -0.224223   \n",
              "           Utilities                                   0.219515   -0.202845   \n",
              "\n",
              "                                                       Government Activity  \\\n",
              "date       factor                                                            \n",
              "2025-04-01 Market                                                -1.000000   \n",
              "           Dividend                                               1.000000   \n",
              "           Growth                                                -1.000000   \n",
              "           Leverage                                               1.000000   \n",
              "           Momentum                                              -1.000000   \n",
              "...                                                                    ...   \n",
              "2026-03-31 Industrials                                           -0.058272   \n",
              "           Institutions, Associations & Organizations            -0.243807   \n",
              "           Real Estate                                            0.047162   \n",
              "           Technology                                            -0.082265   \n",
              "           Utilities                                              0.138822   \n",
              "\n",
              "                                                       Healthcare  \\\n",
              "date       factor                                                   \n",
              "2025-04-01 Market                                       -1.000000   \n",
              "           Dividend                                      1.000000   \n",
              "           Growth                                       -1.000000   \n",
              "           Leverage                                      1.000000   \n",
              "           Momentum                                     -1.000000   \n",
              "...                                                           ...   \n",
              "2026-03-31 Industrials                                  -0.078380   \n",
              "           Institutions, Associations & Organizations   -0.393242   \n",
              "           Real Estate                                   0.137231   \n",
              "           Technology                                   -0.395049   \n",
              "           Utilities                                     0.044974   \n",
              "\n",
              "                                                       Industrials  \\\n",
              "date       factor                                                    \n",
              "2025-04-01 Market                                         1.000000   \n",
              "           Dividend                                      -1.000000   \n",
              "           Growth                                         1.000000   \n",
              "           Leverage                                      -1.000000   \n",
              "           Momentum                                       1.000000   \n",
              "...                                                            ...   \n",
              "2026-03-31 Industrials                                    1.000000   \n",
              "           Institutions, Associations & Organizations     0.210333   \n",
              "           Real Estate                                    0.056403   \n",
              "           Technology                                    -0.482192   \n",
              "           Utilities                                      0.034718   \n",
              "\n",
              "                                                       Institutions, Associations & Organizations  \\\n",
              "date       factor                                                                                   \n",
              "2025-04-01 Market                                                                        1.000000   \n",
              "           Dividend                                                                     -1.000000   \n",
              "           Growth                                                                        1.000000   \n",
              "           Leverage                                                                     -1.000000   \n",
              "           Momentum                                                                      1.000000   \n",
              "...                                                                                           ...   \n",
              "2026-03-31 Industrials                                                                   0.210333   \n",
              "           Institutions, Associations & Organizations                                    1.000000   \n",
              "           Real Estate                                                                  -0.175821   \n",
              "           Technology                                                                    0.136438   \n",
              "           Utilities                                                                    -0.046746   \n",
              "\n",
              "                                                       Real Estate  \\\n",
              "date       factor                                                    \n",
              "2025-04-01 Market                                        -1.000000   \n",
              "           Dividend                                       1.000000   \n",
              "           Growth                                        -1.000000   \n",
              "           Leverage                                       1.000000   \n",
              "           Momentum                                      -1.000000   \n",
              "...                                                            ...   \n",
              "2026-03-31 Industrials                                    0.056403   \n",
              "           Institutions, Associations & Organizations    -0.175821   \n",
              "           Real Estate                                    1.000000   \n",
              "           Technology                                    -0.411016   \n",
              "           Utilities                                      0.472253   \n",
              "\n",
              "                                                       Technology  Utilities  \n",
              "date       factor                                                             \n",
              "2025-04-01 Market                                        1.000000   1.000000  \n",
              "           Dividend                                     -1.000000  -1.000000  \n",
              "           Growth                                        1.000000   1.000000  \n",
              "           Leverage                                     -1.000000  -1.000000  \n",
              "           Momentum                                      1.000000   1.000000  \n",
              "...                                                           ...        ...  \n",
              "2026-03-31 Industrials                                  -0.482192   0.034718  \n",
              "           Institutions, Associations & Organizations    0.136438  -0.046746  \n",
              "           Real Estate                                  -0.411016   0.472253  \n",
              "           Technology                                    1.000000  -0.194446  \n",
              "           Utilities                                    -0.194446   1.000000  \n",
              "\n",
              "[5271 rows x 21 columns]"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate the ewma on the outer product (vcov with mean zero)\n",
        "df_factor_returns_outer = df_factor_returns.groupby(\"date\").apply(\n",
        "    lambda df: pd.DataFrame(np.outer(df, df), df.columns, df.columns)\n",
        ")\n",
        "df_factor_returns_outer.index.names = [\"date\", \"factor\"]\n",
        "df_cor_tieout = (\n",
        "    pd.DataFrame(df_factor_returns_outer)\n",
        "    .unstack()\n",
        "    .ewm(halflife=report_settings.factor_cov_settings.halflife_cor)\n",
        "    .mean()\n",
        "    .stack(future_stack=True)\n",
        "    .reindex(df_factor_returns_outer.columns, axis=1)\n",
        "    .groupby(\"date\")\n",
        "    .apply(\n",
        "        lambda df: df.droplevel(\"date\")\n",
        "        / np.outer(np.diag(df) ** 0.5, np.diag(df) ** 0.5)\n",
        "    )\n",
        "    .astype(np.float32)\n",
        ")\n",
        "\n",
        "df_cor_tieout"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "id": "bfb94e34",
      "metadata": {},
      "outputs": [],
      "source": [
        "pd.testing.assert_frame_equal(df_cor, df_cor_tieout, check_index_type=False, check_categorical=False, check_like=True, atol=1e-5)"
      ]
    }
  ],
  "metadata": {
    "jupytext": {
      "text_representation": {
        "extension": ".py",
        "format_name": "percent",
        "format_version": "1.3",
        "jupytext_version": "1.19.3"
      }
    },
    "kernelspec": {
      "display_name": ".venv",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.15"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}