{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Factor Covariance Matrix Forecasts\n",
        "\n",
        "In this tutorial we are going to create several covariance matrices of factor returns. This covariance matrix follows from a standard factor model. We will also tie out the numbers against a pandas/numpy-based reimplementation. More specifically, we will:\n",
        "- Create a basic risk model.\n",
        "- Extract the factor returns.\n",
        "- Use the covariance matrix forecast report to compute the covariance matrix.\n",
        "- Use pandas to extract the factor volatility and correlation matrix time-series.\n",
        "- Replicate the volatility forecast.\n",
        "- Replicate the correlation forecast.\n",
        "\n",
        "Throughout this notebook we work with a randomly generated dataset. The results should generalize to real data, but for legal reasons we do not show any real data on our public API. Bayesline clients can run this notebook on real data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Imports & Setup\n",
        "\n",
        "For this tutorial notebook, you will need to import the following packages."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import polars as pl\n",
        "import numpy as np\n",
        "\n",
        "from bayesline.api.equity import (\n",
        "    ReportSettings,\n",
        "    FactorCovarianceReportSettings,\n",
        "    ExposureSettings,\n",
        "    CategoricalExposureGroupSettings,\n",
        "    ContinuousExposureGroupSettings,\n",
        "    FactorRiskModelSettings,\n",
        "    ModelConstructionSettings,\n",
        "    ReportSettings,\n",
        "    UniverseSettings,\n",
        ")\n",
        "from bayesline.apiclient import BayeslineApiClient"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We will also need to have a Bayesline API client configured."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "skip-execution"
        ]
      },
      "outputs": [],
      "source": [
        "bln = BayeslineApiClient.new_client(\n",
        "    endpoint=\"https://[ENDPOINT]\",\n",
        "    api_key=\"[API-KEY]\",\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Creating the covariance matrix forecasts\n",
        "\n",
        "Let's first set up a basic risk model and use it to generate the forecasts. We choose to run with mostly default settings. The steps involved are:\n",
        "1. Creating the settings of the risk model.\n",
        "2. Loading the report model engine.\n",
        "3. Running the engine to generate the covariance report.\n",
        "\n",
        "The first step is creating the risk model settings."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {},
      "outputs": [],
      "source": [
        "factorriskmodel_settings = FactorRiskModelSettings(\n",
        "    universe=UniverseSettings(dataset=\"Bayesline-US-All-1y\"),\n",
        "    exposures=ExposureSettings(\n",
        "        exposures=[\n",
        "            ContinuousExposureGroupSettings(hierarchy=\"market\"),\n",
        "            CategoricalExposureGroupSettings(hierarchy=\"trbc\"),\n",
        "            CategoricalExposureGroupSettings(hierarchy=\"continent\"),\n",
        "            ContinuousExposureGroupSettings(hierarchy=\"style\"),\n",
        "        ]\n",
        "    ),\n",
        "    modelconstruction=ModelConstructionSettings(\n",
        "        estimation_universe=None,\n",
        "        zero_sum_constraints={\"trbc\": \"mcap_weighted\", \"continent\": \"mcap_weighted\"},\n",
        "    ),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Next, we create the report engine from the report settings. We run with the defaults here."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {},
      "outputs": [],
      "source": [
        "report_settings = ReportSettings(\n",
        "    report=FactorCovarianceReportSettings(),\n",
        "    risk_model=factorriskmodel_settings,\n",
        ")\n",
        "report_engine = bln.equity.portfolioreport.load(report_settings)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Let's see what these settings really are by printing them out."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"report_type\": \"Factor Covariance report\",\n",
            "  \"measures\": [\n",
            "    {\n",
            "      \"type\": \"FactorCovariance\"\n",
            "    }\n",
            "  ],\n",
            "  \"halflife_factor_vol\": 42,\n",
            "  \"halflife_factor_vra\": null,\n",
            "  \"halflife_factor_cor\": 126,\n",
            "  \"shrink_factor_cor_method\": null,\n",
            "  \"shrink_factor_cor_length\": 1008,\n",
            "  \"shrink_factor_cor_standardized\": false,\n",
            "  \"nw_lags_factor_vol\": 0,\n",
            "  \"nw_lags_factor_vol_halflife_override\": null,\n",
            "  \"nw_lags_factor_cor\": 0\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "print(report_settings.report.model_dump_json(indent=2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The different settings that jointly make up the covariance matrix are:\n",
        "1. **halflife_factor_vol** The halflife of the factor volatility. The default is a 42-day halflife.\n",
        "2. **halflife_factor_vra** The halflife of the cross-sectional factor volatility adjustment. The default is to not do any adjustment.\n",
        "3. **halflife_factor_cor** The halflife of the factor correlation. The default is a 126-day halflife.\n",
        "4. **nw_lags_factor_vol** The overlap or Newey-West lags to incluce on the factor volatility forecast. The default is zero, meaning no autocorrelation correction is performed.\n",
        "5. **nw_lags_factor_cor** The overlap or Newey-West lags to incluce on the factor correlation forecast. The default is zero, meaning no autocorrelation correction is performed.\n",
        "\n",
        "Now we get the actual time-series of the covariance matrices."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>...</th>\n",
              "      <th>Asia</th>\n",
              "      <th>Europe</th>\n",
              "      <th>Oceania</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Leverage</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2024-08-27</th>\n",
              "      <th>Market</th>\n",
              "      <td>0.007288</td>\n",
              "      <td>0.008562</td>\n",
              "      <td>0.004093</td>\n",
              "      <td>0.003134</td>\n",
              "      <td>0.001097</td>\n",
              "      <td>0.000990</td>\n",
              "      <td>-0.000468</td>\n",
              "      <td>0.000922</td>\n",
              "      <td>-0.002703</td>\n",
              "      <td>0.007861</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-0.001047</td>\n",
              "      <td>0.002271</td>\n",
              "      <td>-0.000959</td>\n",
              "      <td>0.004910</td>\n",
              "      <td>-0.001885</td>\n",
              "      <td>0.002355</td>\n",
              "      <td>-0.001289</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Energy</th>\n",
              "      <td>0.008562</td>\n",
              "      <td>0.010058</td>\n",
              "      <td>0.004808</td>\n",
              "      <td>0.003681</td>\n",
              "      <td>0.001289</td>\n",
              "      <td>0.001164</td>\n",
              "      <td>-0.000549</td>\n",
              "      <td>0.001083</td>\n",
              "      <td>-0.003176</td>\n",
              "      <td>0.009236</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-0.001231</td>\n",
              "      <td>0.002668</td>\n",
              "      <td>-0.001127</td>\n",
              "      <td>0.005768</td>\n",
              "      <td>-0.002214</td>\n",
              "      <td>0.002766</td>\n",
              "      <td>-0.001514</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Basic Materials</th>\n",
              "      <td>0.004093</td>\n",
              "      <td>0.004808</td>\n",
              "      <td>0.002299</td>\n",
              "      <td>0.001760</td>\n",
              "      <td>0.000616</td>\n",
              "      <td>0.000556</td>\n",
              "      <td>-0.000263</td>\n",
              "      <td>0.000518</td>\n",
              "      <td>-0.001518</td>\n",
              "      <td>0.004415</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-0.000588</td>\n",
              "      <td>0.001275</td>\n",
              "      <td>-0.000539</td>\n",
              "      <td>0.002757</td>\n",
              "      <td>-0.001058</td>\n",
              "      <td>0.001322</td>\n",
              "      <td>-0.000724</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Industrials</th>\n",
              "      <td>0.003134</td>\n",
              "      <td>0.003681</td>\n",
              "      <td>0.001760</td>\n",
              "      <td>0.001347</td>\n",
              "      <td>0.000472</td>\n",
              "      <td>0.000426</td>\n",
              "      <td>-0.000201</td>\n",
              "      <td>0.000396</td>\n",
              "      <td>-0.001162</td>\n",
              "      <td>0.003380</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-0.000450</td>\n",
              "      <td>0.000976</td>\n",
              "      <td>-0.000413</td>\n",
              "      <td>0.002111</td>\n",
              "      <td>-0.000810</td>\n",
              "      <td>0.001012</td>\n",
              "      <td>-0.000554</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <td>0.001097</td>\n",
              "      <td>0.001289</td>\n",
              "      <td>0.000616</td>\n",
              "      <td>0.000472</td>\n",
              "      <td>0.000165</td>\n",
              "      <td>0.000149</td>\n",
              "      <td>-0.000070</td>\n",
              "      <td>0.000139</td>\n",
              "      <td>-0.000407</td>\n",
              "      <td>0.001184</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-0.000158</td>\n",
              "      <td>0.000342</td>\n",
              "      <td>-0.000144</td>\n",
              "      <td>0.000739</td>\n",
              "      <td>-0.000284</td>\n",
              "      <td>0.000355</td>\n",
              "      <td>-0.000194</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2025-08-25</th>\n",
              "      <th>Growth</th>\n",
              "      <td>0.000080</td>\n",
              "      <td>0.000169</td>\n",
              "      <td>-0.000185</td>\n",
              "      <td>0.000021</td>\n",
              "      <td>-0.000191</td>\n",
              "      <td>-0.000259</td>\n",
              "      <td>0.000141</td>\n",
              "      <td>-0.000552</td>\n",
              "      <td>0.000177</td>\n",
              "      <td>-0.000217</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000129</td>\n",
              "      <td>-0.000073</td>\n",
              "      <td>0.000262</td>\n",
              "      <td>0.000137</td>\n",
              "      <td>0.000099</td>\n",
              "      <td>-0.000046</td>\n",
              "      <td>-0.000021</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Volatility</th>\n",
              "      <td>0.017138</td>\n",
              "      <td>-0.001166</td>\n",
              "      <td>-0.001979</td>\n",
              "      <td>0.000364</td>\n",
              "      <td>-0.001114</td>\n",
              "      <td>-0.006802</td>\n",
              "      <td>0.003978</td>\n",
              "      <td>-0.005433</td>\n",
              "      <td>0.001183</td>\n",
              "      <td>-0.004732</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.001714</td>\n",
              "      <td>0.000648</td>\n",
              "      <td>0.000137</td>\n",
              "      <td>0.012298</td>\n",
              "      <td>0.000635</td>\n",
              "      <td>-0.000529</td>\n",
              "      <td>-0.000849</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Momentum</th>\n",
              "      <td>-0.001979</td>\n",
              "      <td>0.000658</td>\n",
              "      <td>-0.001427</td>\n",
              "      <td>-0.000559</td>\n",
              "      <td>-0.002316</td>\n",
              "      <td>-0.000270</td>\n",
              "      <td>0.000374</td>\n",
              "      <td>-0.000985</td>\n",
              "      <td>0.000700</td>\n",
              "      <td>0.002264</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-0.000181</td>\n",
              "      <td>-0.001034</td>\n",
              "      <td>0.000099</td>\n",
              "      <td>0.000635</td>\n",
              "      <td>0.003505</td>\n",
              "      <td>-0.000324</td>\n",
              "      <td>-0.000532</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Dividend</th>\n",
              "      <td>-0.000451</td>\n",
              "      <td>0.000139</td>\n",
              "      <td>0.000268</td>\n",
              "      <td>-0.000065</td>\n",
              "      <td>0.000105</td>\n",
              "      <td>0.000443</td>\n",
              "      <td>-0.000276</td>\n",
              "      <td>0.000611</td>\n",
              "      <td>-0.000142</td>\n",
              "      <td>0.000263</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-0.000124</td>\n",
              "      <td>0.000082</td>\n",
              "      <td>-0.000046</td>\n",
              "      <td>-0.000529</td>\n",
              "      <td>-0.000324</td>\n",
              "      <td>0.000482</td>\n",
              "      <td>0.000141</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Leverage</th>\n",
              "      <td>-0.000454</td>\n",
              "      <td>-0.000013</td>\n",
              "      <td>0.000448</td>\n",
              "      <td>0.000352</td>\n",
              "      <td>0.000819</td>\n",
              "      <td>0.000645</td>\n",
              "      <td>-0.000035</td>\n",
              "      <td>0.000345</td>\n",
              "      <td>-0.000543</td>\n",
              "      <td>-0.000018</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000052</td>\n",
              "      <td>0.000278</td>\n",
              "      <td>-0.000021</td>\n",
              "      <td>-0.000849</td>\n",
              "      <td>-0.000532</td>\n",
              "      <td>0.000141</td>\n",
              "      <td>0.000590</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>6474 rows × 26 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                 Market    Energy  Basic Materials  \\\n",
              "date       factor                                                    \n",
              "2024-08-27 Market              0.007288  0.008562         0.004093   \n",
              "           Energy              0.008562  0.010058         0.004808   \n",
              "           Basic Materials     0.004093  0.004808         0.002299   \n",
              "           Industrials         0.003134  0.003681         0.001760   \n",
              "           Consumer Cyclicals  0.001097  0.001289         0.000616   \n",
              "...                                 ...       ...              ...   \n",
              "2025-08-25 Growth              0.000080  0.000169        -0.000185   \n",
              "           Volatility          0.017138 -0.001166        -0.001979   \n",
              "           Momentum           -0.001979  0.000658        -0.001427   \n",
              "           Dividend           -0.000451  0.000139         0.000268   \n",
              "           Leverage           -0.000454 -0.000013         0.000448   \n",
              "\n",
              "                               Industrials  Consumer Cyclicals  \\\n",
              "date       factor                                                \n",
              "2024-08-27 Market                 0.003134            0.001097   \n",
              "           Energy                 0.003681            0.001289   \n",
              "           Basic Materials        0.001760            0.000616   \n",
              "           Industrials            0.001347            0.000472   \n",
              "           Consumer Cyclicals     0.000472            0.000165   \n",
              "...                                    ...                 ...   \n",
              "2025-08-25 Growth                 0.000021           -0.000191   \n",
              "           Volatility             0.000364           -0.001114   \n",
              "           Momentum              -0.000559           -0.002316   \n",
              "           Dividend              -0.000065            0.000105   \n",
              "           Leverage               0.000352            0.000819   \n",
              "\n",
              "                               Consumer Non-Cyclicals  Financials  Healthcare  \\\n",
              "date       factor                                                               \n",
              "2024-08-27 Market                            0.000990   -0.000468    0.000922   \n",
              "           Energy                            0.001164   -0.000549    0.001083   \n",
              "           Basic Materials                   0.000556   -0.000263    0.000518   \n",
              "           Industrials                       0.000426   -0.000201    0.000396   \n",
              "           Consumer Cyclicals                0.000149   -0.000070    0.000139   \n",
              "...                                               ...         ...         ...   \n",
              "2025-08-25 Growth                           -0.000259    0.000141   -0.000552   \n",
              "           Volatility                       -0.006802    0.003978   -0.005433   \n",
              "           Momentum                         -0.000270    0.000374   -0.000985   \n",
              "           Dividend                          0.000443   -0.000276    0.000611   \n",
              "           Leverage                          0.000645   -0.000035    0.000345   \n",
              "\n",
              "                               Technology  Utilities  ...  Asia  Europe  \\\n",
              "date       factor                                     ...                 \n",
              "2024-08-27 Market               -0.002703   0.007861  ...   0.0     0.0   \n",
              "           Energy               -0.003176   0.009236  ...   0.0     0.0   \n",
              "           Basic Materials      -0.001518   0.004415  ...   0.0     0.0   \n",
              "           Industrials          -0.001162   0.003380  ...   0.0     0.0   \n",
              "           Consumer Cyclicals   -0.000407   0.001184  ...   0.0     0.0   \n",
              "...                                   ...        ...  ...   ...     ...   \n",
              "2025-08-25 Growth                0.000177  -0.000217  ...   0.0     0.0   \n",
              "           Volatility            0.001183  -0.004732  ...   0.0     0.0   \n",
              "           Momentum              0.000700   0.002264  ...   0.0     0.0   \n",
              "           Dividend             -0.000142   0.000263  ...   0.0     0.0   \n",
              "           Leverage             -0.000543  -0.000018  ...   0.0     0.0   \n",
              "\n",
              "                               Oceania      Size     Value    Growth  \\\n",
              "date       factor                                                      \n",
              "2024-08-27 Market                  0.0 -0.001047  0.002271 -0.000959   \n",
              "           Energy                  0.0 -0.001231  0.002668 -0.001127   \n",
              "           Basic Materials         0.0 -0.000588  0.001275 -0.000539   \n",
              "           Industrials             0.0 -0.000450  0.000976 -0.000413   \n",
              "           Consumer Cyclicals      0.0 -0.000158  0.000342 -0.000144   \n",
              "...                                ...       ...       ...       ...   \n",
              "2025-08-25 Growth                  0.0  0.000129 -0.000073  0.000262   \n",
              "           Volatility              0.0  0.001714  0.000648  0.000137   \n",
              "           Momentum                0.0 -0.000181 -0.001034  0.000099   \n",
              "           Dividend                0.0 -0.000124  0.000082 -0.000046   \n",
              "           Leverage                0.0  0.000052  0.000278 -0.000021   \n",
              "\n",
              "                               Volatility  Momentum  Dividend  Leverage  \n",
              "date       factor                                                        \n",
              "2024-08-27 Market                0.004910 -0.001885  0.002355 -0.001289  \n",
              "           Energy                0.005768 -0.002214  0.002766 -0.001514  \n",
              "           Basic Materials       0.002757 -0.001058  0.001322 -0.000724  \n",
              "           Industrials           0.002111 -0.000810  0.001012 -0.000554  \n",
              "           Consumer Cyclicals    0.000739 -0.000284  0.000355 -0.000194  \n",
              "...                                   ...       ...       ...       ...  \n",
              "2025-08-25 Growth                0.000137  0.000099 -0.000046 -0.000021  \n",
              "           Volatility            0.012298  0.000635 -0.000529 -0.000849  \n",
              "           Momentum              0.000635  0.003505 -0.000324 -0.000532  \n",
              "           Dividend             -0.000529 -0.000324  0.000482  0.000141  \n",
              "           Leverage             -0.000849 -0.000532  0.000141  0.000590  \n",
              "\n",
              "[6474 rows x 26 columns]"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# generate the report data\n",
        "order = {\n",
        "    \"date\": [\"date\"],\n",
        "    \"factor\": [\"factor_group\", \"factor\"],\n",
        "    \"factor_col\": [\"factor_group_col\", \"factor_col\"],\n",
        "}\n",
        "report = report_engine.get_report(order=order)\n",
        "\n",
        "# massage the data into a more usable format\n",
        "df_report = (\n",
        "    report.get_data(\n",
        "        [],\n",
        "        expand=(\"date\", \"factor\"),\n",
        "        pivot_cols=(\"factor_col\",),\n",
        "        value_cols=(\"FactorCovariance\",)\n",
        "    )\n",
        ").with_columns(pl.col(\"date\").cast(pl.Date))  # string to date\n",
        "\n",
        "# convert to pandas\n",
        "df_vcov = df_report.to_pandas().set_index([\"date\", \"factor\"]).rename(columns=lambda c: c.split(\"^\")[0])\n",
        "df_vcov"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "For downstream comparisons, we split these into factor volatilities and correlations."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>...</th>\n",
              "      <th>Asia</th>\n",
              "      <th>Europe</th>\n",
              "      <th>Oceania</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Leverage</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2025-08-19</th>\n",
              "      <td>0.174342</td>\n",
              "      <td>0.161448</td>\n",
              "      <td>0.093997</td>\n",
              "      <td>0.052590</td>\n",
              "      <td>0.076704</td>\n",
              "      <td>0.084888</td>\n",
              "      <td>0.054344</td>\n",
              "      <td>0.114819</td>\n",
              "      <td>0.047862</td>\n",
              "      <td>0.112037</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.031732</td>\n",
              "      <td>0.030392</td>\n",
              "      <td>0.016548</td>\n",
              "      <td>0.110538</td>\n",
              "      <td>0.059152</td>\n",
              "      <td>0.022407</td>\n",
              "      <td>0.024967</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-20</th>\n",
              "      <td>0.173077</td>\n",
              "      <td>0.162287</td>\n",
              "      <td>0.093217</td>\n",
              "      <td>0.052428</td>\n",
              "      <td>0.076728</td>\n",
              "      <td>0.085163</td>\n",
              "      <td>0.053946</td>\n",
              "      <td>0.115067</td>\n",
              "      <td>0.047750</td>\n",
              "      <td>0.111100</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.031467</td>\n",
              "      <td>0.030165</td>\n",
              "      <td>0.016422</td>\n",
              "      <td>0.109734</td>\n",
              "      <td>0.059249</td>\n",
              "      <td>0.022219</td>\n",
              "      <td>0.024770</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-21</th>\n",
              "      <td>0.171665</td>\n",
              "      <td>0.161461</td>\n",
              "      <td>0.093135</td>\n",
              "      <td>0.051995</td>\n",
              "      <td>0.076168</td>\n",
              "      <td>0.084529</td>\n",
              "      <td>0.053641</td>\n",
              "      <td>0.114350</td>\n",
              "      <td>0.047357</td>\n",
              "      <td>0.110308</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.031347</td>\n",
              "      <td>0.029928</td>\n",
              "      <td>0.016440</td>\n",
              "      <td>0.108825</td>\n",
              "      <td>0.058777</td>\n",
              "      <td>0.022040</td>\n",
              "      <td>0.024621</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-22</th>\n",
              "      <td>0.181212</td>\n",
              "      <td>0.160500</td>\n",
              "      <td>0.092386</td>\n",
              "      <td>0.051829</td>\n",
              "      <td>0.076564</td>\n",
              "      <td>0.084793</td>\n",
              "      <td>0.053753</td>\n",
              "      <td>0.115357</td>\n",
              "      <td>0.046980</td>\n",
              "      <td>0.109743</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.031145</td>\n",
              "      <td>0.032185</td>\n",
              "      <td>0.016314</td>\n",
              "      <td>0.111832</td>\n",
              "      <td>0.059516</td>\n",
              "      <td>0.022124</td>\n",
              "      <td>0.024468</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-25</th>\n",
              "      <td>0.180353</td>\n",
              "      <td>0.160313</td>\n",
              "      <td>0.091774</td>\n",
              "      <td>0.051436</td>\n",
              "      <td>0.075977</td>\n",
              "      <td>0.084825</td>\n",
              "      <td>0.053378</td>\n",
              "      <td>0.115683</td>\n",
              "      <td>0.046637</td>\n",
              "      <td>0.109232</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.030897</td>\n",
              "      <td>0.031992</td>\n",
              "      <td>0.016185</td>\n",
              "      <td>0.110898</td>\n",
              "      <td>0.059203</td>\n",
              "      <td>0.021951</td>\n",
              "      <td>0.024282</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 26 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "              Market    Energy  Basic Materials  Industrials  \\\n",
              "date                                                           \n",
              "2025-08-19  0.174342  0.161448         0.093997     0.052590   \n",
              "2025-08-20  0.173077  0.162287         0.093217     0.052428   \n",
              "2025-08-21  0.171665  0.161461         0.093135     0.051995   \n",
              "2025-08-22  0.181212  0.160500         0.092386     0.051829   \n",
              "2025-08-25  0.180353  0.160313         0.091774     0.051436   \n",
              "\n",
              "            Consumer Cyclicals  Consumer Non-Cyclicals  Financials  \\\n",
              "date                                                                 \n",
              "2025-08-19            0.076704                0.084888    0.054344   \n",
              "2025-08-20            0.076728                0.085163    0.053946   \n",
              "2025-08-21            0.076168                0.084529    0.053641   \n",
              "2025-08-22            0.076564                0.084793    0.053753   \n",
              "2025-08-25            0.075977                0.084825    0.053378   \n",
              "\n",
              "            Healthcare  Technology  Utilities  ...  Asia  Europe  Oceania  \\\n",
              "date                                           ...                          \n",
              "2025-08-19    0.114819    0.047862   0.112037  ...   0.0     0.0      0.0   \n",
              "2025-08-20    0.115067    0.047750   0.111100  ...   0.0     0.0      0.0   \n",
              "2025-08-21    0.114350    0.047357   0.110308  ...   0.0     0.0      0.0   \n",
              "2025-08-22    0.115357    0.046980   0.109743  ...   0.0     0.0      0.0   \n",
              "2025-08-25    0.115683    0.046637   0.109232  ...   0.0     0.0      0.0   \n",
              "\n",
              "                Size     Value    Growth  Volatility  Momentum  Dividend  \\\n",
              "date                                                                       \n",
              "2025-08-19  0.031732  0.030392  0.016548    0.110538  0.059152  0.022407   \n",
              "2025-08-20  0.031467  0.030165  0.016422    0.109734  0.059249  0.022219   \n",
              "2025-08-21  0.031347  0.029928  0.016440    0.108825  0.058777  0.022040   \n",
              "2025-08-22  0.031145  0.032185  0.016314    0.111832  0.059516  0.022124   \n",
              "2025-08-25  0.030897  0.031992  0.016185    0.110898  0.059203  0.021951   \n",
              "\n",
              "            Leverage  \n",
              "date                  \n",
              "2025-08-19  0.024967  \n",
              "2025-08-20  0.024770  \n",
              "2025-08-21  0.024621  \n",
              "2025-08-22  0.024468  \n",
              "2025-08-25  0.024282  \n",
              "\n",
              "[5 rows x 26 columns]"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate actual factor volatilities from vcovs\n",
        "df_vol = df_vcov.groupby(level=\"date\").apply(\n",
        "    lambda df: pd.Series(np.diag(df) ** 0.5, df.index.droplevel(\"date\"))\n",
        ")\n",
        "df_vol.columns.name = None\n",
        "df_vol.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>...</th>\n",
              "      <th>Asia</th>\n",
              "      <th>Europe</th>\n",
              "      <th>Oceania</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Leverage</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2025-08-25</th>\n",
              "      <th>Growth</th>\n",
              "      <td>0.027298</td>\n",
              "      <td>0.065028</td>\n",
              "      <td>-0.124795</td>\n",
              "      <td>0.025007</td>\n",
              "      <td>-0.155016</td>\n",
              "      <td>-0.188488</td>\n",
              "      <td>0.162800</td>\n",
              "      <td>-0.294809</td>\n",
              "      <td>0.233843</td>\n",
              "      <td>-0.122661</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.258081</td>\n",
              "      <td>-0.141794</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.076467</td>\n",
              "      <td>0.103424</td>\n",
              "      <td>-0.128167</td>\n",
              "      <td>-0.054107</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Volatility</th>\n",
              "      <td>0.856853</td>\n",
              "      <td>-0.065603</td>\n",
              "      <td>-0.194439</td>\n",
              "      <td>0.063828</td>\n",
              "      <td>-0.132216</td>\n",
              "      <td>-0.723050</td>\n",
              "      <td>0.671954</td>\n",
              "      <td>-0.423478</td>\n",
              "      <td>0.228823</td>\n",
              "      <td>-0.390604</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.500156</td>\n",
              "      <td>0.182544</td>\n",
              "      <td>0.076467</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.096767</td>\n",
              "      <td>-0.217297</td>\n",
              "      <td>-0.315350</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Momentum</th>\n",
              "      <td>-0.185332</td>\n",
              "      <td>0.069289</td>\n",
              "      <td>-0.262668</td>\n",
              "      <td>-0.183582</td>\n",
              "      <td>-0.514879</td>\n",
              "      <td>-0.053805</td>\n",
              "      <td>0.118316</td>\n",
              "      <td>-0.143826</td>\n",
              "      <td>0.253623</td>\n",
              "      <td>0.350147</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>-0.098723</td>\n",
              "      <td>-0.546127</td>\n",
              "      <td>0.103424</td>\n",
              "      <td>0.096767</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.248990</td>\n",
              "      <td>-0.369793</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Dividend</th>\n",
              "      <td>-0.114013</td>\n",
              "      <td>0.039374</td>\n",
              "      <td>0.133062</td>\n",
              "      <td>-0.057912</td>\n",
              "      <td>0.062695</td>\n",
              "      <td>0.237979</td>\n",
              "      <td>-0.235805</td>\n",
              "      <td>0.240609</td>\n",
              "      <td>-0.138234</td>\n",
              "      <td>0.109572</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>-0.182794</td>\n",
              "      <td>0.116952</td>\n",
              "      <td>-0.128167</td>\n",
              "      <td>-0.217297</td>\n",
              "      <td>-0.248990</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.264753</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Leverage</th>\n",
              "      <td>-0.103686</td>\n",
              "      <td>-0.003341</td>\n",
              "      <td>0.200817</td>\n",
              "      <td>0.281899</td>\n",
              "      <td>0.444195</td>\n",
              "      <td>0.313135</td>\n",
              "      <td>-0.027029</td>\n",
              "      <td>0.122807</td>\n",
              "      <td>-0.479819</td>\n",
              "      <td>-0.006972</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.069087</td>\n",
              "      <td>0.358233</td>\n",
              "      <td>-0.054107</td>\n",
              "      <td>-0.315350</td>\n",
              "      <td>-0.369793</td>\n",
              "      <td>0.264753</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 26 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                         Market    Energy  Basic Materials  Industrials  \\\n",
              "date       factor                                                         \n",
              "2025-08-25 Growth      0.027298  0.065028        -0.124795     0.025007   \n",
              "           Volatility  0.856853 -0.065603        -0.194439     0.063828   \n",
              "           Momentum   -0.185332  0.069289        -0.262668    -0.183582   \n",
              "           Dividend   -0.114013  0.039374         0.133062    -0.057912   \n",
              "           Leverage   -0.103686 -0.003341         0.200817     0.281899   \n",
              "\n",
              "                       Consumer Cyclicals  Consumer Non-Cyclicals  Financials  \\\n",
              "date       factor                                                               \n",
              "2025-08-25 Growth               -0.155016               -0.188488    0.162800   \n",
              "           Volatility           -0.132216               -0.723050    0.671954   \n",
              "           Momentum             -0.514879               -0.053805    0.118316   \n",
              "           Dividend              0.062695                0.237979   -0.235805   \n",
              "           Leverage              0.444195                0.313135   -0.027029   \n",
              "\n",
              "                       Healthcare  Technology  Utilities  ...  Asia  Europe  \\\n",
              "date       factor                                         ...                 \n",
              "2025-08-25 Growth       -0.294809    0.233843  -0.122661  ...   NaN     NaN   \n",
              "           Volatility   -0.423478    0.228823  -0.390604  ...   NaN     NaN   \n",
              "           Momentum     -0.143826    0.253623   0.350147  ...   NaN     NaN   \n",
              "           Dividend      0.240609   -0.138234   0.109572  ...   NaN     NaN   \n",
              "           Leverage      0.122807   -0.479819  -0.006972  ...   NaN     NaN   \n",
              "\n",
              "                       Oceania      Size     Value    Growth  Volatility  \\\n",
              "date       factor                                                          \n",
              "2025-08-25 Growth          NaN  0.258081 -0.141794  1.000000    0.076467   \n",
              "           Volatility      NaN  0.500156  0.182544  0.076467    1.000000   \n",
              "           Momentum        NaN -0.098723 -0.546127  0.103424    0.096767   \n",
              "           Dividend        NaN -0.182794  0.116952 -0.128167   -0.217297   \n",
              "           Leverage        NaN  0.069087  0.358233 -0.054107   -0.315350   \n",
              "\n",
              "                       Momentum  Dividend  Leverage  \n",
              "date       factor                                    \n",
              "2025-08-25 Growth      0.103424 -0.128167 -0.054107  \n",
              "           Volatility  0.096767 -0.217297 -0.315350  \n",
              "           Momentum    1.000000 -0.248990 -0.369793  \n",
              "           Dividend   -0.248990  1.000000  0.264753  \n",
              "           Leverage   -0.369793  0.264753  1.000000  \n",
              "\n",
              "[5 rows x 26 columns]"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate actual factor correlations from vcovs\n",
        "df_cor = df_vcov.groupby(level=\"date\").apply(\n",
        "    lambda df: df.droplevel(\"date\")\n",
        "    / np.outer(np.diag(df) ** 0.5, np.diag(df) ** 0.5)\n",
        ")\n",
        "df_cor.tail()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Manually replicating the covariance forecasts\n",
        "\n",
        "We can also estimated the risk model and get the returns directly. From these returns we can construct the covariance forecasts. Bayesline returns dataframes in `polars`, but they can be easily converted to `pandas` dataframes. We also remove the factor group (market, style, industry, etc.) for convenience."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>...</th>\n",
              "      <th>Asia</th>\n",
              "      <th>Europe</th>\n",
              "      <th>Oceania</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Leverage</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2025-08-19</th>\n",
              "      <td>-0.012457</td>\n",
              "      <td>-0.001213</td>\n",
              "      <td>0.004926</td>\n",
              "      <td>0.004431</td>\n",
              "      <td>0.008365</td>\n",
              "      <td>0.003666</td>\n",
              "      <td>-0.002158</td>\n",
              "      <td>0.003093</td>\n",
              "      <td>-0.004364</td>\n",
              "      <td>0.003283</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.001825</td>\n",
              "      <td>0.000976</td>\n",
              "      <td>0.001007</td>\n",
              "      <td>-0.014873</td>\n",
              "      <td>-0.007365</td>\n",
              "      <td>-0.000221</td>\n",
              "      <td>0.004215</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-20</th>\n",
              "      <td>-0.003994</td>\n",
              "      <td>0.012966</td>\n",
              "      <td>0.000518</td>\n",
              "      <td>-0.002631</td>\n",
              "      <td>-0.004921</td>\n",
              "      <td>0.006301</td>\n",
              "      <td>0.001206</td>\n",
              "      <td>0.008118</td>\n",
              "      <td>-0.002558</td>\n",
              "      <td>0.000043</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000089</td>\n",
              "      <td>-0.000620</td>\n",
              "      <td>-0.000318</td>\n",
              "      <td>-0.002513</td>\n",
              "      <td>0.004078</td>\n",
              "      <td>-0.000022</td>\n",
              "      <td>0.000370</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-21</th>\n",
              "      <td>0.001681</td>\n",
              "      <td>0.006386</td>\n",
              "      <td>0.005551</td>\n",
              "      <td>0.000366</td>\n",
              "      <td>-0.001721</td>\n",
              "      <td>-0.001779</td>\n",
              "      <td>-0.001929</td>\n",
              "      <td>0.003653</td>\n",
              "      <td>0.000378</td>\n",
              "      <td>-0.002681</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-0.001460</td>\n",
              "      <td>0.000473</td>\n",
              "      <td>-0.001100</td>\n",
              "      <td>0.000664</td>\n",
              "      <td>0.000814</td>\n",
              "      <td>0.000266</td>\n",
              "      <td>-0.000823</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-22</th>\n",
              "      <td>0.030332</td>\n",
              "      <td>-0.005454</td>\n",
              "      <td>0.001132</td>\n",
              "      <td>0.002571</td>\n",
              "      <td>0.006121</td>\n",
              "      <td>-0.006245</td>\n",
              "      <td>0.003781</td>\n",
              "      <td>-0.010345</td>\n",
              "      <td>-0.000635</td>\n",
              "      <td>-0.004322</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000946</td>\n",
              "      <td>0.006080</td>\n",
              "      <td>-0.000289</td>\n",
              "      <td>0.014323</td>\n",
              "      <td>-0.005876</td>\n",
              "      <td>-0.001676</td>\n",
              "      <td>0.000784</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-25</th>\n",
              "      <td>-0.007501</td>\n",
              "      <td>0.009374</td>\n",
              "      <td>0.002649</td>\n",
              "      <td>-0.000985</td>\n",
              "      <td>0.001371</td>\n",
              "      <td>-0.005463</td>\n",
              "      <td>0.001370</td>\n",
              "      <td>-0.008411</td>\n",
              "      <td>0.001051</td>\n",
              "      <td>-0.004592</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000423</td>\n",
              "      <td>0.001079</td>\n",
              "      <td>-0.000251</td>\n",
              "      <td>0.000190</td>\n",
              "      <td>0.002282</td>\n",
              "      <td>-0.000347</td>\n",
              "      <td>0.000468</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 26 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "              Market    Energy  Basic Materials  Industrials  \\\n",
              "date                                                           \n",
              "2025-08-19 -0.012457 -0.001213         0.004926     0.004431   \n",
              "2025-08-20 -0.003994  0.012966         0.000518    -0.002631   \n",
              "2025-08-21  0.001681  0.006386         0.005551     0.000366   \n",
              "2025-08-22  0.030332 -0.005454         0.001132     0.002571   \n",
              "2025-08-25 -0.007501  0.009374         0.002649    -0.000985   \n",
              "\n",
              "            Consumer Cyclicals  Consumer Non-Cyclicals  Financials  \\\n",
              "date                                                                 \n",
              "2025-08-19            0.008365                0.003666   -0.002158   \n",
              "2025-08-20           -0.004921                0.006301    0.001206   \n",
              "2025-08-21           -0.001721               -0.001779   -0.001929   \n",
              "2025-08-22            0.006121               -0.006245    0.003781   \n",
              "2025-08-25            0.001371               -0.005463    0.001370   \n",
              "\n",
              "            Healthcare  Technology  Utilities  ...  Asia  Europe  Oceania  \\\n",
              "date                                           ...                          \n",
              "2025-08-19    0.003093   -0.004364   0.003283  ...   0.0     0.0      0.0   \n",
              "2025-08-20    0.008118   -0.002558   0.000043  ...   0.0     0.0      0.0   \n",
              "2025-08-21    0.003653    0.000378  -0.002681  ...   0.0     0.0      0.0   \n",
              "2025-08-22   -0.010345   -0.000635  -0.004322  ...   0.0     0.0      0.0   \n",
              "2025-08-25   -0.008411    0.001051  -0.004592  ...   0.0     0.0      0.0   \n",
              "\n",
              "                Size     Value    Growth  Volatility  Momentum  Dividend  \\\n",
              "date                                                                       \n",
              "2025-08-19  0.001825  0.000976  0.001007   -0.014873 -0.007365 -0.000221   \n",
              "2025-08-20  0.000089 -0.000620 -0.000318   -0.002513  0.004078 -0.000022   \n",
              "2025-08-21 -0.001460  0.000473 -0.001100    0.000664  0.000814  0.000266   \n",
              "2025-08-22  0.000946  0.006080 -0.000289    0.014323 -0.005876 -0.001676   \n",
              "2025-08-25  0.000423  0.001079 -0.000251    0.000190  0.002282 -0.000347   \n",
              "\n",
              "            Leverage  \n",
              "date                  \n",
              "2025-08-19  0.004215  \n",
              "2025-08-20  0.000370  \n",
              "2025-08-21 -0.000823  \n",
              "2025-08-22  0.000784  \n",
              "2025-08-25  0.000468  \n",
              "\n",
              "[5 rows x 26 columns]"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "risk_model = bln.equity.riskmodels.load(factorriskmodel_settings).get_model()\n",
        "df_factor_returns = risk_model.fret().to_pandas().set_index(\"date\").rename(columns=lambda c: c.split(\".\")[1])\n",
        "df_factor_returns.tail()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "From these returns we can run standard pandas functions to get the EWMAs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>...</th>\n",
              "      <th>Asia</th>\n",
              "      <th>Europe</th>\n",
              "      <th>Oceania</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Leverage</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2025-08-19</th>\n",
              "      <td>0.174342</td>\n",
              "      <td>0.161448</td>\n",
              "      <td>0.093997</td>\n",
              "      <td>0.052590</td>\n",
              "      <td>0.076704</td>\n",
              "      <td>0.084888</td>\n",
              "      <td>0.054344</td>\n",
              "      <td>0.114819</td>\n",
              "      <td>0.047862</td>\n",
              "      <td>0.112037</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.031732</td>\n",
              "      <td>0.030392</td>\n",
              "      <td>0.016548</td>\n",
              "      <td>0.110538</td>\n",
              "      <td>0.059152</td>\n",
              "      <td>0.022407</td>\n",
              "      <td>0.024967</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-20</th>\n",
              "      <td>0.173077</td>\n",
              "      <td>0.162287</td>\n",
              "      <td>0.093217</td>\n",
              "      <td>0.052428</td>\n",
              "      <td>0.076728</td>\n",
              "      <td>0.085163</td>\n",
              "      <td>0.053946</td>\n",
              "      <td>0.115067</td>\n",
              "      <td>0.047750</td>\n",
              "      <td>0.111100</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.031467</td>\n",
              "      <td>0.030165</td>\n",
              "      <td>0.016422</td>\n",
              "      <td>0.109734</td>\n",
              "      <td>0.059249</td>\n",
              "      <td>0.022219</td>\n",
              "      <td>0.024770</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-21</th>\n",
              "      <td>0.171665</td>\n",
              "      <td>0.161461</td>\n",
              "      <td>0.093135</td>\n",
              "      <td>0.051995</td>\n",
              "      <td>0.076168</td>\n",
              "      <td>0.084529</td>\n",
              "      <td>0.053641</td>\n",
              "      <td>0.114350</td>\n",
              "      <td>0.047357</td>\n",
              "      <td>0.110308</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.031347</td>\n",
              "      <td>0.029928</td>\n",
              "      <td>0.016440</td>\n",
              "      <td>0.108825</td>\n",
              "      <td>0.058777</td>\n",
              "      <td>0.022040</td>\n",
              "      <td>0.024621</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-22</th>\n",
              "      <td>0.181212</td>\n",
              "      <td>0.160500</td>\n",
              "      <td>0.092386</td>\n",
              "      <td>0.051829</td>\n",
              "      <td>0.076564</td>\n",
              "      <td>0.084793</td>\n",
              "      <td>0.053753</td>\n",
              "      <td>0.115357</td>\n",
              "      <td>0.046980</td>\n",
              "      <td>0.109743</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.031145</td>\n",
              "      <td>0.032185</td>\n",
              "      <td>0.016314</td>\n",
              "      <td>0.111832</td>\n",
              "      <td>0.059516</td>\n",
              "      <td>0.022124</td>\n",
              "      <td>0.024468</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2025-08-25</th>\n",
              "      <td>0.180353</td>\n",
              "      <td>0.160313</td>\n",
              "      <td>0.091774</td>\n",
              "      <td>0.051436</td>\n",
              "      <td>0.075976</td>\n",
              "      <td>0.084825</td>\n",
              "      <td>0.053378</td>\n",
              "      <td>0.115683</td>\n",
              "      <td>0.046637</td>\n",
              "      <td>0.109232</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.030897</td>\n",
              "      <td>0.031992</td>\n",
              "      <td>0.016185</td>\n",
              "      <td>0.110898</td>\n",
              "      <td>0.059203</td>\n",
              "      <td>0.021951</td>\n",
              "      <td>0.024282</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 26 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "              Market    Energy  Basic Materials  Industrials  \\\n",
              "date                                                           \n",
              "2025-08-19  0.174342  0.161448         0.093997     0.052590   \n",
              "2025-08-20  0.173077  0.162287         0.093217     0.052428   \n",
              "2025-08-21  0.171665  0.161461         0.093135     0.051995   \n",
              "2025-08-22  0.181212  0.160500         0.092386     0.051829   \n",
              "2025-08-25  0.180353  0.160313         0.091774     0.051436   \n",
              "\n",
              "            Consumer Cyclicals  Consumer Non-Cyclicals  Financials  \\\n",
              "date                                                                 \n",
              "2025-08-19            0.076704                0.084888    0.054344   \n",
              "2025-08-20            0.076728                0.085163    0.053946   \n",
              "2025-08-21            0.076168                0.084529    0.053641   \n",
              "2025-08-22            0.076564                0.084793    0.053753   \n",
              "2025-08-25            0.075976                0.084825    0.053378   \n",
              "\n",
              "            Healthcare  Technology  Utilities  ...  Asia  Europe  Oceania  \\\n",
              "date                                           ...                          \n",
              "2025-08-19    0.114819    0.047862   0.112037  ...   0.0     0.0      0.0   \n",
              "2025-08-20    0.115067    0.047750   0.111100  ...   0.0     0.0      0.0   \n",
              "2025-08-21    0.114350    0.047357   0.110308  ...   0.0     0.0      0.0   \n",
              "2025-08-22    0.115357    0.046980   0.109743  ...   0.0     0.0      0.0   \n",
              "2025-08-25    0.115683    0.046637   0.109232  ...   0.0     0.0      0.0   \n",
              "\n",
              "                Size     Value    Growth  Volatility  Momentum  Dividend  \\\n",
              "date                                                                       \n",
              "2025-08-19  0.031732  0.030392  0.016548    0.110538  0.059152  0.022407   \n",
              "2025-08-20  0.031467  0.030165  0.016422    0.109734  0.059249  0.022219   \n",
              "2025-08-21  0.031347  0.029928  0.016440    0.108825  0.058777  0.022040   \n",
              "2025-08-22  0.031145  0.032185  0.016314    0.111832  0.059516  0.022124   \n",
              "2025-08-25  0.030897  0.031992  0.016185    0.110898  0.059203  0.021951   \n",
              "\n",
              "            Leverage  \n",
              "date                  \n",
              "2025-08-19  0.024967  \n",
              "2025-08-20  0.024770  \n",
              "2025-08-21  0.024621  \n",
              "2025-08-22  0.024468  \n",
              "2025-08-25  0.024282  \n",
              "\n",
              "[5 rows x 26 columns]"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate expected factor volatilities using the ewma\n",
        "df_vol_tieout = (\n",
        "    pd.DataFrame(df_factor_returns**2)\n",
        "    .ewm(halflife=report_settings.report.halflife_factor_vol)\n",
        "    .mean()\n",
        "    .astype(np.float32)\n",
        "    ** 0.5\n",
        "    * 252**0.5\n",
        ")\n",
        "df_vol_tieout.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {},
      "outputs": [],
      "source": [
        "pd.testing.assert_frame_equal(df_vol, df_vol_tieout, check_column_type=False, check_categorical=False, check_like=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The correlation are a bit more involved. We need to create the outer products and then run EWMAs on each cell in the outer product matrix."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>Market</th>\n",
              "      <th>Energy</th>\n",
              "      <th>Basic Materials</th>\n",
              "      <th>Industrials</th>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <th>Consumer Non-Cyclicals</th>\n",
              "      <th>Financials</th>\n",
              "      <th>Healthcare</th>\n",
              "      <th>Technology</th>\n",
              "      <th>Utilities</th>\n",
              "      <th>...</th>\n",
              "      <th>Asia</th>\n",
              "      <th>Europe</th>\n",
              "      <th>Oceania</th>\n",
              "      <th>Size</th>\n",
              "      <th>Value</th>\n",
              "      <th>Growth</th>\n",
              "      <th>Volatility</th>\n",
              "      <th>Momentum</th>\n",
              "      <th>Dividend</th>\n",
              "      <th>Leverage</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>date</th>\n",
              "      <th>factor</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2024-08-27</th>\n",
              "      <th>Market</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Energy</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Basic Materials</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Industrials</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Consumer Cyclicals</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th rowspan=\"5\" valign=\"top\">2025-08-25</th>\n",
              "      <th>Growth</th>\n",
              "      <td>0.027299</td>\n",
              "      <td>0.065028</td>\n",
              "      <td>-0.124795</td>\n",
              "      <td>0.025007</td>\n",
              "      <td>-0.155016</td>\n",
              "      <td>-0.188488</td>\n",
              "      <td>0.162799</td>\n",
              "      <td>-0.294809</td>\n",
              "      <td>0.233843</td>\n",
              "      <td>-0.122661</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.258081</td>\n",
              "      <td>-0.141794</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.076467</td>\n",
              "      <td>0.103423</td>\n",
              "      <td>-0.128167</td>\n",
              "      <td>-0.054107</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Volatility</th>\n",
              "      <td>0.856853</td>\n",
              "      <td>-0.065603</td>\n",
              "      <td>-0.194439</td>\n",
              "      <td>0.063828</td>\n",
              "      <td>-0.132216</td>\n",
              "      <td>-0.723050</td>\n",
              "      <td>0.671954</td>\n",
              "      <td>-0.423478</td>\n",
              "      <td>0.228824</td>\n",
              "      <td>-0.390604</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.500156</td>\n",
              "      <td>0.182544</td>\n",
              "      <td>0.076467</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.096767</td>\n",
              "      <td>-0.217297</td>\n",
              "      <td>-0.315350</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Momentum</th>\n",
              "      <td>-0.185332</td>\n",
              "      <td>0.069289</td>\n",
              "      <td>-0.262668</td>\n",
              "      <td>-0.183582</td>\n",
              "      <td>-0.514879</td>\n",
              "      <td>-0.053805</td>\n",
              "      <td>0.118316</td>\n",
              "      <td>-0.143826</td>\n",
              "      <td>0.253623</td>\n",
              "      <td>0.350147</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>-0.098723</td>\n",
              "      <td>-0.546126</td>\n",
              "      <td>0.103423</td>\n",
              "      <td>0.096767</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.248990</td>\n",
              "      <td>-0.369793</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Dividend</th>\n",
              "      <td>-0.114013</td>\n",
              "      <td>0.039374</td>\n",
              "      <td>0.133062</td>\n",
              "      <td>-0.057912</td>\n",
              "      <td>0.062695</td>\n",
              "      <td>0.237979</td>\n",
              "      <td>-0.235805</td>\n",
              "      <td>0.240609</td>\n",
              "      <td>-0.138234</td>\n",
              "      <td>0.109573</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>-0.182794</td>\n",
              "      <td>0.116952</td>\n",
              "      <td>-0.128167</td>\n",
              "      <td>-0.217297</td>\n",
              "      <td>-0.248990</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.264753</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Leverage</th>\n",
              "      <td>-0.103686</td>\n",
              "      <td>-0.003340</td>\n",
              "      <td>0.200817</td>\n",
              "      <td>0.281899</td>\n",
              "      <td>0.444195</td>\n",
              "      <td>0.313135</td>\n",
              "      <td>-0.027029</td>\n",
              "      <td>0.122807</td>\n",
              "      <td>-0.479819</td>\n",
              "      <td>-0.006972</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.069087</td>\n",
              "      <td>0.358233</td>\n",
              "      <td>-0.054107</td>\n",
              "      <td>-0.315350</td>\n",
              "      <td>-0.369793</td>\n",
              "      <td>0.264753</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>6474 rows × 26 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                 Market    Energy  Basic Materials  \\\n",
              "date       factor                                                    \n",
              "2024-08-27 Market              1.000000  1.000000         1.000000   \n",
              "           Energy              1.000000  1.000000         1.000000   \n",
              "           Basic Materials     1.000000  1.000000         1.000000   \n",
              "           Industrials         1.000000  1.000000         1.000000   \n",
              "           Consumer Cyclicals  1.000000  1.000000         1.000000   \n",
              "...                                 ...       ...              ...   \n",
              "2025-08-25 Growth              0.027299  0.065028        -0.124795   \n",
              "           Volatility          0.856853 -0.065603        -0.194439   \n",
              "           Momentum           -0.185332  0.069289        -0.262668   \n",
              "           Dividend           -0.114013  0.039374         0.133062   \n",
              "           Leverage           -0.103686 -0.003340         0.200817   \n",
              "\n",
              "                               Industrials  Consumer Cyclicals  \\\n",
              "date       factor                                                \n",
              "2024-08-27 Market                 1.000000            1.000000   \n",
              "           Energy                 1.000000            1.000000   \n",
              "           Basic Materials        1.000000            1.000000   \n",
              "           Industrials            1.000000            1.000000   \n",
              "           Consumer Cyclicals     1.000000            1.000000   \n",
              "...                                    ...                 ...   \n",
              "2025-08-25 Growth                 0.025007           -0.155016   \n",
              "           Volatility             0.063828           -0.132216   \n",
              "           Momentum              -0.183582           -0.514879   \n",
              "           Dividend              -0.057912            0.062695   \n",
              "           Leverage               0.281899            0.444195   \n",
              "\n",
              "                               Consumer Non-Cyclicals  Financials  Healthcare  \\\n",
              "date       factor                                                               \n",
              "2024-08-27 Market                            1.000000   -1.000000    1.000000   \n",
              "           Energy                            1.000000   -1.000000    1.000000   \n",
              "           Basic Materials                   1.000000   -1.000000    1.000000   \n",
              "           Industrials                       1.000000   -1.000000    1.000000   \n",
              "           Consumer Cyclicals                1.000000   -1.000000    1.000000   \n",
              "...                                               ...         ...         ...   \n",
              "2025-08-25 Growth                           -0.188488    0.162799   -0.294809   \n",
              "           Volatility                       -0.723050    0.671954   -0.423478   \n",
              "           Momentum                         -0.053805    0.118316   -0.143826   \n",
              "           Dividend                          0.237979   -0.235805    0.240609   \n",
              "           Leverage                          0.313135   -0.027029    0.122807   \n",
              "\n",
              "                               Technology  Utilities  ...  Asia  Europe  \\\n",
              "date       factor                                     ...                 \n",
              "2024-08-27 Market               -1.000000   1.000000  ...   NaN     NaN   \n",
              "           Energy               -1.000000   1.000000  ...   NaN     NaN   \n",
              "           Basic Materials      -1.000000   1.000000  ...   NaN     NaN   \n",
              "           Industrials          -1.000000   1.000000  ...   NaN     NaN   \n",
              "           Consumer Cyclicals   -1.000000   1.000000  ...   NaN     NaN   \n",
              "...                                   ...        ...  ...   ...     ...   \n",
              "2025-08-25 Growth                0.233843  -0.122661  ...   NaN     NaN   \n",
              "           Volatility            0.228824  -0.390604  ...   NaN     NaN   \n",
              "           Momentum              0.253623   0.350147  ...   NaN     NaN   \n",
              "           Dividend             -0.138234   0.109573  ...   NaN     NaN   \n",
              "           Leverage             -0.479819  -0.006972  ...   NaN     NaN   \n",
              "\n",
              "                               Oceania      Size     Value    Growth  \\\n",
              "date       factor                                                      \n",
              "2024-08-27 Market                  NaN -1.000000  1.000000 -1.000000   \n",
              "           Energy                  NaN -1.000000  1.000000 -1.000000   \n",
              "           Basic Materials         NaN -1.000000  1.000000 -1.000000   \n",
              "           Industrials             NaN -1.000000  1.000000 -1.000000   \n",
              "           Consumer Cyclicals      NaN -1.000000  1.000000 -1.000000   \n",
              "...                                ...       ...       ...       ...   \n",
              "2025-08-25 Growth                  NaN  0.258081 -0.141794  1.000000   \n",
              "           Volatility              NaN  0.500156  0.182544  0.076467   \n",
              "           Momentum                NaN -0.098723 -0.546126  0.103423   \n",
              "           Dividend                NaN -0.182794  0.116952 -0.128167   \n",
              "           Leverage                NaN  0.069087  0.358233 -0.054107   \n",
              "\n",
              "                               Volatility  Momentum  Dividend  Leverage  \n",
              "date       factor                                                        \n",
              "2024-08-27 Market                1.000000 -1.000000  1.000000 -1.000000  \n",
              "           Energy                1.000000 -1.000000  1.000000 -1.000000  \n",
              "           Basic Materials       1.000000 -1.000000  1.000000 -1.000000  \n",
              "           Industrials           1.000000 -1.000000  1.000000 -1.000000  \n",
              "           Consumer Cyclicals    1.000000 -1.000000  1.000000 -1.000000  \n",
              "...                                   ...       ...       ...       ...  \n",
              "2025-08-25 Growth                0.076467  0.103423 -0.128167 -0.054107  \n",
              "           Volatility            1.000000  0.096767 -0.217297 -0.315350  \n",
              "           Momentum              0.096767  1.000000 -0.248990 -0.369793  \n",
              "           Dividend             -0.217297 -0.248990  1.000000  0.264753  \n",
              "           Leverage             -0.315350 -0.369793  0.264753  1.000000  \n",
              "\n",
              "[6474 rows x 26 columns]"
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# calculate the ewma on the outer product (vcov with mean zero)\n",
        "df_factor_returns_outer = df_factor_returns.groupby(\"date\").apply(\n",
        "    lambda df: pd.DataFrame(np.outer(df, df), df.columns, df.columns)\n",
        ")\n",
        "df_factor_returns_outer.index.names = [\"date\", \"factor\"]\n",
        "df_cor_tieout = (\n",
        "    pd.DataFrame(df_factor_returns_outer)\n",
        "    .unstack()\n",
        "    .ewm(halflife=report_settings.report.halflife_factor_cor)\n",
        "    .mean()\n",
        "    .stack(future_stack=True)\n",
        "    .reindex(df_factor_returns_outer.columns, axis=1)\n",
        "    .groupby(\"date\")\n",
        "    .apply(\n",
        "        lambda df: df.droplevel(\"date\")\n",
        "        / np.outer(np.diag(df) ** 0.5, np.diag(df) ** 0.5)\n",
        "    )\n",
        "    .astype(np.float32)\n",
        ")\n",
        "\n",
        "df_cor_tieout"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {},
      "outputs": [],
      "source": [
        "pd.testing.assert_frame_equal(df_cor, df_cor_tieout, check_index_type=False, check_categorical=False, check_like=True, atol=1e-6)"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": ".venv",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.14"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}