Portfolios#

Prerequisites

  • Uploaders API Tutorial

In this tutorial we are going to demonstrate the usage of the Bayesline Portfolios API. The Portfolios API provides a unified access mechanism to bring portfolio holdings data into the system such that it can be used downstream (e.g. for risk analytics).

Specifically, we will introduce and explore:

  • Portfolio Sources

  • Listing existing portfolio sources

  • Uploading new portfolio data (adding to a source, creating a new source)

  • Reading portfolio data

  • Forward filling holdings data with drift correction

  • Funds of funds structures

  • Portfolio Schemas (advanced)

Imports & Setup#

For this tutorial notebook, you will need to import the following packages.

import datetime as dt

import polars as pl

from bayesline.apiclient import BayeslineApiClient

from bayesline.api.equity import (
    PortfolioSettings,
    PortfolioOrganizerSettings,
)

We will also need to have a Bayesline API client configured.

bln = BayeslineApiClient.new_client(
    endpoint="https://[ENDPOINT]",
    api_key="[API-KEY]",
)

The main entrypoint for the Portfolios API sits on bln.equity.portfolios. All portfolios functionality can be reached from here on out.

See here for relevant docs:

portfolios_loader = bln.equity.portfolios

Portfolio Sources#

A portfolio source is an isolated dataset that contains holdings information for different portfolios. This could be a system source (e.g. from a database) or user uploaded data.

For each source it is guaranteed that the data is free of duplications and otherwise consistent.

Listing Available Portfolio Sources#

Below demonstrates how to obtain the list of available sources using the settings menu.

portfolios_loader.settings.available_settings(dataset_name="bayesline/Bayesline-US-All-1y")
PortfolioSettingsMenu(sources=[], schemas=[])

Uploading a Portfolio Source#

A new portfolio source can be added using the portfolios uploader, which can be obtained through the uploader property (note that this is a shortcut for using bln.equity.uploaders.get_data_type("portfolios"), which yields the same uploader).

Given above showed no existing portfolio sources we should find that the uploader has no datasets.

uploader = portfolios_loader.uploader
uploader.get_datasets()
[]

Next let’s create a new portfolios dataset and upload some sample data. For a more detailed walk through on the uploader infrastructure (including parsers, example inputs, versioning, etc.) see the Bayesline Uploaders Tutorial.

demo_portfolio_dataset = uploader.get_or_create_dataset("demo-portfolios")
df = pl.DataFrame({
    "portfolio_id": [
        "Test-Portfolio", "Test-Portfolio", "Test-Portfolio",
        "Test-Portfolio-2", "Test-Portfolio-2", "Test-Portfolio-2",
    ],
    "asset_id": [
        "02079K305", "02079K305", "2588173",
        "85371710", "85371710", "85371710"
    ],
    "asset_id_type": [
        "cusip9", "cusip9", "sedol7",
        "cusip8", "cusip8", "cusip8"
    ],
    "date": [
        dt.date(2026, 1, 1), dt.date(2026, 1, 31), dt.date(2026, 1, 15),
        dt.date(2026, 1, 1), dt.date(2026, 1, 13), dt.date(2026, 2, 15),
    ],
    "currency": [None]*6,
    "share_qty": [None]*6,
    "nav": [
        100, 110, 200,
        50, 55, 54
    ]
}).with_columns(pl.col("currency").cast(pl.String), pl.col("share_qty").cast(pl.Float64))

df
shape: (6, 7)
portfolio_idasset_idasset_id_typedatecurrencyshare_qtynav
strstrstrdatestrf64i64
"Test-Portfolio""02079K305""cusip9"2026-01-01nullnull100
"Test-Portfolio""02079K305""cusip9"2026-01-31nullnull110
"Test-Portfolio""2588173""sedol7"2026-01-15nullnull200
"Test-Portfolio-2""85371710""cusip8"2026-01-01nullnull50
"Test-Portfolio-2""85371710""cusip8"2026-01-13nullnull55
"Test-Portfolio-2""85371710""cusip8"2026-02-15nullnull54
demo_portfolio_dataset.fast_commit(df, mode="append")
UploadCommitResult(version=1, committed_names=[])

Reading Portfolio Data#

Having uploaded a portfolio source we can now use the portfolio loader to obtain the portfolio.

portfolios_loader.settings.available_settings(dataset_name="bayesline/Bayesline-US-All-1y")
PortfolioSettingsMenu(sources=['demo-portfolios'], schemas=[])
portfolios_api = portfolios_loader.load(
    PortfolioSettings.from_source(
        "demo-portfolios"
    ).with_dataset("bayesline/Bayesline-US-All-1y")
)

Why use a separate Portfolios API to obtain this data, as opposed to just using the uploader infrastructure to read the data back? The Portfolios API adds plenty of functionality that is specific to the domain of portfolios, e.g. coverage statistics, forward filling and drifting, etc.

Below demonstrates this functionality.

portfolios_api.get_portfolio_names()
['Test-Portfolio', 'Test-Portfolio-2']
portfolios_api.get_dates()
{'Test-Portfolio': [datetime.date(2026, 1, 1),
  datetime.date(2026, 1, 15),
  datetime.date(2026, 1, 31)],
 'Test-Portfolio-2': [datetime.date(2026, 1, 1),
  datetime.date(2026, 1, 13),
  datetime.date(2026, 2, 15)]}
portfolios_api.get_coverage()
shape: (180, 6)
portfolio_groupportfolio_iddateasset_id_typeinputbayesid
strstrdatestru32u32
"demo-portfolios""Test-Portfolio"2026-01-01"cusip9"11
"demo-portfolios""Test-Portfolio"2026-01-02"cusip9"11
"demo-portfolios""Test-Portfolio"2026-01-03"cusip9"11
"demo-portfolios""Test-Portfolio"2026-01-04"cusip9"11
"demo-portfolios""Test-Portfolio"2026-01-05"cusip9"11
"demo-portfolios""Test-Portfolio-2"2026-03-27"cusip8"10
"demo-portfolios""Test-Portfolio-2"2026-03-28"cusip8"10
"demo-portfolios""Test-Portfolio-2"2026-03-29"cusip8"10
"demo-portfolios""Test-Portfolio-2"2026-03-30"cusip8"10
"demo-portfolios""Test-Portfolio-2"2026-03-31"cusip8"10
portfolios_api.get_portfolio(names=["Test-Portfolio"])
shape: (61, 10)
dateportfolio_groupportfolio_idinput_asset_id_typeinput_asset_idasset_id_typeasset_idcurrencyshare_qtynav
datestrstrstrstrstrstrstrf32f32
2026-01-02"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489100.686859
2026-01-05"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489101.131004
2026-01-06"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489100.428162
2026-01-07"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489102.869011
2026-01-08"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489103.974426
2026-03-25"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566794.746223
2026-03-26"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566791.486298
2026-03-27"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566789.343475
2026-03-30"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566789.069862
2026-03-31"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566793.648773

The input identifier columns retain the uploaded IDs, while asset_id_type and asset_id contain the resolved Bayesline identifier. Other output ID spaces are not currently supported.

portfolios_api.get_id_types()
{'Test-Portfolio': ['bayesid'], 'Test-Portfolio-2': ['bayesid']}
portfolios_api.get_portfolio(names=["Test-Portfolio"], id_type="bayesid")
shape: (61, 10)
dateportfolio_groupportfolio_idinput_asset_id_typeinput_asset_idasset_id_typeasset_idcurrencyshare_qtynav
datestrstrstrstrstrstrstrf32f32
2026-01-02"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489100.686859
2026-01-05"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489101.131004
2026-01-06"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489100.428162
2026-01-07"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489102.869011
2026-01-08"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489103.974426
2026-03-25"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566794.746223
2026-03-26"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566791.486298
2026-03-27"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566789.343475
2026-03-30"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566789.069862
2026-03-31"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566793.648773

Report-backed Holdings#

The demo portfolio data was uploaded monthly. Public holdings are always forward-filled across the report date axis, revalued with current market data, and fully looked through.

portfolios_api.get_portfolio(names=["Test-Portfolio"])
shape: (61, 10)
dateportfolio_groupportfolio_idinput_asset_id_typeinput_asset_idasset_id_typeasset_idcurrencyshare_qtynav
datestrstrstrstrstrstrstrf32f32
2026-01-02"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489100.686859
2026-01-05"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489101.131004
2026-01-06"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489100.428162
2026-01-07"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489102.869011
2026-01-08"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489103.974426
2026-03-25"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566794.746223
2026-03-26"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566791.486298
2026-03-27"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566789.343475
2026-03-30"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566789.069862
2026-03-31"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566793.648773

Funds of Funds#

We can represent fund of funds structures easily by using the portfolio-id as the respective asset ids and portfolio_id as the asset id type.

Below we’re uploading a fund of funds portfolios to our existing dataset by appending it to the existing data.

fof_portfolio_df = pl.DataFrame({
    "portfolio_id": [
        "My-FOF", "My-FOF", "My-FOF", "My-FOF"
    ],
    "asset_id": [
        "Test-Portfolio", "Test-Portfolio", "Test-Portfolio-2", "Test-Portfolio-2"
    ],
    "asset_id_type": [
        "portfolio_id", "portfolio_id", "portfolio_id", "portfolio_id"
    ],
    "date": [
        dt.date(2026, 1, 1), dt.date(2026, 1, 31), dt.date(2026, 1, 1), dt.date(2026, 1, 31)
    ],
    "currency": [None]*4,
    "share_qty": [None]*4,
    "nav": [
        .5, .55, .5, .45
    ]
}).with_columns(pl.col("currency").cast(pl.String), pl.col("share_qty").cast(pl.Float64))

fof_portfolio_df
shape: (4, 7)
portfolio_idasset_idasset_id_typedatecurrencyshare_qtynav
strstrstrdatestrf64f64
"My-FOF""Test-Portfolio""portfolio_id"2026-01-01nullnull0.5
"My-FOF""Test-Portfolio""portfolio_id"2026-01-31nullnull0.55
"My-FOF""Test-Portfolio-2""portfolio_id"2026-01-01nullnull0.5
"My-FOF""Test-Portfolio-2""portfolio_id"2026-01-31nullnull0.45
demo_portfolio_dataset.fast_commit(fof_portfolio_df, mode="append")
UploadCommitResult(version=2, committed_names=[])

Below we’re obtaining the fund of funds portfolio without any alterations.

portfolios_api = portfolios_loader.load(
    PortfolioSettings.from_source(
        "demo-portfolios"
    ).with_dataset("bayesline/Bayesline-US-All-1y")
)
portfolios_api.get_portfolio(names=["My-FOF"])
shape: (61, 10)
dateportfolio_groupportfolio_idinput_asset_id_typeinput_asset_idasset_id_typeasset_idcurrencyshare_qtynav
datestrstrstrstrstrstrstrf32f32
2026-01-02"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0015970.503434
2026-01-05"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0015970.505655
2026-01-06"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0015970.502141
2026-01-07"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0015970.514345
2026-01-08"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0015970.519872
2026-03-25"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0016280.473731
2026-03-26"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0016280.457432
2026-03-27"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0016280.446717
2026-03-30"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0016280.445349
2026-03-31"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0016280.468244

Fund-of-funds holdings are always looked through and forward-filled. USD is the default output currency, and another target currency can be requested explicitly.

portfolios_api = portfolios_loader.load(
    PortfolioSettings.from_source(
        "demo-portfolios"
    ).with_dataset("bayesline/Bayesline-US-All-1y")
)
portfolios_api.get_portfolio(names=["My-FOF"], currency="USD")
shape: (61, 10)
dateportfolio_groupportfolio_idinput_asset_id_typeinput_asset_idasset_id_typeasset_idcurrencyshare_qtynav
datestrstrstrstrstrstrstrf32f32
2026-01-02"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0015970.503434
2026-01-05"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0015970.505655
2026-01-06"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0015970.502141
2026-01-07"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0015970.514345
2026-01-08"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0015970.519872
2026-03-25"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0016280.473731
2026-03-26"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0016280.457432
2026-03-27"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0016280.446717
2026-03-30"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0016280.445349
2026-03-31"demo-portfolios""My-FOF""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.0016280.468244

Portfolio Schemas#

Portfolio Schemas are an advanced topic that sit on top of the Portfolio Sources we previously explored.

Using Portfolio Schemas we can cherry-pick which portfolios should be sourced from what underlying portfolio source. We do this by providing a mapping from portfolio id -> portfolio source. This flexibility allows for a powerful means to arbitrarily override portfolio holdings while keeping everything else the same, e.g. to perform what-if analyses.

Below we are creating a new what-if portfolio dataset where we override holdings for our Test-Portfolio-2. We then create a Portfolio Schema where we source Test-Portfolio-2 from our new dataset, keeping the rest the same.

what_if_dataset = uploader.create_dataset("what-if")
what_if_portfolio_df = pl.DataFrame({
    "portfolio_id": ["Test-Portfolio-2", "Test-Portfolio-2"],
    "asset_id": ["67066G10", "67066G10"],
    "asset_id_type": ["cusip8", "cusip8"],
    "date": [dt.date(2026, 1, 1), dt.date(2026, 1, 31)],
    "currency": [None]*2,
    "share_qty": [None]*2,
    "nav": [100, 125],
}).with_columns(pl.col("currency").cast(pl.String), pl.col("share_qty").cast(pl.Float64))

what_if_portfolio_df
shape: (2, 7)
portfolio_idasset_idasset_id_typedatecurrencyshare_qtynav
strstrstrdatestrf64i64
"Test-Portfolio-2""67066G10""cusip8"2026-01-01nullnull100
"Test-Portfolio-2""67066G10""cusip8"2026-01-31nullnull125
what_if_dataset.fast_commit(what_if_portfolio_df, mode="append")
UploadCommitResult(version=1, committed_names=[])
portfolios_loader.settings.available_settings(dataset_name="bayesline/Bayesline-US-All-1y")
PortfolioSettingsMenu(sources=['demo-portfolios', 'what-if'], schemas=[])
schema = PortfolioOrganizerSettings(
    enabled_portfolios={
        "Test-Portfolio": "demo-portfolios",
        "Test-Portfolio-2": "what-if"
    },
)

portfolios_api = portfolios_loader.load(
    PortfolioSettings(
        portfolio_schema=schema
    ).with_dataset("bayesline/Bayesline-US-All-1y")
)
portfolios_api.get_portfolio_names()
['Test-Portfolio', 'Test-Portfolio-2']

Note below how Test-Portfolio-2 has the updated values.

portfolios_api.get_portfolio(names=['Test-Portfolio', "Test-Portfolio-2"])
shape: (122, 10)
dateportfolio_groupportfolio_idinput_asset_id_typeinput_asset_idasset_id_typeasset_idcurrencyshare_qtynav
datestrstrstrstrstrstrstrf32f32
2026-01-02"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489100.686859
2026-01-02"what-if""Test-Portfolio-2""cusip8""67066G10""bayesid""IC46D464E0""USD"0.536193101.26004
2026-01-05"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489101.131004
2026-01-05"what-if""Test-Portfolio-2""cusip8""67066G10""bayesid""IC46D464E0""USD"0.536193100.868629
2026-01-06"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.319489100.428162
2026-03-27"what-if""Test-Portfolio-2""cusip8""67066G10""bayesid""IC46D464E0""USD"0.65404109.564827
2026-03-30"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566789.069862
2026-03-30"what-if""Test-Portfolio-2""cusip8""67066G10""bayesid""IC46D464E0""USD"0.65404108.027824
2026-03-31"demo-portfolios""Test-Portfolio""cusip9""02079K305""bayesid""ICA17F00B9""USD"0.32566793.648773
2026-03-31"what-if""Test-Portfolio-2""cusip8""67066G10""bayesid""IC46D464E0""USD"0.65404114.064613

Saving the Schema#

We can also save a portfolio schema for later use.

Below we demonstrate how to save the schema we previosly created and then use downstream.

portfolios_loader.organizer_settings.save("my-schema", schema.with_dataset("bayesline/Bayesline-US-All-1y"))
SaveItemResult(name_or_id='my-schema', status=<ItemStatus.success: 'success'>, message='Settings created successfully', name='my-schema', id=1, short_message='')
portfolios_api = portfolios_loader.load(
    PortfolioSettings(
        portfolio_schema="my-schema"
    ).with_dataset("bayesline/Bayesline-US-All-1y")
)

portfolios_api.get_portfolio_names()
['Test-Portfolio', 'Test-Portfolio-2']

Housekeeping#

Lastly, we clean up by deleting our portfolio datasets.

portfolios_loader.organizer_settings.delete("my-schema")
uploader.get_dataset("demo-portfolios").destroy()
uploader.get_dataset("what-if").destroy()