Skip to content

feat(ai): add forecast_dataframe and impute_dataframe - #2876

Open
miladh wants to merge 3 commits into
ai-ts-forecasting/03-low-level-apifrom
ai-ts-forecasting/04-dataframes
Open

miladh wants to merge 3 commits into
ai-ts-forecasting/03-low-level-apifrom
ai-ts-forecasting/04-dataframes

Conversation

@miladh

@miladh miladh commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #2875.

Description

DataFrame variants of forecast and impute, following the insert_dataframe / retrieve_dataframe convention:

  • Each numeric column is one time series, labelled by its column name.
  • NaN values are marked missing: hidden from the model, and for impute_dataframe, predicted.
  • cohort= takes one cohort for every column, or a {column: cohort} mapping. Columns not in the mapping are processed independently.
  • quantile_levels selects the quantile levels in the result: the median (0.5) by default, a list of levels, or None for every level the API returns.
  • The result has a DatetimeIndex and a (column, quantile level) column for every column and included quantile level. It keeps the input's column names and timezone; a naive index is taken to be in UTC, as in insert_dataframe.
  • A DataFrame without a DatetimeIndex, with repeated or non-numeric columns or infinite values, or a cohort mapping that names a column not in the DataFrame, is rejected before anything is sent.

Usage

import numpy as np
import pandas as pd
from cognite.client import CogniteClient

client = CogniteClient()
df = pd.DataFrame(
    {
        "23-PT-1101": [42.1, 42.4, 42.9],
        "23-PT-1201": [39.8, 40.1, 40.0],
        "24-TT-3001": [31.0, np.nan, 30.9],   # NaN is marked missing
    },
    index=pd.date_range("2026-10-01", periods=3, freq="1min"),
)

# Every column independently
forecast = client.ai.time_series.forecast_dataframe(df)

# The two compressor pressures jointly; 24-TT-3001 stays independent
forecast = client.ai.time_series.forecast_dataframe(
    df, cohort={"23-PT-1101": "compression-train-a", "23-PT-1201": "compression-train-a"}
)
forecast["23-PT-1101"][0.5]   # the median forecast, the default
all_levels = client.ai.time_series.forecast_dataframe(df, quantile_levels=None)

# Predict the NaNs, and fill them with the predicted median
imputed = client.ai.time_series.impute_dataframe(df)
filled = df.fillna(imputed.xs(0.5, axis=1, level=1))

pandas is an optional extra, so the new tests are marked dsl. The docstrings note that the endpoints are in beta. The ai.time_series docstring test moves to its own dsl-marked test, like datapoints and raw.

Checklist:

  • Tests added/updated: TestDataFrames in test_api/test_ai_time_series.py.
  • Documentation updated: docstrings.
  • The PR title follows the Conventional Commit spec.

🤖 Generated with Claude Code

@miladh
miladh force-pushed the ai-ts-forecasting/04-dataframes branch from bc78e48 to 8e32224 Compare November 2, 2026 11:15
@miladh
miladh force-pushed the ai-ts-forecasting/03-low-level-api branch from 9d62a18 to 1bb7339 Compare November 2, 2026 11:15
@miladh
miladh marked this pull request as ready for review November 2, 2026 11:23
@miladh
miladh requested review from a team as code owners November 2, 2026 11:23

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces forecast_dataframe and impute_dataframe methods to both the async and sync AI Time Series APIs, allowing users to forecast and reconstruct missing values directly from pandas DataFrames. Feedback on the implementation of the _dataframe_to_input helper function highlights the need for improved robustness, specifically regarding DatetimeIndex validation, timezone-aware index handling, and validating that all columns are numeric to prevent cryptic runtime errors.

Comment thread cognite/client/_api/ai/time_series.py Resolved
@miladh
miladh force-pushed the ai-ts-forecasting/03-low-level-api branch from 1bb7339 to 0fd9b72 Compare November 4, 2026 21:41
@miladh
miladh force-pushed the ai-ts-forecasting/04-dataframes branch from 8802d44 to bf68252 Compare November 4, 2026 21:50
@miladh
miladh force-pushed the ai-ts-forecasting/03-low-level-api branch from 0fd9b72 to c20d0a9 Compare November 5, 2026 09:23
@miladh
miladh force-pushed the ai-ts-forecasting/04-dataframes branch from bf68252 to 9118e3c Compare November 5, 2026 09:23
@miladh
miladh force-pushed the ai-ts-forecasting/03-low-level-api branch from c20d0a9 to 71a6259 Compare November 6, 2026 11:40
@miladh
miladh force-pushed the ai-ts-forecasting/04-dataframes branch from 9118e3c to 567f58e Compare November 6, 2026 11:40
miladh and others added 3 commits November 6, 2026 11:46
DataFrame versions of forecast and impute: each numeric column is one
time series, labelled by its column name, and NaN values are marked
missing, so impute predicts them. cohort is one cohort for every column,
or a mapping from column name to cohort.

The result has a (column, quantile level) column for every column, with the
median as the only level, like to_pandas, and the input's column names and
timezone; a naive index is taken to be in UTC, as in insert_dataframe. A
DataFrame without a DatetimeIndex, with repeated or non-numeric columns or
infinite values, or a cohort mapping that names a column not in the
DataFrame, is rejected before anything is sent. The docstrings note that
the endpoints are in beta.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Like to_pandas, they return the median only by default. quantile_levels
selects other levels: a list of levels, or None for every level the API
returns. It is passed on to to_pandas.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…aframe

Forecasts a DataFrame with the median and with every quantile level, and
imputes its NaN values: the result has one row per NaN timestamp, and
filling the gaps with the median leaves no NaN.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@miladh
miladh force-pushed the ai-ts-forecasting/04-dataframes branch from 567f58e to d0fb961 Compare November 6, 2026 11:48

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant