Data Requirements
This section explains the data requirements for
TimeGPT
.
TimeGPT
accepts pandas
and polars
dataframes in long format with the following necessary columns:
ds
(timestamp): timestamp in formatYYYY-MM-DD
orYYYY-MM-DD HH:MM:SS
.y
(numeric): The target variable to forecast.
(Optionally, you can also pass a DataFrame without the ds
column as long as it has DatetimeIndex)
TimeGPT
also works with distributed dataframes like dask
, spark
and ray
.
You can also include exogenous features in the DataFrame as additional columns. For more information, follow this tutorial.
Below is an example of a valid input dataframe for TimeGPT
.
import pandas as pd
df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/air_passengers.csv')
df.head()
timestamp | value | |
---|---|---|
0 | 1949-01-01 | 112 |
1 | 1949-02-01 | 118 |
2 | 1949-03-01 | 132 |
3 | 1949-04-01 | 129 |
4 | 1949-05-01 | 121 |
Note that in this example, the ds
column is named timestamp
and the y
column is named value
. You can either:
-
Rename the columns to
ds
andy
, respectively, or -
Keep the current column names and specify them when using any method from the
NixtlaClient
class with thetime_col
andtarget_col
arguments.
For example, when using the forecast
method from the NixtlaClient
class, you must instantiate the class and then specify the columns names as follows.
from nixtla import NixtlaClient
nixtla_client = NixtlaClient(
api_key = 'my_api_key_provided_by_nixtla'
)
fcst = nixtla_client.forecast(df=df, h=12, time_col='timestamp', target_col='value')
fcst.head()
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Inferred freq: MS
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Querying model metadata...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
timestamp | TimeGPT | |
---|---|---|
0 | 1961-01-01 | 437.83792 |
1 | 1961-02-01 | 426.06270 |
2 | 1961-03-01 | 463.11655 |
3 | 1961-04-01 | 478.24450 |
4 | 1961-05-01 | 505.64648 |
In this example, the NixtlaClient
is infereing the frequency, but you can explicitly specify it with the freq
argument.
To learn more about how to instantiate the NixtlaClient
class, refer to the TimeGPT Quickstart
Multiple Series
If youโre working with multiple time series, make sure that each series has a unique identifier. You can name this column unique_id
or specify its name using the id_col
argument when calling any method from the NixtlaClient
class. This column should be a string, integer, or category.
In this example, we have five series representing hourly electricity prices in five different markets. The columns already have the default names, so itโs unnecessary to specify the id_col
, time_col
, or target_col
arguments. If your columns have different names, specify these arguments as required.
df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/electricity-short.csv')
df.head()
unique_id | ds | y | |
---|---|---|---|
0 | BE | 2016-10-22 00:00:00 | 70.00 |
1 | BE | 2016-10-22 01:00:00 | 37.10 |
2 | BE | 2016-10-22 02:00:00 | 37.10 |
3 | BE | 2016-10-22 03:00:00 | 44.75 |
4 | BE | 2016-10-22 04:00:00 | 37.10 |
fcst = nixtla_client.forecast(df=df, h=24) # use id_col, time_col and target_col here if needed.
fcst.head()
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Inferred freq: h
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Querying model metadata...
INFO:nixtla.nixtla_client:Restricting input...
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
unique_id | ds | TimeGPT | |
---|---|---|---|
0 | BE | 2016-12-31 00:00:00 | 45.190582 |
1 | BE | 2016-12-31 01:00:00 | 43.244987 |
2 | BE | 2016-12-31 02:00:00 | 41.958897 |
3 | BE | 2016-12-31 03:00:00 | 39.796680 |
4 | BE | 2016-12-31 04:00:00 | 39.204865 |
When working with a large number of time series, consider using a distributed computing framework to handle the data efficiently. TimeGPT
supports frameworks such as Spark, Dask, and Ray.
Exogenous Variables
TimeGPT
also accepts exogenous variables. You can add exogenous variables to your dataframe by including additional columns after the y
column.
df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/electricity-short-with-ex-vars.csv')
df.head()
unique_id | ds | y | Exogenous1 | Exogenous2 | day_0 | day_1 | day_2 | day_3 | day_4 | day_5 | day_6 | |
---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | BE | 2016-10-22 00:00:00 | 70.00 | 57253.0 | 49593.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 |
1 | BE | 2016-10-22 01:00:00 | 37.10 | 51887.0 | 46073.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 |
2 | BE | 2016-10-22 02:00:00 | 37.10 | 51896.0 | 44927.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 |
3 | BE | 2016-10-22 03:00:00 | 44.75 | 48428.0 | 44483.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 |
4 | BE | 2016-10-22 04:00:00 | 37.10 | 46721.0 | 44338.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 |
When using exogenous variables, you also need to provide its future values.
future_ex_vars_df = pd.read_csv('https://raw.githubusercontent.com/Nixtla/transfer-learning-time-series/main/datasets/electricity-short-future-ex-vars.csv')
future_ex_vars_df.head()
unique_id | ds | Exogenous1 | Exogenous2 | day_0 | day_1 | day_2 | day_3 | day_4 | day_5 | day_6 | |
---|---|---|---|---|---|---|---|---|---|---|---|
0 | BE | 2016-12-31 00:00:00 | 70318.0 | 64108.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 |
1 | BE | 2016-12-31 01:00:00 | 67898.0 | 62492.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 |
2 | BE | 2016-12-31 02:00:00 | 68379.0 | 61571.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 |
3 | BE | 2016-12-31 03:00:00 | 64972.0 | 60381.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 |
4 | BE | 2016-12-31 04:00:00 | 62900.0 | 60298.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 |
fcst = nixtla_client.forecast(df=df, X_df=future_ex_vars_df, h=24)
fcst.head()
INFO:nixtla.nixtla_client:Validating inputs...
INFO:nixtla.nixtla_client:Inferred freq: h
INFO:nixtla.nixtla_client:Preprocessing dataframes...
INFO:nixtla.nixtla_client:Using future exogenous features: ['Exogenous1', 'Exogenous2', 'day_0', 'day_1', 'day_2', 'day_3', 'day_4', 'day_5', 'day_6']
INFO:nixtla.nixtla_client:Calling Forecast Endpoint...
unique_id | ds | TimeGPT | |
---|---|---|---|
0 | BE | 2016-12-31 00:00:00 | 51.632830 |
1 | BE | 2016-12-31 01:00:00 | 45.750877 |
2 | BE | 2016-12-31 02:00:00 | 39.650543 |
3 | BE | 2016-12-31 03:00:00 | 34.000072 |
4 | BE | 2016-12-31 04:00:00 | 33.785370 |
To learn more about how to use exogenous variables with TimeGPT
, consult the Exogenous Variables tutorial.
Important Considerations
When using TimeGPT
, the data cannot contain missing values. This means that for every series, there should be no gaps in the timestamps and no missing values in the target variable.
For more, please refer to the tutorial on Dealing with Missing Values in TimeGPT.
Minimum Data Requirements (for AzureAI)
TimeGPT
currently supports any amount of data for generating point forecasts. That is, the minimum size per series to expect results from this call nixtla_client.forecast(df=df, h=h, freq=freq)
is one, regardless of the frequency.
For Azure AI, when using the arguments level
, finetune_steps
, X_df
(exogenous variables), or add_history
, the API requires a minimum number of data points depending on the frequency. Here are the minimum sizes for each frequency:
Frequency | Minimum Size |
---|---|
Hourly and subhourly (e.g., โHโ, โminโ, โ15Tโ) | 1008 |
Daily (โDโ) | 300 |
Weekly (e.g., โW-MONโ,โฆ, โW-SUNโ) | 64 |
Monthly and other frequencies (e.g., โMโ, โMSโ, โYโ) | 48 |
For cross-validation, you need to consider these numbers as well as the forecast horizon (h
), the number of windows (n_windows
), and the gap between windows (step_size
). Thus, the minimum number of observations per series in this case would be determined by the following relationship:
Minimum number described previously + h + step_size + (n_windows - 1)
Updated 19 days ago