kats.models.metalearner.metalearner_hpt module

A module for meta-learner hyper-parameter selection.

This module contains two classes, including:

• MetaLearnHPT for recommending hyper-parameters of forecasting models;

• MultitaskNet for multi-task neural network built with pytorch.

class kats.models.metalearner.metalearner_hpt.MetaLearnHPT(data_x: Optional[pandas.core.frame.DataFrame] = None, data_y: Optional[pandas.core.frame.DataFrame] = None, categorical_idx: Optional[List[str]] = None, numerical_idx: Optional[List[str]] = None, default_model: Optional[str] = None, scale: bool = True, load_model: bool = False, **kwargs)

Bases: object

A class for meta-learner framework on hyper-parameters tuning.

MetaLearnHPT is a framework for choosing hyper-parameter for forecasting models. It uses a multi-task neural network for recommendation, with time series features as inputs, and the hyper-parameters of a given model as outputs. For training, it uses time series features as inputs and the corresponding best hyper-parameters as labels. For prediction, it takes time series or time series features to predict the best hyper-parameters. MetaLearnHPT provides get_default_model, build_network, train, pred, pred_by_feature, save_model, load_model and plot.

data_x

Optional; A pandas.DataFrame object of time series features. data_x should not be None unless load_model is True. Default is None.

data_y

Optional; A pandas.DataFrame object of the corresponding best hyper-parameters. data_y should not be None unless load_model is True. Default is None.

categorical_idx

Optional; A list of strings of the names of the categorical hyper-parameters. Default is None.

numerical_idx

Optional; A list of strings of the names of the numerical hyper-parameters. Default is None.

default_model

Optional; A string of the name of the forecast model whose default settings will be used. Can be ‘arima’, ‘sarima’, ‘theta’, ‘prophet’, ‘holtwinters’, ‘stlf’ or None. Default is None.

scale

Optional; A boolean to specify whether or not to normalize time series features to zero mean and unit variance. Default is True.

load_model

Optional; A boolean to specify whether or not to load a trained model. Default is False.

Sample Usage:

>>> mlhpt_hw = MetaLearnHPT(X, Y, default_model='holtwinters') # Use default Holt-Winter's model as an example.
>>> mlhpt_hw.build_network()
>>> mlhpt_hw.train()
>>> mlhpt_hw.pred(ts=TSdata) # Recommend hyper-parameters for TSdata.
>>> mlhpt_hw.save_model('my_model_binary.pkl') # Save trained model to a binary
>>> mlhpt_hw2=MetaLearnHPT(load_model=True) # Load a trained model
>>> mlhpt_hw2.load_model('my_model_binary.pkl')
>>> mlhpt_hw = MetaLearnHPT(X, Y_holtwinters, ['trend','damped', 'seasonal'], ['seasonal_periods']) # Example for building customized MetaLearnHPT object.
>>> mlhpt_hw.build_network(n_hidden_shared=[30], n_hidden_cat_combo=[[2], [3], [5]],n_hidden_num=[3])
>>> mlhpt_hw.train(loss_scale=30, lr=0.001)

build_network(n_hidden_shared: Optional[List[int]] = None, n_hidden_cat_combo: Optional[List[int]] = None, n_hidden_num: Optional[List[int]] = None) → None

Build a multi-task neural network.

This function builds a multi-task neural network according to given neural network structure (i.e., n_hidden_shared, n_hidden_cat_combo, n_hidden_num). If the MetaLearnHPT object is initiated as a default_model (i.e., default_model is not None), then a default neural network structure will be built and cannot accept customized.

Parameters

• n_hidden_shared – Optional; A list of numbers of hidden neurons in each shared hidden layer. For example, n_hidden_shared = [first_shared_hid_dim, second_shared_hid_dim, ….]. Default is None.

• n_hidden_cat_combo – Optional; A list of lists of task-specific hidden layers’ sizes of each categorical response variables. For example, if we have 3 categorical y, then n_hidden_cat_combo = [[first_spec_hid_dim_cat1, second_spec_hid_dim_cat1, …], [first_spec_hid_dim_cat2, second_spec_hid_dim_cat2, …], [first_spec_hid_dim_cat3, second_spec_hid_dim_cat3, …]]. Length of n_hidden_cat_combo must match the number of categorical y. Default is None.

• n_hidden_num – Optional; A list of task-specific hidden layers’ sizes of numerical response variables. For example, n_hidden_num = [first_spec_hid_dim, second_spec_hid_dim, …]. Default is None.

Returns

None.

get_default_model() → Optional[str]

Get the name of default_model. It the instance is a customized model, return None.

Returns

A string reprsenting the default model or None.

load_model(file_path: str) → None

Load a pre-trained model from a binary.

Parameters

file_path – A string representing the path to load a pre-trained model.

Returns

None.

plot()

Plot loss paths of classification/regression on both training and validation set.

pred(source_ts: kats.consts.TimeSeriesData, ts_scale: bool = True) → pandas.core.frame.DataFrame

Predict hyper-parameters for a new time series data.

Parameters

• source_ts – kats.consts.TimeSeriesData object representing the time series for which to generate hyper-parameters

• ts_scale – A boolean to specify whether or not to rescale time series data (i.e., divide its value by its maximum value) before calculating its features. Default is True.

Returns

A pandas.DataFrame object storing the recommended hyper-parameters.

pred_by_feature(source_x: Union[numpy.ndarray, List[numpy.ndarray], pandas.core.frame.DataFrame]) → List[Dict[str, Any]]

Predict hyper-parameters for time series features.

Parameters

source_x – Time series features.

Returns

A list of dictionaries storing the recommended hyper-parameters.

save_model(file_path: str) → None

Save trained model to a binary.

Parameters

file_path – A string representing the path to save a trained model, which should contain either ‘.p’ or ‘.pkl’ file extension.

Returns

None.

train(loss_scale: float = 1.0, lr: float = 0.001, n_epochs: int = 1000, batch_size: int = 128, method: str = 'SGD', val_size: float = 0.1, momentum: float = 0.9, n_epochs_stop: Union[int, float] = 20) → None

Train the pre-built multi-task neural network.

Parameters

• loss_scale – Optional; A float to specify the hyper-parameter to scale regression loss and classification loss, which controls the trade-off between the accuracy of regression task and classification task. A larger loss_scale value gives a more accurate prediction for classification part, and a lower value gives a more accurate prediction for regression part. Default is 1.0.

• lr – Optional; A float for learning rate. Default is 0.001.

• n_epochs – Optional; An integer for the number of epochs. Default is 1000.

• batch_size – Optional; An integer for the batch size. Default is 128.

• method – Optional; A string for the name of optimizer. Can be ‘SGD’ or ‘Adam’. Default is ‘SGD’.

• val_size – Optional; A float for the proportion of validation set of. It should be within (0, 1). Default is 0.1.

• momentum – Optional; A fload for the momentum for SGD. Default value is 0.9.

• n_epochs_stop – Optional; An integer or a float for early stopping condition. If the number of epochs is larger than n_epochs_stop and there is no improvement on validation set, we stop training. One can turn off the early stopping feature by setting n_epochs_stop = np.inf. Default is 20.

Returns

None.

class kats.models.metalearner.metalearner_hpt.MultitaskNet(input_and_n_hidden_shared: List[int], n_hidden_and_output_cat_combo: List[List[int]], n_hidden_and_output_num: List[int])

Bases: torch.nn.modules.module.Module

A class for multi-task neural network.

Build a multi-task neural network used by MetaLearnHPT. It can also be used to single-task learning. Currently only support Relu activation function.

input_and_n_hidden_shared

A list of integers contains the dimension of input and numbers of hidden neurons in each shared hidden layer. The first value in this list is dimension of input, which is dimension of feature vector. For example, input_and_n_hidden_shared = [input_dim, first_shared_hid_dim, second_shared_hid_dim, ….].

n_hidden_and_output_cat_combo

A list of lists of task-specific hidden layers’ sizes of each categorical response variables and their dimension of output. For example, if we have 3 categorical y with 3, 2, 4 classes respectively, then n_hidden_and_output_cat_combo = [[first_spec_hid_dim_cat1, second_spec_hid_dim_cat1, …, 3], [first_spec_hid_dim_cat2, second_spec_hid_dim_cat2, …, 2], [first_spec_hid_dim_cat3, second_spec_hid_dim_cat3, …, 4]].

n_hidden_and_output_num

A list of integers contains task-specific hidden layers’ sizes of numerical response variables and the dimension of output, which is the number of numerical response variables in data_y. For example, if we have three numerical response variables in y, then n_hidden_and_output_num = [first_spec_hid_dim, second_spec_hid_dim, …, 3]

forward(x)

Forward function in neural networks.