zoo.automl.model package

Subpackages

• zoo.automl.model.tcmf package
  • Submodules
  • zoo.automl.model.tcmf.DeepGLO module
  • zoo.automl.model.tcmf.data_loader module
  • zoo.automl.model.tcmf.local_model module
  • zoo.automl.model.tcmf.local_model_distributed_trainer module
  • zoo.automl.model.tcmf.time module
  • Module contents

Submodules

zoo.automl.model.MTNet_keras module

class zoo.automl.model.MTNet_keras.AttentionRNNWrapper(layer, weight_initializer='glorot_uniform', **kwargs)

Bases: tensorflow.python.keras.layers.wrappers.Wrapper

This class is modified based on https://github.com/zimmerrol/keras-utility-layer-collection/blob/master/kulc/attention.py. The idea of the implementation is based on the paper: “Effective Approaches to Attention-based Neural Machine Translation” by Luong et al. This layer is an attention layer, which can be wrapped around arbitrary RNN layers. This way, after each time step an attention vector is calculated based on the current output of the LSTM and the entire input time series. This attention vector is then used as a weight vector to choose special values from the input data. This data is then finally concatenated to the next input time step’s data. On this a linear transformation in the same space as the input data’s space is performed before the data is fed into the RNN cell again. This technique is similar to the input-feeding method described in the paper cited

build(input_shape)

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Arguments: input_shape: Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

call(x, constants=None, mask=None, initial_state=None)

This is where the layer’s logic lives.

Arguments: inputs: Input tensor, or list/tuple of input tensors. **kwargs: Additional keyword arguments.

Returns: A tensor or list/tuple of tensors.

compute_output_shape(input_shape)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Arguments: input_shape: Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns: An input shape tuple.

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns: Python dictionary.

get_constants(x)

non_trainable_weights

step(x, states)

trainable_weights

class zoo.automl.model.MTNet_keras.MTNetKeras(check_optional_config=False, future_seq_len=1)

Bases: zoo.automl.model.abstract.BaseModel

apply_config(rs=False, config=None)

build()

build MTNet model :param config: :return:

evaluate(x, y, metrics=['mse'])

Evaluate on x, y :param x: input :param y: target :param metric: a list of metrics in string format :return: a list of metric evaluation results

fit_eval(x, y, validation_data=None, mc=False, metrics=None, epochs=10, verbose=0, **config)

optimize and evaluate for one iteration for tuning :param config: tunable parameters for optimization :return:

predict(x, mc=False)

Prediction. :param x: input :return: result

predict_with_uncertainty(x, n_iter=100)

restore(model_path, **config)

restore model from file :param model_path: the model file :param config: the trial config

save(model_path, config_path)

save model to file. :param model_path: the model file path to be saved to. :param config_path: the config file path to be saved to. :return:

zoo.automl.model.Seq2Seq module

class zoo.automl.model.Seq2Seq.LSTMSeq2Seq(check_optional_config=True, future_seq_len=2)

Bases: zoo.automl.model.abstract.BaseModel

evaluate(x, y, metric=['mse'])

Evaluate on x, y :param x: input :param y: target :param metric: a list of metrics in string format :return: a list of metric evaluation results

fit_eval(x, y, validation_data=None, mc=False, verbose=0, **config)

fit for one iteration :param x: 3-d array in format (no. of samples, past sequence length, 2+feature length), in the last dimension, the 1st col is the time index (data type needs to be numpy datetime type, e.g. “datetime64”), the 2nd col is the target value (data type should be numeric) :param y: 2-d numpy array in format (no. of samples, future sequence length) if future sequence length > 1, or 1-d numpy array in format (no. of samples, ) if future sequence length = 1 :param validation_data: tuple in format (x_test,y_test), data used for validation. If this is specified, validation result will be the optimization target for automl. Otherwise, train metric will be the optimization target. :param config: optimization hyper parameters :return: the resulting metric

predict(x, mc=False)

Prediction on x. :param x: input :return: predicted y (expected dimension = 2)

predict_with_uncertainty(x, n_iter=100)

restore(model_path, **config)

restore model from file :param model_path: the model file :param config: the trial config :return: the restored model

save(model_path, config_path)

save model to file. :param model_path: the model file path to be saved to. :param config_path: the config file path to be saved to. :return:

zoo.automl.model.Seq2Seq_pytorch module

class zoo.automl.model.Seq2Seq_pytorch.Decoder(output_dim, hidden_dim, layer_num, dropout)

Bases: torch.nn.modules.module.Module

forward(decoder_input, hidden, cell)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class zoo.automl.model.Seq2Seq_pytorch.Encoder(input_dim, hidden_dim, layer_num, dropout)

Bases: torch.nn.modules.module.Module

forward(input_seq)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class zoo.automl.model.Seq2Seq_pytorch.Seq2Seq(encoder, decoder, target_seq_len=1)

Bases: torch.nn.modules.module.Module

forward(source, target=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class zoo.automl.model.Seq2Seq_pytorch.Seq2SeqPytorch(check_optional_config=True, future_seq_len=1)

Bases: zoo.automl.model.abstract.BaseModel

evaluate(x, y, metric=['mse'])

Evaluate on x, y :param x: input :param y: target :param metric: a list of metrics in string format :return: a list of metric evaluation results

fit_eval(x, y, validation_data=None, mc=False, verbose=0, **config)

optimize and evaluate for one iteration for tuning :param config: tunable parameters for optimization :return:

predict(x, mc=False)

Prediction on x. :param x: input :return: predicted y

predict_with_uncertainty(x, n_iter=100)

restore(model_path, **config)

restore model from file :param model_path: the model file :param config: the trial config :return: the restored model

save(model_path, config_path)

save model to file. :param model_path: the model file. :param config_path: the config file :return:

zoo.automl.model.VanillaLSTM module

class zoo.automl.model.VanillaLSTM.VanillaLSTM(check_optional_config=False, future_seq_len=1)

Bases: zoo.automl.model.abstract.BaseModel

evaluate(x, y, metric=['mse'])

Evaluate on x, y :param x: input :param y: target :param metric: a list of metrics in string format :return: a list of metric evaluation results

fit_eval(x, y, validation_data=None, mc=False, verbose=0, **config)

fit for one iteration :param x: 3-d array in format (no. of samples, past sequence length, 2+feature length), in the last dimension, the 1st col is the time index (data type needs to be numpy datetime type, e.g. “datetime64”), the 2nd col is the target value (data type should be numeric) :param y: 2-d numpy array in format (no. of samples, future sequence length) if future sequence length > 1, or 1-d numpy array in format (no. of samples, ) if future sequence length = 1 :param validation_data: tuple in format (x_test,y_test), data used for validation. If this is specified, validation result will be the optimization target for automl. Otherwise, train metric will be the optimization target. :param config: optimization hyper parameters :return: the resulting metric

predict(x, mc=False)

Prediction on x. :param x: input :return: predicted y

predict_with_uncertainty(x, n_iter=100)

restore(model_path, **config)

restore model from file :param model_path: the model file :param config: the trial config :return: the restored model

save(model_path, config_path)

save model to file. :param model_path: the model file. :param config_path: the config file :return:

zoo.automl.model.VanillaLSTM_pytorch module

class zoo.automl.model.VanillaLSTM_pytorch.LSTMModel(input_dim, hidden_dim, layer_num, dropout, output_dim)

Bases: torch.nn.modules.module.Module

forward(input_seq)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class zoo.automl.model.VanillaLSTM_pytorch.VanillaLSTMPytorch(check_optional_config=True, future_seq_len=1)

Bases: zoo.automl.model.abstract.BaseModel

evaluate(x, y, metric=['mse'])

Evaluate on x, y :param x: input :param y: target :param metric: a list of metrics in string format :return: a list of metric evaluation results

fit_eval(x, y, validation_data, mc=False, verbose=0, **config)

optimize and evaluate for one iteration for tuning :param config: tunable parameters for optimization :return:

predict(x, mc=False)

Prediction on x. :param x: input :return: predicted y

predict_with_uncertainty(x, n_iter=100)

restore(model_path, **config)

restore model from file :param model_path: the model file :param config: the trial config :return: the restored model

save(model_path, config_path)

save model to file. :param model_path: the model file. :param config_path: the config file :return:

zoo.automl.model.XGBoostRegressor module

zoo.automl.model.abstract module

class zoo.automl.model.abstract.BaseModel

Bases: abc.ABC

base model for automl tuning

check_optional_config = False

evaluate(x, y, metric=None)

Evaluate the model :param x: input :param y: target :param metric: :return: a list of metric evaluation results

fit_eval(x, y, validation_data=None, mc=False, verbose=0, **config)

optimize and evaluate for one iteration for tuning :param config: tunable parameters for optimization :return:

future_seq_len = None

predict(x, mc=False)

Prediction. :param x: input :return: result

restore(model_path, **config)

restore model from model file and config. :param model_path: the model file :param config: the config :return: the restored model

save(model_path, config_path, **config)

save model to file. :param model_path: the model file path to be saved to. :param config_path: the config file path to be saved to. :return:

zoo.automl.model.tcmf_model module

class zoo.automl.model.tcmf_model.ModelWrapper

Bases: object

evaluate(**kwargs)

fit(**kwargs)

is_distributed(**kwargs)

load(**kwargs)

predict(**kwargs)

save(**kwargs)

class zoo.automl.model.tcmf_model.TCMF

Bases: zoo.automl.model.abstract.BaseModel

MF regularized TCN + TCN. This version is not for automated searching yet.

evaluate(x=None, y=None, metrics=None, num_workers=None)

Evaluate on the prediction results and y. We predict horizon time-points ahead the input x in fit_eval before evaluation, where the horizon length equals the second dimension size of y. :param x: We don’t support input x currently. :param y: target. We interpret the second dimension of y as the horizon length for evaluation. :param metrics: a list of metrics in string format :param num_workers: the number of workers to use in evaluate. It defaults to 1. :return: a list of metric evaluation results

fit_eval(x, y=None, verbose=0, num_workers=None, **config)

Fit on the training data from scratch. Since the rolling process is very customized in this model, we enclose the rolling process inside this method.

Parameters:x – training data, an array in shape (nd, Td),

nd is the number of series, Td is the time dimension :param y: None. target is extracted from x directly :param verbose: :param num_workers: number of workers to use. :return: the evaluation metric value

fit_incremental(x)

Incremental fitting given a pre-trained model. :param x: incremental data :param config: fitting parameters :return:

static get_default_num_workers()

predict(x=None, horizon=24, mc=False, num_workers=None)

Predict horizon time-points ahead the input x in fit_eval :param x: We don’t support input x currently. :param horizon: horizon length to predict :param mc: :param num_workers: the number of workers to use. Note that there has to be an activate RayContext if num_workers > 1. :return:

restore(model_file)

restore model from model file and config. :param model_path: the model file :param config: the config :return: the restored model

save(model_file)

save model to file. :param model_path: the model file path to be saved to. :param config_path: the config file path to be saved to. :return:

set_params(**config)

class zoo.automl.model.tcmf_model.TCMFDistributedModelWrapper(config)

Bases: zoo.automl.model.tcmf_model.ModelWrapper

evaluate(x, y, metric=None, num_workers=None)

Evaluate the model :param x: input :param y: target :param metric: :param num_workers: :return: a list of metric evaluation results

fit(x, incremental=False, num_workers=None)

is_distributed()

load(model_path, minPartitions=None)

restore model from model file and config. :param model_path: the model file :return: the restored model

predict(x, horizon=24, num_workers=None)

Prediction. :param x: input :param horizon: :param num_workers :return: result

save(model_path)

save model to file. :param model_path: the model file path to be saved to. :return:

class zoo.automl.model.tcmf_model.TCMFLocalModelWrapper(config)

Bases: zoo.automl.model.tcmf_model.ModelWrapper

evaluate(x, y, metric=None, num_workers=None)

Evaluate the model :param x: input :param y: target :param metric: :param num_workers: :return: a list of metric evaluation results

fit(x, incremental=False, num_workers=None)

is_distributed()

load(model_path)

restore model from model file and config. :param model_path: the model file :return: the restored model

predict(x, horizon=24, num_workers=None)

Prediction. :param x: input :param horizon :param num_workers :return: result

save(model_path)

save model to file. :param model_path: the model file path to be saved to. :return:

zoo.automl.model.tcmf_model.split_id_and_train_data(d, is_distributed)

zoo.automl.model.time_sequence module

class zoo.automl.model.time_sequence.TimeSequenceModel(check_optional_config=False, future_seq_len=None)

Bases: zoo.automl.model.abstract.BaseModel

Time Sequence Model is used to do model selection.

evaluate(x, y, metric=['mse'])

Evaluate on x, y :param x: input :param y: target :param metric: a list of metrics in string format :return: a list of metric evaluation results

fit_eval(x, y, validation_data=None, mc=False, metric='mse', verbose=0, **config)

fit for one iteration :param x: 3-d array in format (no. of samples, past sequence length, 2+feature length), in the last dimension, the 1st col is the time index (data type needs to be numpy datetime type, e.g. “datetime64”), the 2nd col is the target value (data type should be numeric) :param y: 2-d numpy array in format (no. of samples, future sequence length) if future sequence length > 1, or 1-d numpy array in format (no. of samples, ) if future sequence length = 1 :param validation_data: tuple in format (x_test,y_test), data used for validation. If this is specified, validation result will be the optimization target for automl. Otherwise, train metric will be the optimization target. :param metric: the way to measure the performance of model :param config: optimization hyper parameters :return: the resulting metric

predict(x, mc=False)

Prediction on x. :param x: input :return: predicted y

predict_with_uncertainty(x, n_iter=100)

restore(model_path, **config)

restore model from model file and config. :param model_path: the model file :param config: the config :return: the restored model

save(model_path, config_path)

save model to file. :param model_path: the model file. :param config_path: the config file :return:

Module contents