Neural Networks and pyWATTS

In the following, we will examine how we can use neural networks within pyWATTS. In the following, we will learn

• how to use the Keras and pyTorch Wrappers, and
• how to implement a Keras based network in a own module.

Usage of Keras an pyTorch Wrappers

One possibility to integrate neural networks into pyWATTS is to use the provided wrappers for Keras and pyTorch. In general, both are used in the same way. Examples of the usage of both wrappers are in the examples folder.

1. You need a Keras or torch model. Note, that for Keras you should use the functional API. Moreover, your input and output layers should be named.

def get_keras_model():
    # write the model with the Functional API, Sequential does not support multiple input tensors
    input_1 = layers.Input(shape=(24,), name='lag_features')  # layer name must match time series name
    hidden = layers.Dense(10, activation='tanh', name='hidden')(input_1)
    output = layers.Dense(24, activation='linear', name='target')(hidden)  # layer name must match time series name
    model = Model(inputs=[input_1], outputs=output)
    return model


def get_torch_model():
    model = torch.nn.Sequential(
        torch.nn.Linear(24, 10),
        torch.nn.ReLU(),
        torch.nn.Linear(10, 24),
    )
return model

2. You initialize the Keras or pyTorch Wrapper with the predefined neural networks. Additionaly, you can add the compile, and fit keyword-arguments of the model as Dict.

keras_wrapper = KerasWrapper(keras_model,
                             fit_kwargs={"batch_size": 8, "epochs": 1},
                             compile_kwargs={"loss": "mse", "optimizer": "Adam", "metrics": ["mse"]})

torch_wrapper = PyTorchWrapper(get_sequential_model(),
                                 fit_kwargs={"batch_size": 8, "epochs": 1},
                                 compile_kwargs={"loss": "mse", "optimizer": "Adam", "metrics": ["mse"]})

3. You have to add the wrapper to the pipeline just as a normal module. Note, if you use keras you have to assign the inputs to the keywords with the same name as you have them defined in the previously defined keras neural network.

keras_wrapper(lag_features=lag_features, target=target)

torch_wrapper(lag_features=lag_features, target=target)

Neural Network Module

As an alternative to the usage of the Wrapers you can implement a new module, which contains a neural network. Therefore, you need a model definition and a reference to a defined model in your module. Afterwards, you should implement the transform, fit, get_param, and set_param method.

The transform method should call the predict function of the model. The fit method has to train the model. The get_param and the set_param method should implement the functionality to storing and loading the module.

Additionally, you have also to define your own save and load methods, which should contain all functionality for storing and loading your neural network.

As an example you can look at the profile_neural_network module. It implements a neural network using Keras without using the KerasWrapper.