This is the Neural Network Ontology. Designed by the AIFB (http://www.aifb.kit.edu).
It enables to formalize Artificial Neural Networks in RDF, making it possible to represent existing Neural Networks in a machine-readable structure.
2019-03-22
The Neural Network Ontology
nno
https://w3id.org/nno/ontology#
This is the Neural Network Ontology. Designed by the AIFB (http://www.aifb.kit.edu).
It enables to formalize Artificial Neural Networks in RDF, making it possible to represent existing Neural Networks in a machine-readable structure.
The Neural Network Ontology
1.0
stable
Activation function used by layer.
has activation function
Layer of the Neural Network.
has layer
Loss function used by Neural Network.
has loss function
Neuron in the respective layer.
has neuron
Optimization function used by Neural Network (e.g. SGD, Adam or RMSprop).
has optimizer
Describes the used dataset for training and evaluating
data set
Dataset for evaluating the trained model. Can be the same like 'trainingdata'.
evaluation data
Number of downloads for this particular model.
has download count
Specifies the sequence of the layers. Starts at 1, the number of layers is n-1, at n: Number of layers.
has layer sequence
Reflects potential real-world impact of the Model.
Determines the Accuracy, Precision etc. of the model.
has metric
Describes the number of neurons that the layers provide.
has neurons
Link to the respository (e.g. github) where the model and further information can be found.
has repository link
Primary intended use and users (domain) for which the Neural Network was trained for.
has intended use
Dataset for training the model.
training data
Framework (e.g. Keras, Pandas, numpy, etc.) used to create this model.
used framework
Organization the person belongs to.
Person who makes this ontology.
Applies an activation function to an output.
Activation Layer
Activation function used by layer.
Activation Function
https://keras.io/activations/
Layer that applies an update to the cost function based input activity.
Activity Regularization Layer
Average pooling for temporal data.
Average Pooling 1D Layer
Average pooling operation for spatial data.
Average Pooling 2D Layer
Average pooling operation for 3D data (spatial or spatio-temporal).
Average Pooling 3D Layer
Batch normalization layer (Ioffe and Szegedy, 2014).
Normalize the activations of the previous layer at each batch, i.e. applies a transformation that maintains the mean activation close to 0 and the activation standard deviation close to 1.
Batch Normalization Layer
Loss function typically used for classification problems.
Classification Loss Function
1D convolution layer (e.g. temporal convolution).
This layer creates a convolution kernel that is convolved with the layer input over a single spatial (or temporal) dimension to produce a tensor of outputs.
Convolutional 1D Layer
2D convolution layer (e.g. spatial convolution over images).
This layer creates a convolution kernel that is convolved with the layer input to produce a tensor of outputs.
Convolutional 2D Layer
Transposed convolution layer (sometimes called Deconvolution).
The need for transposed convolutions generally arises from the desire to use a transformation going in the opposite direction of a normal convolution, i.e., from something that has the shape of the output of some convolution to something that has the shape of its input while maintaining a connectivity pattern that is compatible with said convolution.
Convolutional 2D Transpose Layer
3D convolution layer (e.g. spatial convolution over volumes).
This layer creates a convolution kernel that is convolved with the layer input to produce a tensor of outputs.
Convolutional 3D Layer
Transposed convolution layer (sometimes called Deconvolution).
The need for transposed convolutions generally arises from the desire to use a transformation going in the opposite direction of a normal convolution, i.e., from something that has the shape of the output of some convolution to something that has the shape of its input while maintaining a connectivity pattern that is compatible with said convolution.
Convolutional 3D Transpose Layer
Convolutional LSTM.
It is similar to an LSTM layer, but the input transformations and recurrent transformations are both convolutional.
Convolutional LSTM 2D Layer
Cell class for the ConvLSTM2D layer.
Convolutional LSTM 2D Cell Layer
Convolutional Layer
https://keras.io/layers/convolutional/
This Class represents all Convolutional Neural Networks.
Convolutional Neural Network
Core Layer
https://keras.io/layers/core/
Cropping layer for 1D input (e.g. temporal sequence).
It crops along the time dimension (axis 1).
Cropping 1D Layer
Cropping layer for 2D input (e.g. picture).
It crops along spatial dimensions, i.e. height and width.
Cropping 2D Layer
Cropping layer for 3D data (e.g. spatial or spatio-temporal).
Cropping 3D Layer
Fast GRU implementation backed by CuDNN.
Can only be run on GPU, with the TensorFlow backend.
CuDNN GRU Layer
https://developer.nvidia.com/cudnn
Fast LSTM implementation with CuDNN.
Can only be run on GPU, with the TensorFlow backend.
CuDNN LSTM Layer
https://developer.nvidia.com/cudnn
Just your regular densely-connected NN layer.
Dense Layer
Depthwise separable 2D convolution.
Depthwise Separable convolutions consists in performing just the first step in a depthwise spatial convolution (which acts on each input channel separately).
Depthwise Convolutional 2D Layer
Applies Dropout to the input.
Dropout Layer
Turns positive integers (indexes) into dense vectors of fixed size.
Embedding Layer
Embedding Layer
https://keras.io/layers/embeddings/
This Class represents all Feed Forward Neural Networks.
Feed Forward Neural Network
Flattens the input. Does not affect the batch size.
Flatten Layer
Gated Recurrent Unit - Cho et al. 2014.
There are two variants. The default one is based on 1406.1078v3 and has reset gate applied to hidden state before matrix multiplication. The other one is based on original 1406.1078v1 and has the order reversed.
GRU Layer
Cell class for the GRU layer.
GRU Cell Layer
Global average pooling operation for temporal data.
Global Average Pooling 1D Layer
Global average pooling operation for spatial data.
Global Average Pooling 2D Layer
Global Average pooling operation for 3D data.
Global Average Pooling 3D Layer
Global max pooling operation for temporal data.
Global Max Pooling 1D Layer
Global max pooling operation for spatial data.
Global Max Pooling 2D Layer
Global Max pooling operation for 3D data.
Global Max Pooling 3D Layer
Input() is used to instantiate a Keras tensor.
A Keras tensor is a tensor object from the underlying backend (Theano, TensorFlow or CNTK), which we augment with certain attributes that allow us to build a Keras model just by knowing the inputs and outputs of the model.
Input Layer
Input (e.g. data set).
Input Layer
Long Short-Term Memory layer - Hochreiter 1997.
LSTM Layer
Cell class for the LSTM layer.
LSTM Cell Layer
Wraps arbitrary expression as a Layer object.
Lambda Layer
Layer of the neural network.
Layer
Locally-connected Layer
https://keras.io/layers/local/
Locally-connected layer for 1D inputs.
Locally-connected 1D Layer
Locally-connected layer for 2D inputs.
Locally-connected 2D Layer
Loss function used by Neural Network.
Loss Function
Masks a sequence by using a mask value to skip timesteps.
Masking Layer
Max pooling operation for temporal data.
Max Pooling 1D Layer
Max pooling operation for spatial data.
Max Pooling 2D Layer
Max pooling operation for 3D data (spatial or spatio-temporal).
Max Pooling 3D Layer
Neural Network
Neuron of the layer.
Neuron
Normalization Layer
https://keras.io/layers/normalization/
Optimizer used by Neural Network.
Optimizer
Permutes the dimensions of the input according to a given pattern.
Useful for e.g. connecting RNNs and convnets together.
Permute Layer
Pooling Layer
https://keras.io/layers/pooling/
Recurrent Layer
https://keras.io/layers/recurrent/
This Class represents all Recurrent Neural Networks.
Recurrent Neural Network
Loss function typically used for regression problems.
Regression Loss Function
Repeats the input n times.
Repeat Vector Layer
Reshapes an output to a certain shape.
Reshape Layer
Depthwise separable 1D convolution.
Separable convolutions consist in first performing a depthwise spatial convolution (which acts on each input channel separately) followed by a pointwise convolution which mixes together the resulting output channels.
Intuitively, separable convolutions can be understood as a way to factorize a convolution kernel into two smaller kernels, or as an extreme version of an Inception block.
Separable Convolutional 1D Layer
Depthwise separable 2D convolution.
Separable convolutions consist in first performing a depthwise spatial convolution (which acts on each input channel separately) followed by a pointwise convolution which mixes together the resulting output channels.
Intuitively, separable convolutions can be understood as a way to factorize a convolution kernel into two smaller kernels, or as an extreme version of an Inception block.
Separable Convolutional 2D Layer
Fully-connected RNN where the output is to be fed back to input.
Simple RNN Layer
Cell class for SimpleRNN.
Simple RNN Cell Layer
Spatial 1D version of Dropout.
This version performs the same function as Dropout, however it drops entire 1D feature maps instead of individual elements. If adjacent frames within feature maps are strongly correlated (as is normally the case in early convolution layers) then regular dropout will not regularize the activations and will otherwise just result in an effective learning rate decrease. In this case, SpatialDropout1D will help promote independence between feature maps and should be used instead.
Spatial Dropout 1D Layer
Spatial 2D version of Dropout.
This version performs the same function as Dropout, however it drops entire 2D feature maps instead of individual elements. If adjacent pixels within feature maps are strongly correlated (as is normally the case in early convolution layers) then regular dropout will not regularize the activations and will otherwise just result in an effective learning rate decrease. In this case, SpatialDropout2D will help promote independence between feature maps and should be used instead.
Spatial Dropout 2D Layer
Spatial 3D version of Dropout.
This version performs the same function as Dropout, however it drops entire 3D feature maps instead of individual elements. If adjacent pixels within feature maps are strongly correlated (as is normally the case in early convolution layers) then regular dropout will not regularize the activations and will otherwise just result in an effective learning rate decrease. In this case, SpatialDropout3D will help promote independence between feature maps and should be used instead.
Spatial Dropout 3D Layer
Upsampling layer for 1D inputs.
Upsampling 1D Layer
Upsampling layer for 2D inputs.
Repeats the rows and columns of the data by size[0] and size[1] respectively.
Upsampling 2D Layer
Upsampling layer for 3D inputs.
Repeats the 1st, 2nd and 3rd dimensions of the data by size[0], size[1] and size[2] respectively.
Upsampling 3D Layer
Zero-padding layer for 1D input (e.g. temporal sequence).
Zero-padding 1D Layer
Zero-padding layer for 2D input (e.g. picture).
This layer can add rows and columns of zeros at the top, bottom, left and right side of an image tensor.
Zero-padding 2D Layer
Zero-padding layer for 3D data (spatial or spatio-temporal).
Zero-padding 3D Layer
Anna Nguyen
http://www.aifb.kit.edu/web/Anna_Nguyen
Tobias Weller
http://www.aifb.kit.edu/web/Tobias_Weller
Web Science (AIFB)
http://www.aifb.kit.edu/web/Web_Science
York Sure-Vetter
http://www.aifb.kit.edu/web/York_Sure-Vetter
Adadelta is a more robust extension of Adagrad that adapts learning rates based on a moving window of gradient updates, instead of accumulating all past gradients. This way, Adadelta continues learning even when many updates have been done. Compared to Adagrad, in the original version of Adadelta you don't have to set an initial learning rate. In this version, initial learning rate and decay factor can be set, as in most other Keras optimizers.
Adadelta Optimizer
Adagrad is an optimizer with parameter-specific learning rates, which are adapted relative to how frequently a parameter gets updated during training. The more updates a parameter receives, the smaller the learning rate.
Adagrad Optimizer
Adam Optimizer
Adamax Optimizer
Functional interface to the Add layer.
Add
Adds a bias vector to a tensor.
Bias Add
Binary Crossentropy
Categorical Crossentropy
Categorical Hinge
Cosine Proximity
Exponential Linear Unit
Exponential
Hard Sigmoid
Hinge
Returns a tensor with the same content as the input tensor.
Identity
Kullback Leibler Divergence
Linear
Log Cosh Loss
Maxpool
Mean Absolute Error
Mean Absolute Percentage Error
Mean Squared Error
Mean Squared Logarithmic Error
Merge
Much like Adam is essentially RMSprop with momentum, Nadam is Adam RMSprop with Nesterov momentum.
Nesterov Adam Optimizer
Poisson
Rectified Linear Unit
Reshapes a tensor to the specified shape.
Reshape
Resize Nearest Neighbor
This optimizer is usually a good choice for recurrent neural networks.
RMSProp Optimizer
Scaled Exponential Linear Unit
Includes support for momentum, learning rate decay, and Nesterov momentum.
Stochastic Gradient Descent Optimizer
Sigmoid
Softmax
Softplus
Softsign
Sparse Categorical Crossentropy
Squared Hinge
Hyperbolic Tangent
Tensor Array Read V3
Transposes a tensor and returns it.
Transpose