Keras Models: Sequential vs. Functional

There are two ways to build Keras models: sequential and functional.

The sequential API allows you to create models layer-by-layer for most problems. It is limited in that it does not allow you to create models that share layers or have multiple inputs or outputs.

Alternatively, the functional API allows you to create models that have a lot more flexibility as you can easily define models where layers connect to more than just the previous and next layers. In fact, you can connect layers to (literally) any other layer. As a result, creating complex networks such as siamese networks and residual networks become possible.


Sequential Models

from keras.models import Sequential
from keras.layers import Dense

model = Sequential()
model.add(Dense(2, input_dim=1))
model.add(Dense(1))

In the example above, layers are added piecewise via the Sequential object.

The Sequential model API is great for developing deep learning models in most situations, but it also has some limitations.

For example, it is not straightforward to define models that may have:

  1. multiple different input sources,
  2. produce multiple output destinations, or
  3. models that re-use layers.

Functional Models (⭐️)

from keras.models import Model
from keras.layers import Input
from keras.layers import Dense

# Define the input
#   Unlike the Sequential model, you must create and define 
#   a standalone "Input" layer that specifies the shape of input 
#   data. The input layer takes a "shape" argument, which is a 
#   tuple that indicates the dimensionality of the input data.
#   When input data is one-dimensional, such as the MLP, the shape 
#   must explicitly leave room for the shape of the mini-batch size 
#   used when splitting the data when training the network. Hence, 
#   the shape tuple is always defined with a hanging last dimension.
#   For instance, "(2,)", as in the example below:
visible = Input(shape=(2,))  

# Connecting layers
#   The layers in the model are connected pairwise.
#   This is done by specifying where the input comes from when 
#   defining each new layer. A bracket notation is used, such that 
#   after the layer is created, the layer from which the input to 
#   the current layer comes from is specified.
#   Note how the "visible" layer connects to the "Dense" layer:
hidden = Dense(2)(visible)  

# Create the model
#   After creating all of your model layers and connecting them 
#   together, you must then define the model.
#   As with the Sequential API, the model is the thing that you can
#   summarize, fit, evaluate, and use to make predictions.
#   Keras provides a "Model" class that you can use to create a model 
#   from your created layers. It requires that you only specify the 
#   input and output layers. For example:
model = Model(inputs=visible, outputs=hidden)

The Keras functional API provides a more flexible way for defining models.

Specifically, it allows you to define multiple input or output models as well as models that share layers. More than that, it allows you to define ad hoc acyclic network graphs.

Models are defined by creating instances of layers and connecting them directly to each other in pairs, and then defining a Model that specifies the layers to act as the input and output to the model, via the parameters inputs and outputs, respectively.


Examples

Take a look at how Sequential and Functional models are being used in the examples featured in the post "Embeddings in Keras: Train vs. Pretrained".


If you enjoyed this post and want to buy me a cup of coffee...

The thing is, I'll always accept a cup of coffee. So feel free to buy me one.

Cheers! ☕️