Chapter 9. Models for Sequence Analysis
Analyzing Variable-Length Inputs
Up until now, weâve worked only with data with fixed sizes: images from MNIST, CIFAR-10, and ImageNet. These models are incredibly powerful, but there are many situations in which fixed-length models are insufficient. The vast majority of interactions in our daily lives require a deep understanding of sequencesâwhether itâs reading the morning newspaper, making a bowl of cereal, listening to the radio, watching a presentation, or deciding to execute a trade on the stock market. To adapt to variable-length inputs, weâll have to be a little bit more clever about how we approach designing deep learning models.
Figure 9-1 illustrates how our feed-forward neural networks break when analyzing sequences. If the sequence is the same size as the input layer, the model can perform as expected. Itâs even possible to deal with smaller inputs by padding zeros to the end of the input until itâs the appropriate length. However, the moment the input exceeds the size of the input layer, naively using the feed-forward network no longer works.
Feed-forward networks thrive on fixed input size problems. Zero padding can address the handling of smaller inputs, but when naively utilized, these models break when inputs exceed the fixed input size.
Not all hope ...
Get Fundamentals of Deep Learning, 2nd Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.