Essential Math for AI

Chapter 7. Natural Language and Finance AI: Vectorization and Time Series

They. Can. Read.

H.

One of the hallmarks of human intelligence is our mastery of language at a very early age: comprehension of written and spoken language, written and spoken expression of thoughts, conversation between two or more people, translation from one language to another, and the use of language to express empathy, convey emotions, and process visual and audio data perceived from our surroundings. Leaving the philosophical question of consciousness aside, if machines acquire the ability to perform these language tasks, deciphering the intent of words, at a level similar to humans, or above humans, then it is a major propeller toward general artificial intelligence. These tasks fall under the umbrellas of natural language processing, computational linguistics, machine learning, and/or probabilistic language modeling. These fields are vast and it is easy to find interested people wandering aimlessly in a haze of various models with big promises. We should not get lost. The aim of this chapter is to lay out the natural processing field all at once so we can have a bird’s-eye view without getting into the weeds.

The following questions guide us at all times:

• What type of task is at hand? In other words, what is our goal?

• What type of data is at hand? What type of data do we need to collect?

• What state-of-the-art models are out there that deal with similar tasks and similar types of data? If ...