Three tips for getting started with NLU

What makes a cartoon caption funny? As one algorithm found: a simple readable sentence, a negation, and a pronoun—but not “he” or “she.” The algorithm went on to pick the funniest captions for thousands of the New Yorker’s cartoons, and in most cases, it matched the intuition of its editors.

Algorithms are getting much better at understanding language, and we are becoming more aware of this through stories like that of IBM Watson winning the Jeopardy quiz. Thankfully, large corporations aren’t keeping the latest breakthroughs in natural language understanding (NLU) for themselves. Google released the word2vec tool, and Facebook followed by publishing their speed optimized deep learning modules. Since language is at the core of many businesses today, it’s important to understand what NLU is, and how you can use it to meet some of your business goals. In this article, you will learn three key tips on how to get into this fascinating and useful field.

Learn faster. Dig deeper. See farther.

Join the O'Reilly online learning platform. Get a free trial today and find answers on the fly, or master something new and useful.

Learn more

But first things first: what does “natural language understanding” actually mean? Whenever a computer understands written language—or in other words, derives the meaning of words, sentences or text—we call it natural language understanding. When understanding spoken language, such as voice commands or recorded speech, a process called automatic speech recognition transforms it first into written words.

NLU is technically a sub-area of the broader area of natural language processing (NLP), which is a sub-area of artificial intelligence (AI). Many NLP tasks, such as part-of-speech or text categorization, do not always require actual understanding in order to perform accurately, but in some cases they might, which leads to confusion between these two terms. As a rule of thumb, an algorithm that builds a model that understands meaning falls under natural language understanding, not just natural language processing.

Examples of natural language understanding

Let’s look of some examples of what we mean by “understanding meaning,” in a non-philosophical way. For our first example, we’ll look at relation extraction. The meaning of “London,” for example, could be a multitude of relations, such as: “is a City,” “is a British capital,” “is the location of Buckingham Palace,” “is the headquarter of HSBC.” These are semantic relations, and NLU algorithms are pretty good at extracting such relations from unstructured text. For example, the Open Information Extraction system at the University of Washington extracted more than 500 million such relations from unstructured web pages, by analyzing sentence structure. Another example is Microsoft’s ProBase, which uses syntactic patterns (“is a,” “such as”) and resolves ambiguity through iteration and statistics. It then merges all these relations into knowledge taxonomy. Similarly, businesses can extract knowledge bases from web pages and documents relevant to their business.

Meaning can also be expressed through emotions. Sentiment analysis can determine emotions from text. For businesses, it’s important to know the sentiment of their users and customers overall, and the sentiment attached to specific themes, such as areas of customer service or specific product features.

Another popular application of NLU is chat bots, also known as dialogue agents, who make our interaction with computers more human-like. At the most basic level, bots need to understand how to map our words into actions and use dialogue to clarify uncertainties. At the most sophisticated level, they should be able to hold a conversation about anything, which is true artificial intelligence.  Anybody who has used Siri, Cortana, or Google Now while driving will attest that dialogue agents are already proving useful, and going beyond their current level of understanding would not necessarily improve their function. Most other bots out there are nothing more than a natural language interface into an app that performs one specific task, such as shopping or meeting scheduling. Interestingly, this is already so technologically challenging that humans often hide behind the scenes. Keeping it simple is the key when it comes to building bots.

Begin using NLU—here’s why and how

The good news is that despite many challenges, NLU is breaking out. This means that whether you work with product reviews, receive user feedback, or interact with customers, you can start using NLU methods today. Here are three tips for how and why to get started:

• You can choose the smartest algorithm out there without having to pay for itMost algorithms are publicly available as open source. It’s truly mind-bending that if you want, you can download and start using the same algorithms Google used to beat the world’s Go champion, right now. Many machine learning toolkits come with an array of algorithms; which one is the best depends on what you are trying to predict and the amount of data available. While there may be some general guidelines, it’s often best to loop through them to choose the right one.
• Your deep knowledge of the business is more important than the best algorithmWhen integrating NLU into a new product or service, it is more important to understand the specific business field—how it works and its priorities—than to have the best algorithm at hand.  Consider this example: an app that lets you query Salesforce in natural language. Knowing which questions users are likely to ask and which data can be queried from Salesforce is more important than having the most accurate language parser. After all, it’s more important to solve the right problem with an “okay” algorithm than the wrong problem with the best algorithm out there.
• It’s likely that you already have enough data to train the algorithmsGoogle may be the most prolific producer of successful NLU applications. The reason why its search, machine translation, and ad recommendation work so well is because Google has access to huge data sets. For the rest of us, current algorithms like word2vec require significantly less data to return useful results. Indeed, companies have already started integrating such tools into their workflows. If your business has as a few thousand product reviews or user comments, you can probably make this data work for you using word2vec, or other language modeling methods available through tools like Gensim, Torch, and TensorFlow.

Judging the accuracy of an algorithm

These are all good reasons for giving natural language understanding a go, but how do you know if the accuracy of an algorithm will be sufficient? Consider the type of analysis it will need to perform and the breadth of the field. Analysis ranges from shallow, such as word-based statistics that ignore word order, to deep, which implies the use of ontologies and parsing. Deep learning, despite the name, does not imply a deep analysis, but it does make the traditional shallow approach deeper. Field stands for the application area, and narrow means a specialist domain or a specific task. Broad implies no restrictions on how varied language may be.

As a rule of thumb, try to classify your problem according to this grid:

If accuracy is paramount, go only for specific tasks that need shallow analysis. If accuracy is less important, or if you have access to people who can help where necessary, deepening the analysis or a broader field may work. In general, when accuracy is important, stay away from cases that require deep analysis of varied language—this is an area still under development in the field of AI.