While computers are powerful devices that can do incredibly complex things very quickly, they are actually surprisingly simple, in that they can only understand and act upon a relatively small set of commands. This is why computer programmers exist, as well as all sorts of other computer professions. They essentially translate human needs into a system that computers can both understand and respond to.
In today’s world, though — where many of us are connected online at all times —it has become increasingly important for humans to communicate directly with computers, so that computers can serve them at once and without the need of intermediaries. This is where natural language processing (also known as NLP) comes in to play. Using artificial intelligence (AI) and machine learning techniques, natural language processing translates languages such as English on the fly into commands computers can understand and process.
In this natural language processing tutorial, we will provide an introduction to natural language processing while detailing exactly what is natural language processing and describing how does natural language processing work. We will also present examples of it in action.
Natural Language Processing: An Introduction
Computer languages, such as JavaScript and C++, have certain similarities to natural languages such as English. Both have well structured syntaxes, for example. But the syntactical rules for computer languages are few and simple, while syntactical rules for natural languages are not only numerous but also can get quite complex. Also, computer languages have no irregularities while natural languages often have lots of irregularities.
Another key difference between the two types of languages is something called semantics. Semantics determine the meaning of a word or phrase based on its context within a sentence or a group of sentences. This concept does not really exist in computer languages but plays an important role in natural languages, and it is the most difficult aspect of them for computers to comprehend. This is where AI and NLP-based machine learning comes into play, as the computer requires a certain intelligence in order to understand the context of words and phrases.
Natural Language Processing: How It Works
You can look at natural language processing as requiring the following 4-step process:
1. Human to Computer Communication
The first step in natural language processing is also the simplest: a human must convey to a computer a desire for something. This could be information or an action it wants the computer to initiate, and the human can convey this by typing a request, by providing the computer natural language data, such as a web page, or by speaking directly to the computer.
2. Computer Parsing
In the next step, the computer must parse the data it receives from the human so that it is in a form that it can understand and process. Sometimes this is simple. When someone types a request to a computer, typically no parsing is needed. But if the human provides the computer a data file, such as a web page, the computer must parse the data to get the text it needs to act upon.
When a human actually speaks to a computer, this makes things even more complicated, as the computer must first properly transcribe the audio into text. This, in itself, requires AI, as languages often have what are called homonyms, which are words that sound alike but are different.
Whenever the computer comes upon one of these words, it must choose the correct one using semantic processing. That is, it must understand the context of the word being used to understand which word was actually spoken.
3. Language Processing
This is the most important and complex step in the process, in which the AI software applies a set of natural language processing algorithms to the data it has received and converts it into language that the computer can both understand and process. This entails breaking down both the syntax and the semantics of the data’s language.
Among the many syntactical techniques that are commonly in use are the following:
- Reducing inflected word forms through a process called lemmatization.
- Breaking large words apart into segments that are known as morphemes.
- Segmenting large chunks of text into small units.
- Identifying and tagging parts of speech.
- Performing grammatical analysis of sentences.
Among the many semantical techniques that are commonly in use are the following:
- Identifying and categorizing pieces of text into groups.
- Using something called disambiguation to provide meaning to words based on their context.
- Applying a set of data databases to determine semantic intention of a given word.
When something goes wrong in natural language processing, it is usually in this step. If the software misinterprets the meaning of a sentence, it can lead to poor and even nonsensical results. But as natural processing algorithms get better and better in the future, the results will get better, too, and so will user experience.
4. Computer to Human Communication
The final step in the process is the reverse of the opening step, and it is just as simple. In this step, the computer returns the result to the human. The result could be in the form of a textual reply, a data file (such as a web page) or the playing of an audio file after the text has first been converted into speech.
Natural Language Processing: Applications Now and in the Future
Today, natural language processing applications are everywhere. They are so ubiquitous that you may be using them every day without even knowing it. Here are some examples of some of the most common natural language processing applications available.
1. Translation Apps
For many years now, there have been translation apps available, like Google Translate, which allow you to translate text (or web pages) from one natural language to another. In the beginning, these programs were so bad that they were literally comical and the butt of many jokes. But no one is laughing today. They have improved exponentially over time and primarily because of advances in natural language processing.
While these apps today are still far from perfect — and their output would unlikely fool a native speaker — you can now count on them to provide you proper meaning of an original source. In the future, they will only improve, to the extent that one day they just may fool a native speaker.
2. Grammar Checker
Like translation apps, grammar checkers have been around for many decades, going back to the early years of personal computers and Microsoft Word. Like early translation apps, the first grammar checkers were notoriously poor. They were so poor that most people did not even bother using them. But that is not the case today.
Not only has Word’s grammar checker improved to the point that it has become an essential tool for users of the application, but a company called Grammarly has a free browser plugin that allows people to check their grammar on the fly within just about any web app. Like with today’s translation apps, Grammarly and Word’s grammar checker are still far from perfect.
But as natural language processing technology improves, so will they. It is quite plausible to believe that, in the very near future, grammar checkers will be able to flawlessly find and correct any grammar problem you have, and in a multitude of languages.
3. Interactive Voice Response (IVR) Applications and Chatbots
In the not-so-distant past, people despised automated phone systems, because they were so poor and inefficient. But that is not the case today, in large part to natural language processing. In many instances, you can now communicate with call centers using these IVR apps and chatbots as effectively as you would with an actual human being.
Sometimes, they are even better than humans, as users no longer have to deal with people who might not speak the language fluently or whose accents are difficult to understand. Like with other natural language processing applications currently available, these systems are not yet perfect. But in the future it just may be possible to completely eliminate the need for human customer support personnel.
4. Personal Assistant Applications
If you have any type of computing device, you almost certainly have used a personal assistant, such as Siri, OK Google, Cortana or Alexa. At the heart of all these systems is natural language processing, which takes your commands and provides you information or acts upon your requests.
As natural language processing technology improves, so will they. In the future, not only will there be little need for human personal assistants, but the benefits of having a personal assistant will be available to everyone, regardless of means or geographical location.
One day, natural language processing may eliminate the need for user interfaces (UI) in computer applications. UIs are simply a means of translating what you want to do into something that the computer can understand. But if computers can fully understand you without the need of any intermediaries, UIs will become redundant. This makes it easier and faster to get the information that you want.
