What Is Natural Language Processing?

Natural Language Processing, or NLP for short, is broadly defined as the automatic manipulation of natural language, like speech and text, by software.

The study of natural language processing has been around for more than 50 years and grew out of the field of linguistics with the rise of computers.

In this post, you will discover what natural language processing is and why it is so important.

After reading this post, you will know:

  • What natural language is and how it is different from other types of data.
  • What makes working with natural language so challenging.
  • Where the field of NLP came from and how it is defined by modern practitioners.

Kick-start your project with my new book Deep Learning for Natural Language Processing, including step-by-step tutorials and the Python source code files for all examples.

Let’s get started.

What Is Natural Language Processing?

What Is Natural Language Processing?
Photo by pedrik, some rights reserved.

Natural Language

Natural language refers to the way we, humans, communicate with each other.

Namely, speech and text.

We are surrounded by text.

Think about how much text you see each day:

  • Signs
  • Menus
  • Email
  • SMS
  • Web Pages
  • and so much more…

The list is endless.

Now think about speech.

We may speak to each other, as a species, more than we write. It may even be easier to learn to speak than to write.

Voice and text are how we communicate with each other.

Given the importance of this type of data, we must have methods to understand and reason about natural language, just like we do for other types of data.

Need help with Deep Learning for Text Data?

Take my free 7-day email crash course now (with code).

Click to sign-up and also get a free PDF Ebook version of the course.

Challenge of Natural Language

Working with natural language data is not solved.

It has been studied for half a century, and it is really hard.

It is hard from the standpoint of the child, who must spend many years acquiring a language … it is hard for the adult language learner, it is hard for the scientist who attempts to model the relevant phenomena, and it is hard for the engineer who attempts to build systems that deal with natural language input or output. These tasks are so hard that Turing could rightly make fluent conversation in natural language the centerpiece of his test for intelligence.

— Page 248, Mathematical Linguistics, 2010.

Natural language is primarily hard because it is messy. There are few rules.

And yet we can easily understand each other most of the time.

Human language is highly ambiguous … It is also ever changing and evolving. People are great at producing language and understanding language, and are capable of expressing, perceiving, and interpreting very elaborate and nuanced meanings. At the same time, while we humans are great users of language, we are also very poor at formally understanding and describing the rules that govern language.

— Page 1, Neural Network Methods in Natural Language Processing, 2017.

From Linguistics to Natural Language Processing

Linguistics

Linguistics is the scientific study of language, including its grammar, semantics, and phonetics.

Classical linguistics involved devising and evaluating rules of language. Great progress was made on formal methods for syntax and semantics, but for the most part, the interesting problems in natural language understanding resist clean mathematical formalisms.

Broadly, a linguist is anyone who studies language, but perhaps more colloquially, a self-defining linguist may be more focused on being out in the field.

Mathematics is the tool of science. Mathematicians working on natural language may refer to their study as mathematical linguistics, focusing exclusively on the use of discrete mathematical formalisms and theory for natural language (e.g. formal languages and automata theory).

Computational Linguistics

Computational linguistics is the modern study of linguistics using the tools of computer science. Yesterday’s linguistics may be today’s computational linguist as the use of computational tools and thinking has overtaken most fields of study.

Computational linguistics is the study of computer systems for understanding and generating natural language. … One natural function for computational linguistics would be the testing of grammars proposed by theoretical linguists.

— Pages 4-5, Computational Linguistics: An Introduction, 1986.

Large data and fast computers mean that new and different things can be discovered from large datasets of text by writing and running software.

In the 1990s, statistical methods and statistical machine learning began to and eventually replaced the classical top-down rule-based approaches to language, primarily because of their better results, speed, and robustness. The statistical approach to studying natural language now dominates the field; it may define the field.

Data-Drive methods for natural language processing have now become so popular that they must be considered mainstream approaches to computational linguistics. … A strong contributing factor to this development is undoubtedly the increase amount of available electronically stored data to which these methods can be applied; another factor might be a certain disenchantment with approaches relying exclusively on hand-crafted rules, due to their observed brittleness.

— Page 358, The Oxford Handbook of Computational Linguistics, 2005.

The statistical approach to natural language is not limited to statistics per-se, but also to advanced inference methods like those used in applied machine learning.

… understanding natural language require large amounts of knowledge about morphology, syntax, semantics and pragmatics as well as general knowledge about the world. Acquiring and encoding all of this knowledge is one of the fundamental impediments to developing effective and robust language systems. Like the statistical methods … machine learning methods off the promise of automatic the acquisition of this knowledge from annotated or unannotated language corpora.

— Page 377, The Oxford Handbook of Computational Linguistics, 2005.

Statistical Natural Language Processing

Computational linguistics also became known by the name of natural language process, or NLP, to reflect the more engineer-based or empirical approach of the statistical methods.

The statistical dominance of the field also often leads to NLP being described as Statistical Natural Language Processing, perhaps to distance it from the classical computational linguistics methods.

I view computational linguistics as having both a scientific and an engineering side. The engineering side of computational linguistics, often called natural language processing (NLP), is largely concerned with building computational tools that do useful things with language, e.g., machine translation, summarization, question-answering, etc. Like any engineering discipline, natural language processing draws on a variety of different scientific disciplines.

How the statistical revolution changes (computational) linguistics, 2009.

Linguistics is a large topic of study, and, although the statistical approach to NLP has shown great success in some areas, there is still room and great benefit from the classical top-down methods.

Roughly speaking, statistical NLP associates probabilities with the alternatives encountered in the course of analyzing an utterance or a text and accepts the most probable outcome as the correct one. … Not surprisingly, words that name phenomena that are closely related in the world, or our perception of it, frequently occur close to one another so that crisp facts about the world are reflected in somewhat fuzzier facts about texts. There is much room for debate in this view.

— Page xix, The Oxford Handbook of Computational Linguistics, 2005.

Natural Language Processing

As machine learning practitioners interested in working with text data, we are concerned with the tools and methods from the field of Natural Language Processing.

We have seen the path from linguistics to NLP in the previous section. Now, let’s take a look at how modern researchers and practitioners define what NLP is all about.

In perhaps one of the more widely textbooks written by top researchers in the field, they refer to the subject as “linguistic science,” permitting discussion of both classical linguistics and modern statistical methods.

The aim of a linguistic science is to be able to characterize and explain the multitude of linguistic observations circling around us, in conversations, writing, and other media. Part of that has to do with the cognitive size of how humans acquire, produce and understand language, part of it has to do with understanding the relationship between linguistic utterances and the world, and part of it has to do with understand the linguistic structures by which language communicates.

— Page 3, Foundations of Statistical Natural Language Processing, 1999.

They go on to focus on inference through the use of statistical methods in natural language processing.

Statistical NLP aims to do statistical inference for the field of natural language. Statistical inference in general consists of taking some data (generated in accordance with some unknown probability distribution) and then making some inference about this distribution.

— Page 191, Foundations of Statistical Natural Language Processing, 1999.

In their text on applied natural language processing, the authors and contributors to the popular NLTK Python library for NLP describe the field broadly as using computers to work with natural language data.

We will take Natural Language Processing — or NLP for short –in a wide sense to cover any kind of computer manipulation of natural language. At one extreme, it could be as simple as counting word frequencies to compare different writing styles. At the other extreme, NLP involves “understanding” complete human utterances, at least to the extent of being able to give useful responses to them.

— Page ix, Natural Language Processing with Python, 2009.

Statistical NLP has turned another corner and is now strongly focused on the use of deep learning neural networks to both perform inference on specific tasks and for developing robust end-to-end systems.

In one of the first textbooks dedicated to this emerging topic, Yoav Goldberg succinctly defines NLP as automatic methods that take natural language as input or produce natural language as output.

Natural language processing (NLP) is a collective term referring to automatic computational processing of human languages. This includes both algorithms that take human-produced text as input, and algorithms that produce natural looking text as outputs.

— Page xvii, Neural Network Methods in Natural Language Processing, 2017.

Deep Learning for Natural Language Processing

Deep learning techniques show a lot of promise for challenging natural language processing problems. Learn more here:

For an overview of how deep learning neural networks can be harnessed for natural language, see the post:

Further Reading

This section provides more resources on the topic if you are looking go deeper.

Books

Wikipedia

Summary

In this post, you discovered what natural language processing is why it is so important.

Specifically, you learned:

  • What natural language is and how it is different from other types of data.
  • What makes working with natural language so challenging.
  • Where the field of NLP came from and how it is defined by modern practitioners.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Develop Deep Learning models for Text Data Today!

Deep Learning for Natural Language Processing

Develop Your Own Text models in Minutes

...with just a few lines of python code

Discover how in my new Ebook:
Deep Learning for Natural Language Processing

It provides self-study tutorials on topics like:
Bag-of-Words, Word Embedding, Language Models, Caption Generation, Text Translation and much more...

Finally Bring Deep Learning to your Natural Language Processing Projects

Skip the Academics. Just Results.

See What's Inside

28 Responses to What Is Natural Language Processing?

  1. Avatar
    Tim September 22, 2017 at 9:07 pm #

    So do you think Computational Linguistics (CL) is basically the same field as Natural Language Processing (NLP) or do you view them as separate?

    • Avatar
      Jason Brownlee September 23, 2017 at 5:39 am #

      Hmmm, really it is a matter of the perspective on how to view the problem.

      It’s a lot like the way stats and ml can both view predictive modeling problems differently.

      Really, I’m a pragmatist and I’ll draw knowledge/wisdom from anywhere as long as I can get closer to a viable solution.

  2. Avatar
    Colin Goldberg February 13, 2018 at 5:10 am #

    In your blog on deep learning, you make this statement in response to someone’s question:

    “It seems that the niche for deep learning techniques is when you are working with raw analog data, like audio and image data.”

    With this in mind, what can be said about deep learning for natural language processing? What are the pros and cons?

    • Avatar
      Jason Brownlee February 13, 2018 at 8:07 am #

      Nice, I still stand behind that. With text it is the same idea, text is raw analog data.

      There’s tons of it, it’s full of ambiguity, there are few formal rules for language, etc.

  3. Avatar
    vikas jain March 30, 2018 at 5:58 am #

    a more interesting use case of NLP and deep learning combined is sarcasm detector. check this out. https://www.technologyreview.com/s/602639/how-vector-space-mathematics-helps-machines-spot-sarcasm/

    • Avatar
      Jason Brownlee March 30, 2018 at 6:46 am #

      Thanks for sharing.

      • Avatar
        Carilus Oselu August 31, 2021 at 4:19 am #

        Great article.

        • Adrian Tam
          Adrian Tam September 1, 2021 at 8:23 am #

          Thanks

  4. Avatar
    Arif November 18, 2018 at 4:04 am #

    Thank you for sharing your writing…

    I don’t learn language as a degree or research thing so can I still better use NLTK for projects that involved NLP?

    Do I need a great mathematical knowledge, specifically statistics and algorithm knowledge to understand NLP and NLTK?

    What are some examples of real use cases of NLP?

  5. Avatar
    Shivani December 5, 2018 at 3:21 am #

    What are the practical application of nlp sir

  6. Avatar
    Shubhi agrawal January 14, 2019 at 10:48 pm #

    Helpful Article!! I also have written an article on Natural processing language. http://www.nbdigitech.com/resources/what-is-natural-language-processing/

  7. Avatar
    James frederik March 25, 2019 at 10:04 pm #

    It’s too tough.

  8. Avatar
    Praveen S March 11, 2020 at 9:46 pm #

    Great Insight Jason!
    Here is our thoughts about NLP, Hope this will add value to this great post Thank you.

    https://notesnewtech.com/chatbot/what-is-natural-language-processing/

  9. Avatar
    Ashok Sagar June 5, 2020 at 5:36 pm #

    I read this post your post so nice and very informative post thanks for sharing this post

  10. Avatar
    kanha September 28, 2020 at 9:00 pm #

    Propose a mechanism to classify the users into suitable categories such as “arts and
    entertainment, religion and spirituality, automotive and vehicles, business etc.
    Only text data file is given task is create target feature.

    I don’t think we will be able to classify when there is no target variable is given.
    How to solve it.

  11. Avatar
    kanha September 28, 2020 at 9:02 pm #

    currently has over a 1,000 users and aims to increase its reach to 50,000 by the
    end of the fiscal year. Propose a warehousing system that you will implement to ensure timely
    update of their records given its dynamic nature.

  12. Avatar
    Willaim Aleena July 5, 2021 at 4:37 am #

    Yes it right! its seems that article is for deep learning processing language during working about analog images and audio data.

    With this in mind, while working artificial intelligence technology, I found below article…

    https://www.fabbyweb.com/the-working-method-of-artificial-intelligence-technology/

    What are the pros and cons?

  13. Avatar
    Tanedo August 11, 2021 at 3:33 am #

    Hi,

    I am looking same answer from author about pros and cons

  14. Avatar
    Yishai Rasowsky October 20, 2022 at 8:20 pm #

    Thank you for your articles! Where can I learn more from you about the classical NLP pipeline?

Leave a Reply