A Gentle Introduction to Text Summarization

Text summarization is the problem of creating a short, accurate, and fluent summary of a longer text document.

Automatic text summarization methods are greatly needed to address the ever-growing amount of text data available online to both better help discover relevant information and to consume relevant information faster.

In this post, you will discover the problem of text summarization in natural language processing.

After reading this post, you will know:

• Why text summarization is important, especially given the wealth of text available on the internet.
• Examples of text summarization you may encounter every single day.
• The application and promise of deep learning methods for automatic text summarization.

Kick-start your project with my new book Deep Learning for Natural Language Processing, including step-by-step tutorials and the Python source code files for all examples.

Let’s get started.

A Gentle Introduction to Text Summarization
Photo by Dmitry Sumin, some rights reserved.

Overview

This post is divided into 5 parts; they are:

1. Text Summarization
2. What is Automatic Text Summarization?
3. Examples of Text Summaries
4. How to Summarize Text
5. Deep Learning for Text Summarization

Text Summarization

There is an enormous amount of textual material, and it is only growing every single day.

Think of the internet, comprised of web pages, news articles, status updates, blogs and so much more. The data is unstructured and the best that we can do to navigate it is to use search and skim the results.

There is a great need to reduce much of this text data to shorter, focused summaries that capture the salient details, both so we can navigate it more effectively as well as check whether the larger documents contain the information that we are looking for.

Textual information in the form of digital documents quickly accumulates to huge amounts of data. Most of this large volume of documents is unstructured: it is unrestricted and has not been organized into traditional databases. Processing documents is therefore a perfunctory task, mostly due to the lack of standards.

— Page xix, Automatic Text Summarization, 2014.

We cannot possibly create summaries of all of the text manually; there is a great need for automatic methods.

In their 2014 book on the subject titled “Automatic Text Summarization,” the authors provide 6 reasons why we need automatic text summarization tools.

1. Summaries reduce reading time.
2. When researching documents, summaries make the selection process easier.
3. Automatic summarization improves the effectiveness of indexing.
4. Automatic summarization algorithms are less biased than human summarizers.
5. Personalized summaries are useful in question-answering systems as they provide personalized information.
6. Using automatic or semi-automatic summarization systems enables commercial abstract services to increase the number of texts they are able to process.

— Pages 4-5, Automatic Text Summarization, 2014.

Now that we know that we need automatic summaries of text, let’s better define what we mean by text summarization.

What is Automatic Text Summarization?

Automatic text summarization, or just text summarization, is the process of creating a short and coherent version of a longer document.

Text summarization is the process of distilling the most important information from a source (or sources) to produce an abridged version for a particular user (or users) and task (or tasks).

— Page 1, Advances in Automatic Text Summarization, 1999.

We (humans) are generally good at this type of task as it involves first understanding the meaning of the source document and then distilling the meaning and capturing salient details in the new description.

As such, the goal of automatically creating summaries of text is to have the resulting summaries as good as those written by humans.

The ideal of automatic summarization work is to develop techniques by which a machine can generate summarize that successfully imitate summaries generated by human beings.

— Page 2, Innovative Document Summarization Techniques: Revolutionizing Knowledge Understanding, 2014.

It is not enough to just generate words and phrases that capture the gist of the source document. The summary should be accurate and should read fluently as a new standalone document.

Automatic text summarization is the task of producing a concise and fluent summary while preserving key information content and overall meaning

— Text Summarization Techniques: A Brief Survey, 2017.

Next, let’s make this understanding concrete with some examples.

Examples of Text Summaries

There are many reasons and uses for a summary of a larger document.

One example that might come readily to mind is to create a concise summary of a long news article, but there are many more cases of text summaries that we may come across every day.

In their 1999 book on the topic titled “Advances in Automatic Text Summarization,” the authors provide a useful list of every-day examples of text summarization.

• headlines (from around the world)
• outlines (notes for students)
• minutes (of a meeting)
• previews (of movies)
• synopses (soap opera listings)
• reviews (of a book, CD, movie, etc.)
• digests (TV guide)
• biography (resumes, obituaries)
• abridgments (Shakespeare for children)
• bulletins (weather forecasts/stock market reports)
• sound bites (politicians on a current issue)
• histories (chronologies of salient events)

— Page 1, Advances in Automatic Text Summarization, 1999.

It is clear that we are reading and using summaries a more than we might first believe.

How to Summarize Text

There are two main approaches to summarizing text documents; they are:

1. Extractive Methods.
2. Abstractive Methods.

The different dimensions of text summarization can be generally categorized based on its input type (single or multi document), purpose (generic, domain specific, or query-based) and output type (extractive or abstractive).

— A Review on Automatic Text Summarization Approaches, 2016.

Extractive text summarization involves the selection of phrases and sentences from the source document to make up the new summary. Techniques involve ranking the relevance of phrases in order to choose only those most relevant to the meaning of the source.

Abstractive text summarization involves generating entirely new phrases and sentences to capture the meaning of the source document. This is a more challenging approach, but is also the approach ultimately used by humans. Classical methods operate by selecting and compressing content from the source document.

… there are two different approaches for automatic summarization: extraction and abstraction. Extractive summarization methods work by identifying important sections of the text and generating them verbatim; […] abstractive summarization methods aim at producing important material in a new way. In other words, they interpret and examine the text using advanced natural language techniques in order to generate a new shorter text that conveys the most critical information from the original text

— Text Summarization Techniques: A Brief Survey, 2017.

Classically, most successful text summarization methods are extractive because it is an easier approach, but abstractive approaches hold the hope of more general solutions to the problem.

Deep Learning For Text Summarization

Recently deep learning methods have shown promising results for text summarization.

Approaches have been proposed inspired by the application of deep learning methods for automatic machine translation, specifically by framing the problem of text summarization as a sequence-to-sequence learning problem.

Abstractive text summarization is the task of generating a headline or a short summary consisting of a few sentences that captures the salient ideas of an article or a passage. […] This task can also be naturally cast as mapping an input sequence of words in a source document to a target sequence of words called summary.

— Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond, 2016.

These deep learning approaches to automatic text summarization may be considered abstractive methods and generate a wholly new description by learning a language generation model specific to the source documents.

… the recent success of sequence-to-sequence models, in which recurrent neural networks (RNNs) both read and freely generate text, has made abstractive summarization viable

— Get To The Point: Summarization with Pointer-Generator Networks, 2017.

The results of deep learning methods are not yet state-of-the-art compared to extractive methods, yet impressive results have been achieved on constrained problems such as generating headlines for news articles that rival or out-perform other abstractive methods.

The promise of the approach is that the models can be trained end-to-end without specialized data preparation or submodels and that the models are entirely data-driven, without the preparation of specialized vocabulary or expertly pre-processed source documents.

… we propose a fully data-driven approach to abstractive sentence summarization. […] the model is structurally simple, it can easily be trained end-to-end and scales to a large amount of training data.

— A Neural Attention Model for Abstractive Sentence Summarization, 2015

Further Reading

This section provides more resources on the topic if you are looking go deeper.

Text Summarization Papers

• A Review on Automatic Text Summarization Approaches, 2016.
• A Review Paper on Text Summarization, 2016.
• Text Summarization Techniques: A Brief Survey, 2017.

Deep Learning Text Summarization Papers

• A Neural Attention Model for Abstractive Sentence Summarization, 2015
• Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond, 2016.
• Get To The Point: Summarization with Pointer-Generator Networks, 2017.

Books

• Advances in Automatic Text Summarization, 1999.
• Automatic Text Summarization, 2014.
• Innovative Document Summarization Techniques: Revolutionizing Knowledge Understanding, 2014.

Articles

• Automatic summarization
• Text summarization with TensorFlow, 2016
• Has Deep Learning been applied to automatic text summarization (successfully)?
• Taming Recurrent Neural Networks for Better Summarization, 2017.
• Deep Learning for Text Summarization

Summary

In this post, you discovered the problem of text summarization in natural language processing.

Specifically, you learned:

• Why text summarization is important, especially given the wealth of text available on the internet.
• Examples of text summarization you may encounter every single day.
• The application and promise of deep learning methods for automatic text summarization.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Develop Deep Learning models for Text Data Today!

Develop Your Own Text models in Minutes

...with just a few lines of python code

Discover how in my new Ebook:
Deep Learning for Natural Language Processing

It provides self-study tutorials on topics like:
Bag-of-Words, Word Embedding, Language Models, Caption Generation, Text Translation and much more...

Finally Bring Deep Learning to your Natural Language Processing Projects

Skip the Academics. Just Results.

See What's Inside