Text summarization is the problem of creating a short, accurate, and fluent summary of a longer text document.
Automatic text summarization methods are greatly needed to address the ever-growing amount of text data available online to both better help discover relevant information and to consume relevant information faster.
In this post, you will discover the problem of text summarization in natural language processing.
After reading this post, you will know:
- Why text summarization is important, especially given the wealth of text available on the internet.
- Examples of text summarization you may encounter every single day.
- The application and promise of deep learning methods for automatic text summarization.
Kick-start your project with my new book Deep Learning for Natural Language Processing, including step-by-step tutorials and the Python source code files for all examples.
Let’s get started.

A Gentle Introduction to Text Summarization
Photo by Dmitry Sumin, some rights reserved.
Overview
This post is divided into 5 parts; they are:
- Text Summarization
- What is Automatic Text Summarization?
- Examples of Text Summaries
- How to Summarize Text
- Deep Learning for Text Summarization
Need help with Deep Learning for Text Data?
Take my free 7-day email crash course now (with code).
Click to sign-up and also get a free PDF Ebook version of the course.
Text Summarization
There is an enormous amount of textual material, and it is only growing every single day.
Think of the internet, comprised of web pages, news articles, status updates, blogs and so much more. The data is unstructured and the best that we can do to navigate it is to use search and skim the results.
There is a great need to reduce much of this text data to shorter, focused summaries that capture the salient details, both so we can navigate it more effectively as well as check whether the larger documents contain the information that we are looking for.
Textual information in the form of digital documents quickly accumulates to huge amounts of data. Most of this large volume of documents is unstructured: it is unrestricted and has not been organized into traditional databases. Processing documents is therefore a perfunctory task, mostly due to the lack of standards.
— Page xix, Automatic Text Summarization, 2014.
We cannot possibly create summaries of all of the text manually; there is a great need for automatic methods.
In their 2014 book on the subject titled “Automatic Text Summarization,” the authors provide 6 reasons why we need automatic text summarization tools.
- Summaries reduce reading time.
- When researching documents, summaries make the selection process easier.
- Automatic summarization improves the effectiveness of indexing.
- Automatic summarization algorithms are less biased than human summarizers.
- Personalized summaries are useful in question-answering systems as they provide personalized information.
- Using automatic or semi-automatic summarization systems enables commercial abstract services to increase the number of texts they are able to process.
— Pages 4-5, Automatic Text Summarization, 2014.
Now that we know that we need automatic summaries of text, let’s better define what we mean by text summarization.
What is Automatic Text Summarization?
Automatic text summarization, or just text summarization, is the process of creating a short and coherent version of a longer document.
Text summarization is the process of distilling the most important information from a source (or sources) to produce an abridged version for a particular user (or users) and task (or tasks).
— Page 1, Advances in Automatic Text Summarization, 1999.
We (humans) are generally good at this type of task as it involves first understanding the meaning of the source document and then distilling the meaning and capturing salient details in the new description.
As such, the goal of automatically creating summaries of text is to have the resulting summaries as good as those written by humans.
The ideal of automatic summarization work is to develop techniques by which a machine can generate summarize that successfully imitate summaries generated by human beings.
— Page 2, Innovative Document Summarization Techniques: Revolutionizing Knowledge Understanding, 2014.
It is not enough to just generate words and phrases that capture the gist of the source document. The summary should be accurate and should read fluently as a new standalone document.
Automatic text summarization is the task of producing a concise and fluent summary while preserving key information content and overall meaning
— Text Summarization Techniques: A Brief Survey, 2017.
Next, let’s make this understanding concrete with some examples.
Examples of Text Summaries
There are many reasons and uses for a summary of a larger document.
One example that might come readily to mind is to create a concise summary of a long news article, but there are many more cases of text summaries that we may come across every day.
In their 1999 book on the topic titled “Advances in Automatic Text Summarization,” the authors provide a useful list of every-day examples of text summarization.
- headlines (from around the world)
- outlines (notes for students)
- minutes (of a meeting)
- previews (of movies)
- synopses (soap opera listings)
- reviews (of a book, CD, movie, etc.)
- digests (TV guide)
- biography (resumes, obituaries)
- abridgments (Shakespeare for children)
- bulletins (weather forecasts/stock market reports)
- sound bites (politicians on a current issue)
- histories (chronologies of salient events)
— Page 1, Advances in Automatic Text Summarization, 1999.
It is clear that we are reading and using summaries a more than we might first believe.
How to Summarize Text
There are two main approaches to summarizing text documents; they are:
1. Extractive Methods.
2. Abstractive Methods.
The different dimensions of text summarization can be generally categorized based on its input type (single or multi document), purpose (generic, domain specific, or query-based) and output type (extractive or abstractive).
— A Review on Automatic Text Summarization Approaches, 2016.
Extractive text summarization involves the selection of phrases and sentences from the source document to make up the new summary. Techniques involve ranking the relevance of phrases in order to choose only those most relevant to the meaning of the source.
Abstractive text summarization involves generating entirely new phrases and sentences to capture the meaning of the source document. This is a more challenging approach, but is also the approach ultimately used by humans. Classical methods operate by selecting and compressing content from the source document.
… there are two different approaches for automatic summarization: extraction and abstraction. Extractive summarization methods work by identifying important sections of the text and generating them verbatim; […] abstractive summarization methods aim at producing important material in a new way. In other words, they interpret and examine the text using advanced natural language techniques in order to generate a new shorter text that conveys the most critical information from the original text
— Text Summarization Techniques: A Brief Survey, 2017.
Classically, most successful text summarization methods are extractive because it is an easier approach, but abstractive approaches hold the hope of more general solutions to the problem.
Deep Learning For Text Summarization
Recently deep learning methods have shown promising results for text summarization.
Approaches have been proposed inspired by the application of deep learning methods for automatic machine translation, specifically by framing the problem of text summarization as a sequence-to-sequence learning problem.
Abstractive text summarization is the task of generating a headline or a short summary consisting of a few sentences that captures the salient ideas of an article or a passage. […] This task can also be naturally cast as mapping an input sequence of words in a source document to a target sequence of words called summary.
— Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond, 2016.
These deep learning approaches to automatic text summarization may be considered abstractive methods and generate a wholly new description by learning a language generation model specific to the source documents.
… the recent success of sequence-to-sequence models, in which recurrent neural networks (RNNs) both read and freely generate text, has made abstractive summarization viable
— Get To The Point: Summarization with Pointer-Generator Networks, 2017.
The results of deep learning methods are not yet state-of-the-art compared to extractive methods, yet impressive results have been achieved on constrained problems such as generating headlines for news articles that rival or out-perform other abstractive methods.
The promise of the approach is that the models can be trained end-to-end without specialized data preparation or submodels and that the models are entirely data-driven, without the preparation of specialized vocabulary or expertly pre-processed source documents.
… we propose a fully data-driven approach to abstractive sentence summarization. […] the model is structurally simple, it can easily be trained end-to-end and scales to a large amount of training data.
— A Neural Attention Model for Abstractive Sentence Summarization, 2015
Further Reading
This section provides more resources on the topic if you are looking go deeper.
Text Summarization Papers
- A Review on Automatic Text Summarization Approaches, 2016.
- A Review Paper on Text Summarization, 2016.
- Text Summarization Techniques: A Brief Survey, 2017.
Deep Learning Text Summarization Papers
- A Neural Attention Model for Abstractive Sentence Summarization, 2015
- Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond, 2016.
- Get To The Point: Summarization with Pointer-Generator Networks, 2017.
Books
- Advances in Automatic Text Summarization, 1999.
- Automatic Text Summarization, 2014.
- Innovative Document Summarization Techniques: Revolutionizing Knowledge Understanding, 2014.
Articles
- Automatic summarization
- Text summarization with TensorFlow, 2016
- Has Deep Learning been applied to automatic text summarization (successfully)?
- Taming Recurrent Neural Networks for Better Summarization, 2017.
- Deep Learning for Text Summarization
Summary
In this post, you discovered the problem of text summarization in natural language processing.
Specifically, you learned:
- Why text summarization is important, especially given the wealth of text available on the internet.
- Examples of text summarization you may encounter every single day.
- The application and promise of deep learning methods for automatic text summarization.
Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.
Thank you, Jason. Very interest work.
Jason, help me please. I try to work with NLP tasks.. not especially with text summarization.. but I think my question is relevant here too.. relevant for NLP.
I don’t find a way how can I save accuracy of my neural network, when the data, phrases have very different length.
The net fits on training data, and often feels collapse when she sees new data.
Perhaps use padding and masking to ignore padded values?
https://machinelearningmastery.com/handle-missing-timesteps-sequence-prediction-problems-python/
Thank you, Jason. I tried use masking too. I should continue….
If it is interest for you, you can try to solve NLP problem for Mercari price suggestion competition on Kaggle with me and other participants.
Sorry, I don’t have the capacity for new projects Alexander.
Do you cover text summarization in any of your books?
Not at this stage. I do have some posts on it though:
https://machinelearningmastery.com/?s=text+summarization&submit=Search
Hi,
Which top conferences you would recommend looking at the state of the art on Summarization?
Good question, I don’t know off hand.
Probably a good bet is Gensim’s summarizer
Good intro and useful links. Will also take a look at your book when I return to this topic in May. Could you explain “trained end-to-end without specialized data preparation”? What is “end-to-end” in this context? What do mean by “data prep”? Are you referring to structuring data in some way or doing named entity recognition, etc?
Exactly.
Hi Jason, I am new to all of this – a colleague forwarded your article and it’s very helpful. I was just wondering if you could point me to an example where text summarization has been used in a legal treatise to create chapter summaries? Thanks!
I don’t know off hand, I would recommend a search on google or google scholar.
Hi,
I am currently working on a problem to generate tags for movies that are due to be released in the following week so that users can search for these tags to get the movie that they want to go for. The tags such as cast, genre etc can be easily extracted using simple web scraping off of sites like imdb. However, I was looking to extract much more detailed tags for the movies that may explain intricate details which may not be attainable by web scraping.
An approach that I was considering is to use text summarisation to summarise the various reviews , descriptions, tweets and other content that is posted about the movie to generate the said tags. Do you think that this approach can work?
Sounds fun, sure try it.
Maybe cut scope to a specific subset of the problem and prototype models for that.
Hi Jason. Thanks for this really nice article about Text Summarization.
Jason I have question. I am interested in working NLP based Text Summarization process. How and where should I start ?
Start by defining your problem then gathering the data.
Sir, I am a final year undergraduate who wanted to make a project on this. But I do not have been into this field of line. Is it possible for me to work on something like this? If yes, then where should I begin?
A good place to start might be to read up on the problem, recent papers, etc.
Hi Jason, Thank you for this article. A few years ago, I wrote a “zero-install’ web summarizer which you could activate while browsing the Internet. It worked decently with articles that had a single theme (like news reports, as you mentioned).
For testing purposes, I did ask it to identify the three main sentences in the entire King James Bible. It did report back 3 sentences but they looked pretty random. In general, text summarization doesn’t work well with articles that talk about many events (e.g. a novel) or contain “funny text” (poems, riddles).
Here is a link to my program, if interested: http://ShowSummary.com
Thanks for sharing.
Hi Jason, I want to implement document text summarization using Python.
Please tell me how to do it?
I have a number of tutorials, you can get started here:
https://machinelearningmastery.com/?s=text+summarization&post_type=post&submit=Search
Hi Jason,
I am working on Organisation Behavior. I would like to classify the sentences in 5 / 10 different categories from the text generated from the Participants. In addition i would like to understand the overall theme related to 3 major categories.
1. Text (contain 10 -15 sentences) – Classify each sentence into minor category
2. Text – understand the overall theme – classify into major category
3. Summarize the report
I have training data where everything has been done manually. Now i have to do it automatically using machine learning. I can generate enough training data as much as required.
Can you please suggest the techniques? or where i can start
manivannan
Perhaps two separate models would be a good place to start.
Hi Jason can i ask what algorith should i use when creating a short story summarize system?
I recommend a number of architectures here:
https://machinelearningmastery.com/encoder-decoder-models-text-summarization-keras/
Hi,
nice article and wrap up, thank you! However, I do recommend to add the problems existing with the (current) evaluation of automatic text summarization systems. I good start might be Sebastian Ruder’s collection on summarization here:
http://nlpprogress.com/english/summarization.html
Maybe it’s enough to add it to the ‘Further Reading’ section.
All the best,
Tilman
Thanks Tilman.
Dear Jason,
I want to do Phd in this area what topic should I choose ? Please suggest.
I don’t know sorry.
Perhaps read some papers and consider where the gaps in understanding might be.
For web document summarization what is best to use on extractive and abstractive techniques?
Perhaps experiment with a few methods on your dataset and discover what works best?
Which library do you suggest for Text Summarization?
Is using spaCy library reduces the performance for large chunks of text?
Perhaps try a few approaches and discover what works best for your specific dataset.
Just great. Makes me want to go for the deep dive. Next up keyword-ology. Thanks
Thanks!
nice article and wrap up, thank you. Jason!
I was wondering how do we evaluate the performance of text summarization methods or how do I know the Text Summaries generated are good or not. Thanks!!
You’re welcome.
Good question, often measures such as bleu and rogue. I recommend checking the literature, specifically recent papers on the topic.