How to Load and Manipulate Images for Deep Learning in Python With PIL/Pillow

Before you can develop predictive models for image data, you must learn how to load and manipulate images and photographs.

The most popular and de facto standard library in Python for loading and working with image data is Pillow. Pillow is an updated version of the Python Image Library, or PIL, and supports a range of simple and sophisticated image manipulation functionality. It is also the basis for simple image support in other Python libraries such as SciPy and Matplotlib.

In this tutorial, you will discover how to load and manipulate image data using the Pillow Python library.

After completing this tutorial, you will know:

  • How to install the Pillow library and confirm it is working correctly.
  • How to load images from file, convert loaded images to NumPy arrays, and save images in new formats.
  • How to perform basic transforms to image data such as resize, flips, rotations, and cropping.

Discover how to build models for photo classification, object detection, face recognition, and more in my new computer vision book, with 30 step-by-step tutorials and full source code.

Let’s get started.

Tutorial Overview

This tutorial is divided into six parts; they are:

  1. How to Install Pillow
  2. How to Load and Display Images
  3. How to Convert Images to NumPy Arrays and Back
  4. How to Save Images to File
  5. How to Resize Images
  6. How to Flip, Rotate, and Crop Images

How to Install Pillow

The Python Imaging Library, or PIL for short, is an open source library for loading and manipulating images.

It was developed and made available more than 25 years ago and has become a de facto standard API for working with images in Python. The library is now defunct and no longer updated and does not support Python 3.

Pillow is a PIL library that supports Python 3 and is the preferred modern library for image manipulation in Python. It is even required for simple image loading and saving in other Python scientific libraries such as SciPy and Matplotlib.

The Pillow library is installed as a part of most SciPy installations; for example, if you are using Anaconda.

For help setting up your SciPy environment, see the step-by-step tutorial:

If you manage the installation of Python software packages yourself for your workstation, you can easily install Pillow using pip; for example:

For more help installing Pillow manually, see:

Pillow is built on top of the older PIL and you can confirm that the library was installed correctly by printing the version number; for example:

Running the example will print the version numbers for PIL and Pillow; your version numbers should be the same or higher.

Now that your environment is set up, let’s look at how to load an image.

Want Results with Deep Learning for Computer Vision?

Take my free 7-day email crash course now (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

Download Your FREE Mini-Course

How to Load and Display Images

We need a test image to demonstrate some important features of using the Pillow library.

In this tutorial, we will use a photograph of the Sydney Opera House, taken by Ed Dunens and made available on Flickr under a creative commons license, some rights reserved.

Sydney Opera House

Sydney Opera House

Download the photograph and save it in your current working directory with the file name “opera_house.jpg“.

Images are typically in PNG or JPEG format and can be loaded directly using the open() function on Image class. This returns an Image object that contains the pixel data for the image as well as details about the image. The Image class is the main workhorse for the Pillow library and provides a ton of properties about the image as well as functions that allow you to manipulate the pixels and format of the image.

The ‘format‘ property on the image will report the image format (e.g. JPEG), the ‘mode‘ will report the pixel channel format (e.g. RGB or CMYK), and the ‘size‘ will report the dimensions of the image in pixels (e.g. 640×480).

The show() function will display the image using your operating systems default application.

The example below demonstrates how to load and show an image using the Image class in the Pillow library.

Running the example will first load the image, report the format, mode, and size, then show the image on your desktop.

The image is shown using the default image preview application for your operating system, such as Preview on MacOS.

Sydney Opera House Displayed Using the Default Image Preview Application

Sydney Opera House Displayed Using the Default Image Preview Application

Now that you know how to load an image, let’s look at how you can access the pixel data of images.

How to Convert Images to NumPy Arrays and Back

Often in machine learning, we want to work with images as NumPy arrays of pixel data.

With Pillow installed, you can also use the Matplotlib library to load the image and display it within a Matplotlib frame.

This can be achieved using the imread() function that loads the image an array of pixels directly and the imshow() function that will display an array of pixels as an image.

The example below loads and displays the same image using Matplotlib that, in turn, will use Pillow under the covers.

Running the example first loads the image and then reports the data type of the array, in this case, 8-bit unsigned integers, then reports the shape of the array, in this case, 360 pixels wide by 640 pixels high and three channels for red, green, and blue.

Finally, the image is displayed using Matplotlib.

Sydney Opera House Displayed Using Matplotlib

Sydney Opera House Displayed Using Matplotlib

The Matplotlib wrapper functions can be more effective than using Pillow directly.

Nevertheless, you can access the pixel data from a Pillow Image. Perhaps the simplest way is to construct a NumPy array and pass in the Image object. The process can be reversed converting a given array of pixel data into a Pillow Image object using the Image.fromarray() function. This can be useful if image data is manipulated as a NumPy array and you then want to save it later as a PNG or JPEG file.

The example below loads the photo as a Pillow Image object and converts it to a NumPy array, then converts it back to an Image object again.

Running the example first loads the photo as a Pillow image then converts it to a NumPy array and reports the shape of the array. Finally, the array is converted back into a Pillow image and the details are reported.

Both approaches are effective for loading image data into NumPy arrays, although the Matplotlib imread() function uses fewer lines of code than loading and converting a Pillow Image object and may be preferred.

For example, you could easily load all images in a directory as a list as follows:

Now that we know how to load images as NumPy arrays, let’s look at how to save images to file.

How to Save Images to File

An image object can be saved by calling the save() function.

This can be useful if you want to save an image in a different format, in which case the ‘format‘ argument can be specified, such as PNG, GIF, or PEG.

For example, the code listing below loads the photograph in JPEG format and saves it in PNG format.

Running the example loads the JPEG image, saves it in PNG format, then loads the newly saved image again, and confirms that the format is indeed PNG.

Saving images is useful if you perform some data preparation on the image before modeling. One example is converting color images (RGB channels) to grayscale (1 channel).

There are a number of ways to convert an image to grayscale, but Pillow provides the convert() function and the mode ‘L‘ will convert an image to grayscale.

Running the example loads the photograph, converts it to grayscale, saves the image in a new file, then loads it again and shows it to confirm that the photo is now grayscale instead of color.

Example of Grayscale Version of Photograph

Example of Grayscale Version of Photograph

How to Resize Images

It is important to be able to resize images before modeling.

Sometimes it is desirable to thumbnail all images to have the same width or height. This can be achieved with Pillow using the thumbnail() function. The function takes a tuple with the width and height and the image will be resized so that the width and height of the image are equal or smaller than the specified shape.

For example, the test photograph we have been working with has the width and height of (640, 360). We can resize it to (100, 100), in which case the largest dimension, in this case, the width, will be reduced to 100, and the height will be scaled in order to retain the aspect ratio of the image.

The example below will load the photograph and create a smaller thumbnail with a width and height of 100 pixels.

Running the example first loads the photograph and reports the width and height. The image is then resized, in this case, the width is reduced to 100 pixels and the height is reduced to 56 pixels, maintaining the aspect ratio of the original image.

We may not want to preserve the aspect ratio, and instead, we may want to force the pixels into a new shape.

This can be achieved using the resize() function that allows you to specify the width and height in pixels and the image will be reduced or stretched to fit the new shape.

The example below demonstrates how to resize a new image and ignore the original aspect ratio.

Running the example loads the image, reports the shape of the image, then resizes it to have a width and height of 200 pixels.

The sized of the image is shown and we can see that the wide photograph has been compressed into a square, although all of the features are still quite visible and obvious.

Standard resampling algorithms are used to invent or remove pixels when resizing, and you can specify a technique, although default is a bicubic resampling algorithm that suits most general applications.

Resized Photograph That Does Not Preserve the Original Aspect Ratio

Resized Photograph That Does Not Preserve the Original Aspect Ratio

How to Flip, Rotate, and Crop Images

Simple image manipulation can be used to create new versions of images that, in turn, can provide a richer training dataset when modeling.

Generally, this is referred to as data augmentation and may involve creating flipped, rotated, cropped, or other modified versions of the original images with the hope that the algorithm will learn to extract the same features from the image data regardless of where they might appear.

You may want to implement your own data augmentation schemes, in which case you need to know how to perform basic manipulations of your image data.

Flip Image

An image can be flipped by calling the flip() function and passing in a method such as FLIP_LEFT_RIGHT for a horizontal flip or FLIP_TOP_BOTTOM for a vertical flip. Other flips are also available

The example below creates both horizontal and vertical flipped versions of the image.

Running the example loads the photograph and creates horizontal and vertical flipped versions of the photograph, then plots all three versions as subplots using Matplotlib.

You will note that the imshow() function can plot the Image object directly without having to convert it to a NumPy array.

Plot of Original, Horizontal, and Vertical Flipped Versions of a Photograph

Plot of Original, Horizontal, and Vertical Flipped Versions of a Photograph

Rotate Image

An image can be rotated using the rotate() function and passing in the angle for the rotation.

The function offers additional control such as whether or not to expand the dimensions of the image to fit the rotated pixel values (default is to clip to the same size), where to center the rotation the image (default is the center), and the fill color for pixels outside of the image (default is black).

The example below creates a few rotated versions of the image.

Running the example plots the original photograph, then a version of the photograph rotated 45 degrees, and another rotated 90 degrees.

You can see that in both rotations, the pixels are clipped to the original dimensions of the image and that the empty pixels are filled with black color.

Plot of Original and Rotated Version of a Photograph

Plot of Original and Rotated Version of a Photograph

Cropped Image

An image can be cropped: that is, a piece can be cut out to create a new image, using the crop() function.

The crop function takes a tuple argument that defines the two x/y coordinates of the box to crop out of the image. For example, if the image is 2,000 by 2,000 pixels, we can clip out a 100 by 100 box in the middle of the image by defining a tuple with the top-left and bottom-right points of (950, 950, 1050, 1050).

The example below demonstrates how to create a new image as a crop from a loaded image.

Running the example creates a cropped square image of 100 pixels starting at 100,100 and extending down and left to 200,200. The cropped square is then displayed.

Example of a Cropped Version of a Photograph

Example of a Cropped Version of a Photograph

Extensions

This section lists some ideas for extending the tutorial that you may wish to explore.

  • Your Own Images. Experiment with Pillow functions for reading and manipulating images with your own image data.
  • More Transforms. Review the Pillow API documentation and experiment with additional image manipulation functions.
  • Image Pre-processing. Write a function to create augmented versions of an image ready for use with a deep learning neural network.

If you explore any of these extensions, I’d love to know.

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Summary

In this tutorial, you discovered how to load and manipulate image data using the Pillow Python library.

Specifically, you learned:

  • How to install the Pillow library and confirm it is working correctly.
  • How to load images from file, convert loaded images to NumPy arrays, and save images in new formats.
  • How to perform basic transforms to image data such as resize, flips, rotations, and cropping.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.


Develop Deep Learning Models for Vision Today!

Deep Learning for Computer Vision

Develop Your Own Vision Models in Minutes

…with just a few lines of python code

Discover how in my new Ebook:
Deep Learning for Computer Vision

It provides self-study tutorials on topics like: classification, object detection (yolo and rcnn), face recognition (vggface and facenet), data preparation and much more…

Finally Bring Deep Learning to your Vision Projects

Skip the Academics. Just Results.

Click to learn more.


32 Responses to How to Load and Manipulate Images for Deep Learning in Python With PIL/Pillow

  1. Carlos March 22, 2019 at 7:55 am #

    Hello Jason,

    Thanks a lot for making all of us very accessible all this material.
    Computer vision has a lot of potential for you to apply all your previous work about deep learning.

    Thanks again,
    Carlos.

    • Jason Brownlee March 22, 2019 at 8:45 am #

      I agree, thanks.

      • prd March 22, 2019 at 5:17 pm #

        SIR i am working computer vision field .I have purchase your book on machine learning algorithms.I have gone through some selected topics.I am working on person re-identification.I have gone through some review paper and well as some other papers using deep learning.I want to write my own review on person reid but till now I could not able to write my own texonomy. Pls help to design my own taxonomy

  2. Mike March 22, 2019 at 8:03 pm #

    Another great article.
    Just so you know: your blog, ebooks and tutorials enabled me to get into machine learning. After two years I am now at a point where I am able to create commercial applications and am a certified professional.

    I don’t visit here as much as before as my projects mostly consist of stuff thats not your focus, but I still recommend your site to anyone asking for tutorials and guides on the subject.

    Great work. Thank you!

  3. Nirmala March 23, 2019 at 4:37 pm #

    Great post. Thanks for making it accessible to all of us.
    A quick question, if there is any text content written on the image, would it be possible to extract the text ?

    Thank you

    • Jason Brownlee March 24, 2019 at 7:03 am #

      Great suggestion, thanks.

      I hope to cover the topic in the future.

  4. JG March 24, 2019 at 5:40 am #

    Nice and valuable image tutorial as usual from Mr. Brownlee. Thanks!

    Because you can manipulate images with different libraries such as PIL (and PILLOW) and MATPLOTLIB, at the beginning you can get confused how to read, manipulate, save, show, etc. Images.
    In addition you have now Keras equivalent functions and methods such as load_image, image_to_array, array_to_image, preprocessing images such as ImageDataGenerator for data_augmentation, etc….so decided which one to use having so many parallels or equivalents ways to do it it is some time confused. Anyway, I think, Keras is now the more comprehensible, once you are working in Machine/Deep Learning.

    Something remarkable of imaging, at least was for me, is that when you read a image into a numpy array, that is you convert some .jpg format into a numpy array (later on you can save the np array in a “.npy” numpy format) , the volume of the file get multiply by 40 times in general. For example a 15 KB .jpg image get converted into 0.6 MB in np …so the only explanation it is the capacity of JPEG format to compress this digital numpy array !

    • Jason Brownlee March 24, 2019 at 7:09 am #

      Thanks, nearly of them build on and require PIL/Pillow.

      Nevertheless, I will be demonstrating each API in coming tutorials.

      Yes, I have this too. You must carefully choose precision (e.g. 1632 bit floats) and save using compressed numpy arrays.

  5. Gilles March 28, 2019 at 10:21 am #

    Easy, practical to the point. Thanks!

  6. Ishwarya April 4, 2019 at 12:41 pm #

    Hi there,

    Thanks for the useful post. I want to crop a part of an image, say for instance draw a rectangle at a desired angle over a small portion in a large image and then crop that part.
    Can you please suggest how i can crop it. I have the center point of the rectangle , height , width and angle at which it is tilted

  7. Vijayalakshmi April 7, 2019 at 2:01 am #

    I have the four coordinates of the rectangle. How to use this to crop the image. Can you please provide me code example to do them
    Thanks

    • Jason Brownlee April 7, 2019 at 5:31 am #

      I recommend referring to the example of using the crop() function in the above tutorial.

  8. Chiranjibi Sitaula April 13, 2019 at 11:55 am #

    Dear Sir,
    I am wondering to slice an image into two triangles with diagonal. How to do that? I am wondering about it. I need to slice an image with diagonals. Thanks

    • Jason Brownlee April 13, 2019 at 1:49 pm #

      Good question. Sorry, I don’t have an example of this.

  9. murat May 9, 2019 at 5:05 pm #

    hey,

    i need that how to load and manipulate LIST images for deep learning. You have worked just one image, but i need multiple images process. multiple images load, manipulate etc.

    please help me will you?

  10. Rumeli Depolama May 10, 2019 at 10:29 am #

    This is very useful article, thank you very much for machine learning

  11. poornima June 18, 2019 at 5:30 pm #

    sir, i am working on image comparison can you please help to how to compare two images in python and modules to be installed.

  12. Injamamul Haque June 26, 2019 at 12:42 pm #

    sir,is it possible to determine the speed of a object using pixel value ?

  13. Sunil July 14, 2019 at 6:56 pm #

    I want to give the labels to the images read from the folder, for example reading images from 4 folders and give labels for each folder.

    • Jason Brownlee July 15, 2019 at 8:16 am #

      Yes, the examples in this tutorial will provide an excellent starting point.

  14. Ahmet TORTUMLU July 20, 2019 at 4:59 pm #

    Thank your, really helpfull for beginners,

    i have questions

    1. How can we divede equal parts(for example 8 or 9) with this ways

    2.While i am managing images i am encountring error that image sizes are string 🙂

Leave a Reply