What is Deep Learning?

A lot is happening in the world of AI at the moment. Some of you may be wondering how machines have the ability to do what they can do. How can they recognise images, understand speech, and even reply to my requests???

Welcome to the world of Deep Learning. 

Deep Learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks.

Yes, I understand, that sounds very technical and overwhelming, right? 

If you are just starting out in the field of deep learning or you had some experience with neural networks some time ago, you may be confused. I know I was confused initially and so were many of my colleagues and friends who learned and used neural networks in the 1990s and early 2000s.

The leaders and experts in the field have ideas of what deep learning is and these specific and nuanced perspectives shed a lot of light on what deep learning is all about.

But what is deep learning? What is it all about? What did deep learning mean to the pioneers and thought leaders of today? If you’re thinking about these questions, then you’re in the right place. 

This article will explore what deep learning is by hearing from a range of experts and leaders in the field.

If you’re really keen to learn about deep learning, kick-start your project with my new book Deep Learning With Python, which includes step-by-step tutorials and the Python source code files for all examples.

Kick-start your project with my new book Deep Learning With Python, including step-by-step tutorials and the Python source code files for all examples.

Let’s dive in.

What is Deep Learning?

What is Deep Learning?
Photo by Kiran Foster, some rights reserved.

Andrew Ng on the Essence of Deep Learning

Renowned for his contributions to the field, Andrew Ng, founder of DeepLearning.AI and other platforms such as Coursera formally founded Google Brain. This eventually resulted in the productization of deep learning technologies across a large number of Google services.

Andrew Ng has frequently spoken and written a lot about what deep learning is, making him a great starting point for those who wish to learn more about the field.

In the early stages of deep learning, Andrew described deep learning in the context of traditional artificial neural networks. In the 2013 talk titled “Deep Learning, Self-Taught Learning and Unsupervised Feature Learning” he described the idea of deep learning as:

Using brain simulations, hope to:

– Make learning algorithms much better and easier to use.

– Make revolutionary advances in machine learning and AI.

I believe this is our best shot at progress towards real AI

Later his comments became more nuanced.

As time progressed, Andrew Ng’s insights into deep learning became more refined and nuanced.

According to Andrew, the core of deep learning is the availability of modern computational power and the vast amount of available data to actually train large neural networks. When discussing why now is the time that deep learning is taking off at ExtractConf 2015 in a talk titled “What data scientists should know about deep learning“, he commented:

very large neural networks we can now have and … huge amounts of data that we have access to

He also commented on the significance of scale in the world of deep learning. As we construct larger neural networks and train them with more and more data, their performance continues to increase. Unlike many traditional machine learning methods that reach a plateau in performance, deep learning stands out. 

Crafting bigger neural networks and furnishing them with increasing volumes of data will lead to a rise in efficacy. 

Andrew noted:

for most flavors of the old generations of learning algorithms … performance will plateau. … deep learning … is the first class of algorithms … that is scalable. … performance just keeps getting better as you feed them more data

Here is a nice cartoon explaining this from one of his slides:

Why Deep Learning?

Why Deep Learning?
Slide by Andrew Ng, all rights reserved.

Another point that Andrew Ng highlights is the importance of supervised learning within deep learning. Speaking at the 2015 ExtractConf, he pointed out:

almost all the value today of deep learning is through supervised learning or learning from labeled data

Echoing similar sentiments, during a 2014 lecture at Stanford University titled “Deep Learning,” he stated:

one reason that deep learning has taken off like crazy is because it is fantastic at supervised learning

Andrew often mentions that we should and will see more benefits coming from the unsupervised side of the tracks as the field matures to deal with the abundance of unlabeled data available.

Jeff Dean: The Architect Behind Google’s Deep Learning Infrastructure

Jeff Dean, a driving force behind the Systems and Infrastructure Group at Google, has now been appointed Google’s chief scientist and played a pivotal role and is perhaps partially responsible for the scaling and adoption of deep learning within Google. Jeff was involved in the Google Brain project and the development of large-scale deep learning software DistBelief and later TensorFlow.

In his 2016 presentation “Deep Learning for Building Intelligent Computer Systems” Jeff commented in a similar vein, that deep learning is really all about large neural networks.

When you hear the term deep learning, just think of a large deep neural net. Deep refers to the number of layers typically and so this kind of the popular term that’s been adopted in the press. I think of them as deep neural networks generally.

He has given this talk a few times, and in a modified set of slides for the same talk, Jeff emphasizes the scalability of neural networks indicating that results get better with more data and larger models, which in turn require more computation to train.

It seems like Andrew Ng and Jeff Dean were definitely having the same conversations.

Results Get Better With More Data, Larger Models, More Compute

Results Get Better With More Data, Larger Models, More Compute
Slide by Jeff Dean, All Rights Reserved.

Deep Learning: The Art of Hierarchical Feature Learning

Another characteristic of deep learning models is their ability to perform automatic feature extraction from raw data, also known as feature learning.

Yoshua Bengio: A Pioneer’s Perspective

Yoshua Bengio is another significant figure in the deep learning domain. Starting out with an interest in automatic feature learning capabilities that large neural networks are capable of achieving.

In his 2012 paper, “Deep Learning of Representations for Unsupervised and Transfer Learning” Bengio commented:

Deep learning algorithms seek to exploit the unknown structure in the input distribution in order to discover good representations, often at multiple levels, with higher-level learned features defined in terms of lower-level features

He then further expands on this idea in his 2009 technical report titled “Learning Deep Architectures for AI” where he emphasizes the importance of the hierarchy in feature learning.

Deep learning methods aim at learning feature hierarchies with features from higher levels of the hierarchy formed by the composition of lower level features. Automatically learning features at multiple levels of abstraction allow a system to learn complex functions mapping the input to the output directly from data, without depending completely on human-crafted features.

Bengio, in collaboration with Ian Goodfellow and Aaron Courville, published a book titled “Deep Learning” which defines deep learning in terms of the depth of the architecture of the models.

The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones. If we draw a graph showing how these concepts are built on top of each other, the graph is deep, with many layers. For this reason, we call this approach to AI deep learning.

This book has become a definitive resource within the field, presenting multilayer perceptrons as a core algorithm in deep learning, suggesting that deep learning has effectively integrated artificial neural networks.

Peter Norvig: Google’s Take on Depth and Abstraction

The quintessential example of a deep learning model is the feedforward deep network or multilayer perceptron (MLP).

Peter Norvig, the Director of Research at Google is well-known for his textbook on AI titled “Artificial Intelligence: A Modern Approach“.

In his 2016 presentation “Deep Learning and Understandability versus Software Engineering and Verification” Norvig resonates with Bengio’s views on deep learning. He defined deep learning with a focus on the power of abstraction permitted by using a deeper network structure.

a kind of learning where the representation you form has several levels of abstraction, rather than a direct input to output

The Evolution of the Term “Deep Learning”. Why Not Simply “Artificial Neural Networks”?

Geoffrey Hinton, a pioneer in the field of artificial neural networks co-published the first paper on the backpropagation algorithm for training multilayer perceptron networks.

In 2006, Hinton co-authored “A Fast Learning Algorithm for Deep Belief Nets” in which the term “deep” signified networks with multiple layers, particularly restricted Boltzmann machines.

Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.

This, along with another seminal paper Geoff co-authored titled “Deep Boltzmann Machines” on an undirected deep network were well received by the community as they were shown to be successful examples of greedy layer-wise training of networks, allowing many more layers in feedforward networks.

In another co-authored article in “Reducing the Dimensionality of Data with Neural Networks“,  the term “deep” persisted in describing their approach to developing networks with many more layers than was previously typical.

We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.

Echoing Andrew Ng’s sentiments about the fusion of computational power, using vast datasets, and optimal weight initialization, the article conveys:

It has been obvious since the 1980s that backpropagation through deep autoencoders would be very effective for nonlinear dimensionality reduction, provided that computers were fast enough, data sets were big enough, and the initial weights were close enough to a good solution. All three conditions are now satisfied.

Talking with the Royal Society in 2016 titled “Deep Learning“, Geoff commented that Deep Belief Networks were the start of deep learning in 2006. This success, especially in speech recognition, prompted both the neural network and speech recognition sectors to take heed in 2009 titled “Acoustic Modeling using Deep Belief Networks“, achieving state of the art results. 

It was the results that made the speech recognition and the neural network communities take notice of the use of “deep” as a differentiator on previous neural network techniques that probably resulted in the name change.

The descriptions of deep learning in the Royal Society talk are very backpropagation-centric as you would expect. Interestingly, Hinton gives 4 reasons why backpropagation (read “deep learning”) did not take off last time around in the 1990s. The first two points resonated highly with Andrew Ng’s comments about datasets being too small and computers being too slow.

What Was Actually Wrong With Backpropagation in 1986?

What Was Actually Wrong With Backpropagation in 1986?
Slide by Geoff Hinton, all rights reserved.

Deep Learning as Scalable Learning Across Domains

Deep learning has shown to particularly excel in scenarios where inputs and often outputs are analog. This means that rather than relying on a few quantities values in tabular format, deep learning has shown to thrive when dealing with pixel data from images, documents of text data or files of audio data.

Yann LeCun: The Visionary Behind Convolutional Neural Networks (CNNs)

Currently serving as the Vice-President, Chief AI Scientist at Meta, Yann LeCun is the father of the network architecture that excels at object recognition in image data called the Convolutional Neural Network (CNN). This technique has and continues to see great success because like multilayer perceptron feedforward neural networks, the technique scales with data and model size and can be trained with backpropagation.

This biases his definition of deep learning as the development of very large CNNs, which have had great success on object recognition in photographs.

During a 2016 presentation at Lawrence Livermore National Laboratory titled “Accelerating Understanding: Deep Learning, Intelligent Applications, and GPUs” LeCun described deep learning as the pursuit of hierarchical representations and further goes on to say that it as a scalable approach to building object recognition systems:

deep learning [is] … a pipeline of modules all of which are trainable. … deep because [has] multiple stages in the process of recognizing an object and all of those stages are part of the training

Deep Learning = Learning Hierarchical Representations

Deep Learning = Learning Hierarchical Representations
Slide by Yann LeCun, all rights reserved.

Jurgen Schmidhuber: The Innovator Behind Long Short-Term Memory Networks (LSTMs)

We bring another father to the discussion, Jurgen Schmidhuber, celebrated for his creation of the Long Short-Term Memory Network (LSTM), a type of recurrent neural network.

Schmidhuber has expressed his reservations about labeling the domain as “deep learning” in his 2014 paper titled “Deep Learning in Neural Networks: An Overview“. He comments on the problematic naming of the field and the differentiation of deep from shallow learning. He also interestingly describes depth in terms of the complexity of the problem rather than the model used to solve the problem.

At which problem depth does Shallow Learning end, and Deep Learning begin? Discussions with DL experts have not yet yielded a conclusive response to this question. […], let me just define for the purposes of this overview: problems of depth > 10 require Very Deep Learning.

Demis Hassabis and the Rise of DeepMind

Demis Hassabis, the visionary behind DeepMind, which was later acquired by Google, heralded a revolutionary fusion of deep learning and reinforcement learning. This combined breakthrough was able to handle complex learning problems like game playing, famously demonstrated in playing Atari games and the game Go with Alpha Go.

In keeping with the naming, they called their new technique a Deep Q-Network, combining Deep Learning with Q-Learning, subsequently dubbing the expansive domain as “Deep Reinforcement Learning”.

In their 2015 nature paper titled “Human-level control through deep reinforcement learning” they comment on the importance of deep neural networks and how it played a pivotal role in their breakthrough and highlighted the need for hierarchical abstraction.

To achieve this,we developed a novel agent, a deep Q-network (DQN), which is able to combine reinforcement learning with a class of artificial neural networks known as deep neural networks. Notably, recent advances in deep neural networks, in which several layers of nodes are used to build up progressively more abstract representations of the data, have made it possible for artificial neural networks to learn concepts such as object categories directly from raw sensory data.

Now coming to bring great minds together, what stands as a landmark paper in the realm of deep learning, Yann LeCun, Yoshua Bengio and Geoffrey Hinton published a paper in Nature titled simply “Deep Learning“. In it, they open with a clean definition of deep learning highlighting the multi-layered approach.

Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction.

Later the multi-layered approach is described in terms of representation learning and abstraction.

Deep-learning methods are representation-learning methods with multiple levels of representation, obtained by composing simple but non-linear modules that each transform the representation at one level (starting with the raw input) into a representation at a higher, slightly more abstract level. […] The key aspect of deep learning is that these layers of features are not designed by human engineers: they are learned from data using a general-purpose learning procedure.

Although it is a nice and generic description, this holistic description encapsulates the essence of most artificial neural network algorithms, offering a good note to end on.

Summary

In this post you discovered that deep learning is just very big neural networks on a lot more data, requiring bigger computers.

Although early approaches published by Hinton and collaborators focus on greedy layerwise training and unsupervised methods like autoencoders, modern state-of-the-art deep learning is focused on training deep (many layered) neural network models using the backpropagation algorithm. The most popular techniques are:

  • Multilayer Perceptron Networks.
  • Convolutional Neural Networks.
  • Long Short-Term Memory Recurrent Neural Networks.

I hope this has cleared up what deep learning is and how leading definitions fit together under the one umbrella.

If you have any questions about deep learning or about this post, ask your questions in the comments below and I will do my best to answer them.

295 Responses to What is Deep Learning?

  1. Avatar
    Gibachan August 16, 2016 at 7:24 am #

    If the deep learning is such great algorithm, do you think that other older algorithms (like SVM) are no longer efficient to solve our problems?

    • Avatar
      Jason Brownlee August 16, 2016 at 8:59 am #

      I think that SVM and similar techniques still have their place. It seems that the niche for deep learning techniques is when you are working with raw analog data, like audio and image data.

    • Avatar
      Tooba February 17, 2017 at 2:43 am #

      first of all I would like to appreciate your effort. This is one of the best blog on deep learning I have read so far.
      Well I would like to ask you if we need to extract some data like advertising boards from image, what you suggest is better SVM or CNN or do you have any better algorithm than these two in your mind?

      • Avatar
        Swapnil Pote March 17, 2017 at 1:22 am #

        CNN will give better result as compare to svm in image classification

        • Avatar
          Ibrahim Amer September 6, 2017 at 10:11 pm #

          CNN would be extremely better than SVM if and only if you have enough data. The reason that CNN would be better is that CNN work as an automatic feature extractor and you won’t need to bother yourself of feature selection and wondering if the extracted feature would weather work with the model or not. CNN extracts all possible features, from low-level features like edges to higher-level features like faces and objects.

    • Avatar
      Dudley Jude March 23, 2018 at 4:35 pm #

      As an Adult Education instructor (Andragogy), how can I apply deep learning in the conventional classroom environment?

    • Avatar
      Mike Rauner November 4, 2020 at 1:41 am #

      To answer your question, I would suggest you read this article – https://www.simplilearn.com/tutorials/deep-learning-tutorial/deep-learning-frameworks as it clearly explains the Deep Learning algorithms

      • Avatar
        Jason Brownlee November 4, 2020 at 6:44 am #

        Thanks for sharing.

        It summarises deep learning libraries, not algorithms.

  2. Avatar
    Alan Beckles MD MS August 16, 2016 at 11:12 am #

    Can CNNs perform tasks such as Medical Diagnosis or should they be

    combined with another technique such as Reinforement Learning to

    optimize performance?

    • Avatar
      Jason Brownlee August 16, 2016 at 11:20 am #

      Generally, CNNs are really good at working with image data.

      Medical Diagnosis seems like a really broad domain. You may want to narrow your scope and clearly define and frame your problem before selecting specific algorithms.

  3. Avatar
    Alan Beckles MD MS August 16, 2016 at 12:03 pm #

    ECG interpretation may be a good problem for CNNs in that they are images. Another project is the development of a Consultant in Cardiovascular Disease analogous to MYCIN, an Infectious Disease Consultatant developed by Shortliffe & Buchanan @ Stanford ~ 40 years ago which was Rule Based.

  4. Avatar
    napoleon Boakye September 9, 2016 at 1:24 am #

    So Jason, what is the next discovery after “deep learning”?

    • Avatar
      Jason Brownlee September 9, 2016 at 7:22 am #

      No idea Napoleon. Deep learning has enough potential to keep us busy for a long while.

      • Avatar
        salma June 6, 2018 at 3:20 am #

        sir plz let me know on what basis cnn is extracting features from an image….

  5. Avatar
    napoleon Boakye September 12, 2016 at 8:13 am #

    Okay

  6. Avatar
    Francesco D'Amore September 14, 2016 at 11:04 pm #

    Good overview.

    Take a look at this:

    http://deeplearning4j.org/

    It could be a good tool for DL?

  7. Avatar
    Jason Wills October 4, 2016 at 10:10 pm #

    hello, may deep learning apply to use in the stock market ?
    What I mean : it doesn’t just only use to draw with old data diagram and use the old model but also write down how is the next day to give the number forecast ?

    • Avatar
      Jason Brownlee October 5, 2016 at 8:28 am #

      Hi Jason, deep learning may apply to the stock market.

      I am not an expert in finance so I cannot give you expert advice. Try it and see.

      You may be interested in this post on time series forecasting with deep learning:
      https://machinelearningmastery.com/time-series-prediction-with-deep-learning-in-python-with-keras/

      • Avatar
        Jason Wills October 5, 2016 at 2:16 pm #

        Thank for your reply, I have read some your posts and I am very impressed with your work. About myself , I just start to find out what is this filed and you have many experiences about them. I hope if you have some experiences about the finance especially in stock market…pls help me some reference to learn it by myself or find the “Tribute”as you mentioned 🙂

  8. Avatar
    maisie Badami October 15, 2016 at 4:47 pm #

    loved it , thanks for the overview , answered to a lot of my question

    I am trying to find a topic for my Master-PHD proposal in Deep Learning in medical diagnosis and just wondering if there is any hot topic in this field at the moment ? and how can I learn more about this special field of Deep Learning

    • Avatar
      Jason Brownlee October 17, 2016 at 10:18 am #

      I’m glad to hear it was useful Maisie.

      I would suggest talking to medical diagnosis people about big open problems where there is access to lots of data.

    • Avatar
      Radya February 24, 2019 at 12:39 am #

      Hi maisie
      I am looking for some information for my master thesis either about CNN and deep learning in medical diagnosis
      So plz if u find anything usefull help me cause I’m new in the feild

      • Avatar
        Jason Brownlee February 24, 2019 at 9:10 am #

        Sounds like a great area.

        Perhaps start by reviewing recent papers on the topic?

        • Avatar
          Radya February 26, 2019 at 9:19 am #

          I am already but it seem a bit hard
          I am looking for advices so I can continue

  9. Avatar
    neha rahman October 20, 2016 at 5:15 am #

    i am looking for M tech thesis in this topic…help me explore new areas….

    • Avatar
      Jason Brownlee October 20, 2016 at 8:40 am #

      Hi neha, the best person to talk to about research topic ideas is your advisor. Best of luck.

  10. Avatar
    Abbey November 14, 2016 at 4:39 am #

    Hi Jason,

    Thank you so much for your post. I am trying to solve an open problem with regards to embedded short text messages on the social media which are abbreviation, symbol and others. For instance, take bf can be interpret as boy friend or best friend. The input can be represent as character but how can someone encode this as input in neural network, so it can learn and output the target at the same time. Please help.

    Regards
    Abbey

    • Avatar
      Jason Brownlee November 14, 2016 at 7:46 am #

      Very cool problem Abbey.

      I would suggest starting off by collecting a very high-quality dataset of messages and expected translation.

      I would then suggest encoding the words as integers and use a word embedding to project the integer vectors into a higher dimensional space.

      Let me know how you go.

  11. Avatar
    Sam Wilson January 5, 2017 at 4:14 am #

    Hi, thanks for the good overview.

    In your opinion, on what field CNN could be used in developing countries?
    Because there seems less raw data than developed countries, i couldn’t think of any use of CNN in developing countries.

    • Avatar
      Jason Brownlee January 5, 2017 at 9:41 am #

      Sorry Sam, I don’t know.

      CNNs are state of the art on many problems that have spatial structure (or structure that can be made spatial).

      Anything with images is a great start, domains like text and time series are also interesting.

    • Avatar
      Danie Truter March 21, 2017 at 1:10 am #

      Hi… I am an average developer in a developing country and my opinion is “yes”… if you find a way to get all these “disconnected” data together than you can help on gathering these data to make it easier for developing countries not to make the same mistakes as developed countries… thus bringing the cost down on “becoming” a developed country without the cost… the “research” exist… the implementation is the problem…

  12. Avatar
    Muhammad Faisal February 26, 2017 at 12:42 am #

    Hello Jason,

    a very well and nicely explained article for the beginners.
    I would like to ask one question, Please tell me any specific example in the area of computer vision, where shallow learning (Conventional Machine Learning) is much better than Deep Learning.

    • Avatar
      Jason Brownlee February 26, 2017 at 5:30 am #

      Great question, I’m not sure off hand. Computer Vision is not really my area of expertise.

  13. Avatar
    priyanka yemul March 13, 2017 at 9:23 pm #

    This article is useful for learning deep learning .Nice article

  14. Avatar
    Chris Jarvis March 14, 2017 at 8:36 am #

    Wonderful summary of Deep Learning – I am doing an undergraduate dissertation/thesis on applying Artificial Intelligence to solving Engineering problems.

  15. Avatar
    Danie Truter March 21, 2017 at 1:01 am #

    Hi… I am just an average normal developer, but I find this article very informative…

    May I please ask one question:

    If the “internet” and “line speed” was fast enough, would it mean these algorithms could learn itself or are the “programs” currently limited to human interaction during the learning stage…

    So my actual question: the “data” according to me is available -> “internet” BUT do we (humanity currently) already have the computational ability to make “sense” of the data via these algorithms AND are the software developed in such a way to ignore human approval?

    • Avatar
      Jason Brownlee March 21, 2017 at 8:42 am #

      The data needed to learn for a given problem varies from problem to problem. As does the source of data and the transmission of data from the source to the learning algorithm.

  16. Avatar
    Cyriac Peter March 22, 2017 at 5:56 am #

    Dr Jason, this is an immensely helpful compilation. I researched quite a bit today to understand what Deep Learning actually is. I must say all articles were helpful, but yours make me feel satisfied about my research today. Thanks again.

    Based on my readings so far, I feel predictive analytics is at the core of both machine learning and deep learning is an approach for predictive analytics with accuracy that scales with more data and training. Would like to hear your thoughts on this.

  17. Avatar
    Tran Anh Tuan March 30, 2017 at 6:23 pm #

    This article is very interesting and useful for a beginner in machine learning like me.

    I am thinking about a project (just for my hobby) of designing a stabilization controller for a DIY Quadrotor. Do you have any advice on how and where I should start off? Can algorithms like SVM be used in this specific purpose? Is micro controller (like Arduino) able to handle this problem?

    Thank you in advance

  18. Avatar
    Murali April 19, 2017 at 8:39 pm #

    hi
    Is the Deep Learning is suitable for prediction of any diseases like Diabetes using data mining algorithms?
    If yes give some ideas to work in it

    • Avatar
      Jason Brownlee April 20, 2017 at 9:24 am #

      It may be good, but try a suite of algorithms to see what works best on your problem.

      See this post:
      https://machinelearningmastery.com/a-data-driven-approach-to-machine-learning/

      • Avatar
        arif May 7, 2017 at 6:29 pm #

        hi jason are you fine i read your article that help me out and the comment section also
        may i know that which is the latest algorithm in deep neural network

        i need your help about neural network . i am working on neural network thanks

        • Avatar
          Jason Brownlee May 8, 2017 at 7:43 am #

          There are many deep learning algorithms.

          The most popular are MLPs for tabular data, CNNs for image data and LSTMs for sequence data.

  19. Avatar
    arif May 8, 2017 at 3:09 pm #

    thanks jason in research base which algorithm you suggest for me to work

    • Avatar
      Jason Brownlee May 9, 2017 at 7:38 am #

      I would suggest you pick an area that most excites you.

  20. Avatar
    arif May 12, 2017 at 1:25 am #

    JASON I WANT TO WORK IN MEDICAL AREA OR IMPLEMENT IN TO MEDICAL SITE

  21. Avatar
    shivam tripathi May 15, 2017 at 3:44 am #

    does deep learning is a solution of over-fitting problem in machine learning?

    • Avatar
      Jason Brownlee May 15, 2017 at 5:54 am #

      No, deep learning methods can overfit like any other.

  22. Avatar
    Mani May 19, 2017 at 4:32 pm #

    Thanks for the great article. What is the best approach for classifying products based on product description?

  23. Avatar
    Tofa May 26, 2017 at 6:30 pm #

    Lots of unnecessary points your explained which make difficult to understand what is actually deep learning is, also unnecessary explanaiton meke me bouring to read the document.

  24. Avatar
    Anthony June 6, 2017 at 7:19 pm #

    Jason,
    What do you think is the future of deep learning?
    How many years do you think will it take before a new algorithm becomes popular?

    • Avatar
      Jason Brownlee June 7, 2017 at 7:12 am #

      Hi Anthony,

      There is not one algorithm, but a population with the headliners: MLP, LSTM and CNN.

      I do not know where we are headed, sorry.

  25. Avatar
    Dubem June 16, 2017 at 10:02 pm #

    I am a student of computer science and am to present a seminar on deep learning, I av no idea of what is all about…..can I get articles dat can aid Me

  26. Avatar
    Neel June 26, 2017 at 3:00 pm #

    Hi Jason, I have been referring to a few of your blogs for my Machine Learning stuff. One striking feature of your blogs is simplicity which draws me regularly to this place! This is very helpful.:) Talking about Deep Learning vs traditional ML, the general conception is that Deep Learning beats a human being at its ability to do feature abstraction. Is this true?
    Also, could you tell me why Deep Learning fails to achieve more than many of the traditional ML algorithms for different datasets despite the assumed superiority of DL in feature abstraction over other algorithms?

    • Avatar
      Jason Brownlee June 27, 2017 at 8:27 am #

      Deep learning is great at feature extraction and in turn state of the art prediction on what I call “analog data”, e.g. images, text, audio, etc.

      It can be used on tabular data (e.g. spreadsheet of numbers) but this is not it’s sweet spot and often can be beaten by other methods, like gradient boosting.

      There is no one algorithm to rule them all, just different algorithms for different problems and our job is to discover what works best on a given problem.

      • Avatar
        Neel June 28, 2017 at 8:17 pm #

        Thanks for your response Jason! I appreciate your clarification.

  27. Avatar
    Yukin July 27, 2017 at 6:29 pm #

    I am wondering that if I use a convolutional neural work in my train model, could I say it is deep learning?

  28. Avatar
    Suchithra August 4, 2017 at 1:43 pm #

    Sir , it’s a great review about deep learning

    My question is what is the difference between deep neural network and CNN.

    Is deep learning is applicable to quantitative data( tabular data)

    I made a deep neural network model for bulk quantitative data and get a better result than traditional​ neural method.
    What it means sir ?

    Is my deep learning technique right?

    • Avatar
      Jason Brownlee August 4, 2017 at 3:45 pm #

      A CNN is a type of neural network. It can be made deep. Therefore, it is a type of deep neural network.

      Yes, neural nets require all input data to be tabular (vectorized).

      I cannot know if your model is right. Evaluate it carefully and compare it to other models.

  29. Avatar
    mera August 17, 2017 at 11:29 pm #

    if we use hierarchal training algorithm such as we use unsupervised learning autoencoder with bottleneck (hidden layer, 10,2,10) for training then use the supervised learning with same autoencoder architecture ( hidden layer, 10,2,10) to tune the unsupervised model parameter (weights, bias). These training processes are performed separately.

    now the unsupervised autoencoder works as dimension reduction and extract features. does the supervised model work in the same way and extract the feature or just this step conducted in the unsupervised learning

    • Avatar
      Jason Brownlee August 18, 2017 at 6:20 am #

      he supervised model will interpret the features and use them to make predictions.

  30. Avatar
    jayaram September 23, 2017 at 3:57 pm #

    can i say deeplerning == cnn, Do we have types in deep learning

    • Avatar
      Jason Brownlee September 24, 2017 at 5:14 am #

      CNN is a type of deep learning. So is RNN and MLP.

  31. Avatar
    Sudha September 23, 2017 at 8:15 pm #

    Sir, It is a good intro to deep learning. i’m planning to do phd in diagnosis of heart disease using deep learning. I have data’s of features. I don’t know how to classify those data. Can you please refer some material for numerical data classification using tensor flow.

  32. Avatar
    mhamed kassem October 3, 2017 at 11:28 pm #

    hi , i have started in my graduation project , and i am creating a medical system based on data comes from medical sensors , and i hope to use deep learning in detecting the disease or the health case based on that data , but i don’t know from where i should start in deep learning
    can you help me in doing it ?

  33. Avatar
    Cathy October 5, 2017 at 9:37 am #

    Hi Jason, thank you for the excellent overview. May I know how to apply deep learning in predicting adverse drug reactions, particularly in drug-drug interaction?

  34. Avatar
    Sudha October 21, 2017 at 7:48 pm #

    Hi Jason, Can you please tell me the unsupervised deep learning algorithms available? Please refer some link to learn about it.

    • Avatar
      Jason Brownlee October 22, 2017 at 5:18 am #

      Sorry, I don’t have material on unsupervised learning algorithms, I don’t find them useful in practice.

  35. Avatar
    naivebae October 31, 2017 at 2:56 pm #

    What does “bigger models” mean? Are there more equations in the model? Are there more variables in the model? Are there more for loops? How exactly is the model “bigger”? What’s an example of a “not big model” and why is that worse? And what exactly is meant by the term “model” in this field? I’m still not clear on that. Like what exactly are the characteristics that make something a “model” and why is another general programming algorithm like, say, a loop that divides numbers successively to determine if an integer is prime, not a “model”? To me that sounds like a “model” for determining if a number is prime, so what is meant in this field by “model”? What are the inherent properties that make something a “model”? Is a model a type of algorithm? Is it a class in object-oriented design? What’s a “model” and what;s a “bigger model”?

    • Avatar
      Jason Brownlee October 31, 2017 at 2:59 pm #

      Too many questions for one comment, sorry.

      • Avatar
        naivebae October 31, 2017 at 4:30 pm #

        Ok, let’s start by what exactly is meant by the term “model” in this field? Because I’m still absolutely not clear on what that means. If I write up some code to solve a problem, how do I know whether or not I’ve “built a model”?

        • Avatar
          Jason Brownlee November 1, 2017 at 5:43 am #

          A model is often referred to as the weights + structure after the training algorithm has been run on data.

          It is the “program” that can make predictions on new data.

          • Avatar
            naivebae November 1, 2017 at 7:55 am #

            And then what’s the “bigger model” you refer to in this article? Are there more weights and more structure in the training algorithm? How is that achieved? Do you plug in more equations and more variables and decision parameters/whatever? How do you know what additional equations and parameters to plug in, and how do you know those are the right ones as opposed to others? Or does “bigger models” simply refer to running on more overall data, and the training algorithm is the same with the same amount of equations/variables/computations per training example?

          • Avatar
            Jason Brownlee November 1, 2017 at 4:13 pm #

            “Larger Models” refers to deeper (more layers) or wider (more neurons per layer), ultimately more representational capacity.

  36. Avatar
    Sukesh Kumar Ranjan November 3, 2017 at 9:10 am #

    Best summary for students to learn basics of Deep Learning. Thank you Jason

  37. Avatar
    Souha November 8, 2017 at 8:01 pm #

    Thank you very much. It is very good summary about deep learning.
    Could you give some algorithms used in deep learning , please.

    • Avatar
      Jason Brownlee November 9, 2017 at 9:57 am #

      The three to focus on are: Multilayer Perceptron, Convolutional Neural Network and Long Short-Term Memory Network.

  38. Avatar
    chithambaram November 21, 2017 at 7:48 pm #

    how deep learning can be applied in music ? is it possible to analyze existing music to compose new music by computers ?? If yes what type of algorithm should be used ?

    • Avatar
      Jason Brownlee November 22, 2017 at 11:11 am #

      Yes, you could classify music or generate music.

  39. Avatar
    Javed Iqbal November 21, 2017 at 8:38 pm #

    Dear Dr Jason Brownlee, I really found this very useful and helpful for beginners to this domain.

  40. Avatar
    Hamed December 6, 2017 at 4:46 pm #

    I am familiar with machine learning and neural networks. My expertise is optimization and I am just interested in this field. What do you suggest as a good starting point? I prefer to learn it through experience and see how it works on different cases.

    • Avatar
      Jason Brownlee December 7, 2017 at 7:50 am #

      I have some advice for getting started here that might help:
      https://machinelearningmastery.com/start-here/#deeplearning

      • Avatar
        Hamed December 7, 2017 at 1:14 pm #

        Thanx a lot 🙂

        • Avatar
          Precious July 17, 2020 at 3:06 am #

          What are the objective study of deep learning

          • Avatar
            Jason Brownlee July 17, 2020 at 6:24 am #

            Some are interested in the limits of the method and pushing those limits, e.g. academics.

            Some are interested in better solutions to hard problem, e.g. engineers.

            I focus on the latter here.

  41. Avatar
    JD Maloy December 8, 2017 at 7:50 am #

    Do you think a deep-learning system could be “taught to read” in a manner similar to a young child? That is:
    1. Visual input of the words on each page
    2. Coordinated (i.e. synchronized) sound input of those words being read
    3. Repeat/reinforce as needed by “reading” the same book multiple times
    4. Expand the data set by “reading” a variety of books

    …with the ultimate goal & test being to present the system with a book it hasn’t “seen” before, and have it “read” that book via a synthesized voice.

    (Never mind the robotics to turn the pages, we can leave that part to a human assist.)

    • Avatar
      Jason Brownlee December 8, 2017 at 2:27 pm #

      A general system of this form could be constructed, not sure about the “like a child” part though.

  42. Avatar
    nazek hassouneh December 11, 2017 at 4:08 am #

    i am nazek hassouneh
    i am a master student and my thesis in deep learning
    i have data set with class label
    i will use deep learning for classification
    data set is an excel format and will convert to CSV form
    data set with 49 attribute and 131,000 records
    how can i use deep learning i need tutorial for that
    i need specific nueral networks suitable for this data set

  43. Avatar
    Alexander January 23, 2018 at 11:05 pm #

    Very Usefull Article In Deep Learning

  44. Avatar
    Tyler January 27, 2018 at 11:10 pm #

    Thanks Jason. Great article as always.

  45. Avatar
    Mat Rawsthorne January 29, 2018 at 6:21 am #

    Hi Jason
    loving your articles on ML and NLP! Apologies if this is a daft question but do the extra layers in deep learning models make them more or less transparent? I am thinking in the context of looking for patterns of helpful comments in forum exchanges – would I be able to recognise the features discovered (e.g. as a series of IFTTT rules) or would they just come out as a series of factor weights?
    Very new to this so any pointers most welcome
    Keep up the good work
    best wishes
    Mat

    • Avatar
      Jason Brownlee January 29, 2018 at 8:20 am #

      Layers add layers of abstraction which makes the model more complex/opaque.

  46. Avatar
    Mat Rawsthorne January 29, 2018 at 10:17 pm #

    Thanks Jason. I see that people have been making progress on increasing transparency in deep learning https://arxiv.org/abs/1710.09511
    I am working my way through your NLP crash course to understand more

  47. Avatar
    Alaa February 6, 2018 at 11:36 pm #

    Hi ..
    It’s a great blog I’ve ever seen , thanks a lot for all your efforts, wish you the best ????
    I am a master student and my thesis is about breast cancer diagnosis , I want to know is deep learning algorithm appropriate for my thesis
    and if the answer is yes which one is the best
    I’m so confused plz help me ????

    • Avatar
      Jason Brownlee February 7, 2018 at 9:24 am #

      I would recommend starting with CNNs for image data.

  48. Avatar
    Imtiyaz February 22, 2018 at 10:37 pm #

    Very nice one.I want to use deep learning in tourism sector. I can manage to get the tourists data.
    Can you tell me how can i use deep learning in tourism sector.

    Thanks

  49. Avatar
    Kelemu February 28, 2018 at 4:23 am #

    Thanks for sharing these types of soul idea especially for like underresourced country. I am trying to do my thesis by “Object detection by Deep learning for autonomous Vehicle” which method prefer for accomplished my thesis.

  50. Avatar
    anna March 2, 2018 at 1:26 pm #

    a very useful for beginner like me. and right now i’m in doing research with title “analysis on deep learning method for heart disease. which suitable technique i should use ?

    • Avatar
      Jason Brownlee March 2, 2018 at 3:24 pm #

      Thanks.

      Perhaps try a suite of methods and see what works best for your specific dataset.

  51. Avatar
    hans March 7, 2018 at 5:51 am #

    thanks very much jason for your valuable information , i have question about features extraction ;
    i know features selection , but what means by features extraction which is a property of deep Learning.

  52. Avatar
    Jomy March 8, 2018 at 9:13 pm #

    Want to learn more about deep neural network and its variants. Could you please tell me how?

  53. Avatar
    Fabio March 15, 2018 at 2:39 am #

    Hello Jason,

    I’m a PhD student working on a decentralized IDS (Intrusion Detection System) platform utilizing Big Data, and I’m using machine learning algorithms to detect some signature based attacks.

    I’m researching if deep learning would be a good AI method to detect non-signature based attacks (anomalies). I’ve used neural networks to train a specific node to detect a specific signature based attack, but I’m new to deep learning and not sure how apply it into my model.

    Would Multilayer Perceptron, Convolutional Neural Network or Long Short-Term Memory Network algorithms applicable at detecting anomalies with gigantic amounts of raw data?

    Thanks in advance and great article, very useful.

    • Avatar
      Jason Brownlee March 15, 2018 at 6:33 am #

      I would recommend testing a suite of methods to see what works best for your specific dataset.

  54. Avatar
    AJITH L M April 15, 2018 at 4:44 pm #

    If i am new to this where can i start , eventhough i read the full article its difficult for me to get some technical terms ? So where can i start if i am starting from scratch?
    THANK YOU

  55. Avatar
    Shreenivas Londhe April 30, 2018 at 12:30 pm #

    What about using Deep Learning in regression problems? Can it be useful for problems like ocean wave forecasting in univariate mode?

  56. Avatar
    Hazem May 1, 2018 at 10:59 pm #

    thank you Doctor Jason Brownlee
     For the great effort you make
    I have several questions and I hope to be as broad as we used to
    CNN is one of the neural networks that can be very deep but my question here is the code that distinguishes between being a normal neural network and being a deep neural network knowing that it can be used in both cases

    I have implemented a deep learning application that predicts the status of my client as he will continue his service or not

    The Code

    import theano
    import theano
    import tensorflow

    # Importing the libraries
    import numpy as np
    import matplotlib.pyplot as plt
    import pandas as pd
    # Importing the dataset
    dataset = pd.read_csv(‘C:/Python34/Churn_Modelling.csv’)
    X = dataset.iloc[:, 3:13].values
    y = dataset.iloc[:, 13].values

    from sklearn.preprocessing import LabelEncoder, OneHotEncoder
    labelencoder_X_1 = LabelEncoder()
    X[:, 1] = labelencoder_X_1.fit_transform(X[:, 1])
    labelencoder_X_2 = LabelEncoder()
    X[:, 2] = labelencoder_X_2.fit_transform(X[:, 2])

    onehotencoder = OneHotEncoder(categorical_features = [1])
    X = onehotencoder.fit_transform(X).toarray()
    X = X[:, 1:]

    # Splitting the dataset into the Training set and Test set
    from sklearn.model_selection import train_test_split
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.4)

    # Feature Scaling
    from sklearn.preprocessing import StandardScaler
    sc = StandardScaler()
    X_train = sc.fit_transform(X_train)
    X_test = sc.transform(X_test)

    # Importing the Keras libraries and packages
    import keras
    from keras.models import Sequential
    from keras.layers import Dense

    #Initializing Neural Network
    classifier = Sequential()

    # Adding the input layer and the first hidden layer
    classifier.add(Dense(activation=”relu”, kernel_initializer=”uniform”, units=7, input_dim=11))
    # Adding the second hidden layer
    classifier.add(Dense(activation=”relu”, kernel_initializer=”uniform”, units=7))
    # Adding the output layer
    classifier.add(Dense(activation=”sigmoid”, kernel_initializer=”uniform”, units=1))

    # Compiling Neural Network
    classifier.compile(optimizer = ‘adam’, loss = ‘binary_crossentropy’, metrics = [‘accuracy’])

    # Fitting our model
    classifier.fit(X_train, y_train, batch_size = 50, nb_epoch = 500)

    # Predicting the Test set results
    y_pred = classifier.predict(X_test)
    y_pred = (y_pred > 0.5)

    # Creating the Confusion Matrix
    from sklearn.metrics import confusion_matrix
    cm = confusion_matrix(y_test, y_pred)

    The Data set

    This application was found on the internet and its code is (Can I specify what functions (code) that make it a deep learning)

    And you have implemented an application for you to learn about God called Your First Machine Learning Project in Python

    What functions are used make me know whether this code is deep learning or machine learning

     In the past it was possible to put a lot of layers and use a large data size in MLP

    MLB is now one of the deep learning algorithms
     What code you have completed makes it a profound learning

    Sorry for the delay I would like to explain to me the application has been applied using the deep learning algorithm which is the same algorithm application is not using deep learning

  57. Avatar
    Hazem May 2, 2018 at 2:00 pm #

    I’m sorry to urge you Dr. Jason
    I would also like a small code showing the use of deep learning about traditional learning

    • Avatar
      Jason Brownlee May 3, 2018 at 6:30 am #

      What is “traditional learning” in the context of deep learning?

  58. Avatar
    Hazem May 3, 2018 at 3:34 pm #

    I mean traditional learning is the algorithms in which we do not use depth but similar in use
    Like RNN was used by the production of deep learning idea
    But I mean what the code will differentiate between RNN and DNN, knowing that RNN and many of the previous algorithms are deep learning algorithms

    Thank you for your patience with me

    • Avatar
      Jason Brownlee May 4, 2018 at 7:33 am #

      Generally, any neural network may be referred to as deep learning now. I don’t see a practical distinction other than for marketing.

  59. Avatar
    ilham May 20, 2018 at 1:32 pm #

    Dear Mr. Jason, there is one thing I don’t understand. In the article it is said that Deep Learning’s performance get better with more data while other, older algorithms tends to reach a plateau. Can you explain more and give an example about the plateau?

    Initially I think the plateau is there because more data can cause overfitting, but after some browsing I found out that more data will decrease the chance of overfitting. It is the number of feature, not the number of data that causes overfitting. The only thing I can think about how more data can create plateau is on heuristic algorithm, which can create more local minima where algorithms can get stuck on.

  60. Avatar
    Rayan June 10, 2018 at 6:51 pm #

    Thanks John. I found the article very useful.
    I am now confident I know what deep learning is.

  61. Avatar
    Deepansh Agrawal June 23, 2018 at 8:15 pm #

    A very good blog John. I am a newbie to the field of Deep Learning and this blog has helped me well.

    Thanks

    • Avatar
      Jason Brownlee June 24, 2018 at 7:31 am #

      I’m glad it helped, also, my name is “Jason”.

  62. Avatar
    Amit July 27, 2018 at 7:50 pm #

    Can we detect Malware Infections/DOS/Brute Force Attacks on any Network using Deep Learning?

  63. Avatar
    Sama August 2, 2018 at 11:13 pm #

    Hi, I want to know what are the deep learning methods using PAC Bayesian. And then compare them with other kind of methods.

    would you please clarify for me this problem?

  64. Avatar
    shano August 13, 2018 at 12:37 am #

    https://www.technoonews.com/2018/08/what-is-neural-network.html
    technology lover

  65. Avatar
    Sean King September 29, 2018 at 3:00 am #

    My brain just imploded.

  66. Avatar
    Karthik Mudlapur October 21, 2018 at 10:56 pm #

    Amazing read and this article has given me a head start to write my Master thesis in this field. Thanks.

  67. Avatar
    Preethi S October 23, 2018 at 7:24 pm #

    Hai Sir,
    I am Preethi, working with Visual Studio C++. My research problem is related to classification and prediction. Now I am trying to use the deep concept (sparse autoencoder ) in my problem, but I didnt get any C++ package for autoencoder. I found so many python packages, but I have to follow VS C++. OpenCV offers modules for CNN ,not for autoencoders.
    Could you pl suggest me any package to use with VS C++?

    Waiting for your kind response.

    • Avatar
      Jason Brownlee October 24, 2018 at 6:27 am #

      Sorry, I’m not familiar with cpp libraries for deep learning.

  68. Avatar
    Dr. Upasana Sharma November 13, 2018 at 6:07 pm #

    Very useful article on Deep Learning

  69. Avatar
    Ankit Sethia November 23, 2018 at 12:21 am #

    Kindly explain Convolutional Nets Model of Deep Learning in detail.

  70. Avatar
    Aarvish November 27, 2018 at 6:04 pm #

    Does Particle Swarm Optimization and ant bee colony algorithms which comes under artificial intelligence is related to deep learning method or unsupervised learning techniques in machine learning

    • Avatar
      Jason Brownlee November 28, 2018 at 7:38 am #

      Not really, they are optimization algorithms.

      Neural networks are function approximation algorithms.

  71. Avatar
    Kumar January 8, 2019 at 5:25 pm #

    This article has given me great knowledge on deep learning. currently doing research on data mining. Could you please suggest me how to apply deep learning for cancer classification. Right now I am applying cuckoo search optimization algorithm.

  72. Avatar
    Jack Liu January 17, 2019 at 9:41 pm #

    Hey I’m wondering what are your thoughts about AI being implemented into healthcare

  73. Avatar
    Md. Abbas Ali Khan January 24, 2019 at 3:52 pm #

    HI,
    I am beginner and I want to make a CVML- Computer Vision and Machine Learning lab. What tools and requirement have I need.

  74. Avatar
    Tingting January 29, 2019 at 3:57 am #

    Thanks Jason. This article is so well written and informative.

  75. Avatar
    Amit Jindia January 31, 2019 at 10:46 pm #

    Thanks. This article is so informative.

  76. Avatar
    Izzy February 24, 2019 at 9:54 am #

    Hi Jason and thank you for such an enlightening article.

    What I understood is that the hidden layers act as feature learners from the data. They achieve
    this by successively applying nonlinear transformations (with the activation functions) on the input data to map them into a new space: the feature space.
    In case of a classification task, the classes become easier (linearly) to separatein this feature space.
    What about in the case of regression? What is the function of the hidden layers in a deep neural network model for regression.

    Thank you in advance!

    • Avatar
      Izzy February 24, 2019 at 10:15 am #

      I would say: In case of regression, there is the nonlinear transformation of the input data to the feature space and there a linear regression in that new feature space can be applied to aproximate the numerical target variable.

    • Avatar
      Jason Brownlee February 25, 2019 at 6:34 am #

      Not quite, the model can learn a non-linear separation of classes.

      Same idea, but it learns a non-linear fit for association between the inputs to the output value.

  77. Avatar
    Izzy February 25, 2019 at 11:11 pm #

    ok, thank you for the response.
    Is what I said the case for SVR?

    • Avatar
      Jason Brownlee February 26, 2019 at 6:22 am #

      Not quite, an SVM can have a non-linear kernel.

      • Avatar
        Izzy March 6, 2019 at 5:04 am #

        yes sure. It is the non linear kernel that enables the non linear transformation of the input data to the feature space.
        Isn’t the same principle as the activation functions in the hidden layers of a deep neural network?

  78. Avatar
    Tejas B February 26, 2019 at 7:12 am #

    Hello Jeson

    this is regarding deep learning mini cource. i have done some research and read some material about deep learning and for deep learning mini cource – lession 1 – task , below are the details as requested

    10 impressive applications of deep learning methods in the field of natural language processing

    1. Text summarization

    https://arxiv.org/pdf/1707.02268.pdf

    2. Robust real-time detection, tracking, and pose estimation of faces in video streams

    https://ieeexplore.ieee.org/abstract/document/1334689

    3. Deep Residual Learning for Image Recognition

    http://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html

    4. Multi-digit Number Recognition from Street View Imagery

    https://arxiv.org/abs/1312.6082

    5. Automatic whale counting in satellite images with deep learning

    https://www.biorxiv.org/content/10.1101/443671v1.abstract

    6. A short-term building cooling load prediction method using deep learning algorithms

    https://www.sciencedirect.com/science/article/pii/S0306261917302921

    7. Deep learning for pixel-level image fusion: Recent advances and future prospects

    https://www.sciencedirect.com/science/article/pii/S1566253517305936

    8. Reading Text in the Wild

    https://link.springer.com/article/10.1007/s11263-015-0823-z

    9. Neural Machine Translation by Jointly Learning to Align and Translate

    https://arxiv.org/abs/1409.0473

    10. Interleaved Text/Image Deep Mining on a Large-Scale Radiology Database for Automated Image Interpretation

    http://www.jmlr.org/papers/volume17/15-176/15-176.pdf

    going through 2nd lession . Thank you for availing this information.

    Highly appreciated !!

    Thanks

  79. Avatar
    Nadjema bouaroui March 6, 2019 at 9:07 am #

    Hi
    Can you help me for my mastery
    Study thé properties of deep learning approximation

  80. Avatar
    Mohanish March 14, 2019 at 1:34 am #

    Hi sir ,I have one question that how to train the cifar 10 dataset with opencv and python such that after training it will generate xml file.
    My aim is to ” Detect objects using OpenCV an python ”.
    As I am new in this field, so please consider me.

    • Avatar
      Jason Brownlee March 14, 2019 at 9:26 am #

      Sorry, I don’t have any tutorials on this topic, I may cover it in the future.

  81. Avatar
    shivan April 21, 2019 at 8:52 am #

    hello dear

    how can I start with (Deep learning in medical image analysis) for my thesis

    thanks

    • Avatar
      Jason Brownlee April 21, 2019 at 9:07 am #

      Perhaps start here:
      https://machinelearningmastery.com/start-here/#dlfcv

      • Avatar
        shivan April 21, 2019 at 10:22 am #

        generally, I should start with computer vision after that and the more specific subject is in medical image analysis, is it right …..thanks

        • Avatar
          Jason Brownlee April 22, 2019 at 6:14 am #

          I recommend that you start working on your project directly, rather than study subjects to get ready to work on your project.

          Perhaps the most appropriate methods will be deep learning models like pre-trained convolutional neural networks.

  82. Avatar
    Sami April 22, 2019 at 10:33 pm #

    Hello Everyone my thesis topic is “CLEFT FACIAL AESTHETIC OUTCOME EVALUATION BASED ON DEEP TRANSFER LEARNING” how can I use deep learning and transfer learning model and to combine them and develop cleft facial aesthetic outcome measure system please help me I am really worried about it

  83. Avatar
    Andrews April 26, 2019 at 1:14 am #

    Thanks for the information, now I understand

  84. Avatar
    Osahon Okoro May 6, 2019 at 6:41 pm #

    Thanks Jason.

    The post really brought me to light about Deep learning. I don’t know if you can be of help for my M.Sc thesis. I intend to use deep learning to obtain sistolic and diastolic data readings from a wearable device then run it through CNN to produce a more accurate value as its output.
    The CNN will run on a parallel architecture to accommodate the processing power.

    And being a consultant for an ICT firm, i will also want to know if you are open to take up some consultancy contract with the firm? You can reach me on my email: if you are interested.

  85. Avatar
    James Lee May 22, 2019 at 6:41 pm #

    Been reading up a lot on machine learning/deep learning, AI thinking how I can use them to UP my marketing ANTE. Will probably need a tech guy to really do it, but just wanted to get a good grasp about the topic and then I came across yours. Great article!

  86. Avatar
    BHAVI June 19, 2019 at 3:34 am #

    Hi James ..your articles are always helping. can you please tell me ..how we can train a DQN model with the help of given dataset

    • Avatar
      Jason Brownlee June 19, 2019 at 8:16 am #

      I hope to cover that topic in the future.

      • Avatar
        BHAVI June 19, 2019 at 2:51 pm #

        I am waiting for your tutorial …….as they are the BEST

  87. Avatar
    Harvey Jones June 21, 2019 at 11:00 pm #

    Thanks for the article. I read a few more articles and decided to work in Tensorflow for deep learning.

  88. Avatar
    Mik June 24, 2019 at 1:40 pm #

    Hi jason
    Which part of deep learning needs to cogitated to improve deep learning?
    Is it approch of weigh choosing or the structure of neurals (number of layers and number of neuron in each layers or relation between each other)…. ?
    Which part it is????

  89. Avatar
    safeharbourship July 3, 2019 at 9:16 pm #

    deep learning in itself is an intense topic the way you have elaborate it is great job.please keep sharing such topic.

  90. Avatar
    Temesgen July 9, 2019 at 3:56 pm #

    how can I start with (Deep learning in bitcoin price prediction) for my thesis

  91. Avatar
    Valarmathi Srinivasan July 22, 2019 at 4:02 pm #

    Hello sir,

    Would like to know that how features are extracted in CNN

  92. Avatar
    Ezz July 26, 2019 at 11:44 pm #

    Hi. Being new to ML, this site is looking promising.Thank you.
    I was considering using Google AutoML (https://cloud.google.com/automl-tables/docs/ ) on structured and labelled (probabaly JSON) but very large sets of textual data, to encode some near-subjective decision making that could possibly be explicitly coded, but with quite some complexity.
    It could just be more elegant and scalable if a machine model could be trained, with human guidance.
    If that sounds at all feasible, should I start with machine-learning-in-r-step-by-step and at least get used to the field using the “shallow” ML algorithms mentioned there, or would deep-learning likely prove more beneficial but still be an intelligible subject to a newcomer?
    Eventually I’d want to employ this in an app.

  93. Avatar
    Luciana Silva September 20, 2019 at 6:17 am #

    Hi. Your posts are really good. I am learning a lot about ML. I would like to know whether deep learning can tacke classification problems when I have an unlabeled or partially labeled dataset.

    • Avatar
      Jason Brownlee September 20, 2019 at 1:32 pm #

      Thanks!

      Yes, this sounds like semi-supervised learning.

      I recommend testing a range of methods on your problem in order to discover what works best, including deep learning techniques.

  94. Avatar
    Andy October 19, 2019 at 7:50 am #

    Jason,
    I am a CS student and have taken other classes in DL, yet the current material in an in-depth class has me challenged. I understand the concepts, but have a hard time completing working code with all the pieces in the time I am given.

    We just finished RNN & CNN and for the next month will do reinforcement and unsupervised. I will be asked to take a network with distinct architecture and successfully train & test, most times with imported data. I am able to run different pieces of the code, but perfectly setting up all the parameters gives me a lot of trouble.

    do you think your new book will help me?

    • Avatar
      Jason Brownlee October 20, 2019 at 6:14 am #

      Setting the right parameters gives everyone trouble.

      The reason is there are no good theories for how to do it – the best we have is careful experimentation which basically means trial and error.

      My books are not a magic bullet, it sounds like you’re getting good guidance from your course already.

      If you need more advice on how to configure neural nets and diagnose faults, the tutorials here will help:
      https://machinelearningmastery.com/start-here/#better

  95. Avatar
    Ravita Mishra November 2, 2019 at 1:26 am #

    Hi Jason, I am Ravita Research scholar and doing research on scalable and efficient Job Recommendation using deep learning technique.
    I am new in deep learning technique, which algorithm is suitable for job recommendation.i am using CareerBuilder dataset. Pls. suggest me which algorithm give better accuracy and scalabilty.

  96. Avatar
    saif hameed November 7, 2019 at 9:19 pm #

    i loved the text
    my question is
    im going to work my PHD continue other relative work
    he work on automatic image annotation he used CNN
    con i continue using hybrid net. like CNN+LSTM+MLP or CNN+RNN something like that ?

  97. Avatar
    Prasannaa January 16, 2020 at 11:54 pm #

    How would you implement Predective mechanism for IT Service Management -Problems using Deep Learning

    • Avatar
      Jason Brownlee January 17, 2020 at 6:01 am #

      I would recommend testing a suite of algorithms for a problem and discover what works best, rather than starting with the solution (deep learning).

  98. Avatar
    Monisha R February 6, 2020 at 6:09 am #

    what are the reason(s) for the recent takeoff of deep learning?

  99. Avatar
    ubie February 26, 2020 at 6:10 am #

    I want to create a speech to text using ANN-based cuckoo search optimisation. Can you guide me through

    • Avatar
      Jason Brownlee February 26, 2020 at 8:29 am #

      I don’t have tutorials on this topic, sorry.

  100. Avatar
    Twenty Oktavia March 11, 2020 at 8:12 am #

    Talking about deep learning, Airlangga University conducted a study of the Multi Projection Deep Learning Network for Segmentation in 3D Medical Radiographic Images. Want to know more? Visit the following link:
    http://news.unair.ac.id/2019/12/19/multi-projection-deep-learning-network-untuk-segmentasi-pada-gambar-radiografi-medis-3d/
    Thank You

  101. Avatar
    JG June 6, 2020 at 9:22 pm #

    Great historical review and research about the persons and conceptual ideas behind the main milestones of Deep Learning development with keys references.

    I can see your are not only a great ML/DL populariser, but also and not less, a great scientific researcher of ML/DL

    Thanks Jason

  102. Avatar
    Medium June 10, 2020 at 7:09 am #

    I did a quick skim over the information and plan on going back and rereading it. I was curious I have experience in HTML, CSS, Javascript, PHP, and C++. I am planning to also learn Binary, Python, and Assembly as well as a few others. The reason why is I want to build A.I. that cross the border of weak and strong A.I. and approaches the threshold of Super A.I.s. In order to do so I plan on creating a computer language but want a bunch of references to do so. So, in the end, my question is. What language do you recommend for Deep Learning and other coding languages?

  103. Avatar
    CArlos cuesta June 29, 2020 at 9:19 am #

    Buen artículo, he tenido que leerlo dos veces.

  104. Avatar
    krishnapriya July 11, 2020 at 1:21 am #

    sir, I want to do research in machine learning for predicting disease. I saw a lot of papers about that. most of the papers using small data sets.

    I want to increase the size of the data sets what is the best method for large data sets. is it possible with MACHINE LEARNING? or deep learning is easy for large data sets

  105. Avatar
    lakshmi July 11, 2020 at 4:24 am #

    Is deep learning is used instead of using machine learning for predicting heart disease. for bigger data sets

    • Avatar
      Jason Brownlee July 11, 2020 at 6:21 am #

      Deep learning is a type of machine learning.

      You must discover what works best for your dataset.

      • Avatar
        Krishna Priya July 11, 2020 at 7:04 pm #

        Sir not about imbalanced data sets. In case if we are using thousands or lakhs of records are using instead of hundreds of recods.

        • Avatar
          Jason Brownlee July 12, 2020 at 5:49 am #

          The best method for large datasets is the method that performs the best on your dataset, and meets the requirements of your project.

  106. Avatar
    Rajesh SANYAL July 28, 2020 at 5:49 pm #

    I am a Physics student….I have no basics for any computer language… But would like to learn about DL and ML …. Where should I start

  107. Avatar
    Renisha August 2, 2020 at 5:55 pm #

    Hello sir, Can you provide an example for using deep learning to classify data that is in the form of CSV files?

  108. Avatar
    Adnan August 31, 2020 at 9:32 pm #

    hi sir : i need to build a project for detected baby crying … so i need a model for sound of crying baby in AI algorithm can you help me plz ..

    • Avatar
      Jason Brownlee September 1, 2020 at 6:29 am #

      Sorry, I don’t have tutorials on working with audio data. I cannot give you good off the cuff advice.

  109. Avatar
    Saeid Rezaei September 27, 2020 at 10:54 pm #

    Hello Jason, Thank you for your amazing blog. I have chosen the Deep Learning course this semester while I have a little information about Machine Learning, (I plan to choose ML course next semester). I wanna know is ML a prerequisite for deep learning? or I can learn Deep Learning algorithms and models and learn other classical ML algorithms simultaneously? Thank you in advance for your answer

  110. Avatar
    Ahmed Hamed January 7, 2021 at 3:53 am #

    Hello Jason,
    Can i use deep learning algorithm (CNN) in the resolution of optimization problem, example: I have a matrix with n*m values, in my case I need to select value from this matrix and turn the other to “0” in order to minimize the error or maximize energy?, Is there any matlab code do that?

    • Avatar
      Jason Brownlee January 7, 2021 at 6:22 am #

      I don’t think this is a deep learning problem, it sounds like straight linalg.

      I don’t have any examples that use matlab code sorry.

      • Avatar
        Ahmed Hamed January 7, 2021 at 7:17 pm #

        Thanks

  111. Avatar
    Roshan January 13, 2021 at 8:37 pm #

    can you provide the MLP algorithm for hand written digit classification using sklearn dataset?

    • Avatar
      Jason Brownlee January 14, 2021 at 6:13 am #

      Yes, there are many on the blog, use the search box.

  112. Avatar
    Shahbaz January 31, 2021 at 5:59 am #

    hi Sir, Hope u fine, my teacher said python is removed from the market in coming 10 year , is this reality in ur vision, and he said js will be the coming language in AI ,…what u think about it.

    • Avatar
      Jason Brownlee January 31, 2021 at 9:39 am #

      Perhaps ask your teacher why they think this and make up your own mind.

  113. Avatar
    KULDEEP March 17, 2021 at 12:49 am #

    (base) C:\Users\226399\Kerasprojects>python beach1.py
    Data Type:uint8
    Min:0.000

    Max: 255.000

    After Normalization
    Min:0.000,Max:1.000

  114. Avatar
    Mary Hill April 16, 2021 at 9:12 pm #

    Thanks for sharing. Informative Article.

  115. Avatar
    Alice June 29, 2021 at 7:55 pm #

    Hello Sir, i have tuned a model for facial emotion recognition and the highest accuracy so far is about 54% on average but i used ResNet50

    could you help me how can i improve the accuracy ..

  116. Avatar
    Alice July 1, 2021 at 6:32 pm #

    thank you so much

  117. Avatar
    Kofi Antwi September 11, 2021 at 7:11 pm #

    How do we cite your 2015 Extract Conference? How do we cite your useful comments? How do we cite extracts from slides you have used in conferences?

    • Adrian Tam
      Adrian Tam September 14, 2021 at 1:12 pm #

      You mean providing URLs?

  118. Avatar
    Capital Time November 24, 2021 at 7:53 pm #

    First, I appreciate your blog; I have read your article carefully, Your content is very valuable to me. I hope people like this blog too. I hope you will gain more experience with your knowledge; That’s why people get more information.

  119. Avatar
    Aakash November 24, 2021 at 11:47 pm #

    Very good explanation.. Really appreciate the efforts!!!

    other people interested to learn more about data science can go to- learnbay.co

  120. Avatar
    deepika January 4, 2022 at 6:09 pm #

    wow its such a great post! im really into digital world and learning been always the best method to keep me motivated. keep sharing ! i recently came to know Amritsar Digital Academy from here you can learn digital marketing course which is such a good and helpful course in today’s era! it worth trying.

    • Avatar
      James Carmichael January 7, 2022 at 8:16 am #

      Thank you for the feedback, Deepika!

  121. Avatar
    Camila January 14, 2022 at 12:06 am #

    Can tell us about the deep learning interview questions?

  122. Avatar
    Fatima March 4, 2022 at 6:22 am #

    Hi Dr. Jason, I have a dataset.
    it is a multi-class label classification, I want to apply Deep Neural networks, all the dataset features are probabilities, How can I prepare the data before applying the model. My question is Should I apply data normalization then build the model or do something else?

    Thanks
    Regards!

  123. Avatar
    wbaynews June 4, 2022 at 8:42 pm #

    First, I appreciate your blog; I have read your article carefully, your content is very valuable to me. I hope people like this blog too. I hope you will gain more experience with your knowledge; That’s why people get more information.

    • Avatar
      James Carmichael June 5, 2022 at 10:21 am #

      Thank you for the support and feedback wbaynews!

  124. Avatar
    Hossam July 16, 2022 at 11:33 pm #

    can you point out how to start project like tax fraud analysis

  125. Avatar
    Prabham November 4, 2022 at 10:05 pm #

    Deep learning in itself is an intense topic the way you have elaborate it is a great job. please keep sharing such topics.

    • Avatar
      James Carmichael November 5, 2022 at 8:11 am #

      Thank you for the support and feedback Prabham! We greatly appreciate it!

  126. Avatar
    Ula Yousef February 14, 2024 at 1:28 am #

    Thanks for such amazing article.
    Can you please give me a clear explanation about the difference between deep learning and machine learning,
    What they mean by shallow learning it is the same as machine learning ?

    • Avatar
      James Carmichael February 14, 2024 at 9:33 am #

      Hi Ula…You are very welcome! Machine Learning or ML is the study of systems that can learn from experience (e.g. data that describes the past). You can learn more about the definition of machine learning in this post:

      What is Machine Learning?
      Predictive Modeling is a subfield of machine learning that is what most people mean when they talk about machine learning. It has to do with developing models from data with the goal of making predictions on new data. You can learn more about predictive modeling in this post:

      Gentle Introduction to Predictive Modeling
      Deep Learning is the application of artificial neural networks in machine learning. As such, it is a subfield of machine learning. You can learn more about deep learning in this post:

      What is Deep Learning?

Leave a Reply