Deep Learning

An example showing how the ICGI uses Convolutional Neural Networks, also known as "Deep learning". The input image (1) is analyzed by applying different adaptive filters (3) on the image, each creating a different representation (4). New filters are then used on these representations to generate a new set of different representations (not shown). In general, this process is repeated until a desired result is produced, which in this case is the probability that a pixel in the input image belongs to a nucleus (7).

That is, for a given task, the performance of the algorithm on the task improves with the number and variety of tasks it solves. (Typically, the task is such that for a given input, an output is produced. For example, “what is this a picture of?” or “translate this sentence from French to English.”)

Animation showing how a Convolutional Neural Network may calculate features

The algorithms are based on characteristic features within the data, and solve the task based on these features. What separates deep learning from other machine learning methods is that these features are found automatically, as opposed to methods using manually designed ways of identifying features.

Deep learning methods are typically built as stacked representation learning methods, hence the term "deep". Each representation level is generated by extracting different features from the previous representation level. This hierarchical stack of representations can extract features ranging from very general to more detailed features. The combination of features is then used in the final classification. Deep artificial neural networks are especially suited for this, and is therefore very often used in deep learning.

The feature extractors are adaptive, and are optimized to minimize a certain cost function measuring the performance of the method on the task at hand. In supervised learning methods, the method is trained to perform well on a set of labeled training data. In order to achieve general results that also perform well on unseen examples, the amount and variety of data is very important. This is partly the reason why "big data" has been such a hype in recent years.

Deep learning is hardly a new phenomenon, but it has had a resurgence in recent years due to impressive results on certain tasks in computer vision and natural language processing. This renaissance can mainly be ­attributed to small improvements in methods, access to more computing power, and massive amounts of annotated data.