The different types of machine learning.

The field of learning is vast and there are many different approaches, and the same applies to Machine Learning models. Machine learning can be divided into several main approaches depending on the tasks to be carried out, but also on the elements available for study.

To prepare an ML model, several steps are necessary. The first step is to define the problem we want to solve. At the same time, we need to analyse the data we have available and prepare it for our context. From this point onwards, several machine learning models can be used, depending on their advantages and disadvantages with regard to the data.

This is followed by the second stage, in which we can evaluate the results of several models and improve the algorithms through optimisation and parameterisation. So it’s after this preliminary study phase that we can run a model.

One of the great strengths of machine learning is the richness of these approaches, which in many cases enable us to find a model to respond to a task. In the plethora of models that have been discovered, several distinctive elements can be used to categorise the models.

The first distinctive element is supervised and unsupervised learning.

Depending on the nature of the data available for the model, there are three main families which are regularly used as a basis for separating the approaches.

The first principle that divides ML processes is whether the input data sets are supervised or unsupervised. Learning consists of storing experience and using it to generate knowledge. But this experience can vary depending on the situation. Sometimes the experience consists of watching a situation unfold to which an answer is given, sometimes the answer is not given.

The rudiments of supervised learning.

If an answer is given for a sufficient number of cases, we are in the context of supervised learning. Finally, when an answer is given, it is as if a supervisor had provided additional information. There is a set of elements for which we know the answer, and we have a so-called test data set to validate our future model.

When data is provided with an expected output value, at least for part of the data, the aim is to define a general rule for matching the data to the output value. The output data, when defined, indicates that the sample is labelled or tagged. The labelled part of the data is generally provided by capturing the real situation in the form of data or by labelling at a later stage by experts in the field of study or by a group of individuals manually labelling large-scale data.

In other words, the aim is to define a predictive model which, for a set of elements (X; y) where X is a set of data and y is the answer given for the set X, can later find the unknown value y for a new element X. The elements of X are generally referred to as features or attributes. They can take many forms, from numerical data to images or text. Obviously, within a set of data X, there can be a mixture of different forms of data.

The element y can also be of several forms, but generally, it is a categorical data defined by a text or numerical format. In this case, we talk about a classification problem: we are trying to define the class in which a set X falls. When y is a non-categorical value, such as a real number for example, we talk about a regression problem. We try to determine the value of y on the basis of the relationships of other known pairs (X; y), we regress on what we know about the behaviour highlighted by the data set.

In the field of supervised learning, there are several methods for speech recognition, recommendations and spam detection.

The basics of unsupervised learning.

In the case where we have data, but no answer has been given, the situation becomes even trickier when it comes to defining a rule. Unfortunately, this is the most common case. In this case, we use unsupervised models.

Here, we no longer have a pair (X; y) but only several X elements without the y part. Data is present, but it is transmitted without a defined structure and no y-value is proposed. It’s up to the model to define the y-values that it considers effective for defining sets. Furthermore, data presented without a categorisation element is said to be unlabelled.

Unsupervised learning is very often used to detect abnormal phenomena such as fraud or rare diseases. It is often necessary to improve understanding of the input data by using dimension reduction or noise cleaning techniques.

To conclude on the two main categories of supervised and unsupervised learning, it is important to point out that there are hybrid approaches, in particular semi-supervised learning when the volume of labelled data is deemed insufficient for supervised learning models. This is very common when it is costly to involve experts in labelling datasets. This is often an effective alternative to the real difficulties of gathering and processing information.

The special case of reinforcement learning.

The last major field in machine learning is reinforcement learning, which uses a completely different paradigm. This approach is less well known, but it is gaining in interest, thanks in particular to the many uses it has been put to recently. Here, at each stage, the ML model will learn whether a given action is positive or negative by means of a reward distribution policy. The model will adapt its behaviour based on the feedback it receives from the data. The model tries to follow the objectives that have been defined. This field includes robotics and autonomous transport. The big difference in this family of models is that the algorithm interacts directly with the environment.

Differences linked to the use of machine learning.

Active and passive learning.

In this section, the question arises as to the action taken by the model. If, during its training, the model enhances the information initially made available by acting on the initial data, we are in the context of active learning. If the model remains within a strict reading of the input data, it is said to be passive.

Online and offline learning.

Another important distinction concerns the learning context and distinguishes between models that have to analyse the situation online, i.e. providing a result at the same time as learning, and those that have the time upstream (offline) to analyse a volume of training data and only rely on prior training to provide results a posteriori. In online learning, it’s easy to imagine that a progression curve is needed for the model to produce conclusive results. There are therefore hybrid approaches where the online model does not start completely without experience (cold start).

Differences in the actions to be performed on machine learning models.

Does the model require an external agent?

A major distinction concerns whether or not the model is supported. This can be done by involving a human expert or another algorithm to help the model achieve its objectives. It is also possible to use a player who is an adversary to the model, trying to mislead it so that it improves.

Setting parameters for Machine Learning algorithms.

Machine learning approaches can also be separated according to whether the model is parametrisable or non-parametrisable. A parametrisable model can be optimised by finding the right parameter configuration, and its performance can be analysed using a function.

The key point in defining the optimum configuration is the nature of the dataset made available. Often, datasets are limited in volume. Sometimes, and this is more serious, the datasets are not representative of the set of studies (overgeneralisation). In this case, it is possible for parameterisable Machine Learning algorithms to partially compensate for this shortcoming.

Differences in the choice of techniques used by the algorithms.

Finally, in addition to the above distinctions, ML algorithms are also divided into several families depending on the techniques used. There are four main families.

Logical learning

This is undoubtedly the approach most quickly put forward. Here, the human proposes rules to the algorithm, which the latter must follow in order to reason logically and make assumptions. Here we are close to the fundamentals of automation.

Statistical training

The data used in these learning algorithms plays a major role, since its nature will guide the search for a function to link the input data to the output data. To do this, the models use concepts from mathematics and, more commonly, statistics.

Learning using artificial neural networks

ANNs (Artificial Neural Networks) are based on the structure of neurons in animal biology to determine complex relationships between input and output data. This group has subsequently been broken down into a new sub-branch called deep learning, which involves using the computing capabilities of modern machines to design increasingly extensive (deep) networks to respond to complex problems.

Les algorithmes génétiques

Finally, genetic algorithms also imitate a biological concept, that of the theory of evolution, with events such as mutations and crossover to produce models that are optimised for a specific situation.

To conclude this initial introduction to the key differences between the various machine learning approaches, it is important to remember that Machine Learning offers a variety of solutions and that choosing the right method upstream is the key to success. We have not gone into the details of which families are the best for a given problem, simply because this question depends enormously on the context and particular cases. In the end, it is only by having a global knowledge of the different techniques that the AI engineer can propose a model after having analysed the framework in which he must work.

Photo Maxime Macé

Maxime Macé

Simple passionné de thématiques diverses et variées. J’apprécie enrichir mes connaissances dans les disciplines techniques comme l’informatique, les sciences et l’ingénierie, mais aussi dans les domaines merveilleux de la philosophie, l’art et la littérature.

Prenons contact

Vous souhaitez me contacter suite à la lecture de cet article ? Merci de renseigner votre adresse électronique et je vous recontacterai dans les plus brefs délais.

CV Ingénieur développeur informatique

CV Étudiant PhD Artificial Intelligence

Portfolio Artist designer