AI terms: Machine Learning (ML), Reinforcement Learning (RL) and Deep Learning (DL)

An article by ScienceInsight
Article AI130

AI vs. ML

Artificial Intelligence (AI) is the overarching term that deals with machines capable of exhibiting human-like intelligence. Machine Learning, on the other hand, is not. The goal of machine learning is to teach a machine to perform a specific task and deliver precise results by identifying patterns. Machine Learning (ML) is a specific subfield of AI that focuses on developing algorithms and models that can learn from data and recognize patterns. Thus, AI is a much broader field than ML. ML is one specific method for achieving AI.

Difference Learning:

AI can attain the desired intelligence using various strategies and technologies, while ML is explicitly based on learning from data.

Difference Applications:

AI applications can include expert systems, process automation, and deep learning, while ML is employed in many AI applications such as image recognition and speech recognition. An AI system could be, for example, an autonomous vehicle, whereas an ML system is an algorithm that learns from image data to recognize objects.

Generative AI

Generative AI is a form of artificial intelligence that is trained to create new content such as text, images, music, audio, and videos. It operates through machine learning on large datasets and, based on algorithms and pre-trained foundational models, independently produces original results.

How generative AI works

Training data:

Generative AI models are trained on vast amounts of data to learn patterns and structures within it

Foundational models:

These models serve as base frameworks, are multitask-capable, and can be fine-tuned for specific use cases

Machine learning:

By leveraging machine learning algorithms and techniques, the AI can generate new content that mimics or extends existing patterns

User interaction:

Users input a “prompt,” and the AI then generates a response, such as text, an image, or a song, based on its learned knowledge

Examples of Applications

Content creation:

Generating texts, emails, reports, and presentations

Creative fields:

Producing images (AI art), music, and videos

Development:

Assisting in programming, creating 3D models and prototypes

Healthcare:

Analyzing medical images to aid diagnosis and drug development

Service:

Providing chatbots for customer support and auto mated responses

ML vs. RL vs. DL

Machine Learning (ML), Reinforcement Learning (RL), and Deep Learning (DL) are not the same. Reinforcement Learning and Deep Learning are specific types of Machine Learning. Machine Learning is a broader field that encompasses various learning methods, including Reinforcement Learning, Deep Learning, Supervised Learning, and Unsupervised Learning.

Machine Learning (ML)
Is a branch of artificial intelligence concerned with developing algorithms that can learn from data without being explicitly programmed.

Reinforcement Learning (RL)
RL is a specific type of Machine Learning where an agent learns to make decisions in a dynamic environment to achieve maximum utility or reward. Learning occurs through interaction with the environment, with the agent receiving feedback in the form of rewards or penalties. Need for simulations:
Training Reinforcement Learning models often requires simulations of the environment to replicate interactions and rewards.

Deep Learning (DL)
Deep Learning is a specific type of Machine Learning based on artificial neural networks with many layers (deep neural networks) and is particularly well-suited for tasks involving large datasets and complex patterns (a method for pattern recognition and prediction). While Machine Learning includes a broader range of algorithms, Deep Learning primarily uses neural networks to detect complex patterns in large datasets. Deep Learning can be utilized as a tool within Reinforcement Learning, for example, to improve state recognition or action selection.

Learning paradigm: Deep Learning is often a form of supervised learning, where models are trained on labeled data, but it can also be employed in unsupervised learning.

The main distinguishing feature between Reinforcement Learning and Deep Learning is that in Deep Learning, the learning process is based on a training dataset, which is then applied to a new dataset. Reinforcement Learning, on the other hand, learns dynamically by adjusting actions based on continuous feedback to maximize rewards. Reinforcement Learning represents a learning paradigm in which an agent learns to make optimal decisions through interaction with its environment.

KI fields

Machine Learning (ML)

Reinforcement Learning (RL)

Deep Learning (DL)

Definiton

ML is a method for creating AI systems that can learn from data and recognize patterns. It encompasses a broad range of different learning methods.

A specific learning method aimed at making decisions in dynamic environments to maximize utility.

A specific learning method that focuses on deep neural networks to identify and simulate complex patterns in data.

Learning process

The learning process can be carried out through various methods, such as Supervised Learning and Unsupervised Learning.

Learns through interaction with the environment (trial-and-error interaction) and receives feedback in the form of rewards/penalties. An agent performs actions and receives a reward or penalty based on the environment's response. It learns an optimal strategy through interaction with its environment to accomplish a specific task.

Learns from large amounts of data by using deep neural networks to identify structures and relationships within the data. Builds computational models composed of multiple layers, enabling the creation of abstraction levels from the data. Encompasses algorithms that learn from data to make predictions or decisions.

Algorithmen

Utilizes a variety of algorithms, such as decision trees, support vector machines, or linear models.

Commonly used algorithms include Q-learning, SARSA, policy gradient methods, and Deep Q-Networks (DQN). The choice of the appropriate algorithm depends on the specific application and the characteristics of the problem, such as the type of environment, the complexity of states and actions, and the available resources.

Relies on artificial neural networks with many layers (deep neural networks).

Datenmenge und Rechenleistung

Machine Learning can be more efficient with smaller datasets.

RL is typically resource-intensive, especially when deep learning is employed or real-world systems are being controlled. The required amounts of data and computational power are often orders of magnitude higher than in classic supervised learning.

Deep learning models often require very large amounts of data to train effectively and need a powerful computational infrastructure.

Feature Engineering*

With ML models, manual feature engineering is often required, meaning the creation of relevant input features for the model.

Enables efficient learning. Construction of a meaningful and compact state representation (s) from observations.

DL models can often extract features autonomously from the raw data.

Datentypen

Can use various data types, e.g., structured data.

Learns from collected experiences in a dynamic environment.

Can handle both structured and unstructured data, such as images, text, and speech.

Komplexität

Less complex than RL or DL

RL models are generally more complex and can also recognize more intricate patterns and structures in data that are difficult for standard machine learning models to capture.

DL models are typically more complex and can recognize even more intricate patterns and structures in data that are difficult for standard machine learning models to capture.

Anwendungen

Various fields of application, e.g., image recognition, speech processing

Robotics, game development (e.g., chess, Go), autonomous vehicles, process optimization

Deep Learning is often used for tasks that require high accuracy and the processing of large amounts of data, such as image recognition, speech recognition, pattern recognition, natural language processing, and autonomous driving.

*) What is meant by 'Feature Engineering', also referred to as "data preprocessing". It describes the processes involved in transforming raw data into features that can be used for developing machine learning models. In other words: The transformation and enrichment of data with the goal of improving the performance of machine learning algorithms used to train models with this data.

Terms in Reinforcement Learning

Term Meaning / Usage Origin / Library
obs Stands for observation – the current state of the environment as seen by the agent gym / gymnasium (OpenAI)
.step() Environment function: executes an action and returns obs, reward, done, info gym / gymnasium
env Stands for environment – the RL environment in which the agent operates gym, also used by stable-baselines3
reward Feedback after an action indicating how good or bad it was gym, but used by all RL frameworks
done Indicates whether an episode has ended gym / gymnasium
OpenAI Gym Library for defining RL environments Developed by OpenAI
stable-baselines3 High-level RL framework for easy use of algorithms such as PPO, DQN, etc. Community project, based on gym
model.learn() Training method for the RL model stable-baselines3
.predict() Makes predictions based on the trained model object stable-baselines3

OpenAI Gym

OpenAI originally developed Gym. OpenAI Gym is a free Python toolkit and API that provides a collection of simulated environments for training and testing reinforcement learning (RL) agents. It serves as an industry standard for developing, comparing, and evaluating RL algorithms by offering a unified interface for communication between learning algorithms and various environments.

Stable_baselines3

Stable-baselines3 is a framework for training agents. It consists of a set of improved implementations of reinforcement learning (RL) algorithms based on Farama Foundation (former OpenAI Baselines). Stable-baselines3 is not directly from former OpenAI, but rather an open-source project built upon it.

Best-known and practically relevant RL frameworks

Framework Type Language Short Description & Special Features
Gym / Gymnasium Low-Level Env Python OpenAI's standard API for RL environments. No algorithms included!
Stable-Baselines3 High-Level Python Easy to use, many algorithms (PPO, DQN, A2C, etc.), actively maintained
RLlib (Ray) High-Level Python Scalable, distributed RL, ideal for production environments
CleanRL Medium-Level Python Minimalistic implementations, ideal for learning
Tianshou High-Level Python Highly modular, supports many algorithms
Acme (by DeepMind) High-Level Python Research-oriented but complex. Usable with JAX
PettingZoo Multi-Agent Python Gym-like, but for multi-agent environments
SBX (Stable Baselines X) High-Level (experimental) Python Extension of SB3 with more features, still in development
Keras-RL2 Medium-Level Python Uses Keras/TensorFlow, but less actively maintained
Dopamine (by Google) Research Python Lightweight, specifically for DQN variants, but not very modular
Coach (by Intel) High-Level Python Many algorithms, but no longer actively developed
Unity ML-Agents Env + Training C# / Python Particularly suited for 3D simulations with Unity
TF-Agents High-Level Python TensorFlow-based, official from Google

Current RL learning frameworks

Framework Active? Backend Special Features / Focus
Stable-Baselines3 (SB3) active PyTorch Simple, modular, popular, many algorithms
CleanRL active PyTorch Extremely minimalist, one file per algorithm
Tianshou active PyTorch Highly modular, many algorithms, performant
RLlib (Ray) active PyTorch / TensorFlow Distributed RL, scales to clusters
Acme (DeepMind) less active JAX Research-oriented, structured, complex
SBX (Stable Baselines X) experimental PyTorch Extension of SB3 with features like HER
Keras-RL2 less active TensorFlow Keras-based, but rather outdated
TF-Agents (Google) less active TensorFlow Complex, heavyweight, for TF users
Dopamine (Google) discontinued TensorFlow Research focus on DQN, less modular
Coach (Intel) discontinued TensorFlow Many algorithms, difficult to use
Horizon (Meta) barely maintained PyTorch Meta's RL for production, high effort

Python libraries for RL

numpy:

NumPy is a powerful Python library essential for numerical calculations and scientific computing. It provides efficient data structures such as multidimensional arrays (N-dimensional arrays, or ndarrays for short) and offers a wide variety of functions for working with these arrays. NumPy is a cornerstone for many other Python libraries in the fields of data science, machine learning, and scientific programming.

pandas:

The Pandas library is a powerful open-source Python library specifically designed for data analysis and manipulation. It provides flexible data structures and functions that facilitate working with tabular and time-series data.

seaborn:

Seaborn is a Python library used for creating statistical graphics. It is built on top of Matplotlib and is tightly integrated with Pandas data structures. Seaborn simplifies the creation of complex diagrams and enables efficient data analysis and presentation by providing a user-friendly interface for statistical visualizations.

sklearn:

Scikit-learn, also known as sklearn, is a Python library for machine learning, supporting both supervised and unsupervised learning. It also provides capabilities for analyzing models using statistical methods. It is an open-source project for machine learning. This data science toolkit optimizes artificial intelligence (AI), ML, and statistical modeling with a consistent interface.

error: Content is protected
Scroll to Top