A brief about
Machine Learning
Prof. Kailash Singh
Department of Chemical Engineering
MNIT Jaipur
Introduction
• Machine Learning (ML) is the field of study that gives computers the
capability to learn.
• ML is data driven technology: Large amount of data is required for this.
• It can learn itself from past data and automatically improve prediction.
• From the given dataset, it can detect various patterns on data.
• ML is a subdomain of artificial intelligence (AI) that focuses on
developing systems that learn on the data they ingest.
• AI is a broad word that refers to systems or machines that resemble
human intelligence.
2
Contd…Definition of ML
• Arthur Samuel coined the term “Machine Learning ” in 1959.
• He defined machine learning as “the field of study that gives
computers the ability to learn without being explicitly programmed”
• Example:Handwriting recognition
• Task: Recognizing and classifying handwritten words within images
• Performance : Percent of words correctly classified
• Training: A dataset of handwritten words with given classifications
3
Contd…Example of ML learning
• A robot driving learning problem
• Task : Driving on highways using
vision sensors
• Performance : Average distance
traveled before an error
• Training : A sequence of images and
steering commands recorded while
observing a human driver
4
Difference between AI and ML
• AI can perform tasks typically requiring human intelligence. These tasks
include reasoning, problem-solving, perception, and decision-making.
• AI encompasses various subfields such as natural language processing,
robotics, computer vision, and machine learning.
• ML is a subset of AI that involves the development of algorithms enabling
computers to learn and make decisions based on data.
• Instead of being explicitly programmed for every task, ML systems improve
their performance as they are exposed to more data.
• Example: AI is used in virtual assistants like Siri and Alexa. ML is used in
recommendation systems on platforms like Netflix and Amazon, fraud
detection systems, etc.
ML in Chemical Engineering
• The conventional modeling and simulation process traditionally used
in chemical engineering is composed of the following steps:
1. Observe the physical phenomenon;
2. Assume hypotheses;
3. Generate a mathematical model from the fundamental equations;
4. Generate data from the model;
5. Compare the model with experimental data.
• The construction of model by ML reverses this order.
Ref.: Lavor et al. (2024), “Machine learning in chemical engineering: Hands-on activities,” Education for Chemical
Engineers, https://doi.org/10.1016/j.ece.2023.09.005
6
Inversion towards programming
7
Perspective in Chemical Engineering
• AI has made tremendous advancements in the last 15 years or so.
• Robotics, computer vision, speech recognition, and natural language processing
are stunning advances.
• Chemical engineers use AI and ML in various applications such as catalyst design.
• The use of AI in chemical engineering is not new, it is a 35-year-old ongoing
progress with remarkable successes.
• Chemical Engineering is undergoing an unprecedented transition—one that
presents significant challenges and opportunities in modeling and automated
decision-making.
• This has been driven by the convergence of economic and powerful computing
devices, tremendous progress in molecular engineering, the ever-increasing
automation of integrated operations, tightening environmental constraints, and
business demands for speedier delivery of goods and services to market.
Ref.: V. Venkatasubramanian (2019), “The Promise of Artificial Intelligence in Chemical Engineering: Is It Here,
Finally?”, AIChE Journal. 8
Contd…
• Artificial Intelligence is the study of how to make computers do things
at which, at the moment, people are better.
• AI could eventually end up doing all “things” that humans do, and do
them much better—that is, achieve super-human performance as
witnessed recently.
9
AI applications in various disciplines
• Game playing: Chess, Go
• Robotics: self-driving cars, etc.
• Vision: Facial recognition
• Speech recognition
• Natural language processing
• Expert systems or knowledge-based systems—for example, MYCIN,
CONPHYDE
• Clustering
10
Background of AI in Chemical Engineering
Phase-I: Expert Systems era
• The Expert Systems Era (from the early 1980s through the mid-
1990s), saw the first broad effort to exploit AI in chemical
engineering.
• Expert systems, also called knowledge-based systems, rule based
systems, or production systems, are computer programs that mimic
the problem-solving of humans with expertise in a given domain.
• Expert problem-solving typically involves large amounts of
specialized knowledge, called domain knowledge, often in the form
of rules of thumb, called heuristics, typically learned and refined over
years of problem-solving experience.
11
Contd…
• Expert system rapidly narrows down the search by recognizing
patterns and by using the appropriate heuristics.
• A number of impressive expert systems were developed, starting with
MYCIN, an expert system for diagnosing infectious diseases
developed at Stanford University during 1972–82.
• The first expert system application in chemical engineering was
CONPHYDE, developed in 1983 by Bañares-Alcántara, Westerberg,
and Rychner at Carnegie Mellon for predicting thermo-physical
properties of complex fluid mixtures.
• Another expert system DECADE was developed in 1985 at CMU for
catalyst design.
12
Contd…
• The first course on AI in PSE was developed and taught at Columbia
University in 1986 and it was subsequently offered at Purdue University
for many years.
• In 1986, Stephanopoulos published an article titled, “Artificial Intelligence
in Process Engineering”, in which he discussed the potential of AI in process
engineering and outlined a research program to realize it.
• The first AIChE session on AI was organized by Gary Powers (CMU) at the
annual meeting held in Chicago in 1985.
• The first international conference, Intelligent Systems in Process
Engineering (ISPE’95), sponsored by the Computer Aids for Chemical
Engineering (CACHE) Corporation, was organized by Stephanopoulos,
Davis, and Venkatasubramanian, in July 1995.
13
Phase II: Neural Networks Era (1990-2008)
• NN era was started in 1990 onward. NN acquired large amount of
data easing maintenance and development of models.
• Backpropagation algorithm gained popularity for training feedforward
NN to learn hidden patterns in input-ouput patterns. This algorithm
used gradient descent search method to adjust the weights of
connections between nodes iteratively.
14
Contd…
• Researchers made substantial progress on addressing challenging
problems in modeling, fault diagnosis, control, and product design.
• There were hundreds of articles in this domain during this phase.
• Despite the surprising success of neural networks in many practical
applications, some especially challenging problems in vision, natural
language processing, and speech understanding remained beyond
the capabilities of the neural nets of this era.
• Some researchers thought of need of neural nets with more hidden
layers to solve such problems.
15
Current Scenario
• The progress of AI over the last decade or so has been very exciting, and
the resource limitations mentioned above are largely gone now.
• Implementation difficulties have been greatly diminished.
• people have started to trust and accept recommendations from AI-assisted
systems such as ChatGPT, Google Gemini, Alexa, etc.
[Practice question for students: Make a report on the AI assisted systems
available today]
• In 1985, the most powerful computer was the CRAY-2 supercomputer. Its
computational speed was 1.9 gigaflops and it consumed 150 kW of power.
• Today Apple Watch performs at 3 gigaflops, while consuming just 1 W of
power. Normal desktops usually perform at 100 gigaflops.
16
Contd…
• There have been dramatic advances in software—in the performance of
algorithms and in high-level programming environments such as MATLAB,
Mathematica, Python, Julia, and TensorFlow.
• We have also seen great progress in wireless communication technologies.
• The other critical development is the availability of tremendous amounts of
data, “big data”, in many domains, which made possible the stunning
advances in ML.
• history would recognize that there were three early milestones in AI: One is
Deep Blue (IBM) defeating Gary Kasparov in chess in 1997, the second
Watson (IBM) becoming Jeopardy champion in 2011, and the third is the
surprising win by AlphaGO in 2016.
17
Phase III: Deep learning and the data science
era (~2005 to present)
• The era of Data Science or Predictive Analytics was made possible by
three important ideas:
• deep or convolutional neural nets (CNNs),
• reinforcement learning, and
• Statistical ML
• These technologies are behind the recent AI success stories in game
playing, natural language processing, robotics, and vision.
• Unlike neural nets of the 1990s, which typically had only one hidden
layer of neurons, deep neural nets have multiple hidden layers. Such
an architecture has the potential to extract features hierarchically for
complex pattern recognition.
18
Contd…
• The breakthrough came in 2006 by using a layer-by-layer training
strategy coupled with considerable increase in processing speed.
• A procedure called convolution in the training of the neural net made
feature extraction feasible.
• Convolution is a filtering technique, well-known in signal processing.
• Another architectural innovation was the recurrent neural network.
• The second important idea, reinforcement learning, can be thought
of as a scheme for learning a sequence of actions to achieve a desired
outcome, such as maximizing an objective function.
19
Contd…
• The third key idea, statistical ML, is the combination of mathematical
methods from probability and statistics with ML techniques.
• This led to a number of useful techniques such as LASSO, Support
Vector Machines, random forests, clustering, and Bayesian belief
networks.
20
Advice for future chemical engineers
• Developing AI methods is not just a matter of tracking new
developments in computer science and merely applying them in
chemical engineering.
• While there are some easy gains to be made, many intellectually
challenging problems in our industry are not amenable to AI solutions
yet. Nor can they be solved by computer scientists, as they lack our
domain knowledge.
• Since transport phenomena problems are solved by using differential
equations, mathematicians would address these core chemical
engineering problems.
21
Contd…
• To find the solution in chemical engineering, has to be well educated
in AI fundamentals, going beyond merely running ML code.
• Our UG and PG students need to become familiar with applied AI
techniques if we want keeps ourselves in AI race.
• So today’s engineering graduates should be well adapted to AI
techniques but be well acquainted with fundamentals of chemical
engineering to rightly apply these techniques.
22
The future of chemical engineering in the era
of Generative AI
• AI promises to revolutionize the way we work by automating labor-
intensive tasks and allowing complex processes to be completed in a
fraction of the time.
• Recently, ChatGPT provided a boost to pharmaceutical companies by
interacting with other AI tools and providing a more user-friendly
interface.
• This allowed ChatGPT to answer questions based on information
provided by the other tools rather than making users spend time
searching for the information themselves.
• By being integrated into the wider platform of AI tools, ChatGPT has
been able to improve the value of the entire system.
Ref.: Jin Xuan and Thorin Daniel (2023) The Future of Chemical Engineering in the Era of Generative AI, The Chemical Engineer.
23
Contd…
• Large language models (LLMs) have been used by students to solve
chemical engineering problems such as turbine efficiency and
reaction time.
• If these tools are taken from the classroom into industry, a whole
new set of opportunities for chemical engineering design will open
up.
• The necessity of coding ability in chemical engineering has been
established for some time, but many chemical engineers have limited
coding ability.
• Generative AI can automate the creation of small functions and
portions of programs by generating chunks of code.
24
Limitations of using ChatGPT in chemical
engineering
• While AI is powerful, it is limited in several significant ways.
• Many implementations of generative AI are hampered by a lack of
suitable data and an inability to accept different input data types.
• Recent attempts to design generative AI for P&ID automation have
shown promise but are hindered by the need for improved
comprehension of chemical engineering-specific terms and a lack of
available P&IDs to learn from.
• If complex chemical engineering models are to be successfully
developed, much more data is required which may prove difficult to
obtain.
25
• Chemical engineers of the future may use a suite of AI technologies to design
a chemical plant in a fraction of the time it currently takes.
• Future AI will integrate several different kinds of models into one combined
framework, able to parse information and generate results in one go.
• Unit design requires flowchart and P&ID creation, before an economic
analysis occurs. These steps could be condensed with the use of a set of AI
models.
• While the use of AI may speed up the design process, the results will still
need to be understood and the human aspect cannot be removed.
• So, these tools will not replace the skills of an engineer but allow them to
work with greater efficiency and flexibility.
• Overall, the chemical engineers are supposed to use AI tools with complete
understanding of the process.
26
Benefits of AI in Chemical Engineering
AI is the study of models capable of learning, analysing, and solving
problems—tasks that often require human intelligence. The Benefits of AI
are as follows:
• Process optimization: Analyses data such as process variables, raw
material characteristics, and operating conditions to optimize chemical
processes, maximise product yield while minimising energy use and waste.
• Safety Assessment: It can Identify faulty process conditions, anticipating
probable process malfunction and giving operators early warnings, which
can lower risks, prevent accidents, and enhance overall process safety.
• Catalyst Design: AI can assist in designing new catalysts with enhanced
properties, such as higher selectivity, better stability, and lower cost.
Contd…
• Sustainable Process Design: AI can help in the design of eco-friendly
chemical processes such as lowering greenhouse gas emissions, waste
creation, and energy consumption.
• Material Development: AI can help in developing new materials such as
polymers, adsorbents, catalysts, etc. with desirable features by analysing
databases of information about the properties of materials.
• Quality control: AI can help in quality control by forecasting quality
deviations and launching corrective actions by real-time analysis of sensor
data and other process parameters.
• Smart Manufacturing: Chemical companies use AI with IoT sensors to
create smart factories. AI processes real-time data to optimize production
parameters (like temperature, pressure, and flow rates), detect equipment
failures early, and schedule predictive maintenance.
Types of Machine Learning
• Supervised learning
• Unsupervised learning
• Semi-supervised learning
• Reinforcement learning
Ref.: Trinh, C.; Meimaroglou, D.; Hoppe, S. “Machine Learning in Chemical Product Engineering: The State of
the Art and a Guide for Newcomers”, Processes, 2021, 9, 1456. https://doi.org/10.3390/pr9081456
29
(a) Supervised learning
• The model is trained on labelled dataset. It learns by inputs and their
desired outputs and then finds patterns and connections between the
input and the output.
• The goal is to learn a general rule that maps inputs to outputs.
• The training process continues until the model achieves the desired
level of accuracy on the training data.
• There are two categories of supervised learning:
• Classification
• Regression
30
Classification algorithms
• Classification deals with predicting categorical target variables, which represent
discrete classes or labels.
• For example, classifying emails as spam or not spam,
or predicting whether a patient has a high risk of heart disease.
• Classification algorithms learn to map the input features to one of the predefined
classes.
• Some classification algorithms are:
• Logistic Regression
• Support Vector Machine
• Random Forest
• Decision Tree
• K-Nearest Neighbors (KNN)
• Naive Bayes
31
Regression algorithms
• Regression deals with predicting continuous target variables, which
represent numerical values.
• For example, predicting the price of a house based on its size, location, and
amenities, or forecasting the sales of a product.
• Regression algorithms learn to map the input features to a continuous
numerical value.
• Some regression algorithms are:
• Linear Regression
• Polynomial Regression
• Ridge Regression
• Lasso Regression
• Decision tree
• Random Forest
32
Applications of Supervised Learning
• Image classification: Identify objects, faces, and other features in images.
• Natural language processing: Extract information from text, such as sentiment,
entities, and relationships.
• Speech recognition: Convert spoken language into text.
• Recommendation systems: Make personalized recommendations to users.
• Predictive analytics: Predict outcomes, such as sales, customer churn, and stock
prices.
• Medical diagnosis: Detect diseases and other medical conditions.
• Autonomous vehicles: Recognize and respond to objects in the environment.
• Email spam detection
• Weather forecasting
33
(b) Unsupervised Machine Learning
• Unsupervised learning algorithm discovers patterns and relationships using
unlabeled data.
• The primary goal of Unsupervised learning is often to discover hidden
patterns, similarities, or clusters within the data, which can then be used
for various purposes, such as data exploration, visualization, dimensionality
reduction, and more.
• For example, we have mixed data of images of elephant, cats, dogs. Classify
the items based on their similarities.
• The most common problems are related to dimensionality reduction and
clustering
• Dimensionality reduction is used for the compression of large data sets.
Principal component analysis (PCA) is the most popular algorithm of this
family.
34
Contd…
• Clustering refers to the process of identification of existing clusters in
the input data. The clusters are groups of data that present a relative
similarity with respect to a specific characteristic.
• K-means clustering is a popular clustering algorithm, mainly due to its
ease in application and its low level of mathematical complexity.
• Other unsupervised learning algorithms are:
• Autoencoders (AE)
• Hierarchical clustering analysis (HCA),
• Independent component analysis (ICA)
• Gaussian mixture model (GMM)
35
Applications of Unsupervised Learning
• Clustering: Group similar data points into clusters.
• Anomaly detection: Identify outliers or anomalies in data.
• Dimensionality reduction: Reduce the dimensionality of data while preserving its
essential information.
• Recommendation systems: Suggest products, movies, or content to users based
on their historical behavior or preferences.
• Topic modeling: Discover latent topics within a collection of documents.
• Density estimation: Estimate the probability density function of data.
• Image and video compression: Reduce the amount of storage required for
multimedia content.
• Community detection in social networks: Identify communities or groups of
individuals with similar interests or connections.
• Content recommendation: Classify and tag content to make it easier to
recommend similar items to users. 36
(c) Semi-supervised Learning
• In semi-supervised learning, the data set is generally composed of a
small amount of labeled data and a majority of unlabeled data.
• It’s particularly useful when obtaining labeled data is costly, time-
consuming, or resource-intensive.
• This technique is mostly applicable in the case of image data sets
where usually all images are not labeled.
37
Applications of Semi-Supervised Learning
• Image Classification and Object Recognition: Improve the accuracy of models by
combining a small set of labeled images with a larger set of unlabeled images.
• Natural Language Processing (NLP): Enhance the performance of language
models and classifiers by combining a small set of labeled text data with a vast
amount of unlabeled text.
• Speech Recognition: Improve the accuracy of speech recognition by leveraging a
limited amount of transcribed speech data and a more extensive set of unlabeled
audio.
• Recommendation Systems: Improve the accuracy of personalized
recommendations by supplementing a sparse set of user-item interactions
(labeled data) with a wealth of unlabeled user behavior data.
• Healthcare and Medical Imaging: Enhance medical image analysis by utilizing a
small set of labeled medical images alongside a larger set of unlabeled images.
38
(d) Reinforcement Learning
• It focuses on learning a series of actions for optimizing an overall reward—
for example, winning at a game of chess.
• RL is centered around the concept of learning by interaction. It means that
RL learns from interactions with an environment to maximize a reward
function.
• Correct labels for learning a series of actions are not known upfront in RL—
instead, they need to be learned through interactions with the
environment
• With RL, the model (also called an agent) interacts with its environment,
and by doing so generates a sequence of interactions that are together
called an episode.
39
Contd…
• For example, imagine that we want to teach a computer to play the
game of chess and win against human players.
• The labels (rewards) for each individual chess move made by the
computer are not known until the end of the game, because during
the game itself, we don't know whether a particular move will result
in winning or losing that game.
• It would be a positive reward given if the computer won the game,
because the agent had achieved the overall desired outcome; and
vice versa, a negative reward if the computer had lost the game.
40
Application of Reinforcement Learning
• Robotics: Robots with pre-programmed behavior are useful in
structured environments, such as the assembly line of an automobile
manufacturing plant, where the task is repetitive in nature.
• A master chess player makes a move. The choice is informed both by
planning, anticipating possible replies and counter replies.
• An adaptive controller adjusts parameters of a petroleum refinery’s
operation in real time.
41
Stephen Hawking quote
• "I fear that AI may replace
humans altogether. If people
design computer viruses,
someone will design AI that
improves and replicates itself”.
• “AI will eventually reach a level
where it will essentially be a
new form of life that will
outperform humans.”
42
