Nuclear Engineering and Design 415 (2023) 112698

Contents lists available at ScienceDirect

Nuclear Engineering and Design

journal homepage: www.elsevier.com/locate/nucengdes

Utilizing MATLAB machine learning models to categorize transient events

in a nuclear power plant using generic pressurized water reactor simulator
Muhammad Zubair a, *, Yumna Akram b
a
Department of Mechanical & Nuclear Engineering, University of Sharjah, P.O. Box 27272, Sharjah, United Arab Emirates
b
Nuclear Energy Systems Simulation & Safety Research Group, Sustainable Energy & Power Systems Research Centre, RISE, University of Sharjah, P.O. Box 27272,
Sharjah, United Arab Emirates

A R T I C L E I N F O A B S T R A C T

Keywords: Enhancing safety and dependability within nuclear power facilities holds paramount importance in safeguarding
Nuclear Safety both individuals and the environment. The adoption of machine learning for diagnosing faults in these plants is
Machine Learning steadily gaining interest, driven by its capacity to detect faults, alleviate human errors in high-pressure scenarios,
MATLAB
and ensure the secure and consistent operation of these facilities swiftly and accurately.
Artificial Intelligence
This paper examines the use of machine learning models for fault diagnostics, specifically, the identification of
transient events in a nuclear power plant to reduce human errors. The data was collected from WSC’s Generic
Pressurized Water Reactor (GPWR) simulator and processed using MATLAB. The simulator encompasses models
for both the primary and secondary systems of the nuclear power plant (NPP). Additionally, it incorporates
models for the control systems and instrumentation responsible for monitoring and regulating the reactor,
serving as integral components for data extraction and transient modeling. A total of 9 different transient events
were simulated with 12 different initial conditions to create a dataset with 72,000 observations. Nine types of
classification models (33 total preset models) were trained and validated using the classification learners
application. Among them, the Neural Network Classifiers (NNC) displayed the highest average accuracy of 90%.
The Fine Tree, Ensemble Bagged Trees, and Medium Neural Network models were the best-performing individual
models with validation accuracies above 90% and a maximum training time of 8 min. These models were further
analyzed using accuracy, confusion matrix, precision, recall, and F1 score. To optimize these models, techniques
such as different validation schemes and feature selection were utilized to further reduce their training time and
improve their prediction accuracy. The optimized models boasted comparable accuracies with a maximum
training time of under 1.5 min. The results of this study exhibited favorable comparisons with other machine-
learning endeavors in the field of reactor transient detection and diagnostics. Notably, the study maintained
low execution and computation times while preserving high levels of accuracy. This study offers insightful in­
formation on how AI and machine learning can be used to improve nuclear power plant diagnostics, enhance
safety, and provide support to the operator.

1. Introduction Monitoring Techniques for Performance Assessment, n.d.). Nuclear en­

ergy is a critical source of energy that can fulfil the world’s ever-
The world’s increasing population and energy demands have led to increasing need for energy while addressing the pressing need for
the rapid depletion of finite energy sources such as fossil fuels. While clean and sustainable energy sources. Although not renewable in the
these methods provide cheap and fast energy, they also produce harmful traditional sense; nuclear energy presents itself as a sustainable, low-
gases such as CO2, which contribute to global warming by increasing the carbon emitting energy source, distinguished by its unwavering reli­
earth’s temperature (United Nations, n.d.). To reduce greenhouse gas ability and consistency—a notable contrast to other renewables such as
emissions and transition to a low-carbon future; the world must rapidly solar and wind, which are subject to the unpredictable nature of weather
reduce its dependency on fossil fuels and invest in more sustainable and conditions (Visualize and assess Classifier performance in Classification
renewable energy production methods (Technical Review of On-Line Learner - MATLAB & Simulink, n.d.). In 2020; the world’s 442 reactors

* Corresponding author.
E-mail address: [email protected] (M. Zubair).

https://doi.org/10.1016/j.nucengdes.2023.112698
Received 31 July 2023; Received in revised form 5 October 2023; Accepted 19 October 2023
Available online 3 November 2023
0029-5493/© 2023 Elsevier B.V. All rights reserved.
M. Zubair and Y. Akram Nuclear Engineering and Design 415 (2023) 112698

Fig. 1. Proposed methodology diagram.

produced around 10 % of global electricity and 4.3 % of global energy protection of workers, the public, and the environment from undue ra­
(Cross-Validation, n.d.). With the ongoing challenges of global warming diation risks” in the 2022 International Atomic Energy Agency (IAEA)
and the transition to a low-carbon future; the significance of nuclear Nuclear Safety and Security Glossary, making it one of the most
energy has grown as they are a highly efficient and reliable method of important fields in the nuclear industry. Almost every aspect of a nuclear
power generation. power plant has been researched to improve it, as nuclear safety in­
Despite its advantages, it is paramount to consider the safety and cludes the safety of nuclear facilities, radiation safety, radioactive waste
security risks entailed by nuclear power. Nuclear safety is defined as “the management safety, and radioactive material transit safety (Rank fea­
achievement of proper operating conditions, the prevention of acci­ tures for classification using minimum redundancy maximum relevance
dents, and the mitigation of accident consequences, resulting in the (MRMR) algorithm, n.d.). Despite these improvements, faults are

2
M. Zubair and Y. Akram Nuclear Engineering and Design 415 (2023) 112698

Fig. 2. The overview page of the GPWR simulator.

inevitable. 1.2. Machine learning

1.1. Fault detection and diagnosis Recently, AI has garnered significant interest across various in­
dustries due to its potential for enhancing automation processes. It en­
Fault Detection and Diagnosis (FDD) techniques are used to ensure ables organizations to automate complex tasks, simplify operations, and
the proper functioning of systems and their individual components. optimize efficiency, which has resulted in increased productivity,
These methods aid in identifying faults, determining their location, and decreased expenses, and improved decision-making (Kamuda and Sul­
assessing their severity to ensure the safety and reliability of the systems livan, 2019).
by reducing human errors. Fault Detection and Diagnostics (FDD) One of the most popular applications of AI in the nuclear industry is
techniques find extensive applications within the nuclear industry machine learning. It is a process through which a data-driven model
(Angione et al., 2022). These methods serve a multifaceted purpose, classifies or predicts an outcome by generating mathematical correla­
including sensor calibration monitoring and assessing the inherent un­ tions between the system parameters (Huang et al., 2023). Machine
certainty in empirical process variable predictions (Holechek et al., Learning (ML) models are divided into four broad categories: super­
2022). Additionally, they play a critical role in predicting the peak vised, unsupervised, semi-supervised, and reinforced learning. The
cladding temperature (PCT) during scenarios like a large break loss of process of deducing new correlations based on a pattern acquired from a
coolant accident (LBLOCA) (Huang et al., 2023). Furthermore, FDD labeled input–output training set is known as supervised learning. For
techniques are instrumental in classifying signal transients, facilitating regression analysis, the result could be a continuous value or a classifi­
the reliable monitoring and timely diagnosis of nuclear components and cation label for classification analysis (Krūmiņš and Kļaviņš, 2023).
systems (International atomic energy agency, 2022). These versatile Machine Learning models have found diverse applications within the
applications of FDD contribute significantly to enhancing the safety, nuclear industry. Notably, Artificial Neural Network (ANN) models have
efficiency, and overall performance of nuclear facilities. been employed to enhance computational efficiency by accurately pre­
FDD methods can be categorized as model-based and model-free dicting wall temperatures at the Critical Heat Flux (CHF) when inte­
approaches. A mathematical model is used to depict the characteristic grated with the SPACE code (Ma and Jiang, 2009). Additionally, these
behavior of the system in a model-based FDD. Faults in the system are models have been utilized for isotope identification and quantification
detected and diagnosed by comparing the outputs of the mathematical within mixtures, even when using low-resolution gamma-ray spec­
model and the actual system. This approach has limited applications in trometers (MATLAB and Machine Learning And Deep Learning Toolbox
the nuclear sector due to the high level of complexity required to build Release, 2023). Furthermore, classification models have proven valu­
an accurate mathematical model of a nuclear power plant (NPP) (Jo able for identifying Nuclear Power Plant (NPP) transient events based on
et al., 2019). Model-free FDD approaches are further subdivided into hydraulic parameters (Mena et al., 2021; Mena et al., 2022).
data-driven and signal-driven methods. Both these methods require
large amounts of data to be processed and analyzed, which may require
1.3. Research objectives
more computational time. The data-driven FDD approach uses real-time
or historical data correlations to detect and diagnose faults. In contrast,
The core objective of this paper is to use MATLAB to train and
the signal-driven approach compares the extracted data with baseline
validate ML models with the capability to predict transient events using
measurements to diagnose faults. Often at times, model-free FDD ap­
hydraulic parameters derived from the Generic Pressurized Water
proaches are combined with artificial intelligence (AI) for a faster and
Reactor (GPWR) simulator. The paper’s structure is organized as fol­
more efficient diagnosis (Kaličanin et al., 2019).
lows: Section 2 provides a comprehensive overview of the proposed
approach for data extraction and processing. In Section 3, we present the
model’s prediction results, accompanied by a detailed analysis. Section

3
M. Zubair and Y. Akram Nuclear Engineering and Design 415 (2023) 112698

Fig. 3. List of inbuilt scenarios in the GPWR simulator.

Fig. 4. List of all available malfunctions in the GPW simulator.

4 discusses the optimization strategies applied, along with their impact thoroughly assessed and certified to perform plant maneuvers realisti­
on model prediction and training time. Finally, Section 5 concludes the cally and accurately based on the unit it depicts (Reactor Modeling, Inc.,
study by summarizing the findings and providing closing remarks. n.d.). Fig. 2 shows the overview page of the simulator (see Figs. 3 and 4).
The GPWR’s primary loop consists of two circulation loops, two
2 Materials and methodology steam generators (SG), and four reactor coolant pumps (RCP). The
secondary loop connects the two steam generators to the turbines (one
This section will go over the stages of data collection, preprocessing, high-pressure and three low-pressure), condensers, and feedwater
and model training. The software and procedures employed will be pumps. The total thermal power of the GPWR is 3983 MW, and the
described and discussed in detail. Fig. 1 shows an overview of the pro­ turbine electric power is 1400 MW (Park et al., 2020; Zubair et al.,
posed method. 2017).
The simulator employs a range of validated reactor codes to guar­
antee that the GPWR’s simulations accurately reflect real reactor
2.1. GPWR behavior. RELAP5 and MARS are used for thermal–hydraulic engineer­
ing analysis. NESTLE is a two-energy group neutronics code that cal­
Collecting data for model training using actual reactor transients is culates neutron flux and power for each node at every time step.
impractical as there are too many risks associated with it. Hence, the MELCOR is utilized to model the development of severe accidents in
data for this study was collected using the Western Services Co­ light-water reactors, providing a comprehensive, engineering-level
operation’s (WSC) Generic Pressurized Water Reactor (GPWR) simu­ computer code for this purpose (Qi et al., 2023).
lator. The GPWR simulator is based on a real working unit that has been

4
M. Zubair and Y. Akram Nuclear Engineering and Design 415 (2023) 112698

Table 1 Middle Of Life, End Of Life) and plant condition (Startup, shutdown). It
Monitored parameters extracted from the GPWR. is important to note that in the Startup condition, the core is critical,
Normalized Neutron Containment Temperature MS Flow From SG-1 Line- whereas in the Shutdown condition, the core is subcritical and at hot
Flux (%) (oC) 1B (kg/s) standby. Table 2 lists the 12 initial conditions selected for this study and
Containment Pressure RC Loop-1B Norm Flow MS Flow From SG-2 Line- their specifications. For each initial condition, a 600-second simulation
(kg/cm2) (%) 2A (kg/s) was conducted, and the monitored parameters were recorded every
RCS Lvl Loop 1 WR (N/ RC Loop-2A Norm Flow MS Flow From SG-2 Line- second. Each run was exported to a separate Excel file.
A) (%) 2B (kg/s) To simulate the different transient events, a combination of the
RCS Lvl Loop 1 NR (N/ RC Loop-2B Norm Flow SG-1 Pressure (kg/cm2)
A) (%)
following two techniques was used: the list of scenarios and plant mal­
Hot Leg 1 Temperature Pzr Surge Line Temp (oC) SG-2 Pressure (kg/cm2) functions. The GPWR has an inbuilt list of accident scenarios and mal­
(oC) functions that the user may select. Table 3 lists the nine transient events
Cold Leg 1A PORV Discharge Pzr Calculated Average simulated. The events were specifically chosen to try and cover the
Temperature (oC) Temperature (oC) Temperature (oC)
whole power plant.
Cold Leg 1B SG-1 NR Level (%) Pressurizer Pressure
Temperature (oC) Each of the transient scenarios was subjected to simulation across all
Hot Leg 2 Temperature SG-2 NR Level (%) Norm Pressurizer Level 12 initial conditions. Each simulation run spanned a duration of 600 s;
(oC) (%) however, the transient event was intentionally initiated after a 10-sec­
Cold Leg 2A FW Flow To SG-1 (kg/s) Pzr Water Temperature ond period of normal operation. This deliberate delay serves the pur­
Temperature (oC) (oC)
Cold Leg 2B FW Flow To SG-2 (kg/s) Pzr Steam Temperature
pose of allowing the reactor to attain steady-state conditions,
Temperature (oC) (oC) characterized by consistent and stable parameter values. Notably, the
RC Loop-1A Norm Flow MS Flow From SG-1 Line- Generator Power (MW) specific steady-state conditions vary based on the chosen initial condi­
(%) 1A (kg/s) tion for simulation. Consequently, the 10 s of normal operation provide
a valuable baseline comparison point. Each of these simulation runs was
meticulously stored in distinct Excel files, systematically named to
While not a replica of any specific PWR plant, is constructed based on
reflect their content. In total, this process generated 120 CSV files,
the Korea Electric Power Corporation (KEPCO) APR1400 design, a
preserving the data in its raw form for further analysis and investigation.
widely recognized and standardized pressurized water reactor (PWR)
design used in several NPPs worldwide. The decision to employ the
GPWR as the training model is rooted in its ability to offer valuable
2.3. Data analysis
insights into the behavior of PWR systems in general. The initial and
boundary conditions may vary between specific NPPs simulators due to
MATLAB version 2023a was used to modify the raw data and create
unique characteristics, operational differences, and variations that need
the model’s input dataset. A code was written to read the 120 CSV files
to be considered.
and clean the workspace by deleting any unnecessary data, such as the
simulation date and time, measurement units, parameter ranges, and
2.2. Data collection timestamps. It was decided against normalizing the data to keep
execution and computational time as low as possible. Additionally, two
The thermal–hydraulic data acquired from the GPWR served as input new columns were added to each file, indicating the initial condition
features for the ML models. The entire list of monitored parameters (33
features) was extracted for each individual run. Table 1 lists all the Table 3
features exported. While the user can enter more parameters manually, List of transient events simulated using the GPWR.
the list covers all parts of the NPP.
Complete Loss Of Offsite Power
The collected features were then analyzed, and it was noticed that All Feedwater Pump Trip
the Normalized Neutron Flux was just a normalization of the Generator Main Turbine Trip Without Scram
Power hence it was omitted from the list of features used for model 100 % Pressurizer Surge Line Break
training. The RCS Lvl Loop 1 WR and NR were also omitted from the list, All Reactor Coolant Pumps Trip
All MSIVs Closure
as their values stayed consistent throughout each run. Loss Of Coolant Accident
The GPWR simulator has several built-in initial conditions that the Main Steam Line Rupture
user can reset the plant to. The initial conditions are a combination of Primary System Depressurization
the power output (100 %, 50 %, 1 %), core lifetime (Beginning Of Life,

Table 2
Initial conditions used and their specifications.
Initial Normalized Power Generator Power Turbine Speed RCS Pressure (kg/ Pressurizer Water Average Temperature Boron Concentration
Condition (%) (MW) (rpm) cm2) Level (%) (0C) (ppm)

BOL 100 % 99.95 1408.36 1500.01 157.14 52.32 309.09 1505.31

MOL 100 % 99.93 1408.20 1500.01 157.16 53.07 309.11 850.55
EOL 100 % 99.96 1408.55 1500.01 157.03 52.98 309.14 20.00
BOL 50 % 50.40 654.14 1500.01 157.49 38.87 302.69 1505.55
MOL 50 % 50.57 649.23 1500.01 157.41 39.43 302.15 850.84
EOL 50 % 50.68 558.04 1500.01 157.58 40.96 303.80 20.22
BOL Start-Up 0.51 0 3 157.27 27.51 295.78 1900.01
MOL Start-Up 0.52 0 3 157.16 27.88 295.56 1262.02
EOL Start-Up 0.50 0 3 157.25 27.78 295.86 455.03
BOL 0.51 0 3 157.89 29.44 295.76 2000.00
Shutdown
MOL 0.52 0 3 157.21 27.422 295.55 1396.56
Shutdown
EOL 0.50 0 3 157.08 30.20 295.83 600.05
Shutdown

5
M. Zubair and Y. Akram Nuclear Engineering and Design 415 (2023) 112698

Table 4 2.4. Classification Learners Application

Features used as input for the ML model.
Containment Pressure (kg/cm2) Containment Temperature (oC) The Classification Learners Application, from the Machine Learning
o and Deep Learning MATLAB toolbox, was used to train and validate
Hot Leg 1 Temperature ( C) SG-1 NR Level (%)
Cold Leg 1A Temperature (oC) SG-2 NR Level (%) different models. The user selects the Data Set Variable, Response, and
Cold Leg 1B Temperature (oC) FW Flow To SG-1 (kg/s) Predictors when starting a new session in the application. The user also
Hot Leg 2 Temperature (oC) FW Flow To SG-2 (kg/s) specifies the type of validation scheme used. In machine learning,
Cold Leg 2A Temperature (oC) MS Flow From SG-1 Line-1A (kg/s) validation plays an especially significant role. It helps analyze the pre­
Cold Leg 2B Temperature (oC) MS Flow From SG-1 Line-1B (kg/s)
RC Loop-1A Norm Flow (%) MS Flow From SG-2 Line-2A (kg/s)
diction capability of the trained model and guards against overfitting.
RC Loop-1B Norm Flow (%) MS Flow From SG-2 Line-2B (kg/s) Fig. 5 shows the new session window when the Classification Learners
RC Loop-2A Norm Flow (%) SG-1 Pressure (kg/cm2) Application is used.
RC Loop-2B Norm Flow (%) SG-2 Pressure (kg/cm2) The entire data set is used as the input in this study, the transient type
Pzr Surge Line Temp (oC) Calculated Average Temperature (oC)
as the response variable, and all other features (including the initial
PORV Discharge Pzr Temperature (oC) Pressurizer Pressure
Generator Power (MW) Norm Pressurizer Level (%) conditions) as predictors. The chosen validation scheme was 25 %
Initial Condition Pzr Water Temperature (oC) Holdout Validation. The models are trained using 75 % of the total data
Transient Pzr Steam Temperature (oC) set, and their accuracy is assessed using the 25 % held out initially.
The application offers 9 types of classifiers to choose from: decision
trees, discriminant analysis, support vector machines, logistic regres­
and simulated accident. Each data point was labeled from 1 − 12, rep­
sion, nearest neighbors, naive Bayes, kernel approximation, ensembles,
resenting the corresponding initial condition (following the same order
and neural networks. In MATLAB’s classification modeling framework,
as in Table 2). After ensuring that all the columns were in the same order
within these 9 categories, a total of 33 specific models exist, each
with the same headers, the files were vertically concatenated, resulting
employing unique mathematical techniques and configurations. These
in a comprehensive dataset consisting of 72,000 observations and 32
models come with predefined settings optimized for specific data pat­
features (consisting of the 30 monitored parameters, the initial condi­
terns and classification challenges. This comprehensive set of models
tion, and simulated accident). This dataset has 2,304,000 data points in
significantly reduces the need for manual model optimization, stream­
total. Table 4 shows a list of the 32 features found in the complete
lining the selection process and saving valuable time and effort in
dataset.
building and evaluating machine learning classifiers within MATLAB.

Fig. 5. The new session window when using the Classification Learners Application in MATLAB.

6
M. Zubair and Y. Akram Nuclear Engineering and Design 415 (2023) 112698

Table 5 Table 6
Brief overview of the nine classification models. Validation results of the trained models from the Classification Learners
Classifier Description
Application.
Model Type Preset Validation Average
Decision Tree Decision trees operate by iteratively dividing a dataset into
Accuracy (%) Accuracy (%)
subsets based on the most informative features, resulting in a
tree-like structure with leaves representing class labels. Tree Fine Tree 93.45 77.58
These models are straightforward to interpret, but they tend Medium Tree 88.59
to overfit since they heavily rely on the training data. Coarse Tree 50.71
Discriminant The discriminant analysis finds linear combinations of Discriminant Linear Discriminant 83.45 83.45
Analysis features that maximize class separation while minimizing Quadratic NaN
within-class variance. It projects data onto these Discriminant
discriminant axes and assigns data points to classes based on Efficient Efficient Logistic 89.57 89.55
their relative positions to class-specific centroids. However, Regression
this model assumes data follows a Gaussian distribution, Efficient Linear SVM 89.54
which can limit its applicability. Naive Bayes Gaussian Naive NaN 87.32
Support Vector SVMs aim to find the best hyperplane to separate data points Bayes
Machines from different classes with a maximum margin. They can be Kernel Naive Bayes 87.32
computationally intensive and sensitive to the choice of Support Vector Linear SVM 89.85 88.97
kernel function, requiring careful parameter tuning. Machines Quadratic SVM 84.83
K-Nearest Neighbors K-Nearest Neighbors classification assigns a data point to the Cubic SVM 90.11
majority class among its k-nearest neighbors in the feature Fine Gaussian SVM 90.14
space. It relies on distance metrics to measure the similarity Medium Gaussian 89.90
between data points. However, as the number of features SVM
increases, KNN’s performance may degrade since it involves Coarse Gaussian 88.99
the entire dataset, leading to computational challenges. SVM
Neural Networks Neural networks consist of layers of interconnected artificial K-Nearest Fine KNN 87.53 88.33
neurons that process and transform input data. They use Neighbors Medium KNN 88.82
activation functions, weights, and biases to capture complex Coarse KNN 88.69
data relationships but demand significant computational Cosine KNN 88.87
resources and sensitivity to network structure. Cubic KNN 88.62
Kernal Kernel approximation models classify data by mapping it Weighted KNN 87.41
Approximation into a higher-dimensional space using a simpler Ensemble Boosted Trees 88.67 88.02
transformation, often linear. These models expedite Bagged Trees 93.29
computation but are highly reliant on the chosen Subspace 82.51
approximation method. Discriminant
Logistic Regression Logistic regression models the probability of binary Subspace KNN 87.03
outcomes using a logistic function. It identifies the best- RUSBoosted Trees 88.59
fitting line (or hyperplane) to separate classes in the feature Neural Network Narrow Neural 89.98 90.11
space. However, it assumes a linear relationship between Network
predictors and outcomes and is susceptible to outliers. Medium Neural 90.28
Naïve Bayes Naïve Bayes classifiers use Bayes’ theorem and assume Network
feature independence given the class to estimate class Wide Neural 90.12
probabilities. They make predictions based on the highest Network
probability. However, they assume independence among Bilayered Neural 90.09
predictors and can be affected by imbalanced data Network
Ensemble Ensemble methods combine multiple base models to Trilayered Neural 90.09
enhance predictive accuracy and generalization. They offer Network
improved performance but can be computationally Kernel SVM Kernel 89.01 89.01
expensive and challenging to interpret, as results vary based Logistic Regression 89.01
on the choice of base models. Kernel

Table 5 shows a summary of 9 classifiers used in our study (Sallehhudin Formulae to calculate the evaluation metrics:
and Diab, 2021; Generic Simulators).
TP + TN
The trained and validated models are evaluated using their predic­ Accuracy =
Total Predictions
tion Accuracy, Precision, Recall, and the F1 Score. The percentage Ac­
curacy is automatically calculated by the application, whereas the TP
Precision, Recall, and F1 scores require data from the Confusion Matrix. Precision =
TP + FP
The Confusion Matrix provides a clear and detailed summary of how
well the model’s predictions align with the true labels in a classification TP
Recall =
problem. The diagonal values signify the number of correct classifica­ TP + FN
tions, whereas the values in the other cells signify misclassifications. The
matrix is helpful in analyzing the performance of the model and F1 Score =
2*Precision*Recall
concluding areas of weak performance. Key components of a confusion Precision + Recall
matrix (Varghese, 2021):
3 Results and discussion
• True Positives (TP): The number of instances correctly identified as
belonging to the target class. All the models available in the Classification Learners Application
• True Negatives (TN): The number of instances correctly identified as were trained and validated. The average accuracy was calculated for
not belonging to the target class. each model type. The Neural Network models achieved the highest
• False Positives (FP): The number of instances incorrectly identified average accuracy of 90 %, whereas the Decision Tree models got the
as belonging to the target class when they actually do not. lowest value of 77.58 %. The results are displayed in Table 6.
• False Negatives (FN): The number of instances incorrectly identified The overall top three best-performing models were studied in more
as not belonging to the target class when they actually do. detail. Table 7 summarizes the results of these models. The confusion

7
M. Zubair and Y. Akram Nuclear Engineering and Design 415 (2023) 112698

Table 7 matrices were used to calculate the Accuracy, Precision, Recall, and F1
Summary of the top 3 performing models. Score to further analyze the models and their prediction capability.
Model Training Accuracy Precision Recall F1
Time (s) (%) (%) (%) Score 3.1. Fine Tree
(%)

Fine Tree 15.99 93.45 94.90 93.41 94.15 Despite the Decision Tree models having the lowest average accu­
Ensemble 289.47 93.29 93.55 93.57 93.55 racy, the Fine Tree model was the most accurate in identifying transient
Bagged events from the list of 33 models. It was able to accurately predict
Trees
Medium 483.31 90.28 93.20 90.22 91.69
16,821 instances out of the 18,000 data points in the validation set in
Neural 15 s. Fig. 6 shows a breakdown of all the predictions by the model. The
Network model seemed to have the most trouble identifying the All Feedwater
Pump Trip and Main Turbine Trip Without Scram. It misclassified these
transients as Normal operation 895 separate times and 148 times as
other transient events.

Fig. 6. Confusion Matrix for Fine Tree.

Fig. 7. Confusion Matrix for Ensemble Bagged Trees.

8
M. Zubair and Y. Akram Nuclear Engineering and Design 415 (2023) 112698

Fig. 8. Confusion Matrix for Medium Neural Network.

3.2. Ensemble Bagged Trees misclassifications. Identifying and addressing the underlying causes of
these misclassifications can potentially lead to advancements in the
The second most accurate model for identifying the transient events models’ overall performance and predictive accuracy.
was the Ensemble Bagged Trees model. Out of 18,000 data points, it was
able to accurately predict 16,792 instances. Fig. 7 shows the confusion 4 Optimization
matrix for this model. This model took around 5 min to train.
The model had the most difficulty detecting Normal Operation. It In this section, the previously discussed models will be optimized to
misclassified 434 instances as Normal Operations when they were other enhance their transient identification capabilities. The Classification
transient events, whereas 475 were misclassified as other transients Learners Application has several inbuilt methods to optimize the per­
when they were Normal Operations. In total, 909 misclassifications out formance of the models. By using these techniques, we aim to either
of the total 1208 were related to Normal Operation. Just like the pre­ improve the prediction accuracy or reduce the training time, hence
vious model (Fine Tree), it struggled to differentiate between the All increasing the reliability of the machine learning models.
Feedwater Pumps Trip, Main Turbine Trip without Scram, and Normal
Operation.
4.1. Validation schemes

3.3. Medium Neural Network Validation is a method for assessing the performance of machine
learning models. This is accomplished by putting the model’s prediction
In terms of identifying transient occurrences, the Medium Neural abilities to the test with an unknown validation set.
Network was the third most accurate model. Out of the 18,000 data The learner’s application allows the user to choose from 3 different
points in the validation set, it correctly predicted 16,251 instances. It has validation schemes (Naimi et al., 2022):
an accuracy of 90.28 %. Fig. 8 depicts a breakdown of the model’s
predictions. • Holdout Validation: The input dataset is partitioned into two subsets
The model, just like the previous two, had the most misclassifications depending on the specified ratio. One subset is used to train the
when dealing with the All Feedwater Pump Trip and the Main Turbine model, while the other is used to validate it.
Trip Without Scram. Around 1437 instances of each of these transients • K-Fold Validation: The input dataset is divided into the ’k’ number of
were misclassified as Normal Operations. The model took the longest to subsets. During each fold, one of the subsets is set aside to validate
train, around 8 min. the model, while the others are used for training. This procedure is
The results of the model predictions offer promising outcomes to a repeated until all ’k’ folds have been used to validate the model.
considerable extent. The top three validated models have prediction After that, the average validation is computed.
accuracies above 90 %, highlighting their ability to identify transient
events through hydraulic parameters. Additionally, these models pre­ In this section, the best three models are trained using the crossover
sent an advantage in terms of training time, with a maximum duration of validation scheme where k = 5, and the results are compared.
eight minutes required for training.
For the Fine Tree and Medium Neural Network Model, the value for 4.1.1. Output
precision is higher than recall. This means the models have more Type II .
errors than Type I errors. However, the Ensemble Bagged Trees model’s
value for recall is higher than precision, indicating that the model has 4.1.2. Discussion
more Type I errors than Type II ones. When comparing the two validation schemes, holdout and cross-
However, it is noted that all three models struggle with differenti­ validation, across the three datasets, minimal change in the model ac­
ating between Normal Operation, All Feedwater Pumps Trip, and Main curacy is observed. This can be attributed to the comparable size of the
Turbine Trip Without Scram. These transients had the most subsets in both schemes: holdout uses 25 % or 1/4 of the data for

9
M. Zubair and Y. Akram Nuclear Engineering and Design 415 (2023) 112698

Table 8 4.2. Feature reduction

The effect of validation schemes on the top 3 performing models.
Validation Holdout 25 % 5 Fold Cross Validation Feature selection is one of the most crucial parts of data pre­
processing since it directly correlates with the output of ML models. The
Fine Tree
Training Time (s) 15.99 21.10 advantages of employing feature selection are (Which country is
Accuracy (%) 93.45 92.96 currently producing the most nuclear power, 2022):
Ensemble Bagged Trees
Training Time (s) 289.47 884.94 • It eases data exploration and analysis.
Accuracy (%) 93.29 92.98
Medium Neural Network
• It reduces the size of the dataset; hence, the computation costs are
Training Time (s) 483.31 1624.40 lowered.
Accuracy (%) 90.28 89.96 • It reduces data noise.

The Classification Learners Application offers feature selection as a

validation, while cross-validation (k = 5) uses 20 % or 1/5 of the data.
method to further optimize models. The user can select from five
Consequently, the accuracies remained similar, as seen in Table 8.
different feature ranking algorithms: MRMR, Chi2, ReliefF, ANOVA, and
However, it is important to note that although all three models
Kruskal Wallis. In this section, the mRMR algorithm will be used to
demonstrated comparable accuracies, the training time was significantly
analyze and rank them based on their importance. The mRMR algorithm
longer with cross-validation. This is expected as the calculations are
is selected as it is the only one that quantifies the importance scores for
repeated five times and averaged in the end.
the features, allowing them to be ranked. The other algorithms simply
Overall, the Holdout validation scheme outperformed the cross-
assign the feature ’inf’ to indicate high importance.
validation approach. It achieved higher prediction accuracies while
Minimum Redundancy Maximum Relevance (mRMR) is a popular
requiring less training time, making it a more suitable choice for online
feature selection algorithm. It calculates redundancy between features
monitoring or time-sensitive analysis.
and relevance between features and the resulting class. The function
quantifies the importance of a feature and returns a score. A high score

Fig. 9. Feature importance score using mRMR algorithm for overall NPP transients.

10
M. Zubair and Y. Akram Nuclear Engineering and Design 415 (2023) 112698

Table 9 and Medium Neural Network models consistently achieved commend­

The effect of feature selection on the top 3 performing models in overall NPP able levels of accuracy, approximating 90 % across all datasets.
transients. This achievement effectively fulfills the objectives set forth in this
Features 31/31 12/31 paper, underscoring the utility of machine learning techniques in pre­
Fine Tree
dicting transient events within nuclear power plants and emphasizing
Training Time (s) 15.99 8.93 the potential for enhancing nuclear safety and reliability. This study
Accuracy (%) 93.45 92.51 proves that model-free fault detection and diagnosis techniques com­
Ensemble Bagged Trees bined with artificial intelligence can ensure safer and much more effi­
Training Time (s) 289.47 15.60
cient operation of nuclear power plants. For future work, these
Accuracy (%) 93.29 93.35
Medium Neural Network prediction models can be combined with online monitoring techniques
Training Time (s) 483.31 81.91 to further increase the transient event detection and diagnosis rate.
Accuracy (%) 90.28 89.60
CRediT authorship contribution statement
suggests that the corresponding predictor is significant. The features are
Muhammad Zubair: Supervision, Methodology, Conceptualization,
ranked using the importance score generated (Zio and Baraldi, 2005).
Funding acquisition, Project administration, Writing – original draft.
The average of the scores will be calculated, and only the features with a
Yumna Akram: Software, Formal Analysis, Investigation, Writing –
score equal to or better than the average will be used to retrain the best
review & editing.
three models (for each dataset), and the outcomes will be analyzed.

4.2.1. Overall nuclear power plant Declaration of Competing Interest

The overall nuclear power plant transient dataset is loaded on the
Classification Learners Application and the mRMR Feature Selection The authors declare that they have no known competing financial
algorithm is used to rank the 31 features based on their importance. interests or personal relationships that could have appeared to influence
Fig. 9 shows a graph of the features and their scores. the work reported in this paper.
The figure clearly illustrates that containment pressure and the
normalized flow rate in reactor coolant loop 2A stand out as the most Data availability
prominent features in the dataset, boasting significantly higher impor­
tance scores compared to other features. Conversely, the initial condi­ Data will be made available on request.
tions hold an importance score of 0, which aligns with their role in
setting steady-state values rather than aiding in the classification of Acknowledgements
transient events. Taking into account all features, the average score
computes to 0.695152. Based on this value, only the top 12 features are The authors thank the University of Sharjah, UAE, for providing a
retained for retraining the top 3 models. This reduction results in the research facility through grant number: Ref. V. C. R. G/ R 1325/2022
dataset shrinking from 2,304,000 data points to 864,000, nearly halving
its original size. It’s important to note that the dataset still contains the References
same number of observations (72,000). Table 9 provides a summary of
Angione, C., Silverman, E., Yaneske, E., 2022. Using machine learning as a surrogate
the model results as features are reduced. model for agent-based simulations. PLOS ONE 17 (2), e0263150.
Reducing the number of features used to train the models had a very Cross-Validation. (n.d.). MATLAB & Simulink. https://www.mathworks.com/discovery/
slight negative effect on the accuracy of the models but immensely cross-validation.html.
Generic Simulators | 3KEYSTUDENT, Inc. (n.d.). https://www.ws-corp.com/default.asp?
reduced the training time of each model. This can be attributed to the
PageID=19&PageNavigation=Generic-Simulators.
fact that the dataset is now approximately one-third the size it was Holechek, J.L., Geli, H.M.E., Sawalhah, M.N., Valdez, R., 2022. A global assessment: Can
originally. The original models took up to 8 min to be trained, whereas renewable energy replace fossil fuels by 2050? Sustainability 14 (8), 4792. https://
the retrained models only took 1.5 min. The impact on the prediction doi.org/10.3390/su14084792.
Huang, Q., Peng, S., Deng, J., Zeng, H., Zhang, Z., Liu, Y., Yuan, P., 2023. A review of the
accuracy is marginal and can be easily overlooked by the significant application of artificial intelligence to nuclear reactors: Where we are and what’s
reduction in training time. next. Heliyon 9 (3), e13883.
International Atomic Energy Agency. (2022). IAEA Nuclear Safety and Security Glossary.
IAEA. https://www.iaea.org/publications/15236/iaea-nuclear-safety-and-security-
5 Conclusion glossary.
Jo, I., Lee, S., Oh, S., 2019. Improved Measures of Redundancy and Relevance for mRMR
In the scope of this research endeavor, data sourced from the Western Feature Selection. Computers 8 (2), 42. https://doi.org/10.3390/
computers8020042.
Services Cooperation’s Generic Pressurized Water Reactor (GPWR) Kaličanin, K., Čolović, M., Njeguš, A., Mitić, V. (2019). Benefits of Artificial Intelligence
simulator was leveraged to simulate nine distinct transient events and Machine Learning in Marketing. Paper presented at Sinteza 2019 - International
encompassing various aspects of the nuclear power plant, each with Scientific Conference on Information Technology and Data Related Research. doi:
10.15308/Sinteza-2019-472-477.
twelve different initial conditions. MATLAB was harnessed as the key Kamuda, M., Sullivan, C.J., 2019. An automated isotope identification and quantification
tool for data preprocessing in preparation for the training and validation algorithm for isotope mixtures in low-resolution gamma-ray spectra. Radiation Phys.
of classification machine learning models, employing the Classification Chem. 155, 281–286. https://doi.org/10.1016/j.radphyschem.2018.06.017.
Krūmiņš, J., Kļaviņš, M., 2023. Investigating the potential of nuclear energy in achieving
Learners Application. Among the array of models scrutinized, Neural
a Carbon-Free energy future. Energies 16 (9), 3612. https://doi.org/10.3390/
Network models emerged as the standout performers, showcasing an en16093612.
impressive average accuracy rate of 90 %. Conversely, the Decision Tree Ma, J., Jiang, J., 2009. Applications of Fault Diagnosis in Nuclear Power Plants: An
models exhibited comparatively lower accuracy, achieving a rate of Introductory Survey. IFAC Proceedings Volumes 42 (8), 1150–1161. https://doi.org/
10.3182/20090630-4-es-2003.00189.
77.58 %. To further enhance model performance, we implemented the MATLAB and Machine Learning And Deep Learning Toolbox Release 2023a, The
mRMR feature selection algorithm alongside diverse validation MathWorks, Inc., Natick, Massachusetts, United States.
schemes. These optimization techniques not only bolstered model ac­ Mena, P., Borrelli, R., Kerby, L.M., 2021. Nuclear Reactor Transient Diagnostics Using
Classification and AutoML. Nucl. Technol. 208 (2), 232–245. https://doi.org/
curacy but also yielded reductions in dataset size and training time. This, 10.1080/00295450.2021.1905470.
in turn, translated into decreased computational demands and associ­ Mena, P., Borrelli, R., Kerby, L.M., 2022. Expanded analysis of machine learning models
ated costs. Ultimately, the optimized Fine Tree, Ensemble Bagged Trees, for nuclear transient identification using TPOT. Nucl. Eng. Design 390, 111694.
https://doi.org/10.1016/j.nucengdes.2022.111694.

11
M. Zubair and Y. Akram Nuclear Engineering and Design 415 (2023) 112698

Naimi, A., Deng, J., Doney, P., Akbari, A.S., Shimjith, S.R., Arul, A.J., 2022. Machine Visualize and assess Classifier performance in Classification Learner - MATLAB &
Learning-Based Fault Diagnosis for a PWR Nuclear Power Plant. IEEE Access 10, Simulink. (n.d.). https://www.mathworks.com/help/stats/assess-classifier-
126001–126010. https://doi.org/10.1109/access.2022.3225966. performance.html.
Park, H., Lee, J., Kim, K.H., 2020. Wall temperature prediction at critical heat flux using Which country is currently producing the most nuclear power? (2022, September 7).
a machine learning model. Ann. Nucl. Energy 141, 107334. https://doi.org/ World Economic Forum. https://www.weforum.org/agenda/2022/01/energy-
10.1016/j.anucene.2020.107334. nuclear-power-production-electricity-sustainable/.
Qi, B., Liang, J., Tong, J., 2023. Fault Diagnosis Techniques for Nuclear Power Plants: A Zio, E., Baraldi, P.G., 2005. Evolutionary fuzzy clustering for the Classification of
Review from the Artificial Intelligence Perspective. Energies 16 (4), 1850. https:// transients in nuclear components. Prog. Nucl. Energy 46 (3–4), 282–296. https://
doi.org/10.3390/en16041850. doi.org/10.1016/j.pnucene.2005.03.010.
Rank features for classification using minimum redundancy maximum relevance Zubair, M., Ababneh, A., Ishag, A., 2017. Station Black out Concurrent with PORV
(MRMR) algorithm - MATLAB fscmrmr. (n.d.). https://www.mathworks.com/help/ Failure using a Generic Pressurized Water Reactor Simulator. Ann. Nucl. Energy 110,
stats/fscmrmr.html#mw_733b9b36-11f2-4aa2-85fc-0988c425cd95_head. 1081–1090.
Reactor Modeling, Inc. (n.d.). https://www.ws-orp.com/default.asp?
PageID=4&PageNavigation=Reactor-Modeling.
Dr. Muhammad Zubair is working as Associate Professor at the Department of Me­
Sallehhudin, W., Diab, A., 2021. Using Machine Learning to Predict the Fuel Peak
chanical & Nuclear Engineering, University of Sharjah, UAE. He is also leading the Nuclear
Cladding Temperature for a Large Break Loss of Coolant Accident. Front. Energy Res.
Energy Systems, Simulation and Safety Research Group at the Research Centre, RISE,
9 https://doi.org/10.3389/fenrg.2021.755638.
University of Sharjah, P.O. Box 27272, Sharjah, United Arab Emirates
Technical Review of On-Line Monitoring Techniques for Performance Assessment: State-
of-the-Art (NURE. (n.d.). NRC Web. https://www.nrc.gov/reading-rm/doc-
collections/nuregs/contract/cr6895/v1/index.htm. Engineer Yumna Akram has BSc degree in Nuclear Engineering from the Department of
United Nations. (n.d.). Generating power | United Nations. https://www.un.org/en/ Mechanical & Nuclear Engineering, University of Sharjah, UAE. He is also a part-time
climatechange/climate-solutions/cities-pollution. research assistant at the Nuclear Energy Systems, Simulation and Safety Research Group
Varghese, D. (2021, December 6). Comparative Study on Classic Machine learning at the Research Centre, RISE, University of Sharjah, P.O. Box 27272, Sharjah, United Arab
Algorithms. Medium. https://towardsdatascience.com/comparative-study-on- Emirates
classic-machine-learning-algorithms-24f9ff6ab222.

12