Algorithm 2: Decision Tree
Input: Training dataset DDD
Output: Trained Decision Tree classifier
1. Initialization:
o Start with the root node, which is initially empty.
2. Splitting:
o Select the best feature to split the data based on a criterion such as Gini impurity
or entropy.
o Split the data into subsets based on the selected feature and its threshold value.
3. Recursive Splitting:
o For each subset, repeat the splitting process recursively until a stopping criterion
is met (e.g., maximum depth, minimum samples per leaf).
o Each subset becomes a node in the tree.
4. Leaf Node Assignment:
o Assign a class label to each leaf node based on the majority class of the samples
in that node.
5. Pruning (optional):
o Remove branches that have little importance or that may cause overfitting.
Algorithm 3: Support Vector Machine (SVM)
Input: Training dataset DDD, kernel function KKK, regularization parameter CCC
Output: Trained SVM model
1. Initialization:
o Define the hyperplane that best separates the classes. This involves finding the
optimal hyperplane that maximizes the margin between classes.
2. Formulation:
o Solve the optimization problem to find the weight vector www and bias bbb that
define the hyperplane.
o Objective: Minimize 1/2∥w∥2 subject to constraints yi(w⋅xi+b)≥1.
3. Kernel Trick:
o Use a kernel function K(x,x′) to handle non-linear separations by mapping data to
a higher-dimensional space.
4. Training:
o Solve the dual optimization problem to find the Lagrange multipliers α\alphaα.
5. Decision Function:
o The decision function is f(x)=∑iαiyiK(xi,x) + b
Algorithm 4: Neural Network
Input: Training dataset DDD, architecture parameters (layers, neurons, activation functions)
Output: Trained Neural Network model
1. Initialization:
o Define the network architecture (input layer, hidden layers, output layer).
o Initialize weights and biases randomly.
2. Forward Propagation:
o For each layer, compute the output using the activation function.
o Formula: a(l)=σ(W(l)a(l−1)+b(l))
3. Loss Calculation:
o Compute the loss using a loss function (e.g., cross-entropy for classification).
4. Backward Propagation:
o Compute gradients of the loss with respect to weights and biases using
backpropagation.
o Update weights and biases using an optimization algorithm (e.g., gradient
descent).
5. Iteration:
o Repeat forward and backward propagation over multiple epochs until
convergence.
Algorithm 5: Ensemble Model (e.g., Random Forest)
Input: Training dataset DDD, number of trees nnn, tree parameters
Output: Trained Ensemble model
1. Initialization:
o Create n decision trees with randomized subsets of features and data samples.
2. Bootstrap Sampling:
o For each tree, generate a bootstrap sample from the training data with
replacement.
3. Training Trees:
o Train each decision tree on its bootstrap sample.
o Use a random subset of features at each split to introduce diversity.
4. Aggregation:
o For classification: Use majority voting among trees.
o For regression: Use the average prediction of trees.
5. Model Output:
o The final output is the aggregated prediction from all trees.
