Decision Tree Classification Algorithm
○ Decision Tree is a Supervised learning technique that can be used for both
classification and Regression problems, but mostly it is preferred for solving
Classification problems. It is a tree-structured classifier, where internal
nodes represent the features of a dataset, branches represent the
decision rules and each leaf node represents the outcome.
○ In a Decision tree, there are two nodes, which are the Decision Node and
Leaf Node. Decision nodes are used to make any decision and have multiple
branches, whereas Leaf nodes are the output of those decisions and do not
contain any further branches.
○ The decisions or the test are performed on the basis of features of the given
dataset.
More
○ It is a graphical representation for getting all the possible solutions
to a problem/decision based on given conditions.
○ It is called a decision tree because, similar to a tree, it starts with the root
node, which expands on further branches and constructs a tree-like structure.
○ In order to build a tree, we use the CART algorithm, which stands for
Classification and Regression Tree algorithm.
○ A decision tree simply asks a question, and based on the answer (Yes/No), it
further split the tree into subtrees.
Note: A decision tree can contain categorical data (YES/NO) as well as numeric data.
Why use Decision Trees?
There are various algorithms in Machine learning, so choosing the best algorithm for the
given dataset and problem is the main point to remember while creating a machine
learning model. Below are the two reasons for using the Decision tree:
○ Decision Trees usually mimic human thinking ability while making a decision, so it is
easy to understand.
○ The logic behind the decision tree can be easily understood because it shows a tree-
like structure.
Decision Tree Terminologies
Root Node: Root node is from where the decision tree starts. It represents the entire dataset,
which further gets divided into two or more homogeneous sets.
Leaf Node: Leaf nodes are the final output node, and the tree cannot be segregated further
after getting a leaf node.
Splitting: Splitting is the process of dividing the decision node/root node into sub-nodes
according to the given conditions.
Branch/Sub Tree: A tree formed by splitting the tree.
Pruning: Pruning is the process of removing the unwanted branches from the tree.
Parent/Child node: The root node of the tree is called the parent node, and other nodes are
called the child nodes.
Example: Suppose there is a
candidate who has a job offer and
wants to decide whether he should
accept the offer or Not. So, to solve
this problem, the decision tree
starts with the root node (Salary
attribute by ASM). The root node
splits further into the next decision
node (distance from the office) and
one leaf node based on the
corresponding labels. The next
decision node further gets split into
one decision node (Cab facility) and
one leaf node. Finally, the decision
node splits into two leaf nodes
(Accepted offers and Declined
