Statistical Learning
Introduction to Statistics
mypostbag1983@[Link]
6TKSB2XAU9
Proprietary content. ©Great
This file Learning.
is meant for All Rights
personal useReserved. Unauthorized use
by mypostbag1983@[Link] or distribution prohibited
only.
Sharing or publishing the contents in part or full is liable for legal action.
Outline
1. Why Statistics
2. Statistical Methods
3. Types of Statistics - Descriptive and Inferential
Statistics
4. Data Sources and Types of Datasets
5. Attributes of Datasets
mypostbag1983@[Link]
6TKSB2XAU9
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Why Statistics is So Important?
Three significant events triggered the current meteoric growth in the
use of analytical decision making and Statistics is central to all of them.
Event1
• Technological developments, Revolution of Internet and
social networks, data generated from mobile phones and
mypostbag1983@[Link]
6TKSB2XAU9
other electronic devices, produce large amount of data from
which insights will have to be sifted.
• The discovery of pattern and trends from these data for
organizations will pave the way for improving profitability,
understanding customer expectations, and appropriately
pricing their products so that they can gain competitive
advantage in the marketplace.
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Why Statistics is So Important?
Event 2
• Advances in enormous computing power to effectively
process and analyze massive amounts of data
• Sophisticated and faster algorithms for solving
mypostbag1983@[Link]
6TKSB2XAU9
problems
• Data Visualization for Business Intelligence and
Artificial Intelligence
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Why Statistics is So Important?
Event 3
• Large data storage capability
mypostbag1983@[Link]
6TKSB2XAU9
• Parallel computing, and cloud computing coupled with
better computer hardware have enabled businesses
and other organizations to solve large scale problems
faster than ever before without sacrificing
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Big Data
Big data
• A set of data that cannot be managed, processed, or
analyzed with traditional software/algorithms within a
reasonable amount of time.
• Big data revolves around
mypostbag1983@[Link]
6TKSB2XAU9
Volume Velocity Variety Value Veracity
Walmart handles over one million purchase transactions
per hour.
Facebook processes more than 250 million picture uploads
per day.
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Statistics - Methods
Classification
• Classification techniques helps in segmenting the customers into
appropriate groups based on key characteristics.
• For example, using appropriate statistical model, an organization
could easily segment the customers into Long Term Customers,
mypostbag1983@[Link]
Medium Term Customers, and Brand Switchers.
6TKSB2XAU9
• Another application in this context is classifying customers into
“Buyers and Non-Buyers.”
• Classification helps professionals understand the customer
behavior and position their products and brands using
appropriate strategies.
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Statistics - Methods
Pattern Recognition
• “A picture is worth thousand words” and it reveals hidden
pattern in the data that could be leveraged by retail professionals.
Pattern recognition techniques include Histogram, Box Plot, Scatter
Plot and other Visual Analytics.
• For example, histogram drawn for income of a particular class of
mypostbag1983@[Link]
6TKSB2XAU9
customers may reveal a symmetrical bell curve pattern or may be
left or right skewed.
• Relationship between age and expenditure could be captured
using a scatter plot.
• Box Plot enables identification of outliers (extreme
points) apart from providing the distribution pattern.
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Statistics - Methods
Association
• Association Analysis helps in determining which of the items go
together. Association rules include a set of analytics that focuses
on discovering relationships that exist among specific
objects.
• In this context, market basket analysis refers to an association rule
mypostbag1983@[Link]
6TKSB2XAU9
that generates the probability for an outcome.
• For example, market basket analysis may lead to a finding that if
customers buy coffee, there is a 40% probability that they also buy
bread.
• Association rules can be adapted by organizations to store lay
cross selling
out, items among others.
bundling, discount and sales promotion decisions, and
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Statistics - Methods
Predictive Modeling
• Both customer segmentation as well as identifying and targeting
most profitable customers can be facilitated by predictive models.
• Regression can be used for predicting the amount of expenditure on
a particular product based on input variables income, age, and
mypostbag1983@[Link]
gender.
6TKSB2XAU9
• Organizations can leverage on other advanced models that
comprise Logistic Regression, and Neural Networks for predicting a
target variable as well as classifying and predicting into which
group the consumer belongs to.
• For example, these models can classify and predict buyers and
non-buyers, and defaulters and non-defaulters on credit card loan.
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Classical Definition of Statistics
“ By Statistics, we mean methods specially adopted to the
elucidation of quantitative data affected to a marked extent by
multiplicity of causes”.
Yule and Kendal
mypostbag1983@[Link]
6TKSB2XAU9
It is interesting to see what Thomas Davenport means by Business
Analytics and note the similarities and dissimilarities between
the two.
“Business Analytics (BA) can be defined as the broad use of
data and quantitative analysis for decision making within
organizations”.
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Types of Statistics
Descriptive Statistics Inferential Statistics is a
is concerned with Data method used to talk
Summarization, about a Population
mypostbag1983@[Link]
Graphs/Charts, and
6TKSB2XAU9
Parameter from a Sample.
Tables
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Population, Parameter, Sample, Statistic
A Population is the universe of possible data for a specified object. Example:
People who have visited or will visit a website.
A Parameter is a numerical value associated with a population. Example: The
average amount of time people spend on a website.
A Sample is a selection of observations from a population. Example: People (or IP
addresses) who visited a website on a specific day.
mypostbag1983@[Link]
6TKSB2XAU9
A Statistic is a numerical value associated with an observed sample. Example: The
average amount of time people spent on a website on a specific day.
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Data Sources
Primary Data are collected by the organization itself for
a particular purpose. The benefits of primary data are that they
fit the needs exactly, are up to date, and reliable.
Secondary Data are collected by other organizations or for other
purposes. Any data, which are not collected by the organization
for the specified purpose, are secondary data. These may be
mypostbag1983@[Link]
6TKSB2XAU9
published by other organizations, available from research
studies, published by the government, web, social media and so
on.
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Types of Data
Qualitative Data are nonnumeric Quantitative data can be classified
in nature and can't be into discrete type or continuous
measured. Examples are gender, type. Discrete type can take only
religion, and place of birth. certain values, and there are
mypostbag1983@[Link]
6TKSB2XAU9 discontinuities between values,
such as the number of rooms in a
Quantitative Data are numerical
hotel, which cannot be in fraction.
in nature and can be
Continuous type can take any
measured. Examples are balance in
value within a specific interval,
your savings bank account, and
such as the production quantity of a
number of members in your
particular type of paper
family.
(measured in kilograms).
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Types of Data Sets
• Record
• Relational records
• Data matrix, e.g., numerical matrix,
crosstabs
• Document data: text documents: term-
frequency vector
• Transaction data
• Graph and network
mypostbag1983@[Link]
6TKSB2XAU9
• World Wide Web
• Social or information networks
• Molecular Structures
• Ordered
• Video data: sequence of images
• Temporal data: time-series
• Sequential Data: transaction sequences
• Genetic sequence data
• Spatial, image and multimedia:
• Spatial data: maps
• Image data
• Video data
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great
SharingLearning. AlltheRights
or publishing Reserved.
contents Unauthorized
in part or full is liable for legaluse or distribution prohibited
action.
Data Objects
• Data sets are made up of data objects.
• A data object represents an entity.
• Examples:
• sales database: customers, store items, sales
• medical database: patients, treatments
• university database: students, professors, courses
mypostbag1983@[Link]
6TKSB2XAU9
• Also called samples , examples, instances, data points, objects, tuples.
• Data objects are described by attributes.
• Database rows -> data objects; columns ->attributes.
This fileLearning.
Proprietary content. ©Great is meant forAll
personal
Rightsuse by mypostbag1983@[Link]
Reserved. Unauthorized use only.
or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Attributes
• Attribute (or dimensions, features, variables): a data field,
representing a characteristic or feature of a data object.
• E.g., customer _ID, name, address
• Types:
• Nominal
• Binary
mypostbag1983@[Link]
6TKSB2XAU9
• Ordinal
• Numeric: quantitative
• Interval-scaled
• Ratio-scaled
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Attribute Types
• Nominal: categories, states, or “names of things”
• Hair_color = {auburn, black, blond, brown, grey, red, white}
• marital status, occupation, ID numbers, zip codes
• Binary
• Nominal attribute with only 2 states (0 and 1)
• Symmetric binary: both outcomes equally important
• e.g., gender
• Asymmetric binary: outcomes not equally important.
mypostbag1983@[Link]
6TKSB2XAU9
• e.g., medical test (positive vs. negative)
• Convention: assign 1 to most important outcome (e.g., HIV positive)
• Ordinal
• Values have a meaningful order (ranking) but magnitude between
successive values is not known.
• Size = {small, medium, large}, grades, army rankings
This file is meant for personal use by mypostbag1983@[Link] only.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
Sharing or publishing the contents in part or full is liable for legal action.
Numeric Attribute Types
• Quantity (integer or real-valued)
• Interval
• Measured on a scale of equal-sized units
• Values have order
• E.g., temperature in C˚or F˚, calendar dates
• No true zero-point
mypostbag1983@[Link]
6TKSB2XAU9
• Ratio
• Inherent zero-point
• We can speak of values as being an order of
magnitude larger than the unit of measurement
(10 K˚ is twice as high as 5 K˚).
• e.g., temperature in Kelvin, length, counts,
monetary quantities
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by mypostbag1983@[Link] only.
Sharing or publishing the contents in part or full is liable for legal action.
