Bachelor of Science (Honours) in Data Science and Artificial Intelligence

DA 102 Data Analysis Basics

Introduction
Learning Objectives

01 Know various types of data

02 Issues associated with data

03 Understand broad group of analytics

04 Understand what is descriptive analytics

05 Understand what is predictive analytics

06 Tools used to perform analytics

3
4
About Data
Data

● Definition: collection of information, facts, or values

● The data is used as basis for computation, analysis and decision making.

● Data can be in the form of numbers, text, images, audio, video.

● Data is the raw material that is processed, organized, interpreted to extract meaning
and generate insights.

● Data is a fundamental concept that plays a crucial role in various fields such as
science, business, and research.

● Data is considered as the key in this era of digital age

6
Examples of data

7
Examples of data

8
Examples of data

9
About Data
Data 01

Numbers Quantitative Data Boolean Values

Qualitative Data
Quantity: 10 {"Un-married", {1 = Poor, 2 = Fair, 3 = {True, False}
"Married", Divorced"} Good, 4 = Very Good,
Price: 347.73 5 = Excellent}

{"Student", "Faculty", {AA = 10, AB = 9, BB =

"Staff"} 8, …, DD = 4, F = 0}

11
Data 02

Temporal Data Text Data Mixed data

Date: 15-Aug-2023 Sequence of characters such as Combination of any of

words, sentences or paragraphs these data types
Time: 11:30 AM

Time Stamp:
11:30 AM, 15-Aug-2023

12
Data 03

Audio Image Data Video data

An encoded file Data captured through cameras Combination several

corresponding to a song images and an
Data captured through associated audio data
screenshots in computers

13
14
Data – stored in spread sheet

15
Data – stored in a note pad

16
17
Real valued data
Analysis - example 01
• To buy the book titled "The Linux Programming Interface" by Michale Kerrisk from
online website

• The objective is to buy the above book at a website which offers best price.

• To achieve this objective, collect prices from various ecommerce websites

When data is real valued

19
Analysis – example 01

When data is real valued

20
Analysis – example 01

When data is real valued

21
Analysis – example 01

When data is real valued

22
Analysis – example 01

When data is real valued

23
Analysis – example 01

When data is real valued

24
Analysis – example 01

Website Price Additional information

www.amazon.in 5594 In stock

www.flipkart.com 5599 In stock

books.rediff.com 5378 Out of stock

bookswagon.com 7738 Out of stock

www.snapdeal.com Not available Not available

When data is real valued

25
Analysis – example 01

Website Price Additional • Prices are real valued

information numbers
www.amazon.in 5594 In stock
• Price at snapdeal.com is not
available due to
www.flipkart.com 5599 In stock unavailability of product

books.rediff.com 5378 Out of stock • Computing lowest price

from the second column of
the table is a challenging
bookswagon.com 7738 Out of stock task as price information in
unavailable for all websites
www.snapdeal.co Not available Not available
m
When data is real valued
26
Analysis – example 01

Website Price Additional • The price offered

information by snapdeal is made 0 to
make price a real valued
www.amazon.in 5594 In stock
number.

www.flipkart.com 5599 In stock • However, this change leads

to factually incorrect
books.rediff.com 5378 Out of stock information that the lowest
price is offered snapdeal.
bookswagon.com 7738 Out of stock • Handling "not
available" values is very
www.snapdeal.co 0 Not available important in data analysis
m otherwise, decisions swing
significantly resulting in
When data is real valued errors
27
Analysis – example 01

• Lowest price is offered by

books.rediff.com
Website Price Additional
information • Additional information tells
www.amazon.in 5594 In stock us that though this is the
best price offered, the book
www.flipkart.com 5599 In stock is out of stock.

• We therefore must search

books.rediff.com 5378 Out of stock for second lowest price.

bookswagon.com 7738 Out of stock • Sort price in descending

order and pick the second
element in sorted list.
www.snapdeal.co Not available Not available
m • We understand that Amazon
When data is real valued Offers the second lowest
price. 28
Analysis – example 01

Website Price Additional • Sampling – 5 websites are

information visited of many e-commerce
websites.
www.amazon.in 5594 In stock
• Computing minimum price
www.flipkart.com 5599 In stock value lead to decision
making
books.rediff.com 5378 Out of stock
• Is this example too easy?
Well the fact is:
bookswagon.com 7738 Out of stock
• 9 out of 10 consumers price
www.snapdeal.co Not available Not available check a product on Amazon
m
• https://www.bigcommerce.c
When data is real valued om/blog/amazon-statistics/
29
30
Quantitative data
Analysis – example 02
• Visit www.amazon.com
• Search for any product and
access associated product
description page.
• Apart from book description,
we get to see ratings given
by users
• The ratings are quantitative
values such as

When data is Quantitative

https://www.vecteezy.com/ 32
Analysis – example 02
• A total of 1043 users gave
ratings
• 75% of users gave 5 stars
• 17% of users gave 4 stars
and so on
• The ordered values are
presented as histogram
visualization
• Average rating is computed
and presented as 4.6 out of 5
stars

When data is Quantitative

33
Analysis – example 02

User 1 5 stars • Analysis: Computed the

following
User 2 5 stars • Number of users who gave 5
stars
User 3 4 stars • Number of users who gave 4
User 4 1 star starts
• ...
… … • Number of users who gave 1
star
User 1043 5 stars • Average rating across all
users

When data is Quantitative

34
Analysis – example 02

• Product ratings influence in conversions from

viewing to buying
• A correlation between ratings and conversion has
been observed – a product with 3.7 rating has 15%
more click through rate (CTR)
• Analysis involving only ratings

When data is Quantitative

35
Analysis – example 02

User 1 5 stars
• From the data perspective
User 2 5 stars • The rating data may be visualized to be stored as
shown
• When users visits product page, average rating and
User 3 4 stars
histograms are generated from this kind of data.
• Analytics play a pivotal role in transforming data and
User 4 1 star influencing decisions

… …

User 1043 5 stars

When data is Quantitative

36
37
Qualitative data
Analysis – example 03
• Visit www.youtube.com
• Search for any video
• Apart from the video, we get
to see how many users
{liked, disliked} the video
• More likes of a video has
strong correlation to
increase in revenue

https://www.vecteezy.com/

When data is Qualitative

39
Analysis – example 03
• Visit www.youtube.com
• Search for any video
• Apart from the video, we get
to see how many users
{liked, disliked} the video
• More likes of a video has
strong correlation to
increase in revenue

https://www.vecteezy.com/

When data is Qualitative

40
41
Text data
Analysis – example 04
• Visit www.amazon.com
• Search for any product and
access associated product
description page.
• Apart from book description,
we get to read detailed
product review given by
customers who bought the
product
• This information is in the text
form.

43
Analysis – example 04
• Analyze the contents for
finding out the sentiment of
the customer

• In the analytics terms,

perform sentiment analysis
of customer reviews

• If the complete review is in

positive sentiment, then the
it suggests that customer is
happy with the product.

44
45
A computer program
• A computer program is data

• Collection of such programs

• Collection of executables

46
Collection of programs
• A computer program is data

• Collection of such programs

• Collection of executables

47
Collection of executables
• A computer program is data

• Collection of such programs

• Collection of executables

48
49
Issues in Data Collection
Bias and Sampling Issues
• Sampling Bias - When the sample collected does not represent the entire
population

• Selection Bias - When participants are not selected randomly, leading to skewed or
inaccurate results

• Nonresponse Bias - When a significant portion of the selected participants does not
respond, leading to a biased sample

51
Measurement Issues
• Measurement Error: Inaccuracies or inconsistencies in measurement
instruments, leading to incorrect or imprecise data.

• Subjective Measurements: When data is collected through subjective methods like

surveys, individuals might respond based on personal biases or interpretations.

• Social Desirability Bias: Participants may provide responses they believe are
socially acceptable rather than their true opinions or behaviors

52
Data Quality and Integrity Issues
• Data Integrity Issues

• Missing values

• A data record that is not complete.

• Data entry errors.

• Duplications in data.

53
Ethical and Privacy Concerns
• Informed Consent: Obtaining proper informed consent from participants, especially
in sensitive or intrusive research.

• Privacy Protection: Ensuring that personally identifiable information is not leaked

or misused.

• Anonymity: Striking a balance between collecting meaningful data and

protecting participants' identities

54
Technological Challenges
• Data Security: Protecting data from breaches, leaks, or unauthorized access.

• Data Storage: Ensuring proper and secure storage of collected data.

• Technical Glitches: Issues with data collection tools, software, or hardware can
affect data quality.

55
Cultural and Language Related Challenges
• Cultural Bias: Data collection instruments might be biased towards a particular
culture or group.

• Translation Issues: Translating surveys or questionnaires into different languages

can lead to discrepancies in meanings.

56
Resource Limitations
• Financial Constraints: Adequate data collection might require funding for tools,
personnel, and infrastructure.

• Time Constraints: Rushed data collection might lead to errors or incomplete data.

57
Types of analytics
Analytics components
• A business context in order • Pregnant women are likely to be price-
to take up data analytics. insensitive.

• Two examples to support the • Their willingness to spend more is the potential
business context. for business.

• Baby products related market share is 38 billion

dollar.

• Given a female customer predict if the customer

is pregnant or not.

• In the business context, if the prediction is

accurate, offer pregnant women the customized
products and services.

59
Analytics components
• A business context in order • In competitive markets such as
to take up data analytics. telecommunication industry, customers abruptly
leave the services of the service provide.
• Two examples to support the
business context. • Predicting which customer leaves the service
before hand help address core concerns of the
customers.

• In the business context, if the prediction is

accurate, offer discounts to retain
such customers.

60
Analytics types

• Descriptive analytics

• Predictive analytics

61
Descriptive analytics
Descriptive analytics
• In descriptive analytics the focus is on summarizing and interpreting historical data

• It gives insights into past events and trends

• Strives to provide a clear and concise representation of data in order to understand

the data and aid in decision making

• Summarizing large volumes of data into manageable and interpretable forms

• Identify patterns and trends present in the historical data

• Visualize data as visualization make it easier to communicate complex data

63
Descriptive analytics - example
• An e-commerce company want to analyze the sales data for a particular product
over the past year.

• It has data describing each sale such as: date of the sale, the product sold, the
quantity, and the revenue generated.

64
Descriptive analytics - example
• Descriptive analytics include

• Summarization: Computing average revenue for each product

• Summarization: What is the total revenue of each product

• Visualization: Plot sales of product A over timeline

• Visualization: Plot Quantity sold every month for product A

• Determine top selling products based on total revenue generated by each product

• Customer segmentation: grouping customers based on purchase frequency,

total spending by customer, identify high value and low value customers

• Peak sales times: Identify days of the week when sales are highest

65
66
Predictive analytics
Predictive analytics
• In predictive analytics the focus is on finding future trends given historical data and
current data

• The goal is to build models that can learn from historical data.

• Use the learning to make predictions about future events (data).

• The models are trained on known outcomes and patterns.

• The trained models are employed to predict new or unseen data.

68
Example - customer churn
Example - customer churn
• Percentage of customers who stopped using the product or service

• In competitive marketplace identifying potential customer who would discontinue

their services is key.

• If known beforehand the intent of the customer that he/she would leave the
services company may initiate retention plan. Which may include offering a coupon
or providing discount for the previous three months service etc.

• The main challenge is no customer will explicitly state the reason leaving the
services

• The reasons are to be understood from multiple data sources. Here is an example

70
Example - customer churn

Data from billing department Data from service department

71
Example - customer churn
• In this example a correlation exists between number of complaints raised, resolved,
unresolved and rating.

• Obtain an elaborate data for all active customers for past four months a snapshot
of the same is given below.

72
Customer churn
• When large number of customer data is presented, it is hard to establish relations
between churn and the attributes of the data visually.

• When a relationship is established, the relationship is to be validated. In the

previous example, customer ID 1235 is identified as potential to churn. This must be
validated.

• The outline of predictive analytics is

• Perform data analysis and modeling: Analyze historical data and discover
relationships, correlations, and patterns that can be used to make predictions. In
the customer churn examples, complaints raised, number of unresolved
complaints, rating on resolved complains are the information relationships.

73
74
Data analytics tools
76
Data analytics tools
• Several tools which are used for data analytics. Some of them are listed below

• Microsoft Excel: Widely used for basic data analysis, calculations, and charting.

• Tableau: Known for its powerful data visualization capabilities and interactive
dashboards.

• Power BI: Microsoft's business analytics service for interactive visualizations and
business intelligence.

• QlikView/Qlik Sense: Tools for data visualization, reporting, and business

intelligence.

• R: A programming language and software environment for statistical computing

and graphics.

• Python: A versatile programming language used for data analysis and manipulation
with libraries like pandas and NumPy.
77
Data analytics tools
• SAS (Statistical Analysis System): Offers a suite of software for advanced analytics,
business intelligence, and data management

• SPSS (Statistical Package for Social Sciences): Software for statistical analysis,
used in social science research and data mining.

• MATLAB: A programming language and environment for numerical computing,

often used in engineering and scientific research.

• Stata: A software package used for data analysis and statistical purposes

• JMP: Statistical discovery software often used for exploratory data analysis.

78
DA102 – Microsoft Excel
• This course focus on basics of data analysis and business modeling

• Microsoft Excel software tool is used.

• The choice of this tool stems from the fact that students of this online programme
are in their first semester and are yet to introduce to programming languages.

• For those students who are introduced to the programming languages this course
should enrich them with the orientation of data manipulation through software
systems.

80
DA102 – Detailed contents
• Basic spread sheet modeling

• Understanding range names

• LOOKUP functions

• INDEX function

• MATCH function

• Text manipulation functions

• Time – Dates and date functions

• Conditional statements (IF statement)

81
DA102 – Detailed contents
• Three-dimensional formulas

• Sensitivity analysis

• COUNT family

• SUM family

• OFFSET function

• INDIRECT function

• Data validation

• Filtering and removing duplicates

• Consolidating data
82
DA102 – Detailed contents
• Pivot tables

83