Chapter 2

Introduction to Data Science

 Outline
❖ An Overview of Data Science

❖ What are data and information?

❖Data types and their representations

❖Data value Chain

❖Basic concepts of big data

2 Introduction to Data Science

 Objective
After completing this chapter, the students will be able to:
 Describe what data science is and the role of data
scientists.
 Differentiate data and information.
 Describe data processing life cycle
 Understand different data types from diverse
perspectives
 Describe data value chain in emerging era of big data.
 Understand the basics of Big Data.
 Describe the purpose of the Hadoop ecosystem
components.
3 Introduction to Data Science
 Overview of Data Science
 Data science is a multi-disciplinary field that uses
scientific methods, processes, algorithms and systems to
extract knowledge and insights from structured,
semi structured and unstructured data.
 Data science continues to evolve as one of the most
promising and in-demand career paths for skilled
professionals.
 Today, successful data professionals understand that they
must advance past traditional skills of analyzing large
amounts of data, data mining, and programming skills.
4 Introduction to Data Science
 Overview of Data Science
 Data scientists must master the full spectrum of the data
science life cycle in order to uncover useful intelligence
for an organization, .
 Data scientists need to:
 Be curious and result-oriented
 Be good at communication skills that allow them to
explain highly technical results to their non-technical
counterparts.
 Have a strong quantitative background in statistics and
linear algebra as well as programming knowledge with
focuses in data warehousing, mining, and modeling to
build and analyze algorithms.
5 Introduction to Data Science
 What are Data and Information?
What is Data?
 Data can be described as unprocessed facts and
figures.
 Data can be defined as a collection of facts, concepts,
or instructions in a formalized manner
 Data should be suitable or easy for communication
interpretation, or processing by human or electronic
machine.
 Data is represented with the help of characters such as
alphabets (A-Z, a-z), digits (0-9) or special characters
(+,-,/,*,<,>,= etc.)
6 Introduction to Data Science
 What are Data and Information?
What is Information?
 Information is the processed data on which decisions
and actions are based.
 It is data that has been processed into a form that is
meaningful to the recipient and is of real or perceived
value in the current or the prospective action or
decision of recipient.
 Further more, information is interpreted data; created
from organized, structured, and processed data in a
particular context.
7 Introduction to Data Science
 Data Vs Information
Data Information
• Described as unprocessed or raw • Described as processed data
facts and figures
• Cannot help in decision making •
Can help in decision making
• Raw material that can be organized, •
Interpreted data; created from
structured, and interpreted to create
organized, structured, and
useful information. processed data in a particular
context.
• ‘groups of non-random’ symbols in • Processed data in the form of
the form of text, images, and voice text, images, and voice
representing quantities, action and representing quantities, action
objects'. and objects'.
8 Introduction to Data Science
 Data Processing Cycle
 Data processing is the re-structuring or re-ordering of
data by people or machine to increase their usefulness
and add values for a particular purpose.
 Data processing cycle is consisted of three basic steps-
Input, Processing, and Output.

9 Introduction to Data Science

 Data Processing Cycle
 Input step− the input data is prepared in some convenient
form for processing. The form depends on the processing
machine.
 For example - when electronic computers are used, the input data can be
recorded on any one of the several types of input medium, such as
magnetic disks, tapes,
 Processing step − the input data is changed to produce data
in a more useful form.
 For example - pay-checks can be calculated from the time cards, or a summary of
sales for the month can be calculated from the sales orders.
 Output step − the result of the proceeding processing step
is collected.
 The particular form of the output data depends on the use of the data.
 For example - output data may be pay-checks for employees.
10 Introduction to Data Science
 Data Processing Cycle

11 Introduction to Data Science

 Data types and their representation
 Data types can be described from diverse
perspectives.
 In computer science and computer
programming, for instance, a data type is simply
an attribute of data that tells the compiler or
interpreter how the programmer intends to use
the data.

12 Introduction to Data Science

 Data types from Computer
programming perspective
 All programming languages explicitly include the notion
of data type
 Common data types include:
• Integers (int)- is used to store whole numbers, mathematically
known as integers
• Booleans (bool)- is used to represent restricted to one of two
values: true or false
• Characters (char)- is used to store a single character
• Floating-point numbers(float)- is used to store real
numbers
• Alphanumeric strings(string)- used to store a combination
of characters and numbers
13 Introduction to Data Science
 Data types from Data Analytics
perspective
 Data analytics is the science of analyzing raw
data in order to make conclusions about that
information
 From a data analytics point of view, there are
three common data types or structures:
 Structured data
 Semi-structured data
 Unstructured data

15 Introduction to Data Science

 Data types from Data Analytics perspective

Structured, Unstructured, and Semi-structured data types

16
 Structured Data
 Structured data is data that adheres to a pre-defined
data model and is therefore straightforward to
analyze.
 Structured data conforms to a tabular format with a
relationship between the different rows and columns.
 Common examples of structured data are Excel files
or SQL databases.
 Each of these has structured rows and columns that
can be sorted.
17 Introduction to Data Science
 Unstructured Data
 Unstructured data does not have a predefined data model
and is not organized in a pre-defined manner.
 Unstructured information is typically text-heavy but may
contain data such as dates, numbers, and facts as well.
 Unstructured data is difficult to understand using
traditional programs as compared to data stored in
structured databases.
 Common examples of unstructured data include audio
files, video files, PDF, Word file or No-SQL databases.

18 Introduction to Data Science

 Semi-Structured Data
 Semi-structured data is a form of structured
data that does not obey the tabular structure of data
models associated with relational databases or other
forms of data tables
 Semi-structured data contains tags or other markers
to separate semantic elements within the data.
 Therefore, it is also known as self-describing
structure
 Example of semi-structured data: XML, JSON…
19 Introduction to Data Science
20 Introduction to Data Science
 Metadata – Data about Data
 From a technical point of view, this is not a
separate data structure, but it is one of the most
important elements for Big Data analysis and big
data solutions.
 Metadata is data about data.
 It provides additional information about a
specific set of data.

21 Introduction to Data Science

 Metadata – Data about Data
 In a set of photographs, for example, metadata
could describe when and where the photos were
taken.
 The metadata then provides fields for dates and
locations which, by themselves, can be
considered structured data.
 Because of this reason, metadata is frequently
used by Big Data solutions for initial analysis.
22 Introduction to Data Science
 Big Data Value Chain (DVC)
 The Data-Value-Chain describes the information flow
within a big data system that aims to generate values and
useful insights from data.
 The Big Data Value Chain identifies the following key high-
level activities:
✓ Data Acquisition
✓ Data Analysis
✓ Data Curation
✓ Data Storage
✓ Data Usage
23 Introduction to Data Science
 Data Value Chain (DVC)

24 Introduction to Data Science

 Data Acquisition
 Data acquisition is the process of gathering, filtering, and
cleaning data before it is put in a data warehouse or any
other storage solution on which data analysis can be
carried out.
 Data acquisition is one of the major big data challenges in
terms of infrastructure requirements.
 The infrastructure required to support the acquisition of
big data must provide
 low latency
 high volumes of transaction
 flexible and dynamic data structures

25 Introduction to Data Science

 Data Analysis
 It is concerned with making the raw data acquired
amenable to use in decision-making as well as domain-
specific usage.
 Data analysis involves exploring, transforming, and
modeling data with the goal of highlighting relevant
data, synthesizing and extracting useful hidden
information with high potential from a business point of
view.
 Related areas include data mining, business intelligence,
and machine learning.
26 Introduction to Data Science
 Data Curation
 It is the active management of data over its life cycle to
ensure it meets the necessary data quality requirements
for its effective usage.
 Data curation is the processes of Content creation,
Selection, Classification, transformation, validation and
preservation
 Data curation is performed by expert curators (Data curators,
scientific curators, or data annotators) that are responsible for
improving the Accessibility, Quality, Trustworthy, Discoverable,
Accessible and Reusable of data

27 Introduction to Data Science

 Data Storage
 It is the persistence and management of data in a
scalable way that satisfies the needs of applications that
require fast access to the data.
 Relational Database Management Systems (RDBMS)
have been the main solution to data storage

28 Introduction to Data Science

 Data Usage
 It covers the data-driven business activities that need
access to data, its analysis, and the tools needed to
integrate the data analysis within the business activity.

 Data usage in business decision-making can enhance

competitiveness through the reduction of costs,
increased added value, or any other parameter that can
be measured against existing performance criteria.

29 Introduction to Data Science

 Basic Concepts of Big Data
What Is Big Data?
 Big data is the term for a collection of large and complex
data sets that it becomes difficult to process using on-hand
database management tools or traditional data processing
applications.
 In this context, a “large dataset” means a dataset too large
to reasonably process or store with traditional tooling or
on a single computer.
 This means that the common scale of big datasets is
constantly shifting and may vary significantly from
organization to organization.
30 Introduction to Data Science
 Basic Concepts of Big Data
Big data is characterized by 3V and more:
➢Volume: large amounts of data Zeta bytes/Massive datasets
➢Velocity: Data is live streaming or in motion
➢Variety: data comes in many different forms from diverse
sources
➢Veracity: can we trust the data? How accurate is it? etc.

31 Introduction to Data Science

 Clustered Computing
 Because of the qualities of big data, individual
computers are often inadequate for handling the
data at most stages.
 To better address the high storage and
computational needs of big data, computer
clusters are a better fit.
 Big data clustering software combines the
resources of many smaller machines, seeking to
provide a number of benefits.
32 Introduction to Data Science
 Benefits of Clustered Computing
 Resource Pooling: Combining the available storage
space to hold data is a clear benefit, but CPU and
memory pooling are also extremely important.
Processing large datasets requires large amounts of all
three of these resources.
 High Availability: Clusters can provide varying levels
of fault tolerance and availability guarantees to prevent
hardware or software failures from affecting access to
data and processing. This becomes increasingly
important as we continue to emphasize the importance
of real-time analytics.
33 Introduction to Data Science
 Benefits of Clustered Computing
Easy Scalability:
 Clusters make it easy to scale horizontally by
adding additional machines to the group.
 This means the system can react to changes in
resource requirements without expanding the
physical resources on a machine.

34 Introduction to Data Science

 Hadoop Ecosystem
 Hadoop is an open-source framework intended to make
interaction with big data easier.

 It is a framework that allows for the distributed

processing of large datasets across clusters of
computers using simple programming models.

 It is inspired by a technical document published by

Google.

35 Introduction to Data Science

 Hadoop Ecosystem Interface

36 Introduction to Data Science

 Big Data Life Cycle with Hadoop
 Activities or life cycle involved with big data
processing are:
I. Ingesting data into the system

II. Processing data in storage

III. Computing and analyzing data

IV. Visualizing the results

37 Introduction to Data Science
 Ingesting data into the system
 The first stage of Big Data processing is Ingest.
 The data is ingested or transferred to Hadoop
from various sources such as relational databases,
systems, or local files.
 Sqoop transfers data from RDBMS to HDFS,
whereas Flume transfers event data.

38 Introduction to Data Science

 Processing the data in storage
 The second stage is Processing. In this stage, the
data is stored and processed.
 The data is stored in the distributed file system,
HDFS, and the NoSQL distributed data, HBase.
 Spark and MapReduce perform data processing.

39 Introduction to Data Science

 Computing and analyzing data
 The third stage is analyzing. Here, the data is
analyzed by processing frameworks such as Pig,
Hive, and Impala.
 Pig converts the data using a map and reduce and
then analyzes it.
 Hive is also based on the map and reduce
programming and is most suitable for structured
data.
40 Introduction to Data Science
 Visualizing the results
 The fourth stage is Access or Visualizing, which is
performed by tools such as Hue and Cloudera Search.
 In this stage, the analyzed data can be accessed by
users.

41 Introduction to Data Science