BIG DATA ANALYTICS
By: Syed Nawaz Pasha @ SR Univeristy
Professional Elective-5
B.Tech IV-II SEM
Preferred Text Book
Author: Seema Acharya and
subhashini chellapan.
Book Title: Big Data and
Analytics
Publisher : Wiley
Types of digital data
Data is precious and irreplaceable asset.
Data->information
Information-> valuable insights
Classification of digital data
Digital data can be classified into three forms:
– Unstructured
– Semi-structured
– Structured
Unstructured data
This is the data which does not conform to a
data model or is not in a form which can be
used easily by a computer program.
About 80—90% data of an organization is in
this format.
Example: PPT’s, images, videos, letters, body
of an email, etc.
Semi-structured data
This is the data which does not conform to a
data model but has some structure.
However, it is not in a form which can be used
easily by a computer program
Examples: emails, XML, markup languages
like HTML, etc.
Structured data
This is the data which is in an organized form
(e.g., in rows and columns) and can be easily
used by a computer program. Relationships
exist between entities of data, such as
classes and their objects.
Data stored in databases is an example of
structured data.
Structured data continue
When do we say that the data is structured?
When data conforms to predefined
schema/structure.
Ex : RDMS
Relational data model where data is stored as
rows and columns.
Sources of structured data
Ease of working with structured
data
Insert/update/delete
Indexing
Scalability
Transaction processing
Semi structured data
Also referred to as self describing structure.
Characteristics of semi-structured data:
Inconsistent structure
Semi structured data Self describing(label/value pairs)
Often schema information is blended with data values
Data objects may have different attributes not known before h
Sources of semi structured data
XML:web services developed using SOAP
JSON:transmit data between server and web
application using REST.
HTML
JSON
{
Id:9
Booktitle=“BDA”
Author=“seema acharya”
Publisher=“wiley india”
YOP=“2011”
Unstructured data
Does not conform to any predefined data
model.
Issues with terminology of unstructured data
Structure can be implied despite not being formerly
Issues with defined
terminology
Data with some structure may still be labeled
unstructured if the structure doesn’t help with
processing task in hand
Data may have some structure or may even be
highly structured in ways that are unanticipated or
unannounced
Dealing with unstructured data
Data mining
Dealing with
unstructured data NLP
Text analytics
Noisy text
analytics
Techniques used to find patterns in
or interpret unstructured data
1. Data Mining
Association rule mining
Regression analysis
Collaborative filtering
Text analytics or text mining:text categorization,text
clustering,sentimental analysis..
Noisy text analytics: extracting structured or
semistructured information from noisy unstructred
data such as chats,blogs,wikis,email.
Part-of-speech tagging:POS or POST
Remind me
Structured data
Semi-structured data
Unstructured data
Test me(seggregate the below as
structured,semi structured and
unstructured)
Email
Msaccess
Images
Database
Chat conversation
Relations
Facebook‘
Videos
Ms excel
XML
Sources of unstructured data
Definition of big data-By
Gartner
Big data is high volume, high velocity and high
variety information assets that demand cost
effective, innovative forms of information
processing for enhanced insights and decision
making.
Big Data Definition
Big data refers to huge amount of data which
is difficult to store and process using on-hand
database system tools or traditional data
processing applications.
Characteristics of Big data(5 V’s of Big
Data).
volume
velocity
Variety
Value
veracity
5 V’s of big data
Challenges with big data
Exponential growth of data-Today’s BIG may be
normal tomorrow.
Cloud computing and virtualization.Complicates
the decision to host big data solutions outside the
enterprise.
Period of retention of big data.
No skilled professionals of data science.
Difficult to
capture,store,prepare,search,analyze,transfer,sec
ure and visualize big data.
Poor data visualization experts.
Big data analytics
Process of examining large datasets of big data to uncover hidden
patterns,unknown correlations,understand the retionale behind
market trends,recognize customer preferences and other business
information.
Classification of analytics
Descriptive analytics
Diagnostic analytics
Predictive analytics
Prescriptive analytics
Analytics 1.0(mid 1950’s Analytics 2.0(2005 to 2012)
to 2009) Descriptive statistics predictive
Descriptive(report on statistics
events,occurrence etc (use data from past to make
predictions for future).
Analytics 3.0
(2012 to present)
Descriptive+predictive+pr
escriptive)
Analytics 1.0,2.0,3.0
Terminologies used in big data
environment
In memory analytics
In database processing
Symmetric multiprocessor system
Massively parallel processing
Distributed vs parallel systems
Share nothing architectures
Shared memory(SM)
Shared Disk(SD)
Shared Nothing(SN)
CAP Theorem
It is impossible for a distributed system to
simultaneously provide all three of the following
guarantees: (Pick any two)
1. Consistency: All nodes should see the same
data at the same time or reads return latest
written value by any client
2. Availability: Every request receives a
response. The system allows operations all the
time and operations return quickly
3. Partition – Tolerance: the system continues to
operate despite arbitrary partitioning due to
network failures
Samples of databases that follow one of the
possible three combinations:AP,CP,CA
