0% found this document useful (0 votes)
218 views103 pagesData Warehousing
The document outlines the syllabus for a course on Data Warehousing (CS341) at Anna University, detailing various topics such as data warehouse architecture, ETL and OLAP technology, metadata, data marts, dimensional modeling, and system management. It emphasizes the importance of data warehousing in enabling organizations to effectively gather, store, and analyze large volumes of data for decision-making. The course aims to provide a comprehensive understanding of data warehousing concepts and practices through structured chapters and review questions.
Uploaded by
shnmaroCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
0% found this document useful (0 votes)
218 views103 pagesData Warehousing
The document outlines the syllabus for a course on Data Warehousing (CS341) at Anna University, detailing various topics such as data warehouse architecture, ETL and OLAP technology, metadata, data marts, dimensional modeling, and system management. It emphasizes the importance of data warehousing in enabling organizations to effectively gather, store, and analyze large volumes of data for decision-making. The course aims to provide a comprehensive understanding of data warehousing concepts and practices through structured chapters and review questions.
Uploaded by
shnmaroCopyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF or read online on Scribd
/ 103
SUBVECT CODE : €CS341
Strictly 0s per Revised Syllabus of
ANNA UNIVERSITY
Choice Based Credit System (CBCS)
Vertical - 3 (Cloud Computing and Data Centre Technologies) (CSE/IT/AIADS)
Vera - 2 (Cloud Computing and Data Cente Techolosis) (CSABS)
DATA WAREHOUSI
Dr. Arati Dandavate
PhD [Computer Science & Eginoetng
ME Compute, BE lecronee & Communcaton)
HOD [CE Dep),
OPES, Dhol Pol Col
Powor Collage of Eng
One
=> TECHNICAL
= PUBLICATIONS
svctia —_—_AnUpThrst fr Kowiedge SYLLABUS
Data Warehousing - (CCS341)
u INTRODUCTION TO DATA WAREHOUSE
Data warchouseIniroduction- Data warehouse compone tional database Vs data warehouse
+ Data warehouse Architocture - Threo-ier Data Warehouse Architecture - Autonomous Data
‘Warehouse - Autonomous Data Warehouse Vs Snowflake - Modem Data Warehouse, (Chapter - 1)
UNIT ETL AND OLAP TECHNOLOGY
What is ETL - ETL Vs ELT - Types of Data warehouses - Data warehouse Design and Model
Delivery Process - Online Analytical Processing (LAP) - Characteristics of OLAP - Online
‘Transaction Procesing (OLTP) Vs OLAP - OLAP operations - Types of OLAP- ROLAP Vs MOLAP
\VsHOLAP. (Chapter -2)
UNIT META DATA, DATA MART AND PARTITION STRATEGY
Meta Data - Categories of Metadata - Role of Metadata - Metadata Repository - Challenges for Meta
Monagement - Data Mart - Need of Data Mart - Cost Ffective Data Mart - Designing Data Mars
(Cost of Data Marts ~ Partitioning Strategy ~ Vertical partition - Normalization ~ Rave Spliing
Horizootal Patton. (Chapter -3)
UNITIV DIMENSIONAL MODELING AND SCHEMA
Dimensional Modeling - Muti - Dimensional Data Modeling - Data Cube - Star Schema - Snowflake
Schema ~ Star Vs Snowflake schema - Fact constllation Schema - Schema Definition - Process
Architecture - Types of Data Base Parallelism - Datawarehouse Tools. (Chapter -4)
UNITV SYSTEM & PROCESS MANAGERS
Data Warehousing System Managers: System Configuration Manager- System Scheduling Manager
System Event Manager - System Database Manager - System Backup Recovery Manager - Data
Warehousing Process Managers : Load Manager - Warehouse Manager - Query Manager - Tuning,
Testing. (Chapter -§)
TABLE OF CONTENTS
Ghapter-1 Introduction to Data Warehouse (1-1 to (1
11 Introduction,
1.1.1 What is Data Warehouse ?
History of Data Warehouse
1.2.1 Characteristics of Data Warehousing.
Data Warehouse Tools
Need for Data Warehouse.
Benefits of Data Warehouse
Data Warehouse Working ?
Data Warehouse Components.
1.7.1 Source Data Component.
1.72 Data Staging Component
1.7.3 Management and Control Component.
Operational Database Vs Data Warehouse.
Data Warehouse Architecture.
1.9.1 Types of Data Warehouse Architect
1.10 Autonomous Data Warehouse.
1.10.1 Modern Data Warehouse.
1.10.2 Modern Data Warehouse's Key Features
1.11 Snowflake : A Different Data Warehouse Architecture.
Review Question:
1.12 Two Marks Questions with Answers Chapter -2
2a
ETL and OLAP Technology (2-1) to (2-54)
ETL (Extract, Transform, Load) 2-2
211 ETL Process. 2-3
2.1.2 Advantages of ETL Process 2-5
2.1.3 Disadvantages of ETL Process 2-6
ETLVs ELT. 2-8
Types of Data Warehouses, 2 2-9
Data Warehouse Design.
24.1 Data Warehouse Design : Top Down Approach,
24.2 Data Warehouse Design : Bottom Up Approach.
24.3 Top Down and Bottom Up Approach Comparision
Data Warehouse Modeling
25.1 Data Modeling Life Cycle.
Delivery Process
Online Analytical Processing (LAP)
274 OLAP Guidelines (Dr. F-Codd Rule)
2.7.2. Characteristics of OP.
2.7.3 Advantages of OLAP.
2.74 Disadvantages of OLAP
Online Transaction Processing (OLTP) Vs OLAP
LAP Operations.
291 Rollup.
292. DriltDown
293 Sle.
294 Dice,
295 Pivot
Types of OLAP.
2404 ROLAP (Relational OLAP).
210.2 ROLAP Architecture.
2.103 ROLAP Advantages and Disadvantages.
2.10.4 Multidimensional OLAP (MOLAP)
2.10.5. Advantages and Disadvantages of Multidimensional OLAP (MOLAP).
2.10.6. Hybrid OLAP (HOLAP).
2.10.7. Advantages and Disadvantages of Hybrid OLAP (HOLAP)
2.108 Other Types
2.1 Comparison of ROLAP and MOLAP
2.12 Comparison of ROLAP VS MOLAP VS HOLAP.
Review Questions..
2.13. Two Marks Questions with Answers
2-49
2-50
2-51
2-51
2-52
2-53
aaa
Chapter-3 Meta Data, Data Mart and Partition Strategy
(3-1) to (3- 40)
Meta Data
3.1.1 Categories of Metadata
3.12 Role of Metadata
3.13. Metadata Repository
3.13.1 Uses of Metadata Repository
3.14 Challenges for Meta Management
Data Mart
3.21 Cost Effective Data Mart.
3.22 Designing Data Marts
3.23 Cost of Data Marts
3.2.4 Data Mart Versus Data Warehouse
Partitioning : Introduction.
33.1 Panition Strategy
33.2 Vertical Partition
33.2.1 Vertical Partition Advantages
3-2
3-2
3-3
35
3-6
3-7
3-9
10
2 Chapter -4 (4-1) to @- 40)
4a
33.22 Vertical Partition Disadvantages
Normalization
3:34. Advantages of Normalization
33.3.2 Disadvantages of Normalization
Row Spliting
3.341 Advantages of Row Splitting
33.42 Disadvantage of Row Spliting
3.3.5 Comparison of Normalization and Row Splitting
33.6 Horizontal Partition.
3.3.7 Advantages of Horizontal Partition
3.3.8 Disadvantages of Horizontal Partition
3.3.8 Comparison of Horizontal and Vertical Partition
Review Questions...
Two Marks Questions with Answers
Dimensional Modeling and Schema
Dimensional Modeling
444 The key Principles of Dimensional Modeling.
4.1.2 _ The Objectives of Dimensional Modeling
4.13 _ Elements of Dimensional Data Model
4.14 Steps of Dimensional Modeling
4.15 Advantages of Dimensional Modeling
Multi-Dimensional Data Modeling.
42.1 Key Components of Mult - Dimensional Data Modeling
Data Cube
Star Schema
44.1 Features of star Schema
442 Advantages of tar Schema,
443 Disadvantages of star Schema
4-2
4-2
43
4-4
4
4
6
4-8
4-10
4-12
4-14
4-15
407
4-17
Snowflake Schema
45.1 Features of Snowflake Schema
45.2 Advantages of Snowflake Schema
453 _ Disadvantages of Snowflake Schema
Comparison of Star Schema and Snowflake Schema.
Fact Constellation Schema
4.7.1 Features of Fact Constellation Schema
4.7.2 Advantages of Fact Constellation Schema
4.7.3 Disadvantages of Fact Constellation Schema
Schema Definition
Process Architecture
4.9.1 Types of Process Architecture.
492 Centralized Process Architecture
493 Benefits of Centralized Process Architecture.
494 Distributed Process Architecture
435 Benefits of Distributed Process Architecture
‘Types of Data Base Parallelism.
Datawarehouse Tools
Review Questions.
4.12 Two Marks Questions with Answers
Chapter-5 System & Process Managers
5.1 Data Warehousing System Managers
5.2 _ System Configuration Manager.
5.3 _ System Scheduling Manager.
5.44 System Event Manager.
55 System Database Manager.
5.6 System Backup Recovery Manager
5.7 Data Warehousing Process Managers
im
4
4
4
4
4-35
4-36
4-38
4-38
Ta
G- 1) to (5-26)
5-2
5-3
5-5
5-6
5-8
5-9
5-11 Load Manager.
58.1 Load Manager Architecture
Warehouse Manager.
59.1 Warehouse Manager Architecture
59. Functions ot Windies amas Introduction to Data
Query Manager Warehouse
5.10.1 Query Manager Architecture
5.11. Tuning
5.12 Testing Syllabus
Daa warehouse Introduction ~ Data warehouse components- operational davabase Vs data
Review Questions... i x nee warehouse - Data warehouse Architecture - Three-ier Data Warehouse Architecture ~ Autonomous
Data Warehouse- Autonomous Data Warehouse Vs Snowflake - Modern Data Warehouse,
5:13. Two Marks Questions with Answers fs 5-24
‘Solved Model Quest Paar ena = oF (MTA) Hel (MT= 9) 7 Contents
14 Inrosucton
12 History of Data Warchouse
4.3 Data Warehouse Tools
14 Need for Data Warehouse
1.5 Benefs of Data Warehouse
1.8 Data Warehouse Working ?
17 Date Warehouse Components
1.8 Operationel Database Vs Data Warehouse
1.9 Data Warehouse Architecture
‘Autonomous Data Warehouse
‘Snowflake A Different Date Warehouse Architecture
Two Marks Questions with Answors Data Warehousing _______tnvesucbon tata Warehouse
EEE introduction
‘© ‘The business firms often generate billions of bytes of data every day on all parts of
‘company, which contains millions of unique details about their clients, personnel and
operations, However, access to this daa is severely restricted and sealed up. Executives
id decision - makers have access to a very limited portion of the data that is actually
gathered, processed, and stored inthe company,
[A new technology has recently developed from new ideas and tools, enabling all the
essential players in an organization to have access to the knowledge they need to thrive
in 2 world that is becoming more and more competitive. "Data warehousing” is the
name given to this new technology. I will be talking about the Fundamental ideas and
vocabulary of data warehousing in this unt
Our first desktop "What if” processing test was with the Lotus. The goal of a data
‘warehouse is to leverage the information of company has acquired to help it respond
‘more effectively, intelligently, quickly and efficiently.
Modem organizations depend on the efficient gathering, storing and integrating of data
from many sources for analysis and insights. These data analytics roles are now vital to
cost reduction, revenue growth and profit maximization. Therefore, it is not surprising
that both the amount of data that has been collected and analyzed as welt as the variety
‘and number of data sources have expanded,
Data-driven enterprises need robust solutions to handle and analyze massive volumes of
data throughout theit whole organization, These solutions are reliable, scalable and
secure enough for regulated industries while supporting a broad varity of data formats
‘and use cases. Any typical database would struggle to meet the needs. The data
warehouse may be of assistance here.
A data warehouse is created by combining data from several sources. Ad hoc or planned
inquiries, analytical reporting and decision - making are made easier. This course uses a
step-by-step technique to teach every fundamental data warehousing theory
‘A Relational Database Management System (RDBMS) concept called a data warehouse
1vas created to address the needs of transaction processing systems. Any centralized dats
store that may be searched for commercial advantages can be broadly characterized as
such, It is @ database that holds data intended to satiate inquiry - based decision ~
‘making. It ig a collection of technologies designed to aid in decision ~ making and
empower knowledge workers (executive, manager and analyst). In order to help
"ECIAGAL PUBUCATIONS® an pt or ronege
Yarshousing he
Antoun ta Date Warshouse
corporate leaders consistently organize, comprehend and apply their information to
‘make strategic choices, data warehousing supports structures and tools.
‘A Data Warehousing (DW) process is used to gather and manage data from many
sourees in order to provide insightful business information. Business data from many
sourees is often connected and analyzed using a data warehouse. The central component
of the BI system, which is designed for data analysis and reporting, is the data
warchouse,
The combination of several technologies and elements facilitates the strategic use of
data, Large amounts of data are electronically stored by a company and are intended for
analysis and inquiry rather than transaction processing, It is process of converting data
{nto information and promptly making it accessible to people so that it might have an
impact.
‘An OLAP engine, customer analysis tools, an Extraction, Transportation and Loading.
(ETL) solution, as well as additional applications that manage data collection and
delivery to business users are all included inthe data warshouse environment,
What is Data Warehouse 7
‘A data system having the following characteristics might be thought of as a data
warehouse
(© It isa database that uses information from many apps to be used for investigative
activities.
It allows for reasonably lengthy exchanges between a limited number of
customers
© Itcontains both recent and old data to give information a historical context.
© Its use involves a lot of reading,
© There are a couple big tables in there.
"Data Warchouse is a subject - oriented, integrated and time - variant store of
{information to support management's decisions.”
A Data Warehousing (DW) process is used to gather and manage data from many
sources in order to provide pertinent business information, Business data from many
sources is routinely linked together and analyzed using a data warchouse.
Data from many sourees, including as point-of-sale operations, marketing automation,
relationship management and more, is processed and reported on using an enterprise
system called a data warehouse. Ad hoe analysis and customized reporting are both Data Warehousing
appropriate uses for data warchouses. A data warehouse is a crucial part of business
intelligence since it was created to provide a long-term perspective on data throughout
time and can store both current and historical data in one location
EE History of Data Warehouse
+ Let us first review the historical management schemes of the analysis data and the
factors that have led to the evolution ofthe data warehousing application class
Traditional approaches to historical data
+ The operational systems and the data they process have received the mjority of
aatention throughout the development of systems. Since it was impractical to maintain
data in operating systems permanently, a framework for archiving the data such systems
hnnd_ processed was only conceived of after the fact, Operational systems need
performance, but analysis systems need flexibility and a wide range of capabilites.
‘These ae the essential needs that distinguish operational systems from analysis systems,
Data from legacy systems
* Over the last three decades, several platforms have evolved with the evolution of
computer technology. Business system development was carried out on IBM mainframe
computers in the 1970s utilizing programs like Cobol, CICS, IMS, DB2, ete
‘The 1980s saw the development of computer platforms like the AS / 400 and VAX /
VMS. The client/server architecture, whichis still in use today, was introduced by the
server platform UNDX in the late 1980s and early 1990s,
‘The mainfiame environment from the 1970s is still used to operate a significant number
of commercial applications, despite all the changes in platforms, architectures, tools and
technology. The main reason is that through time, these systems have accumulated
business knowledge and regulations that are very challenging to transfer to a new
platform or application. These are often referred to as legacy systems, Such technologies
eventually make the data they store distant and challenging to access
Extracted information on the desktop
‘The personal computer has significantly inereased in use for business analysis during the
- last ten years. Many ofthe tools needed to utilize spreadsheets for analysis and graphical
depiction are now available to business analysts. Advanced users will regularly work
with and store data retrieved from the historical sources using desktop database tools
‘© The above has the drawback of leaving the data fragmented and focused on very
particular requirements. Only the information that each individual user needs has been
acquired. The needs of many consumers and uses cannot be met by the extracts,
TECHOCAL PUBLICATIONS® on patronage
Data Warehousing Inradueton to Dats Warehouse
Addressing the needs of a single user requires a lot of effort and money. The drawbacks
experienced prompted the creation of the new application known as data warehousing.
Factors, which lead to data warehousing
‘The data warehousing field has quickly developed due to a variety of variables. The
development of hardware and software technologies has been the most crucial element
Costs of hardware and software Prices for hardware and software have significantly
decreased. Cheaper memory chips with a higher capacity are readily accessible
Strong preprocessors : Preprocessors nowadays are far more powerful than mainframes
of the past, such as Pentium III and Alpha processors,
Budget-fiiendly disks : As hard disk costs decrease, they can now hold hundreds of
sigabytes. In the 1970s and the early 1980s, it would bave taken an entire roomful of
disk drives to store the amount of data that can now be stored on a single disk drive that
is just ono inch high
Strong desktop analysis tools desktops can do tasks that the old - school mainframe
computers couldn', including multi - tier computing, client server architecture, or
user-friendly GUI interfaces,
Server software : When compared to earlier versions, server software is naw reasonably
priced, robust and simple to maintain,
‘An example of this is Windows NT, which has made the configuration of powerful
‘computers incredibly simple and cheaper.
‘The most significant factor in the development of data warehouses has been the
exponential growth of hardware and software capabilites, together with the accessibility
of low ~ cost, user - friendly reporting and analytical tools.
Emergence of standard business applications
‘© Users may now get popular business application packages fom new suppliers. The
German software companies SAP AG, Baan, PeopleSoft and Oracle have all released
software suites with varying capabilities but similar functions. Standard applications are
offered by these application suites, which may take the place of the old custom
developed apps now in use
This is what has caused such programs to become more popular,
Additionally, compared to mainframes, these applications’ data gathering is far more
straightforward. End - user more technology oriented
‘© The development of a technology - focused business analyst is one of the most
significant outcomes of the tremendous investment in technology and march toward the
powerful personal computer. Even while technology - focused end users are not always
advantageous to all projects, this trend has undoubtedly generated a crop of business
‘analysts who are at the foreffont of technology who are now crucial to today's
businesses. These technologically savvy end customers have typically contributed
Significantly to the design and implementation of data warehouses. They are now the
primary users who are the first to highlight the early advantages of data warehouses,
‘These end users are essential for the growth of the data warehouse model because they
‘each other users as they gain expertise with the system.
Users may conduct more precise analyses and improve the functioning of their
organizations by using the data warehouse, As computers got more advanced and
Tequited to process more and larger amounts of data, the necessity to store data
‘developed. But data warchousing is hardly a brand-new concept.
Following are some key events in evolution of data warehouse -
* 1960 - The phrases: measurements and facts are ereated in 1960 a8 a result of a
cooperative research initiative between Dartmouth and General Mills
‘+ 1970- Retail sales dimensional data marts are introduced by Nielsen and TRI.
‘© 1983 - A system for managing databases ereated with decision assistance in mind is
offered by Tera Data Corporation,
When IBM employees Paul Murphy and Barry Devlin created the Business Data
Warchouse inthe late 1980s, data warehousing was born
However, Inman Bill provided the actual notion. He was revered as the inventor of the
data warehouse, He has written on a range of subjects related to the construction,
‘operation and upkeep of the warehouse and the corporate information factory.
HERI Characteristics of Data Warehousing
‘Typically, a data warchouse has four characteristics, according to W. H. Inmon, author
‘of Building the data warehouse and the expert who is usually regarded as the concept's
‘A data warehouse, according to W. H. Inmon, is "a subject - oriented, integrated,
‘nonvolatile, time - variant collection of data to support management’ decisions." Access
to data for in-depth analysis, knowledge discovery and decision - making is made
possible by data warshouses,
TECHNICAL PUBLICATIONS? en pt or onto
Data Warehouse
The organization of data is subject - based rather than application - based, for example,
‘an insurance business employing a data warehouse would arrange its data by customer,
premium and claim rather than by various produets (car, life, ete.) The subject - specific
data only includes the details required for decision support processing.
Integrated
‘Encoding of data is often uneven when it is stored in different financial apps in the
‘operational environment. For instance, although gender may be classified as "mn" and "{"
in one application, 0 and I in another. When data are transferred from the operational
environment to the data warehouse, they are assumed to follow a uniform coding
scheme, for example, gender data is changed to "m" and "f."
Change in time
+ Data that are five to ten years old or older may be stored in the data warchouse and
utilized for comparisons, trends and forecasts. These statistics are outdated,
Non-volatile
*# Once they are entered into the data warchouse, the data are merely accessed - they are
not updated or altered in any way.
‘+ These specific qualities define data warehouses.
© Aconceptual multidimensional perspective
Universal dimension
Limitless aggregation and dimension levels.
Unrestricted operations across dimensions
Handling of dynamic sparse matrices.
The client-server architecture.
‘Multi-user assistance.
Availability
Openness,
Intelligent data manipulation.
Accurate performance in reporting
© Flexible reporting
Data warehouses are often an order of magnitude (and perhaps two orders of magnitude)
bigger than the source databases since they contain massive amounts of data ta Worahousing gat reduction to Data Werehou:
Enterprise - wide data wazchouses, virtual data warchouses and data marts have all been
used to address the problem of the sheer amount of data, which is most likely to be
measured in terabytes,
Entesprise - wide data warehouses are huge initiatives requiring substantial time and
resource investments,
Virtual data warehouses provide users access fo materialized views of operational
databases.
Data marts are often more narrowly focused and simed to a certain group of the
company, like a dependent.
In order to summarize the aforementioned, the following key clerients about distinct
ata warehouse features should be kept in mind
Subject - focused
(Organized around key topics, such customers, products and sales,
‘Concentrating on data modeling and analysis for decision - making rather than routine
tasks or transaction processing.
Provide a clear and brief overview of a certain topic by excluding information that is not
relevant tothe decision - making process
Integrated
‘Built by combining several, disparate data sources, such as relational databases, flat files
and online transaction records.
Offering methods for data integration and cleansing.
Time variation
‘Compared to operational systems, the data warehouse has a much longer time horizon,
‘= Every key structure in the data warehouse has an explicit or implicit reference to time.
‘Non - volatile data that has been changed from the operational environment that is
physically stored separately and doesn't need transaction processing, recovery, or
‘concurrency control methods
‘Only requires the initial loading of the data and the access of the data (no data updates
are necessary)
EE data Warehouse Tools
‘= The tools that enable peroper sourcing of data formats and contents from external data
storage into the data warehouse must carry out a number of crucial functions, including
© Integration and consolidation of data,
Data Warebousing 2 cduction (o Dats Warehouse
Data transation between different forms.
Based on the fimetion of business rules that need transformation, data
transformation and computation
Synchronization and maintenance of metadata, which includes archiving or
updating information about events, loading formats, transformation activites and
soutee files
‘There are several selection criteria which should be considered while implementing, a
data warehouse
(© Its required to be able to recognize the data thatthe tool can read from the data
Itis crucial to provide suppor for flat files, indexed files and older DBMS.
Jn many setups, the ability 10 combine records ffom several data stores is
necessary.
Conversation and the specification interface, which indicates the information to be
extracted, ae crucial
Itis desirable to be able to read data from repository products or data dictionaries,
The tool's developed code need to be entirely maintained
Users may extract just the necessary data using selective data exteation for both
data items and records
For the conversion of data into information, a field - level data inspection is
required
When transferring data across disparate systems, the capacity to execute datatype
and character - sot translation is necessary
‘Aggregation, summarization and derivation fields and records must be able to be
created
Vendor reliability and product support are important factors that need to be
carefully considered
Goals of data warehousing
‘© To.support reporting and analysis
‘© Keep the organization's historical records up to date
‘© Serve as the basis for decision Data Warehousing
HEE Need for Data Warehouse
+ The following justifies the need for a data warehoute
1, Business user: To access historical data summaries, business users mced a data
warchouse, These non-technical persons could just understand the information.
Archive historical data = A data warehouse is required to archive historic
time-raiable dats. This inputs made todo number of ferent tasks
Make strategic choices: Several approaches may be based onthe data inthe data
warehouse, Data warehouse ids in strategie decision-making asa esl
For uniformity and consistency in data : The wer may effectively work to give
uniformity and consistency in data by merging data ftom multiple sources
Quick reaction times: Data warehouses need tobe flexible and quick to respond in
order to andle etc loads and query pairs.
Benefits of Data Warehouse
Recognize market trends to enhance our forecasts
Data warchouses are designed to manage enormous amounts of data
End users can navigate, understand and query data warehouse architectures more easily
Complex queries that would be challenging to build and handle in many normalised
databases could be made easier by data warehouses.
ata warehousing is a useful method for managing the demand for a fot of information
from a lot of customers.
Data warchousing enables the analysis ofa substantial amount of historical data. What Is
1 Data warehouse used for ?
Here are the industries where data warehouses are most often used.
Aine : It is utilised for operational pusposes in the airline system, such as personne]
assignment, studies of route profitability, frequent flyer programme promotions, ct,
Banking : It is often used in the banking industry to efficiently manage the resources on
the desk, A few banks are also utilized for operations, product performance monitoring,
and market research,
Healtheare : Data warehouses were also utilized by the healthcare industry to plan and
forecast results, provide patient westment reports and communicate data with affiliated
insurance firms, medical assistance organizations, ec
TEOHnACAL PUBLGATON? on
Data Warehousing _ se Introduction o Data Warehouse
© Government sector : Data warchouses are utilised for intelligence collection in the
public sector. It aids in the upkeep and analysis of each person's tax data and health
insurance records by government authorities.
Sectors of investment and insurance
‘© In this industry, warehouses are largely used to watch market trends, analyse consumer
trends and analyse data patterns.
Keep the chain
‘© Data warehouses are often utilised in retail chains for distribution and marketing,
‘Additionally, it aids in keeping track of products, consumer purchasing trends,
promotions and pricing strategy.
Telecommunication :
‘© In this industry, distribution choices, sales decisions and product marketing decisions
are all made using a data warehouse.
The hospitality sector
‘© Based on customer feedback and travel habits, this industry uses warchousing services
to plan and prediet the locations for its advertising and marketing efforts,
[ Data Warehouse Working 7
Steps to implement data warehouse
‘© Using 2 three ~ pronged approach, is the best method to reduce the business risk
connected withthe development of a data warehouse.
Enterprise strategy : In this section, we define technical terms such as existing
architecture and technologies. Additionally, we define facts, dimensions and
characteristics. Additionally passed are data transformation and mapping.
Phased implementation : Data warchouse implementation should be done in accordance
with topic areas. Booking and billing are examples of related business entities that
should be developed first before being connected
Iterative prototyping : Developing and testing the data warehouse iteratively is
preferable than a big bang approach to implementation.
‘One or more data sourees provide their data to a data warchouse, which acts as a central
store for the data, The transactional system and other relational databases send data to a
‘data warehouse
TEANCAL PUBLICATIONS an wat inoweaoe ‘* Data may be
1. Structured
2. Semi-structured
3, Unstructured data,
Users may access the converted data in the data warehouse using business intelligen
tools, SQL clients and spreadsheets once the data has been changed, transformed and
ingested. Data from several sources is merged in a data warehouse to produce a large
database. An organization may analyses its clients more thoroughly by combining all of
this data in one location. This makes sure that all the information is taken into account
Data mining is made feasible by data warehousing, Data mining searches for patterns in
the data that might result in increased revenue and profitability
‘Advantages of data warehouse
+ Business users may easily access crucial data from a variety of sources using data
‘warehouses.
Consistent data on multiple cross-functional operations is provided via data warehouse,
‘Ad hoc reporting and querying are also supported.
To lessen the strain on the production system, data warehouses assist in integrating
several data sources
Using a data warehouse may speed up analysis and reporting overall
The user may utilize it more easily for reporting and analysis thanks to restructuring and
integration
Users may obtain crucial data from several sources in a single location thanks to data
warehouses. Asa result, it saves users’ time while obtaining data from various sourc
A substantial quantity of historical data is kept in data warehouses. Users may use this to
analyze various historical periods and patterns to forecast the future.
Disadvantages of data warehouse
‘© A poor choice for unstructured data
The development and implementation of @ data warchouse 2
consuming tasks,
Data Warehouse may easily become out of date
Changes to data types and ranges, data source structure, indexes and searches are
challenging,
Ineduction to Data Warehouse
ta Warehousing Iarodveten to Osta Werenou
‘© Although the data warehouse may seem simple, most consumers would find it to be
excessively complicate.
The scope of a data warehousing project will constantly expand, even the finest project
management efforts.
Users of warehouses may sometimes create unique business rules,
Organizations must invest a significant amount of their resources in training and
implementation,
Hd Data Warehouse Components
‘Users may access the converted data in the data warehouse using business intelligence
tools, SQL clients and spreadsheets once the data has been changed, transformed and
ingested, Data ffom several sources is merged in a data warehouse to produce a large
database
Fig. 1.7.1 shows the components ofa data warehouse,
é—-S-?
Metadata
Fig. 17.4
Central database : The data warehouse is & database. They were typically local or cloud =
based common relational databases. However, in - memory databases are quickly
gaining acceptance due to big data, the need for true, real - time performance and a
sharp drop in RAM prices.
Data integration : Several data integration methods are used to extract data from source
systems, change it and align it for easy analytical consumption, ETL (Extract,
Transform, Load) and ELT procedures are among them, as well as real-time data
replication, bulk - load processing, data transformation and services for data quality and
enrichmes
Data about our data is known as metadata. All of the characteristics of the data sets in
data warehouse ~ their origin, purpose, values and other details - are recorded. Technical
metadata describes where and how data should be stored, as well as how to get it
Business metadata provides the context of data
no ape rrr Date Warehousing 1 __ tnvoducton to Data Warehouse
* Tools for accessing data warehouse : With the help of these tools, users may interact
‘ith the data inside. A few examples of access tools are data mining, OLAP, query and
reporting tools and application development tools,
‘The best approach to assemble the pieces is via architecture, We combine hardware and
software elements to create a data warehouse, We reserve the right to shift equipment
and services to another site, however these structures are built to meet the needs of our
organizations. They are all dependent on the situation we ar in.
fe
Metadata Data ming
BMS
7 0 sre fl Ne
Archived intemal prod
Report Query
Fig. 17.2 Data warehouse component
+ The graphic shows the essential elements of a typical warehouse, The source data
component is located on the left. The data staging component isthe following building
clement. In the middle isthe data storage component, which controls the data in the data
warehouses. Along with storing and managing data, this component also manages and
tracks data via the metadata repository. The information delivery component on the right
displays the many ways that users can obtain data from data warehouses,
Source Data Component
+ Following are the categories used to clasily the source data entering the data
1, Production dat: These kinds of data come ftom the many operating systems used by
the company. Based on the data requirements in the data warehouse, we choose data
Segments rom the varios operational modes,
Intrmal dat : Databases, repos, cxstomer profiles and occasionally even
TECHNICAL PUBLICATIONS ows node
Inraducton to Data Werehous
spreadsheets are kept “private” by each client organization. This is internal data,
some of which may be included into a data warehouse
Archived information ; Operational systems’ main objective is to manage the current
business, Every operating system has old data that we regularly erase and store in
achieved files,
Extemal information ; For the information they use, the majority of CEOs tely
extensively on data ftom outside sources. They make use of statistics produced by a
different department that are particular tothe sector.
Data Staging Component
* Data must be extracted from several operating systems and extemal sources before
being processed for storage in the data warshouse, For the purpose of querying and
analysis, itis required to modify, change and prepare the data that has been obtained
from various sourees in an acceptable manner
The three main tasks performed inthe staging area will now be covered,
1) Data extraction : This technique must handle a variety of data sources. Fach data
source requires a different set of methodologies, which we must use
2) Data transformation : It is well known that data fora data warehouse comes from a
number of sources. For a data warehouse, data transformation poses even more
challenges than data extraction. We do a variety of different actions to change data
Firs, the information that was obtained from each source is cleansed. When merging
data from several sources, eliminating duplicate entries or setting default values for
missing data elements are all examples of cleaning,
‘Standardizing data component components isa crucial part of data transformation. Data
transformation may take many different forms when combining information from
several sources. We combine relevant data from many sources or data from a single
source record,
Data transformation, on the opposite hand, also entails separating outsourced records
into fresh combinations and deleting pointless source data, Data are massively sorted
and combined at the data staging area, When the data transformation function is
finished, we get a set of integrated, purified, standardized and summarized data,
Data loading : The two sorts of actions that make up data loading functions are different
When the data warehouse's structure and construction are complete and it first goes
online, the data is initially loaded into the storage area of the data warehouse, At first,
large volumes of data must be transported, which takes time,
TECHN. PUBLICATIONS mata ata Data Warehousing ___ troduction t Date Warehouse
Data storage hardware
‘An information split repository is used in data warehousing. Most often, the operating
systems data vaults only hold the most current information. These data repositories also
provide entirely normalized data that is arranged for quick processing. Information
delivery component.
‘The process of subscribing for data warchouse files and having data transported to one
‘or more destinations in accordance with some scheduling algorithm set by the client is
made possible by the information delivery element.
Component of metadata
+ A data warehouse's metadata is equivalent to a database management system's data
ictionary or data catalogue. We save details about logical data structures, records and
addresses, details about indexes and other details in the data dictionary.
Data marts
‘It contains a portion of corporate - wide data that is useful to a particular user group. The
focus is limited to a few carefully chosen topics. Although advancements in the data
warehouse business have made regular and incremental data dumps more feasible, data
in a data warehouse should be at least somewhat current but not necessarily up to the
‘minute, Data marts often have organization and are smaller than data warehouses. A
data warehouse should be constructed together with multiple smaller, related data marts
for certain types of queries and reporting, according to current data warehousing trends,
Management and Control Component
‘The services and operations within the data warehouse arc coordinated by the
management and control elements. These parts regulate how the data is transformed and
hhow it is sent to the data warehouse storage, On the other side, it controls how
customers are sent data. It collaborates with database management systems and grants
permission for data to be properly preserved in repositories. It keeps track of how
information is sent into the staging system and then into the actual data warehouse
storage
Data warehouse tools
© There are many data warehousing tools are available in the market. Here, is some most
prominent one
1, Mark logic : Data warehousing tool Mark Logic uses a variety of corporate
capabilities to simplify and acceferate data integration, This tool helps in carrying out
TECHNICAL PUBLICATIONS? pat hone
Date Vlrohousing a routon to Data Warehouse
very difficult search procedures. It can query several sorts of data, including
relationships, documents and metadata,
2 Oracle : The major database in the sector is Oracle. There are several data
warehouse systems available, both on-premises and in the cloud, By improvi
‘operational effectiveness, it aids in optimizing the client experience
|. Amazon RedShift : A tool for data warehouses is Amazon Redshift, All forms of
data may be easily and affordably analyzed using current BI tools and standard SQL
Additionally, it permits the use of query optimization to conduct complicated
searches against petabytes of structured data
HEM operational Database Vs Data Warehouse
Sr Ne ‘Operational database
Data warehouse
1 High- volume transaction processing is OLAP, or online analytical processing,
supported by operational systems. is often supported by data warehousing
systems. High - volume transaction
processing is supported by operational
systems,
‘The majority ofthe time, data
‘warehousing solutions focus 02
historical dats
‘Current data are often an issue for rs
operational systems.
systems Non volatile data may ofen be
updated. Once added, litle was altered
According to need,
are mainly updated regulary
tis eeated for corporate operations tis intended forthe examination of
business metrics by domain, scope and
characteristics.
and reel - ime transactions
tis designed todo limited number af tis designed to handle heavy
workloeds and sophisticated, erratic
auding or retrieving a single row per queries that access mmerous 10s per
table at time. table,
straightforward operations, often,
employs validation data tables and is No real-time validation is necessary
designed to validate incoming data duet the abundance of reliable,
sing transactions consistent information
Tn-comparison to OLTP,itonly
support ew consume lens
Tian accommodate thousands of
liens at once Date Warehousing 18
Introduction to Data Warehouse
‘The main goal of early database systems was to satisfy the requirements of ops
systems, which aro often transactional in nature.
‘There are many different systems, such as general ledgers, accounts payable, financial
‘management, oder processing, order entry and inventory.
By their very nature, operational systems are largely focused on managing a particular
transaction. When, the consumer, put money. into checking account, the banking
operational system is in charge of documenting the transaction to make sure the
associated debit shows up in account record.
(One order, one account and one inventory item are often the focus of an operational
system, An operational system often encou
s predetermined events that demand for
quick access owing to their nature, Most transactions only
data
clude modest quantities of
‘The busines
requirements for an operational system mostly remain constant. The
programme that records the transaction and the application that restricts access to the
data, or the porting side of the banking company, seldom ever change, The data needed
in this kind of system must be up-to-date when a consumer starts a transaction. A bank
‘must first be confident of your current balance before allowing a withdrawal,
Data "Warehousing" away from operational systems
‘The main idea behind data warehousing is that by isolating the data used for business
‘analysis from the data used in operational systems, it can be accessible more efficiently
Over time, many of the factors causing this divergence have changed. In the past, to
reduce the performance effect on operating systems, older systems archived data onto
tapes when it became inactive and several analytical reports ran from these tapes or
mirror data sources.
With the exception of the fact that they are now taken into more formal consideration
throughout the data warehouse design process, these reasons for separating operational
data from analytical data have not materially altered with the development of data
warehousing systems.
Many business analysis procedures have become considerably more intricate and
sophisticated as a result of technological advancements and changes in the nature of
company. Modem data warehousing systems are capable of creating regular reports as
well as more complex online analysis, such as mult - dimensional analysis.
TEGHIVCAL PUBLCATIONS® an ptt rine
Bota Warehousing 1-19
lo Data Warehouse
[EEE bata Warehouse Architecture
‘+ In essence, a data warehouse gives decision - support programs access to past data
Reporting, OLAP, Executive Information Systems (FIS), and data mining are examples
‘of such applications
+ A data warehouse, according to W. H. Inmon, the person who first coined the phase, is
a centralized, integrated collection of information, Integrate here refers tothe cleaning.
up, merging, and redesigning. Depending on how many systems feed into a warehouse
and how differently they handle comparable information, this may be more of less
difficult.
‘+ However, the majority of businesses already have information repositories in theit
production systems, many of which are centralized
Not data warehouses, then. Actually, no.
‘ In terms of funotion and structure, data warehouses are diff
ent from production
databases or Onli
i Transaction Processing (OLTP) systems. A data warehouse is
geared for data retrieval and reporting and is typically a read-only system, as opposed to
an OLTP system, which is created and optimized for data input and updates,
‘+ While a data warehouse includes the data utilized for business analysis, an OLTP system.
contains the data required for carrying out day-to-day business activities, Data
‘components that may be missing or unknown atthe time of input are common in current
and highly volatile OLPT data. Historical, nonvolatile data that has been corrected for
transaction mistakes may be found in & warehouse. OLPT systems and data warchouses
require various data - modeling techniques since their goals are so dissimilar. Due to the
difficulty of updating OLTP systems with redundant data, redundancy is almost
nonexistent. As a result, OLPT systems are often built using a relational architecture and
are highly normalized. Redundancy, however
facilitates user access and improves performance by reducing the number of tables that
‘must be cont
is preferred in data warehouses because it
‘© Some data warchouses choose a multidimensional ar
approach altogether.
‘© An adequate data model is required to discuss data warehouses and differentiate them
fiom transactional databases. The OLAP and decision - support technologies match the
tecture over a relational
‘multidimensional data model well. A data warehouse is commonly a repository of
integrated data from several sources that has been processed for storage in a
‘multidimensional model, as opposed to multi - databases, which provide access to
disconnected and typically heterogeneous databases, Data warehouses often offer
TEGHNCAL PUBLICATIONS an op roe sing 1-20
lo Data Warehouse
time - series and trend analysis, which both ask for more historical data than is
commonly kept in transactional databases. This is in contrast to the majority of
transactional databases. Data warehouses are nonvolatile in comparison to transactional
databases. As a result, the data war
‘may be thought of as non-real-time with periodic updates,
‘Transactions are the unit and the driving force behind database change in transactional
.ouse's information changes much less often and
systems; by contrast, data warehous
and is updated in accordance with a well - is
{neremental, The warehouse's acquisition component, which offers all necessary
preprocessing, manages warehouse updates,
AA set of decision support tools that are intended at assisting knowledge workers
(executive, manager and analyst) in making quicker and better choices is another way to
efine data warchousing in general. An overview of a data warehouse's conceptual
framework may be seen in the accompanying Fig. 1.9.1 It displays the whole data
warehousing procedure, Before data is stored, this procedure may inelude cleaning and
reformatting, Following the procedure, OLAP, data mining, and DSS may provide fresh,
relevant data, such as rules; this data is shown in the imag
figure demonstrates that files may be used as data sources.
Back fusing
a) _
Va
/ Bass
{ Tocco
gerenne
Returning to the warehouse
Updates
Ne
Fig. 194
Apart from being trendy terms in the current IT sphere, Data Warehousing, Online
Analytical Processing (LAP), and Decision Support Systems are the anticipated
outcomes of IT systems and present requirements. Long before the term “online
transaction processing" (OLTP) was coined, information management systems were
‘nly concemed with collecting and storing data related to routine, basic transactions in
database management systems,
Data Warehousing teas
Inteducton to Date Werehovee
Managers and analysts must now go beyond the basic data storage stage and take use of
IT systems by presenting complicated queries, receiving analytical findings, and making
choices based on those results, The introduction of OLAP and Data Warehousing at this
point provides the corporate world with the system design, guiding principles,
‘methodological approach and - at long last - tools required to support the presentation of
useful Decision Support Systems.
LMF. adopted the design and technique shown in the following image after clos
collaboration with the academic community, which had just lately followed the
develope:
field for the previous ten years. This is the output of the "Foundations of Data
Warehouse Quality - DWQ" Basic Research project, which was financed by
ESPRIT.Apart from being trendy terms in the current IT sphere, Data warehousing,
‘of the commercial sector, which had been advancing and pioneering in the
Online Analytical Processing (OLAP), and Decision Support Systems are the
anticipated outcomes of IT systems and present requirements. Long before the term
“online transaction processing" (OLTP) w:
wore only concemed with collecting and storing data related to routine, basic
coined, information management systems
transactions in database management systems,
Mana alysts must now go beyond the basic data storage stage and take use of
IT systems by presenting complicated queries, receiving analytical findings and making,
choices based on those results. The introduction of OLAP and data warehousing at this
point provides the corporate world with the system design, guiding. principles,
‘methodological approach, and - at long last - tools required to support the presentation
‘of useful decision support systems,
‘A data warehouse, also known as an OLAP system, is created by applying data
warehousing principles 10 conventional database systems and employing the right
design tools since it is fundamentally reliant on architecture. The design and
implementation of data warehouses and OLAP systems follow the IMF's established
approach.
Utilizing specialist data warehouse and OLAP systems, such as MicroStrategy's DSS
Series, the final deployment is carried out. One of the most well - known and respected
‘worldwide leaders in the field of data warehousing tools and systems, Micro strategy
Inc. provides solutions for every ter of
DW architectural
crarchy
‘The data warehouse architecture is a method for defining the overall architecture of data
exchanges, processing and presentation that exists for end - client computing inside the
sanization, Despite the fact that every data warehouse is differen, they nonetheless
ZA PUBLERTIONSS an op rat ln eta Werehousing tae Inteduction to Data Warohouse
share a few crucial components. Online transaction processing (OLTP) is
production systems including payroll, accounts payable, product buying and inventory
‘management
‘These programmes gather extensive data on everyday op
' Data warehouse systems are designed to assist online analytical provessing (OLAP), a
more modem activity. These include forecasting, trend analysis, profiling, and summary
tions
reporting tools,
+ Production databases are routinely updated using OLTP software or manually. A.
‘warehouse database, in contrast, gets regular updates from operating systems, usually
after business hours. As OLTP data accumulates in production databases, it is frequently
extracted, filtered and put onto a dedicated warehouse server that is accessible to users.
As the warchouse is filled with data, tables must be de-normalized, data must be
cleansed of errors and duplication and new fields and keys must be added to match the
user's requirements for sorting, combining and averaging data.
sources
Warehouse
‘Aral
‘Operalonat
‘peratonal
‘stem
Fig. 1.9.2 Data warehouse architecture: basic
Principles of a data warehousing
Load efficiency
‘© Data warehouses need to continuously load new data within constrained time frames.
‘The speed of the load process should be measured in gigabytes and hundreds of millions
of rows per hour and it shouldn't artificially limit the amount of data that can be
processed.
TEGHNEAL PUBLICATIONS? an wat owteige
Daa Warehousing Introduction Data Worehouse
Processing of loads
* To add new or updated data to the data warchouse, a number of processes must be
completed, including data conversion, filtering, reformatting, indexing and metadata
updating
Fact based management requires the greatest level of data quality
Despite “dirty” sources and a large database, the warehouse must provide local
consistency, global consistency and referential integrity.
Query performance The performance of the data warehouse RDBMS must not slow
down fact - based management; huge, sophisticated queries must be finished in seconds,
not days
of terabytes
The size of data warchouses is expanding at startling rates. These days, there are data
warehouses that area few megabytes to hundreds of terabytes in size
Types of Data Warehouse Architecture
4. Threestierarchitacture
The thee-tir architecture is composed ofthe source layer, which comprises several source
systems, the reconciliation layer and the data warehouse layer, which contains data
Warehouses and data mars. The reconciliation layer sits between the data warchouse and the
source data
‘The main advantage of the reconciled layer is that it creates a consistent reference data
todel for the whole business, Additionally
se filing and thse with source data extraction and integration, In some instances, the
reconciled layer is alo used directly to improve the performance of specific operational tasks
Examples include producing daily reports that cannot be adequately prepared using corporate
-pplications or ereatng data flows to regularly feed external processes inorder to benefit fom
cleaning and integration.
it distinguishes between issues with data
warehor
This approach is especially useful for big, enterprise-wide systems, One disadvantage of
this setup is the extra file storage space needed for the duplicate reconciling layer that is
added. As a consequence, the analytical tools are abit less real - time
TECHRICAL PUBLICATIONS pic hee Date Warehousing 4.26 reduction fo Dats Warehouse
Query report Ansty Data mining
Top ter
Frontend
fools
Out
Milo tor
OLAP Server
~ Moniorng
Melaata ropoaTany
rates
eee
Operational database
ae
eas
Exel sources
Fig. 1.9.3 Throe-tler architecture
+ Data warehouses usually have a three-level (tier) architecture that includes
1. Bottom tier (Data warehouse server)
2. Middle ter (OLAP server)
3. Top tier (Front end tool)
* The foundational element ofthe system is a data warek
jouse server running an RDBMS,
Tt could include a repository for metadata and many specialized data mart,
‘= Data is retrieved from operational databases and othe
sources (such as user profile
consultants) through application programmed
erfaces called gateways. SQL code for server execution may be created by client
information provided by externa
Drogrammers using a gateway provided by the underlying DBMS.
‘+ An OLAP server in the intermediate layer for rapid data warehouse queries,
‘In order to construct the OLAP server
1) A Relational OLAP (ROLAP) paradigm, which transforms multidimensional data
functions into standard relational procedures, is an improved relational database
management system,
TECHNICAL PUBLICA TIONG® on ata nape
—.
aa Warehousing
1-25 Introduction t Data We
2) A Multidimensional OLAP (MOLAP) model, a specialized server that natively
handles multidimensional processes and data,
‘A top-tier that contains front - end tools for displaying results from OLAP as well as tools
for data mining the data created by OLAP.
Information that defines DW items is kept in the metadata repository. For middle - tee
and top-tier applications, it contain the following specifications and data
1. An explanation of the data warehouse structure, including the locations, contents,
hierarchies and warehouse schema
2, Operational metadata, which generally
or deleted, as well as warehouse monitoring information like consumption
iicates if apiece of data is active, archived,
information, error reports, audit findings, ete
3. Index - containing system performance information is used to speed up data access
and retrieval
Operational databases, which provide details on data cleansing and transformation
per pr
‘methods, mapping for source RDBMSs, e
4, Business information, such as ownership details, terminologies and pre-configured
queries and reports,
4. Top-down approach
Exiemal 0
Sesing area
© Peers {2s |
Fig. 1.9.4 Top -down approach
en
© The following isa discussion of the crucial elements
‘+ External sources - A location where data of any type is obtained is an extemal source.
Unstructured, semi structured, or both types of data ae all possible.
a7 kr honto Data Warehousing
‘+ Stage area - The data must be checked before being imported into the data warehouse
since it was obtained from external sources and does not follow a certain format. ETL.
software should be used to do this
‘ E(Extracted) : Data was obtained from a third - party data source
© Data is transformed, or T (Transform)
tothe approved forma
* Data is delivered into the data warehouse after being translated into the common format
© Data - warehouse = After being cleansed, data is stored in the data - warehouse as a
central repository. While it really stores the meta data, the true data is saved in data
marts. Remember that this top - down approach preserves the data inthe data warehouse
in its most fundamental configuration
© Data marts : The data mart is a part of the storage component, It keeps track of
information for a single authority - managed organizational function. An organization
‘may have any umber of data marts, depending on the functions. We may also say that
the data mart contains a fraction of the data stored in the data warehouse,
* Studying the enormous amount of data Kept in a data warehouse is known as data
‘mining. Its possible to fi
hhidden patterns in databases or data warehouses by using a
data mining programmed.
‘© When the whole data warehouse has
according to Inmon, who claims that this technique leverages the data warehouse as the
‘completed, data marts are created from it,
organization's primary repository
2. Bottom - up approach
Extomal sources
Staging area
en Datawarenouse
©
Fig. 1.9.5 Bottom - up approach
‘Similar to atop - down approach, the data is initially obtained from external sources.
TECHIVCAL PUBLICATIONS? en wtf wide
|
Dota Warehousing ie
Introduction to Data Warehouse
‘+ Following the staging area (as previously mentioned), the data are then loaded into data
‘marts rather than data warehouses. The data marts, which enable reporting, were created
initially. 1 cone
rates on a certain industry
‘+ Then, these data marts are linked tothe data warchouse,
According to Kinballs description ofthis approach, data marts are constructed intially
to provide analysts a limited viewpoint and a data warchouse is developed when
complete data marts have been established
ERD) Autonomous Data Warehouse
© Oracle Cloud offers a suite ices based on self - driving Oracle
Autonomous Database technology. These services include automating all necessary
database maintenance processes while the system is running, including patching,
upgrading and tuning.
f data management se:
Modern Data Warehouse
+ A-contemporary data warehouse is technique for acquiring and storing the data that is
cloud - based. Organizations may analyze this data to help in decision
making. As a
result, many companies utilize contemporary data warehouses to improve their
operational, administrative and financial procedures. Quality cloud - based warchousin
departments need this information to make better decisions.
+ Data analytics, reporting, data mining, machine leaming and other Business Intelligence
(BD procedures may be performed using a data warehouse, a central data management
system that collects, stores and mixes data from many sources within an organization,
To allow effective analysis that is accessible to everyone within an organization, the
data warehousing system collects, arranges and analyses data from many sources,
Data warehouses have been existed since the 1980s, but as big data has become more
prevalent, they have seen significant transformation recently. Data warchouses are
Progressively including tools for sophisticated analytics and data visualization,
41. Elements of a modern data warehouse
© There are several major
mponents to today’s data warehouses,
Infrastructure
* In the past, businesses have switched their data from databases to file systems to save
‘money. According to EMA's Santaferraro, they are
object storage.
now switching from file systems to
TECPICAL PUBLICATIONS®
