Lecture 5c: Software Design-Data
Intensive Systems
Design and Design Decision Examples
Dec 2024, Orwa
System Architecture
The overall design of a system:
• Computers and networks (e.g., monolithic, distributed)
• Interfaces and protocols (e.g., http, CORBA)
• Databases (e.g., relational, distributed)
• Security (e.g., smart card authentication, SSL)
• Operations (e.g., backup, archiving, audit trails)
• Software environments (e.g., languages, source control tools)
2
Data Intensive Systems
Examples
• Electricity utility customer billing
• Telephone company call recording and billing
• Car rental reservations (e.g., Hertz)
• Stock market brokerage (e.g., Charles Schwab)
• Web sales (e.g., Amazon.com)
3
Example 1: Electricity Utility Billing
First attempt:
Transaction Data input Master file Bill
Each transaction handled as it arrives.
4
Criticisms of First Attempt
Where is this first attempt weak?
The requirements have not been specified!!!
5
Transaction Types
• Create account / close account
• Meter reading
• Payment received
• Other credits / debits
• Check cleared / check bounced
• Account query
• Correction of error
• etc., etc., etc.,
6
Typical Requirements
• All payments to be credited on day received
• Customers must be able to query account by telephone
• Cutting off service for non-payment requires
management authorization
• Data input staff should process n transactions per day
per person
• Error rate must be below 0.01%
• System available 99.9% of business hours
7
Batch Processing: Validation
errors
Incoming Edit & Validated
transactions validation transactions
Data input
read only
Master file
8
Batch Processing: Master File Update
Reports
errors
Validated
transactions Sort by
in batches account
Master file Bills
update
Instructions
9
Benefits of Batch Updating
• All transactions for an account are processed together
• Backup and recovery have fixed checkpoints
• Better management control of operations
• Efficient use of staff and hardware
10
Online Inquiry
Customer
service
read only
Transactions Data input Master file Bills
11
Example 2: A Small-town Stockbroker
• Transactions
Received by mail or over telephone
For immediate or later action
• Complex customer inquiries
• Highly competitive market
12
A Database Architecture
Database(s):
• Customer and account database
• Financial products (e.g., account types, pension plans, savings
schemes)
• External databases (e.g., stock markets, mutual funds,
insurance companies)
13
Database Architecture
Products & Customer & External
services database account database services
14
Real-time Transaction
Real-time
transactions
Products & Customer & External
services database account database services
15
Real-time Transactions & Batch
Processing
Data
Real-time Batch
transactions input processing
Products & Customer & External
services database account database services
16
Architectural considerations
• Real-time service during scheduled hours + batch
processing overnight
• Combine information from several databases
• Database consistency after any type of failure
two-phase commit
reload from checkpoint + log
detailed audit trail
• How will transaction errors be avoided?
• How will transaction errors be corrected?
17
Example: Merger of Two Banks
Each bank has a database with its customer accounts. The databases
are used by staff at many branches and for back-office processing.
The requirement is to integrate the two banks so that they appear to the
customers to be a single organization and to provide integrated service
from all branches.
18
Merger of Two Banks: Options
A ??? B
???
19
Merger of Two Banks: Architectural
Options
I. Convert everything to System A.
convert databases
retrain staff
enhance System A (software and hardware)
discard System B
II. Build an interface between the databases in System A
and System B.
III. Extend client software so that it can interact with either
System A or System B database.
20
Distributed Computing: General Problem
An application that is running on one computer wishes to use data
or services provided by another:
• Network connection
private, public, or virtual private network
location of firewalls
• Protocols
point-to-point, multicast, broadcast
message passing, RPC, distributed objects
stateful or stateless
• Quality of service
21
Network Choices
Public Internet:
Ubiquitous -- worldwide
Low cost
Private network:
Security
Predictable performance
Choice of protocols (e.g., IBM's System Network
Architecture, SNA)
22
Quality of Network Services
Performance
Maximum throughput
Variations in throughput
Real-time media (e.g., audio)
Business
Suppliers
Trouble shooting and maintenance
Upgrades
23
Firewall
Public Private
network network
Firewall
A firewall is a computer at the junction of two network segments that:
• Inspects every packet that attempts to cross the boundary
• Rejects any packet that does not satisfy certain criteria, e.g.,
an incoming request to open a TCP connection
an unknown packet type
24
