Chapter Three: Memory System Design

Outline
Characteristics of Computer Memory Systems
Memory Hierarchy
Cache memory Principle
Elements of Cache design
Internal Memory
External Memory
Virtual Memory

2/3/2025
1
 Characteristics of Computer Memory Systems
• Location : refers to whether memory is internal and external to the computer
 Internal (e.g. processor registers, main memory, cache)
 External (e.g. optical disks, magnetic disks, tapes)
• Capacity
 Number of words
 Number of bytes
• Unit of Transfer : For main memory, this is the number of bits read out of or written
into memory at a time.
 Word : eg. Main Memory
 Block: eg. external memory
• Access Method
 Sequential : eg. Tape units are sequential access
 Direct : eg. Disk units are direct access.
 Random :eg. Main memory and some cache systems are random access
 Associative eg. Cache memories may employ associative access.

2/3/2025 2
Characteristics of Computer Memory Systems
• Performance
 Access time
 Cycle time
 Transfer rate
• Physical Type
 Semiconductor
 Magnetic
 Optical
 Magneto-optical
• Physical Characteristics
 Volatile/nonvolatile
 Erasable/nonerasable
 Organization :- the physical arrangement of bits to form words.

2/3/2025 3
 Memory Hierarchy
 Design Constraints
 How much? “If you build it, they will come.” Applications tend to be built to use any
commonly available amount, so question is open-ended.
 How fast? Must be able to keep up with the CPU -- don’t want to waste cycles waiting
for instructions or operands.
 How expensive? Cost of memory (also associated with “How much?”) must be
reasonable vs. other component costs.
 A variety of technologies are used to implement memory systems, the following
relationships hold:
 Faster access time, greater cost per bit
 Greater capacity, smaller cost per bit
 Greater capacity, slower access time
• The designer’s dilemma
• Would like to use cheaper, large capacity memory technologies
• Good performance requires expensive, lower-capacity, quick-access memories
• Solution: Don’t rely on a single memory component or technology -- use a memory
hierarchy
2/3/2025 4
 Memory Hierarchy

Gap grew 50% per

year

Q. How do architects address this gap?

A. Put smaller, faster “cache” memories between CPU and DRAM. Create a
“memory hierarchy”.

2/3/2025 5
 Memory Hierarchy
• The memory unit is an essential component in any digital computer since it is
needed for storing programs and data
• Not all accumulated information is needed by the CPU at the same time
• Therefore, it is more economical to use low-cost storage devices to serve as
a backup for storing the information that is not currently used by CPU
• The memory unit that directly communicate with CPU is called the main
memory .
• Devices that provide backup storage are calledauxiliary memory .
• The memory hierarchy system consists of all storage devices employed in a
computer system from the slow by high-capacity auxiliary memory to a
relatively faster main memory, to an even smaller and faster cache memory
• The main memory occupies a central position by being able to communicate
directly with the CPU and with auxiliary memory devices through an I/O
processor.
• A special very-high-speed memory called cache is used to increase the speed
of processing by making current programs and data available to the CPU at a
rapid rate.
2/3/2025 6
 Memory Hierarchy

 CPU logic is usually faster than main memory access time, with the result
that processing speed is limited primarily by the speed of main memory
 The cache is used for storing segments of programs currently being
executed in the CPU and temporary data frequently needed in the present
calculations
 The typical access time ratio between cache and main memory is about 1
to 7~10
 Auxiliary memory access time is usually 1000 times that of main memory

2/3/2025 7
 Memory Hierarchy - Diagram

2/3/2025 8
 Storage-Device Hierarchy

Less Cost,
high capacity
and
High slow Speed
Cost
and fast

2/3/2025 9
 Cache memory
• Is the active portions of the program and data are placed in a fast small
memory, the average memory access time can be reduced,
• Thus reducing the total execution time of the program Such a fast small
memory is referred to as cache memory
• The cache is the fastest component in the memory hierarchy and
approaches the speed of CPU component
• When CPU needs to access memory, the cache is examined
• If the word is found in the cache, it is read from the fast memory
• If the word addressed by the CPU is not found in the cache, the main
memory is accessed to read the word

2/3/2025 10
Cache/Main Memory Structure

2/3/2025 11
 Cache memory
• When the CPU refers to memory and finds the word in cache, it is said to
produce a hit
• Otherwise, it is a miss
• The performance of cache memory is frequently measured in terms of a
quantity called hit ratio
• Hit ratio = hit / (hit+miss)

2/3/2025 12
 Cache operation – overview
• CPU requests contents of memory location
• Check cache for this data
• If present, get from cache (fast)
• If not present, read required block from main memory to cache
• Then deliver from cache to CPU
• Cache includes tags to identify which block of main memory is in each
cache slot
 Cache performance
• Although a single cache could try to supply instruction and data, it can be
a bottleneck.
• For example: when a load or store instruction is executed, the pipelined
processor will simultaneously request both data AND instruction
• Hence, a single cache would present a structural hazard for loads and
stores, leading to a stall
• One simple way to conquer this problem is to divide it:
• One cache is dedicated to instructions and another to data.
• Separate caches are found in most recent processors.

2/3/2025 14
 Elements of Cache Design
 Size
 Mapping Function
 Replacement Algorithm
 Write Policy
 Block Size
 Number of Caches
 Elements of Cache Design(cont….)
 Cache Size
 small enough that overall average cost/bit is close to that of main memory alone
 large enough so that overall average access time is close to that of cache alone
 large caches tend to be slightly slower than small ones
 available chip and board area is a limitation
 studies indicate that 1K-512K words is optimum cache size
• The basic characteristic of cache memory is its fast access time,
• Therefore, very little or no time must be wasted when searching the words in the
cache

2/3/2025 16
 Direct Mapping pros & cons
• Simple
• Inexpensive
• Fixed location for given block
• If a program accesses 2 blocks that map to the same
line repeatedly, cache misses are very high
 2. Associative Mapping
• A main memory block can load into any line of cache
• The fastest and most flexible cache organization uses an associative
memory

• Memory address is interpreted as tag and word

• Tag uniquely identifies block of memory
• Every line’s tag is examined for a match
• Cache searching gets expensive
• Associative memory is expensive compared to RAM
 Main Memory
• Most of the main memory in a general purpose computer is made up of RAM
integrated circuits chips, but a portion of the memory may be constructed with ROM
chips
• Operation of a memory cell
• Most commonly, the cell has three functional terminals capable of carrying an
electrical signal.
• The select terminal, as the name suggests, selects a memory cell for a read or write
operation.
• The control terminal indicates read or write.
• For writing, the other terminal provides an electrical signal that sets the state of
the cell to 1 or 0.
• For reading, that terminal is used for output of the cell’s state
• Depicts the operation of a memory cell

2/3/2025 19
• RAM– Random Access memory
• misused term (all these are random access)
• possible both to read data from the memory and to easily and rapidly
write new data into the memory
• volatile - can only be used for temporary storage (all the other types of
random-access memory are non-volatile)
• A RAM chip is better suited for communication with the CPU if it has one
or more control inputs that select the chip when needed
• Integrated RAM are available in two possible operating modes, Static
and Dynamic

2/3/2025 20
 Random-Access Memory (RAM)
• Static RAM (SRAM)
• Each cell stores bit with a six-transistor circuit.
• Retains value indefinitely, as long as it is kept powered.
• Relatively insensitive to disturbances such as electrical noise.
• Faster (8-16 times faster) and more expensive (8-16 times more expensice as
well) than DRAM.

• Dynamic RAM (DRAM)

• Each cell stores bit with a capacitor and transistor.
• Value must be refreshed every 10-100 ms.
• Sensitive to disturbances.
• Slower and cheaper than SRAM.

2/3/2025 21
 ROM– Read Only memory
contains a permanent pattern of data which cannot be changed
 data is actually wired-in to the chip as part of the fabrication process
data insertion step has a large fixed cost
no room for error
cheaper for high-volume production

2/3/2025 22
 ROM
• ROM is used for storing programs that are PERMENTLY resident in the
computer and for tables of constants that do not change in value once
the production of the computer is completed
• The ROM portion of main memory is needed for storing an initial
program calledbootstrap loader, witch is to start the computer software
operating when power is turned off.
• Since a ROM can only ready, the data bus can only be in an output mode.
There is no need for a read or write control because the unit can only
read .
• The two chip select inputs must be CS1=1 and CS2=0 for the unit to
operate. Otherwise, the data bus is in a high-impedance state.

2/3/2025 23
 Types of External Memory
• Magnetic Disk
• RAID
• Removable
• Optical
• CD-ROM
• CD-Recordable (CD-R)
• CD-R/W
• DVD
• Magnetic Tape

2/3/2025 24
 Magnetic Disk
• Disk substrate coated with magnetizable material (iron oxide…rust)
• Substrate used to be aluminium
• Now glass
• Improved surface uniformity
• Increases reliability
• Reduction in surface defects
• Reduced read/write errors
• Lower flight heights (See later)
• Better stiffness
• Better shock/damage resistance

2/3/2025 25
 Read and Write Mechanisms
• Recording & retrieval via conductive coil called a head
• May be single read/write head or separate ones
• During read/write, head is stationary, platter rotates
• Write
• Current through coil produces magnetic field
• Pulses sent to head
• Magnetic pattern recorded on surface below
• Read (traditional)
• Magnetic field moving relative to coil produces current
• Coil is the same for read and write
• Read (contemporary)
• Separate read head, close to write head
• Partially shielded magneto resistive (MR) sensor
• Electrical resistance depends on direction of magnetic field
• High frequency operation
• Higher storage density and speed
2/3/2025 26
 Inductive Write MR Read

2/3/2025 27
 Data Organization and Formatting
• Concentric rings or tracks
• Gaps between tracks
• Reduce gap to increase capacity
• Same number of bits per track (variable packing density)
• Constant angular velocity
• Tracks divided into sectors
• Minimum block size is one sector
• May have more than one sector per block

2/3/2025 28
Disk Data Layout

2/3/2025 29
 Disk Velocity
• Bit near centre of rotating disk passes fixed point slower than bit on
outside of disk
• Increase spacing between bits in different tracks
• Rotate disk at constant angular velocity (CAV)
• Gives pie shaped sectors and concentric tracks
• Individual tracks and sectors addressable
• Move head to given track and wait for given sector
• Waste of space on outer tracks
• Lower data density
• Can use zones to increase capacity
• Each zone has fixed bits per track
• More complex circuitry

2/3/2025 30
Disk Layout Methods Diagram

2/3/2025 31
 Finding Sectors
• Must be able to identify start of track and sector
• Format disk
• Additional information not available to user
• Marks tracks and sectors

2/3/2025 32
 Fixed/Movable Head Disk
• Fixed head
• One read write head per track
• Heads mounted on fixed ridged arm
• Movable head
• One read write head per side
• Mounted on a movable arm

2/3/2025 33
 Removable or Not
• Removable disk
• Can be removed from drive and replaced with another
disk
• Provides unlimited storage capacity
• Easy data transfer between systems
• Nonremovable disk
• Permanently mounted in the drive

2/3/2025 34
 Multiple Platter
• One head per side
• Heads are joined and aligned
• Aligned tracks on each platter form cylinders
• Data is striped by cylinder
• reduces head movement
• Increases speed (transfer rate)

2/3/2025 35
Multiple Platters

2/3/2025 36
Tracks and Cylinders

2/3/2025 37
 DVD – Writable
• Loads of trouble with standards
• First generation DVD drives may not read first generation
DVD-W disks
• First generation DVD drives may not read CD-RW disks
• Wait for it to settle down before buying!

2/3/2025 38
 Magnetic Tape
• Serial access
• Slow
• Very cheap
• Backup and archive

2/3/2025 39