Memory Hierarchy

Computer Organization and Assembly

Language
Air University
• Total memory capacity can be visualized as hierarchy of components
• Slow to fast (down to up)
• High capacity to smaller capacity (down to up)

2
Memory Hierarchy
Main Memory
 Only programs and data currently needed by the processor
Auxiliary Memory
 Devices that provide backup storage
 System programs, large data files etc
 E.g. magnetic disks and tapes
Cache
 Increases the speed of processing
 Rapid rate and small size
 Compensates speed difference between main memory and
processor
 Segments of programs and data currently in use

3
Main Memory
• Central storage unit
• Large and fast memory to store programs and data
during the computer operation
• RAM
• Random access memory
• Volatile
• Stores programs and data that are subject to change
• Read/write memory
• ROM
• Read only memory
• Non-volatile
• Stores tables of constant data that do not change

4
 Main Memory
• Bootstrap loader
• Startup program
• Stored in ROM
• Its function is to load the operating system
• ROM and RAM chips are available in variety of
sizes
• e.g. 128x8

5
 Byte Addressable Memory and Word Addressable Memory

The Memory Chip is divided into equal

parts called as “CELLS”.

Each Cell is uniquely identified by a

binary number called as “ADDRESS”.

The following information can be obtained

From the figure:
1. Total Data Space in the Chip = 64K X 8 = 512bytes (or 64KB ram)
2. Data Space in each Cell = 8 bits (1byte)
3. Address Space in the Chip =16 bits=log2 (64K)
•When the data space in the cell = 8 bits then the corresponding address space is
called as Byte Address.
•When the data space in the cell = word length of CPU then the
corresponding address space is called as Word Address.
6
https://www.geeksforgeeks.org/difference-between-byte-addressable-memory-and-word-addressable-memor
RAM chip
• 7 bit address bus
• 8 bit data bus
• Two chip selects
• Read control signal
• Write control signal

• A 128 * 8 RAM chip has a memory capacity of 128 words( values) of eight bits (one
byte) per word (value).
• This requires a 7-bit address (0000000b to 111 1111b) and an 8-bit bidirectional
data bus.
• The 8-bit bidirectional data bus allows the transfer of data either from memory to
CPU during a read operation or from CPU to memory during a write operation.
• The read and write inputs specify the memory operation, and the two chip select
(CS) control inputs are for enabling the chip only when the microprocessor selects
it.
7
Memory Connection to
CPU
• Let we need to design a sytem which
need Memory Capacity:
512 bytes of RAM and 128x8 bytes of ROM
BUT Chips Available : 128x8 RAM
128x8 ROM
Solution:
• 4 RAMS (each 128x8)
• and 1 ROM
• Each RAM receives 7 lower bits
• Particular RAM chip selected is
determined by lines 8 and 9
• Selection between ROM and Ram is done
by line 10
In the diagram what would be the size of data
bus in bits= ???
8 bits (1 byte)
 Memory
Characteristics
1. Access mode
2. Access time
3. Transfer rate
4. Capacity
5. Cost

10
 1. Access Mode
• Sequential
• Start at the beginning and read through in order like in a single
linked list.
• Access time depends on location of data and previous location
• e.g. tape
• Random
• Individual addresses identify locations exactly
• any location of the memory can be accessed randomly like
accessing in Array.
• e.g. RAM and ROM

https://www.geeksforgeeks.org/memory-access-methods/ 11
 1. Access Mode
• Direct
• Individual blocks have unique address based on physical location.
• Access is by jumping to vicinity plus sequential search
• This method is combination of sequential and random access.
• Access time depends on location and previous location
• e.g. magnetic disk

• Associative
• In this memory, a word is accessed rather than its address. This access access
method is a special type of random access method.
• E.g. Cache memory

12
 2. Access Time
• The average time to reach a storage location in memory
and obtain its content
• Access Time = seek time + transfer time
• Seek Time
• Time required to position the read/write head to a location
• Transfer Time
• Time required to transfer data to or from the device

13
 3. Transfer Rate
• Transfer rate is a standard metric that is used to
measure the speed at which data or information
travels from one location to another.
• Simply: Word or bytes transferred at one time
• For internal memory it is usually governed by
data bus width

14
 Cache
• Small amount of fast memory
• Sits between normal main memory and CPU
• May be located on CPU chip or module
• Faster than main memory by a factor of 5 to 10

15
Levels

16
• Multilevel Caches is one of the techniques to improve Cache
Performance by reducing the “MISS PENALTY”. Miss Penalty refers
to the extra time required to bring the data into cache from the Main
memory whenever there is a “miss” in the cache.
• consider an example where the CPU requires 10 Memory References
for accessing the desired information and consider this scenario in the
following 3 cases of System design

17
Case 1 : System Design without Cache
Memory

Here the CPU directly communicates with the main memory and no caches
are involved.
In this case, the CPU needs to access the main memory 10 times to access the
desired information.

18
 Case 2 : System Design with Cache Memory

Here the CPU at first checks whether the desired data is present in the Cache Memory or
not i.e. whether there is a “hit” in cache or “miss” in the cache. Suppose there is 3 miss
in Cache Memory then the Main Memory will be accessed only 3 times. We can see that
here the miss penalty is reduced because the Main Memory is accessed a lesser number of
times than that in the previous case.

19
 Case 3 : System Design with Multilevel
Cache Memory

Here the Cache performance is optimized further by introducing multilevel Caches. As

shown in the above figure, we are considering 2 level Cache Design. Suppose there is 3
miss in the L1 Cache Memory and out of these 3 misses there is 2 miss in the L2 Cache
Memory then the Main Memory will be accessed only 2 times. It is clear that here the
Miss Penalty is reduced considerably than that in the previous case thereby improving
the Performance of Cache Memory.

20
 Cache operation - overview
• CPU requests contents of memory location
• Check cache for this data
• If present, get from cache (fast)
• If not present, read required block from main memory
to cache
• Then deliver from cache to CPU

21
 Cache Performance
• Hit ratio
• When CPU refers to memory and finds the word in
cache, it is said to produce a hit
• If not found, it is called a miss
The performance of cache memory is frequently measured in terms of a quantity
called Hit ratio.

https://wp-rocket.me/blog/calculate-hit-and-miss-ratios/ 22
• Q1: if you have 51 cache hits and three misses over a period of time,
then Calculate hit ratio?

• Divide 51 by 54. The result would be a hit ratio of 0.944.

• Q2: if you look over a period of time and find that the misses your
cache experienced was11, and the total number of content requests
was 48

• Divide 11 by 48 to get a miss ratio of 0.229.

23
Finding the Average Memory Access Time

• The average memory access time to not only know how many cache
hits are happening, but also how fast they’re happening as well.
• That way, you can further understand where the issue may be if
you’re finding you’re having a high hit ratio, but users are still
complaining about a slow-loading website

24
Example

• CPU with 1ns clock, hit time = 1 cycle, miss penalty = 20 cycles,
cache miss rate = 5% n
AMAT = 1 + ( 0.05 × 20) = 2 cycles per instruction

25
Difference between Virtual memory and Cache
memory

• Virtual Memory increases the capacity of main memory. Virtual

memory is not a storage unit, its a technique. In virtual memory, even
such programs which have a larger size than the main memory are
allowed to be executed.

26
Logical Address Vs Physical Address

• Logical Address is generated by CPU while a program is running.

The logical address is virtual address as it does not exist physically,
therefore, it is also known as Virtual Address. This address is used as
a reference to access the physical memory location by CPU.
• Physical Address identifies a physical location of required data in a
memory. The user never directly deals with the physical address but
can access by its corresponding logical address.

27
task
• Find the Average memory access time for a processor with a 2 ns clock
cycle time, a miss rate of 0.04 misses per instruction, a missed penalty
of 25 clock cycles, and a cache access time (including hit detection) of
1 clock cycle.

• Hit Time = 1 clock cycle (Hit time = Hit rate * access time) but here Hit time
is directly given so,
• Miss rate = 0.04
• Miss Penalty= 25 clock cycle (this is the time taken by the above level of
memory after the hit)
• AMAT = hit time + (miss rate x miss penalty)
• AMAT= 1 + (0.04 * 25)
AMAT= 2 clock cycle

28
 References
• Computer Organization and Architecture
by William Stallings – Chapter 5
• https://www.geeksforgeeks.org/multilevel-cache-organisation/

29