Incisive Coverage Introduction and

RAK Overview
This also includes Incisive 12.2 workshop/labs overview

2013 Cadence Design Systems, Inc.

Coverage Workshop Agenda

Introduction to Metric-Driven Verification
Coverage Metrics
Code coverage
Generating Code Coverage
Analyzing Code Coverage Using IMC
Lab 1: generating & analyzing code coverage

Functional coverage
Building Functional Coverage Model
Analyzing Functional Coverage Using IMC
Lab 2: functional coverage collection & analysis

IMC Merge/Rank/Refine/Report
Lab 3: merge tests & generate report

Summary
Download Workshop Click here -> Coverage Workshop
2

2013 Cadence Design Systems, Inc. All rights reserved.

Labs and DUT Introduction

This workshop includes hands-on labs
This is a verification environment for a simple UART DUT
component that is part of a larger SoC system. We follow the steps
of instrumenting, collecting, and analyzing the coverage.
The labs allow further understanding of the lectures.
To download complete workshop with labs click here - Download
UART block: Converts parallel
data from the APB bus into serial
CPU
data and vice versa

DMA

Memory

AHB
Parallel
APB bus

Bridge
APB

APB

UART
UART
Serial UART
protocol
3

UART
Serial

2013 Cadence Design Systems, Inc. All rights reserved.

Core

USB

Ethernet

UART DUT and Verification Environment

Module Verification
Component

UVM Testbench

Scoreboard

Imitates external
device by
generating frames

Interface & white-box

coverage

Programs the
UART and
transfers traffic

UART DUT (Verilog RTL)

APB
Verification
Component

APB

Tx FIFO

txd

Transmitter

APB
Interface

Mode
Switch
Rx FIFO

Receiver

Control/Status
Registers

UART
Serial

rxd

UART
Verification
Component

Control/Interrupt
Logic

Multi-channel controller

Controls APB and

UART components
Test 1
4

2013 Cadence Design Systems, Inc. All rights reserved.

Test 2

Test 3

Tests indicate which

stimulus sequences to
execute

Functional Verification Methodologies

Directed-Test Driven
Results
Test 1:
Test 2:
Test 3:
Test 4:

DUT

Least effective
in finding the
hidden bugs

Test targets

Coverage Driven
Results
G

Coverage
CG1
CG2
CG3
CG4

DUT

Adds quality &

productivity,
but difficult to
estimate
completion

Coverage targets

Metric Driven

Chip Features
Results

Verif. Plan
Feature A
Feature B
Feature C
Feature D

DUT

B
C D
Feature-based
Verification targets
(Metrics for cov+checks)

2013 Cadence Design Systems, Inc. All rights reserved.

Feature-based
plan with
Feature A 50%
extended
Subset 1 .
20%metrics
Subset 2 .
80% enables
Feature B
efficient
Feature C 10%
and accurate
Feature D 70%
20% project closure

Metric-Driven Verification (MDV) Overview

Planning with Unified Verification Metrics
Metric-Based
Executable
Verification Plan

PDF

Yes

No

Signoff?

Done

Target Metrics Milestones

Actual Metrics Achieved

VE Start

Module
Set One

Module
Set Two

Plan

Measure /
Analyze

Successful Milestone
Missed Milestone

Chip
Integration

Prototype

Incisive
VIP Portfolio

Construct

Production

Failure and
Metric Analysis

Execute
IEM

IFV
IEV

ISX
IES

SN

Testbench Simulation, Formal,

HW/SW Co-Sim, LPV, MSV,
Sim-Acceleration, Emulation
6

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage
Assertions
Checks

Metric-Driven Verification Productivity

Know Where You are Going and Get There Faster
Focus and manage complex
projects using Verification
Planning

Functional
and
Design
Specs

Create Verification
Plan from Spec;
Capturing Metrics

Find the most bugs early, and

use analysis to identify and
work on critical bugs first

vPlan

Scale Verification Throughput:

Automated Parallel Verification Runs
vs. Manual Engineering Work

2013 Cadence Design Systems, Inc. All rights reserved.

Analyze Results
Debug, Fix,
Resubmit

Coverage Is the Key for MDV

Coverage implemented to measure goals
Plan specifies Metrics required for DUT features.
After plan is developed, we need to instrument coverage into the
verification environment.

Coverage implemented to track progress

Plan provides feature-based tracking of progress
Regression results annotated back to Plan Features

Let us dive deeper into coverage

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Workshop Agenda

Introduction to Metric-Driven Verification
Coverage Metrics
Code coverage
Generating Code Coverage
Analyzing Code Coverage Using IMC
Lab 1: generating & analyzing code coverage

Functional coverage
Building Functional Coverage Model
Analyzing Functional Coverage Using IMC
Lab 2: functional coverage collection & analysis

IMC Merge/Rank/Refine/Report
Lab 3: merge tests & generate report

Summary
9

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Metrics
Code Coverage: method of assessing how well the test cases have
exercised the design.
Code Coverage : Block & Branch, Expression,Toggle, FSM
Functional Coverage: focuses on functional aspects of a design and
provides a very good insight on how the verification goals set by a test
plan are being met.
Functional Coverage : Assertion and CoverGroup

10

2013 Cadence Design Systems, Inc. All rights reserved.

Code Coverage vs Functional Coverage

Code coverage
Reflect how thoroughly the DUT code has
been exercised
Enable detection of
Untested areas of DUT
Dead code or unused code
Most of the work are done by coverage
tools automatically

APB
UVC

Functional coverage
Measure what features of DUT were
exercised
Lot of manual work:
Planning
What are the features?
Where/how to measure?
Coding of assertions and covergroups

11

2013 Cadence Design Systems, Inc. All rights reserved.

Scoreboard

DUT

UART
UVC

Coverage Workshop Agenda

Introduction to Metric-Driven Verification
Coverage Metrics
Code coverage
Generating Code Coverage
Analyzing Code Coverage Using IMC
Lab 1: generating & analyzing code coverage

Functional coverage
Building Functional Coverage Model
Analyzing Functional Coverage Using IMC
Lab 2: functional coverage collection & analysis

IMC Merge/Rank/Refine/Report
Lab 3: merge tests & generate report

Summary
12

2013 Cadence Design Systems, Inc. All rights reserved.

Code Coverage : Block Coverage

Identifies the lines of code that are executed during a
simulation run. It helps you determine what areas of the
DUT design are not exercised by the testbenches.
A block is a statement or sequence of statements in
Verilog/VHDL that executes with no branches or delays.
Either none or all of the statements in a block are executed.

block coverage is an essential first step in the overall verification process.

13

2013 Cadence Design Systems, Inc. All rights reserved.

Code Coverage : Block Coverage

always @ (in )
begin
$display("TRUE");
if ( in[1] == 1b1 )
$display("IF1 TRUE");
else
begin
$display("IF1 FALSE");
$display("IF1 FALSE");
end;
$display("TRUE");
if ( in[1] == 1b1 )
$display("IF2 TRUE");
$display("TRUE");
end

BLK1

BLK2
BLK3

BLK4
BLK5
BLK6

Any construct that breaks execution flow creates a new

block, for example:
begin, if, else, case, wait, #, @, for, forever, repeat, while,
disable
14

2013 Cadence Design Systems, Inc. All rights reserved.

Code Coverage : Branch Coverage

Branch Coverage: Yields more precise coverage details than block
coverage by obtaining coverage results for various branches
individually.
With branch coverage, a piece of code is considered 100% covered
when each branch of a conditional statement has been executed at
least once.

assign out2 = (cond1) ? 1b1 : 1b0;

The above statement has the following branches:
- If cond1 evaluates to true, out2 is assigned 1b1
- If cond1 evaluates to false, out2 is assigned 1b0
Considered as a single block for block coverage. With branch coverage this
statement is considered 100% covered when each branch of the statement
has executed.
15

2013 Cadence Design Systems, Inc. All rights reserved.

Block Coverage Analysis Report

Notice that conditional assign

statement was not scored with
block coverage analysis

Score is 0 for the true

part of if statement line 33
Score is 0 for the true
part of if statement line 38

This block of statements

was never executed
This block of statements
was never executed
either

16

2013 Cadence Design Systems, Inc. All rights reserved.

Branch Coverage Analysis Report

With Branch Coverage enable,

conditional assign statement is
now scored.

17

2013 Cadence Design Systems, Inc. All rights reserved.

Code Coverage : Expression Coverage

Expression coverage measures how thoroughly the testbench
exercises expressions in assignments and procedural control
constructs (if/case conditions). It identifies each input condition that
makes the expression true or false and whether that condition
happened in simulation.

Before scoring expression coverage, make sure you have high

block coverage in your regression.

18

2013 Cadence Design Systems, Inc. All rights reserved.

3 types of Expression Coverage

SOP Sum of Products Scoring (Default Mode)
Does each term take a 0 and non-0 value?
Vector inputs are scored as logical (single bit)
Control Scoring
Does each term take a 0 and non-0 value *AND*
Does each term control the output result of the
expression during simulation?
Vector Scoring
Does each term take a 0 and non-0 value *AND*
Does each bit of each term control the result of the
expression?
You can set different scoring modes for selected modules in your design.
19

2013 Cadence Design Systems, Inc. All rights reserved.

SOP Scoring
Reduces expressions to a minimum set of expression inputs that
make the expression both true and false, inherently first-level
b

&

<1>
Simulation
Scoring
Count

Coverage
Hole

(c

d)

<2>

hit

rval

<1>

<3>
<2>

<3>

Resulting value of the expression

(either zero or non-zero)

20

2013 Cadence Design Systems, Inc. All rights reserved.

Expression Inputs
<1>: b
<2>: c
<3>: d

Control Scoring
Checks if each input has controlled the output value of the expression
at some time during the simulation
Known as sensitized condition coverage or focused condition coverage
Breaks an expression into a hierarchy of sub-expressions, more accurate
hit
b
<1>

&

(c

d)

<--

-->

<3>

<4>

&

21

Level 2:
<3>: c
<4>: d

2013 Cadence Design Systems, Inc. All rights reserved.

<2>

--------------

hit

<3>

<4>

|
Level 1:
<1>: b
<2>:(c | d)

<1>

--------------

Vector Scoring
Vector scoring mode is an extension of control scoring mode.
Each bit of a multi-bit signal is scored and reported separately and
lots of data to analyze.
hit

reg [3:0] a, b;

&&

Wire [3:0] val = (a && b);

(a

&&
<1>

Level 1:
<1>: a
<2>: b

Bit table for each

vector operand

22

2013 Cadence Design Systems, Inc. All rights reserved.

b)
<2>

Hit

<1>

<2>

--------------

T1[3] T2[3]
[2] [1]
hit
[2][0]
[1] [0]

0 0 0 0 1 0

0 0 1 0 0 1

1 0 0 1 0 0

0 1 0 0 0 0

0 0 0 0 0 0

Expression Coverage Analysis Report

Source code window & selected
expression

Detailed expression reports

(v1 == 1) was never false

(v2 == 1) was never false
Need additional test cases?

23

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Metrics Toggle Coverage

What does toggle coverage do?
Collect and report design signal toggle activity

What is a design signal toggle?

Normally a binary transition (and return after a finite delay) of a DUT
signal.
Signals may transition through (but not terminate at) an unknown state.

Why bother with toggle coverage?

Its the only code coverage available for a gate-level netlist
Verify that design interconnect is connected and wiggles

Z
0
partial
24

2013 Cadence Design Systems, Inc. All rights reserved.

full

set_toggle_includez

Code Coverage : FSM Coverage

FSMs are critical control logic of DUT
It is important that FSMs logic are exercised thoroughly
by the testbench to ensure that there are no bugs
FSM coverage measures how well this logic is exercised

FSM Coverage Types

- State Coverage
- Transition Coverage
- Arc Coverage

25

: reports what states were visited

: reports what transitions occurred
: reports why each transition occured

2013 Cadence Design Systems, Inc. All rights reserved.

Generating Code Coverage

What coverage metrics should I enable?
Add in stages code coverage to identify holes in test
environment
First-order: block or branch
Second-order: expression

Add in stages the simplest form of functional coverage

in the form of FSM coverage
First-order: states and transitions
Second-order: arcs

Note: There is little value in scoring second-order coverages if

simpler ones like block or branch still show low coverage

26

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Workshop Agenda

Introduction to Metric-Driven Verification
Coverage Metrics
Code coverage
Generating Code Coverage
Analyzing Code Coverage Using IMC
Lab 1: generating & analyzing code coverage

Functional coverage
Building Functional Coverage Model
Analyzing Functional Coverage Using IMC
Lab 2: functional coverage collection & analysis

IMC Merge/Rank/Refine/Report
Lab 3: merge tests & generate report

Summary
27

2013 Cadence Design Systems, Inc. All rights reserved.

Generating Coverage Data

Two-Step Simulation
Design

Testbench
Compile,
Elaborate and
instrument for coverage

coverage
options
OR
coverage
configuration
file

irun -c

irun -r

28

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage
Database(s)

Simulate, score and store

coverage

Generating Coverage Data

IRUN coverage options
-coverage <coverage_types> Enables coverage data generation
B

Enable Block coverage

Enable Toggle coverage

Enable Expression coverage

Enable Functional coverage

Enable FSM coverage

Enable ALL coverage types

-covdut <DUT_module> Specify Module for instrumentation

-cov_file <cov_config_file> Specify coverage Configuration File

Instrument Block/Expression/FSM coverage for Module chipA & chipB
% irun f rtl_debug.f coverage B:E:F covdut chipA covdut chipB

Instrument All coverage types for Module chipA

% irun f rtl_debug.f coverage A covdut chipA

29

2013 Cadence Design Systems, Inc. All rights reserved.

Generating Coverage Data

irun runtime commands
-covworkdir <workdir>

Basename for the work directory

Default work directory is cov_work

-covscope <scope>

Specifies an alternate directory for

storing coverage model files.
Default is scope

-covtest <test>

Test directory name. Default is test

-covoverwrite

Enable overwriting of coverage files

Invoke irun (during run time) with coverage instrumented during

elaboration
/home> irun -r <snapshot>
Coverage model generated in /home/cov_work/scope (.ucm)
Coverage data in /home/cov_work/design/test (.ucd)
30

2013 Cadence Design Systems, Inc. All rights reserved.

Generating Coverage Data

Directory & file structure created for coverage
Module Hierarchy

./

top

cov_work/
scope/

chipA

.ucm

mod2

testA/

mod1

.ucd
testB/

.ucd
% irun c *.v coverage B:E:T covdut chipA

regr_10/

% irun r top covtest testA ...

scope/

.ucm

% irun -r top covtest testB ...

testC/
.ucd
31

% irun r top covworkdir regr_10 covtest testC

2013 Cadence Design Systems, Inc. All rights reserved.

Generating Coverage Data

IRUN Coverage Summary of Commands

32

Switch name

Compile/elab/
runtime

Description

-coverage <string>

elaboration

Enable coverage
instrumentation

-covfile <file>

elaboration

Coverage instrumentation
control file

-covdut <dut>

elaboration

Coverage DUT name

-covworkdir <workdir>

runtime

Coverage work directory

-covscope <scope>

runtime

Coverage scope name

-covtest <test>

runtime

Coverage test name

-covoverwrite

runtime

Overwrite coverage output

files

2013 Cadence Design Systems, Inc. All rights reserved.

Generating Coverage Data

Defining a Coverage configuration file
top
chipA mod2
mod1
mod3

Enable B:E:T in chipA and all its descendents

Matches all instances of mod and all of their descendents

covfile.ccf
select_coverage bet module chipA

Disable Toggle in module mod2

Enable Toggle in module mod3

deselect_coverage t module mod2

select_coverage t module mod3
# This is a comment

Score branches (ternary assignments)

set_branch_scoring

select_coverage -fsm module mod1

select_functional

FSM Extraction in mod1

Score assertions / Covergroups

Run simulation with coverage according to configuration file covfile.ccf

% irun *.v cov_file covfile.ccf
33

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Configuration File

Coverage Configuration File Command

34

Command
select_coverage

Applies to
B:E:T:F

deselect_coverage

B:E:T:F

set_glitch_strobe
set_hit_count_limit
set_subprogram_scoring
set_com
set_com_interface
set_code_fine_grained_m
erging
set_libcell_scoring
set_merge_with_libname
set_parameterized_modul
e_coverage

B:E
B:E
B:E
B:E:T
B:E:T
B:E
ALL
ALL
ALL

2013 Cadence Design Systems, Inc. All rights reserved.

Description
Select the design units or instances to score for coverage
type
Subtract from what is instrumented
-remove_empty_instances remove empty instances from
the coverage hierarchy
Specify the glitch rejection time interval
Sets the upper limit of hit counts
Disable scoring of unused subprograms
Identify items never happen and marks them ignored
All ports of the modules will be marked as variables
Provide merge resilience at concurrent block level instead of
the default module/instance level
Enable scoring of verilog library cells
Match library name of module for merging
Different modules dumped to different parameter values
combinations used to instantiate the module

Coverage Configuration File

CCF Command apply only to Block Coverage
Command

Description

set_implicit_block_scoring

set_branch_scoring

-off: disable scoring of implicit else and/or

default case blocks
-off: disable scoring of explicit else and/or
default case blocks
Enable scoring of verilog continous
assignments
Enable scoring of individual branches

set_statement_scoring

Enable statement coverage scoring

set_explicit_block_scoring
set_assign_scoring

35

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Configuration File

CCF Command apply only to Expression Coverage
Command

Description

set_expr_scoring

-all: enables scoring of all operators and assignments

-sop/control/vector: specifies a scoring mode
-event: enables scoring of verilog events
-no_vhdl_shortcircuit: disables short-circuits
evaluation of VHDL AND/NAND/OR/NOR operators
-vlog_remove_redundancy: disables splitting of
subexpressions
-vhdl_not_as_operator: enables scoring of VHDL not
operator
set_expr_coverable_operators By default, coverage is scored only for Verilog logical
operators (|| and &&) and VHDL logical operators
(OR, AND, NOR, and NAND), and is scored only in
condition expressions. This comand enables scoring
coverage for all operators in condition expressions.

36

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Configuration File

CCF Command apply only to Toggle Coverage
Command

Description

set_toggle_strobe

set_toggle_includex

Define a strobe interval for filtering glitches within the

time window on nets
Set maximum count for rise and fall transitions after
which ncsim stop counting
Record transitions x -> 0 and x -> 1.

set_toggle_includez

Record transitions z -> 0 and z -> 1.

set_toggle_noports

Disable scoring of module ports

set_toggle_portsonly

Score only module ports

set_toggle_excludefile

Exclude specific signals from the design during toggle

simulation

set_toggle_limit

37

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Configuration File

CCF Command apply only to FSM Coverage
Command

Description

set_fsm_attribute

Tag an FSM state vector

set_fsm_reset_scoring Enable scoring of reset states and transitions for

identified FSMs.
set_fsm_arc_scoring
Enable scoring of arcs for identified FSMs.
set_fsm_arc_termlimit
set_fsm_scoring

38

Set the limit on the number of input terms created for an

arc expression
-hold_transition: enable scoring of FSM hold transitions.

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Configuration File

CCF Command apply only to Functional Coverage
Command

Description

set_optimize

-ial_ovm_inst_asrt: treat instances of IAL and OVL components as

individual assertions
Enables scoring of all compiled assertions and covergroups
-ams_control: enables scoring of assertions coverage in AMS
modules
Multiple switches that allow users to fine tune the collection and
generation of coverage data; see ICC User Guide for more details.

select_functional

set_covergroup

39

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Workshop Agenda

Introduction to Metric-Driven Verification
Coverage Metrics
Code coverage
Generating Code Coverage
Analyzing Code Coverage Using IMC
Lab 1: generating & analyzing code coverage

Functional coverage
Building Functional Coverage Model
Analyzing Functional Coverage Using IMC
Lab 2: functional coverage collection & analysis

IMC Merge/Rank/Refine/Report
Lab 3: merge tests & generate report

Summary
40

2013 Cadence Design Systems, Inc. All rights reserved.

Incisive Metrics Center (IMC)

Key Features
Unified metrics center for all languages and
all coverage metrics types
Code (block / expression / toggle)
FSM
Functional (Assertions / covergroup)
State of the art GUI capabilities to browse and
analyze coverage
Instance and Type based views
Robust filtering and sorting capability
Intuitive page navigation
Unified batch mode Command Line Interface (CLI)
Good merge, rank, refinement and report(text, html)
operations.
Exclusion

41

2013 Cadence Design Systems, Inc. All rights reserved.

2 modes to launch IMC

GUI Mode: This invokes Incisive Metric Center, which is a
graphical user interface to observe coverage results,
compare simulations, and refinement file
% imc
% imc -gui

Batch Mode: In this mode, imc command prompt is

launched allowing user to issue IMC commands
interactively. Alternately, all commands can be placed in a
<command_file> which is passed to imc for execution.
% imc batch [command_file]

42

2013 Cadence Design Systems, Inc. All rights reserved.

IMC Help Utilities

How Tos display

topics available
for search

Favorites page
displays list of
pages added as
favorites

Search
How Tos

43

2013 Cadence Design Systems, Inc. All rights reserved.

Analysis History
page displays list
of runs launched
in this or previous
IMC sessions

IMC Summary View

Info Tab
Sub-elements

Activity Center
Page Navigation

Unified
Instance/Type
Verification
Metrics Tree

Sorting / Filtering

Tools & Options

44

2013 Cadence Design Systems, Inc. All rights reserved.

Details Tab

IMC Block Analysis View

Block
Coverage
Analysis

45

2013 Cadence Design Systems, Inc. All rights reserved.

Green Indicates covered ;

Red indicates uncovered

IMC Expression Analysis View

Various
Expressions

Selected
Expression

46

2013 Cadence Design Systems, Inc. All rights reserved.

Source Code

Selected
Expression
Table

IMC Toggle Analysis View

Individual
Vector bits

47

2013 Cadence Design Systems, Inc. All rights reserved.

Source Code

IMC FSM Analysis View

State
Transitions / Arcs

48

2013 Cadence Design Systems, Inc. All rights reserved.

FSM bubble
diagram

Source viewer
with syntax
highlighting

Lab 1: Code Coverage

49

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Workshop Agenda

Introduction to Metric-Driven Verification
Coverage Metrics
Code coverage
Generating Code Coverage
Analyzing Code Coverage Using IMC
Lab 1: generating & analyzing code coverage

Functional coverage
Building Functional Coverage Model
Analyzing Functional Coverage Using IMC
Lab 2: functional coverage collection & analysis

IMC Merge/Rank/Refine/Report
Lab 3: merge tests & generate report

Summary
50

2013 Cadence Design Systems, Inc. All rights reserved.

Functional Coverage What are we measuring?

Stimulus completeness

All address blocks

Packet sizes: short, jumbo, max
All instructions
Important sequences
Read-after-write
Store* - Branch Read*

Scoreboard

APB
UVC

DUT

Typically, implemented with SV covergroups

Where should we implement these covergroups?
In data items (uvm_sequence_item, uvm_sequence) directly?
In monitors?
In scoreboards?

51

2013 Cadence Design Systems, Inc. All rights reserved.

UART
UVC

Implementing Coverage for Stimulus Unique Data Item

If data item definition is unique for each DUT port:
Covergroup(s) can be defined in the data item itself
Type-coverage measures the completeness of the stimulus

Scoreboard

APB
UVC

apb_transferDUT

UART
uart_frame
UVC

In the example above, APB and UART ports have unique data items
Covergroups can be defined directly in the apb_transfer, uart_frame
Type-coverage of apb_transfer measures the apb stimulus coverage
Type-coverage of uart_frame measures the uart stimulus overage

52

2013 Cadence Design Systems, Inc. All rights reserved.

Scoreboard

Covergroup Example in data item

APB
UVC

apb_transfer

DUT

Covergroup in
apb_transfer
data item

53

2013 Cadence Design Systems, Inc. All rights reserved.

UART
UVC

uart_frame

Implementing Coverage for Stimulus Shared Data Item

If data item definition is not unique for each DUT port:
Covergroup should be defined in the monitor or scoreboard
Instance-coverage measures the completeness of the stimulus

Scoreboard
Agent 1

APB Port1
apb_transfer

Mon

APB
UVC

DUT

Agent 2

UART

uart_frame
UVC

APB Port2
apb_transfer

Mon

In the example above, DUT has two APB ports

Do not define covergroup in the apb_transfer
Covergroup of APB stimulus should be defined in the monitor
Instance-coverage of monitor in Agent1 = stimulus coverage of APB Port1
Instance-coverage of monitor in Agent2 = stimulus coverage of APB Port2

54

2013 Cadence Design Systems, Inc. All rights reserved.

Covergroup Example in monitor

Agent1

Scoreboard

apb_transfer

APB Port1

APB
UVC
Agent2

DUT

uart_frame

apb_transfer
APB Port2

Enable instance
coverage

Use set_inst_name()
to specify meaningful
instance name

55

2013 Cadence Design Systems, Inc. All rights reserved.

UART
UVC

Explicit naming of covergroup instances

Do not rely on the default (tool-generated) name
Covergroup instance could have multiple paths (object handles)
==> default name may not always be logical enough for users to follow

Default name could be different between tests => merge issue

Two techniques are available for naming the covergroup

instances explicitly
Use of set_inst_name() built-in method
Use of option.name

We recommend use of set_inst_name() for naming the

covergroup instances explicitly

56

2013 Cadence Design Systems, Inc. All rights reserved.

Functional Coverage What else are we measuring?

DUT features
Operational modes

Scoreboard

1GB, 10GB
720p, 1080p

Corner cases

UVC1

FIFO empty/full
Wrong branch prediction -> flushes of prefetch queues

DUT

Typically, implemented with covergroups and assertions

Where should we implement covergroups & assertions?
Register model (UVM_RGM or UVM_REG)
Monitors, scoreboards
vunit

57

2013 Cadence Design Systems, Inc. All rights reserved.

UVC2

Covergroup Example in Register Model

Generated
automatically by
Register package

58

2013 Cadence Design Systems, Inc. All rights reserved.

Covergroup Example FIFO Corner Cases

Covergroups to
check for corner
cases of FIFOs

59

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Workshop Agenda

Introduction to Metric-Driven Verification
Coverage Metrics
Code coverage
Generating Code Coverage
Analyzing Code Coverage Using IMC
Lab 1: generating & analyzing code coverage

Functional coverage
Building Functional Coverage Model
Analyzing Functional Coverage Using IMC
Lab 2: functional coverage collection & analysis

IMC Merge/Rank/Refine/Report
Lab 3: merge tests & generate report

Summary
60

2013 Cadence Design Systems, Inc. All rights reserved.

Building a Functional Coverage Model

Coverage sampling is expensive

Determine minimum number of variables required to verify a design feature

Dont sample every clock; use sample event or call .sample()

For each feature of the DUT:

Determine what variables we need to sample to verify this feature
Determine where to implement covergroups for these variables
In data items definition?
In monitors?
In scoreboards?

Group the values of each sample variable into meaningful bins

Name them explicitly => easier for report analysis

61

2013 Cadence Design Systems, Inc. All rights reserved.

Building a Functional Coverage Model

For each feature of the DUT (continue):
To determine if certain condition occurred, you might need to cross
the value of two or more variables.
These variables need to be in same covergroup
Cross coverage can generate a lot of data; use ignore_bins and illegal_bins
If you dont do it now, you will have to exclude those during analysis phase

Check the Covergroup type options (e.g., weight, goal)

Name the covergroup using set_inst_name() to help with analysis
and merging
Review your coverage model with the design team to ensure accuracy

62

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Workshop Agenda

Introduction to Metric-Driven Verification
Coverage Metrics
Code coverage
Generating Code Coverage
Analyzing Code Coverage Using IMC
Lab 1: generating & analyzing code coverage

Functional coverage
Building Functional Coverage Model
Analyzing Functional Coverage Using IMC
Lab 2: functional coverage collection & analysis

IMC Merge/Rank/Refine/Report
Lab 3: merge tests & generate report

Summary
63

2013 Cadence Design Systems, Inc. All rights reserved.

IMC Assertion Analysis View

Assertion Finished /
Failed Counts

Source Code

List of Assertions
64

2013 Cadence Design Systems, Inc. All rights reserved.

IMC Covergroup Analysis View

List of Covergroups

Coverpoints
65

2013 Cadence Design Systems, Inc. All rights reserved.

Bins

Incisive Metrics Center Search Capability

Table search bar

Search on a tree

66

2013 Cadence Design Systems, Inc. All rights reserved.

Incisive Metrics Center Refinement Capability

But can be
hidden as well

Exclude items are gray

out by default
Exclusion Ability to save/read
current state to/from a file

Exclude (recursively)
Exclude Local (non-recursive)

Batch Mode also supports ability to save/load refinement files

Refinement can be applied on all metric kinds including block,
Expression (sub-expressions), toggle signal (individual bits), Covergroup,
Coverpoint, bins, cross-bins, assertions ,etc.
67

2013 Cadence Design Systems, Inc. All rights reserved.

Lab 2: Functional Coverage

68

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Workshop Agenda

Introduction to Metric-Driven Verification
Coverage Metrics
Code coverage
Generating Code Coverage
Analyzing Code Coverage Using IMC
Lab 1: generating & analyzing code coverage

Functional coverage
Building Functional Coverage Model
Analyzing Functional Coverage Using IMC
Lab 2: functional coverage collection & analysis

IMC Merge/Rank/Refine/Report
Lab 3: merge tests & generate report

Summary
69

2013 Cadence Design Systems, Inc. All rights reserved.

Merging Coverage Data

Merging Coverage Data

Coverage analysis generally is performed on aggregated
data from multiple tests run on one or more design
configurations.
Coverage data must be merged to provide a
comprehensive view of what is covered.

71

2013 Cadence Design Systems, Inc. All rights reserved.

Merging Coverage Data with IMC

merge command in IMC
merge only supported in IMC batch mode
./
% imc -batch

cov_work/
scope/
test1/

imc> merge test1 test2 test3 out all

OR
imc> merge * out all

test2/
test3/
all/

% imc -batch

./
regr_jan7_12 /
system1/

72

72

IMC assumes tests are under cov_work/scope/

imc> merge regr_jan7_12/system1/* out

regr_jan7_12/system1/all

OR

test1/

imc> preferences set workDir regr_jan7_12

test2/

imc> preferences set scopeDir system1

test3/
all/

imc> merge * out all

2013 Cadence Design Systems, Inc. All rights reserved.

Merge Command
merge
-out <out_path>
[-overwrite]
{<runs> | -runfile <runfile>}
[-metrics <metrics_type>]
[run_order {as_specified | by_timestamp}]
[initial_model {empty | <runpath> | union_all |
primary_run}]
[-message [<message_level>]]
[-covmodeldir <modeldir>]

73

2013 Cadence Design Systems, Inc. All rights reserved.

Merge Command
Initial Model
Initial Model affects how the Merged coverage model looks
like
-initial_model
<runpath>: <runpath> coverage model used as the starting target
model
primary_run: Primary test coverage model used as the starting target
model.
Primary test is the first test used to merge (based on run_order)

union_all
Target model is the union of all test coverage models

empty
User creates a target model
Based on: -target, -targettype, -source, -sourcetype

74

2013 Cadence Design Systems, Inc. All rights reserved.

Initial Model: <runpath> or primary_run

imc> merge Run1 Run2 initial_model primary_run out all

75

2013 Cadence Design Systems, Inc. All rights reserved.

Initial Model: union_all

imc> merge Run1 Run2 initial_model union_all out all

76

2013 Cadence Design Systems, Inc. All rights reserved.

Merge Config System to Block level merge

77

2013 Cadence Design Systems, Inc. All rights reserved.

Merge Config Block to System level merge

merge_config sourcetype A targettype A

merge Run1 Run2 initial_model Run1 out merged_run
merge_config source System.A1 targettype A
merge_config source System.A2 targettype A
merge Run1 Run2 initial_model Run1 out merged_run
78

2013 Cadence Design Systems, Inc. All rights reserved.

Merge Config Create New Hierarchy

79

2013 Cadence Design Systems, Inc. All rights reserved.

Merge Config Create New Hierarchy (one more)

80

2013 Cadence Design Systems, Inc. All rights reserved.

Ranking Coverage Data

Ranking runs in IMC

Ranking runs helps identify a subset of runs that have the lowest cost
and still provide the same coverage grade as the full test suite.
By ranking the runs one can identify the runs that:
Make the most efficient test suite for regression testing eliminating the
redundant runs.
Provide the best possible coverage considering the CPU time or the
simulation time.
Help in achieving the target coverage goal.

Ranking the runs involves:

Setting the Ranking Configuration using the rank_config command (optional)
Ranking the Runs using the rank command

NOTE: Ranking runs does not require loading of runs.

82

2013 Cadence Design Systems, Inc. All rights reserved.

IMC rank_config Command

metric_type_weight_option
increases or decreases the weight of a given metric type. The default
weight if not specified is 1 for all metrics. If weight is 0, the coverage
metric type is not considered while ranking

-w_block <weight>
-w_expression <weight>
-w_toggle <weight>
-w_fsm <weight>
-w_assertion <weight>
-w_covergroup <weight>

-max <integer_value>
specifies the number of runs after which ranking runs must stop
helps identify the best N runs.

-target <cumulative_coverage_goal>
Specify target goal to be achieved while ranking
ranking stops when goal hit

83

2013 Cadence Design Systems, Inc. All rights reserved.

IMC rank_config command (continued)

-initial_runfile <runfile>
Specifies a runfile that contains a list of runs that must be included
always in the ranking results.
The runs specified in the <runfile> appear above the runs
specified for ranking.

-cpu | -sim
Specifies if CPU time or Simulation time should be considered for
ranking. By default, neither CPU time nor simulation time are
considered when ranking

84

2013 Cadence Design Systems, Inc. All rights reserved.

IMC rank command

<runs> | -runfile <runfile>
lists runs to rank or place in <runfile>

-output <output_file>
Specifies the name of the output file where the ranking results are
saved. By default ranking results are written to the standard out,
unless redirected to a file

-run_order {as_specified | by_timestamp}

specifies the order in which the runs are processed during model
union stage
as_specified : process runs in order specified in command line or runfile. This is
the default.
by_timestamp : process runs in reverse chronical order, the latest database is
processed first

85

2013 Cadence Design Systems, Inc. All rights reserved.

IMC rank command (continued)

-use_prev_merge
helps rank runs that were merged earlier

rank_element_option
specifies the elements for which coverage grading is calculated during
ranking. By default ranking is done for all the elements in the design.
-type <types> : specify the module/entity for which coverage grading are
calculated during ranking
-inst <instances> : specify instances for which coverage grading are calculated
during ranking
-covergroup <covergroup_element_full_path> : specify the covergroups for
which coverage grading are calculated during ranking
-coverpoint <coverpoint_element_full_path> : specify the coverpoints for which
coverage grading are calculated during ranking
-assertion <assertion_element_full_path> : specify the assertions for which
coverage grading are calculated during ranking

86

2013 Cadence Design Systems, Inc. All rights reserved.

IMC rank command examples

Rank runs
considering only the
coverage grade of
module/type ALU

imc> rank test1 test2 test3 test4 type ALU

Rank runs to achieve

imc> rank_config -target 85
a cumulative
coverage goal of
imc> rank test1 test2 test3 test4
85%

imc> rank_config -w_block 0 Ignore block and

w_expression 0
expression coverage
imc> rank test1 test2 test3 test4

87

2013 Cadence Design Systems, Inc. All rights reserved.

IMC rank command examples (continued)

imc>rank test1 test2 test3 test4 out rank_result
Doing model union of IUS runs ...
Ranking
With primary run (cov_work/scope/test1): cov_work/scope/icc_59616df2_5bf8faed.ucm
configuration
IUS target model generated successfully.
Coverage ranking options:
============================================================================
Rank elements: Model
Weight: Block = 1, Expression = 1, Toggle = 1, Fsm = 1, Assertion = 1, Covergroup =
1
Ranking Cost: N.A
Target Cumulative Grade(%): 100
Number of Best Runs: 4
Ranking of coverage runs:

Optimized Runs

Run Cum.
Self
Cum. Wt
Self Wt
Contrib.
Run
Id
Grade(%)
Grade(%)
Grade(%) Grade(%) # items Cost Name
============================================================================
Optimized Runs:
Redundant Runs
1
58.18
58.18
58.18
58.18
6634
0
cov_work/scope/test2
2
73.29
64.32
73.29
64.32
500
0
cov_work/scope/test1
Redundant Runs:
3
0.00
58.18
0.00
58.18
0
0
cov_work/scope/test3
4
0.00
58.18
0.00
58.18
0
0
cov_work/scope/test4

88

2013 Cadence Design Systems, Inc. All rights reserved.

HTML reporting

Analyzing Coverage Data

Generating Text Reports in IMC
% imc -batch

Load run all containing merged data from all tests

imc> load -run all

Optional: Load refinement file (if any)

imc> load refinement my_allexclude.vRefine

Generate Summary Report for all instances

imc> report -summary inst [-html] out myreportsum.html
*

Generate Detailed Report for all instances

imc> report -detailed inst [html] -out myreport.det *

90

2013 Cadence Design Systems, Inc. All rights reserved.

IMC: HTML reports

Example of a report Top Level Summary

Key data

Concise useful HELP is

one click away

High-level summary
Total (average)

Navigate hierarchy

Coverage Numbers
Link to flat CoverGroup coverage all
CoverGroups in one table

91

2013 Cadence Design Systems, Inc. All rights reserved.

IMC: HTML reports

Example of a report - Summary

Click to open
detailed report for
instance

Click column header to sort table

Click link to see details

92

2013 Cadence Design Systems, Inc. All rights reserved.

Click instance to descend

IMC: HTML reports

Example of Global CoverGroup Summary Report

Actual
coverage

Covergroup
options

93

2013 Cadence Design Systems, Inc. All rights reserved.

Bin details

Covergroup
instance
name

IMC: HTML reports

Example of Per Covergroup Coverage

Click on item
to view details

94

2013 Cadence Design Systems, Inc. All rights reserved.

Lab 3 Coverage Analysis

95

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Workshop Agenda

Introduction to Metric-Driven Verification
Coverage Metrics
Code coverage
Generating Code Coverage
Analyzing Code Coverage Using IMC
Lab 1: generating & analyzing code coverage

Functional coverage
Building Functional Coverage Model
Analyzing Functional Coverage Using IMC
Lab 2: functional coverage collection & analysis

IMC Merge/Rank/Refine/Report
Lab 3: merge tests & generate report

Summary
96

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Guidelines Summary Page 1

Start with verification planning
Functional coverage design
For each design feature:
Determine minimum number of variables required
Select sampling event carefully
Check the Covergroup type options (e.g., weight, goal) to ensure
correct grading
Name the covergroup using set_inst_name()

Running tests and collect functional coverage

Once functional coverage reaches target & design is stable, start
code coverage

97

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Guidelines Summary Page 2

Code coverage
Start with Block coverage + FSM coverage
Add branch coverage
Expression coverage for critical blocks; work with designers
Is SOP good enough?
If not, work with designers to determine if Control or Vector is required
Toggle coverage for interfaces between sub-blocks

What about SOC level?

Functional coverage determined by the SOC verification plan
What about code coverage?
Since the concerns here are: interconnect between blocks and glue
logic
Toggle coverage for interconnect between blocks
Block/branch/expression? coverage for glue logic

98

2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Guidelines Summary Page 3

Be selective about what you measure & collect

Capture only what you really need otherwise.
you might have to spend lot of time analyzing results
and writing exclusions!

99

2013 Cadence Design Systems, Inc. All rights reserved.

Additional Documentation / Tutorial

IMC User Guide:
<Incisiv_install_dir>/doc/imcuser/imcuser.pdf
IMC Quick Reference Guide:
<Incisiv_install_dir>/doc/imcug/imcqrg.pdf
ICC User Guide:
<Incisiv_install_dir>/doc/iccug/iccug.pdf
SOC and IP level Functional Verification - Rapid Adoption Kits (RAK)
Download Landing Page
To Download this workshop Click here -> Coverage Workshop
100

2013 Cadence Design Systems, Inc. All rights reserved.

Support Contact

http://support.cadence.com
http://www.cadence.com