Cisco ACI

Tenant (Logical Overlay) Policies

www.lumoscloud.com
[email protected]
Agenda
 ACI Logical Model
 Tenant
 L3 Private Network
 Bridge Domain
 Subnet
 End Points Groups (EPG)
 Application Network Profiles (ANP)
and Contracts
 ACI Quality of Service
ACI Logical Model
Defining Terms
 Tenant – Logical separator (i.e. customer, BU, group etc..); separates traffic, admin,
visibility, etc..
 VRF – Also referred to as a context or private L3 network, it provides separation of
routing instances and administration
 Bridge Domain (BD) – Container for subnets (NOT A VLAN); can be used to define an
L2 boundary. Analogous to a primary Private VLAN (PVLAN).
 Subnet – IP addresses within a given Bridge Domain; must be unique within their
associated L3 Private Network.
 Contract – Represent policies between EPGs; “provided” by one EPG and “consumed”
by another.
 End-Point Group (EPG) – Container for objects requiring the same policy treatment (i.e.
app tiers or services).
Logical Model Overview
root/uni
Tenant A Tenant B

VRF A VRF B VRF A

Bridge Domain Bridge Domain Bridge Domain Bridge Domain

Subnet A Subnet B Subnet A Subnet D

Subnet C

EPG EPG EPG EPG EPG EPG EPG EPG

Private-Networks/VRFs and Subnets are independent between tenants

Management Information Model
TENANT
1

n n n n n n
L2/L3 Application
Bridge Contexts
Outside Network Contracts Filters
Domains n 1 (VRF)
Networks Profiles
n n
1 1 1 1
n n
Subnets n
Subjects
n n
 Solid lines indicate objects below contained
EPGs n  Dashed lines indicate a relationship
 1:n indicates one to many
 n:n indicates many to many
Tenant
Tenant
A logical container for application policies

Pre-Configured Tenants
uni/Tenant56
• Common – Policies that can be accessed by
all tenants polUni
uni
• Infra – VxLAN overlay infrastructure
configuration (Private L3, Bridge Domain)
• Mgmt – Inband and OOB configuration of polUni
Tenant Tenant56
fabric nodes
Create Tenant
VRF (Private L3 Network/Context)
L3 Private Network (Context)
Represents application policy domain and L3 forwarding (VRF)
One or more private networks are uni/Tenant56/VRF-Corp root/uni
associated with a tenant
Tenant A
One or more bridge domains are polUni
uni
related to a private network VRF
Equivalent to a virtual routing and Bridge Domain
forwarding (VRF) instance polUni
Tenant Tenant56 Subnet B
IP addresses must be unique
within the private network Subnet C
polUni
VRF Corp
EPG EPG
VRF Settings / Policies
• Policy control
enforcement
• BGP domain
settings
• OSPF domain
settings
• Endpoint
Retention
• Monitoring
Bridge Domain
Bridge Domains
Represents application policy domain and L2 forwarding

NOT a VLAN uni/Tenant56/VRF-Corp/BD-VMData root/uni

Each private network can have polUni
one or more bridge domains uni Tenant A
Each bridge domain must be VRF B
linked to a private network polUni
Tenant Tenant56 Bridge Domain
Each bridge domain must have
at least one subnet Subnet B
polUni
Can span across switch nodes VRF Corp
Subnet C
MAC addresses must be unique
within the Bridge Domain polUni EPG EPG
BD VMData
 Bridge Domain
Settings/ Policies
• Subnets
• IGMP Snoop
• Endpoint
Retention
• DHCP Relay
• Monitoring
• Forwarding
Control
L2 Forwarding Behavior in the Bridge Domain
• ACI Fabric is host routed but what if there is no Layer 3 header?
• If Destination MAC is known, lookup location route to egress leaf (VTEP)
• L2 Unknown Unicast is either
flooded within the BD or sent
to Spine HW Proxy (default)
• L2 Unknown Multicast is set to
flood (default)
• ARP defaults to L3 unicast
lookup but can be set to flood
Subnets
 Subnet
uni/Tenant56/VRF-Corp/BD-vmdata/subnet-10.10.10.0
Each subnet is a child of one
polUni
bridge domain uni
A BD may have more than one root/uni
subnet polUni Tenant A
Tenant Tenant56
Subnets must have unique IP VRF B
addresses within their context polUni
VRF Corp Bridge Domain
Subnets can span multiple
EPGs. Subnet B
polUni
Subnets automatically create BD VMData
Subnet C
SVIs on Leafs where EPs
appear polUni EPG EPG
Subnet-[ip] 10.10.10.x/24
 Subnet
Settings /
Policies
• IP Address/mask
• Scope
• Subnet Control
• L3 Out for Route
Profile
• Route Profile
End Point Groups (EPG)
End Point Group (EPG)
End Point Group – logical entity containing a collection of end-
points with common policy requirements root/uni
• Example: Security, QoS, or L4-L7 services . . Tenant A
VRF B
End Points – entities directly or indirectly connect to the fabric
• Can be physical or virtual; have an address (identity), location, Bridge Domain
and attributes Subnet B

Subnet C
Hypervisor EPG EPG
End Point Group (EPG)
EPG Policy Enforcement and Mapping
• Policy and security enforcement • Network constructs are independent
occurs at the EPG level of Applications

EPG WEB

HTTPS Service HTTPS Service

10.10.11.x
HTTPS Service HTTPS Service

HTTP Service HTTP Service

10.10.10.x
EPGWEB
EPG VM HTTP Service HTTP Service
RH
EPG APP

• Communication is allowed between all End-points in an EPG by default

 End-Point Groups

IP
Current VLAN
Phys VM
Attribute
Virtual VxLAN Address
Port Port NVGRE (E, EX,
End-Points (1.1)
FX)

Future Subnet L4 Ports Application ?

End-Points
Applying Policy to End-Points
APIC
1) End Point attaches to fabric
2) APIC detects End Point, derives source EPG
3) APIC pushes required policy to leaf switch
Src Dst
EPG EPG
Policy pushed to Leaf nodes based on Resolution Immediacy
• On Demand – Policies only pushed to leaf node upon pNIC attachment AND vNIC association with port-group (EPG)
• Immediate – Policies pushed to leaf node upon Hypervisor pNIC attachment; LLDP or OpFlex resolves Hypervisor to
Leaf node attachment
• Pre-Provision – Policy is downloaded to leaf regardless of CDP/LLDP neighborship, even without a host connected to
the VMM switch.
Policy programming in Leaf node hardware based on Deployment Immediacy
• On Demand – Polices programmed into Policy CAM only when reachability is learnt through data path
• Immediate – Policies programmed in Policy CAM once received by APIC as defined by Resolution Immediacy Policy
Policy Table Size Reduction
Sources Destinations Total policy entries = n * m * f
1 Filters
1 n = # sources
2 1 - Allow x
2 - Deny y 2 m = # destinations
3 3 - Allow x 3 f = # filters
4 4 - Deny y 4
5 5 – Allow x Standard model requires 100
n=5 f=5 m=4 policy entries
Source EPG Destination EPG
1 Filters
1
2 1 - Allow x
2 ACI model requires
3 2 - Deny y
3 - Allow x 3
4 4 - Deny y 4 5 policy entries
5 5 – Allow x
n=1 f=5 m=1
Application Network Profiles
(ANP) and Contracts
Defining Terms
Application Network Profiles (ANP) – are a group of EPGs and the Contracts
(policies) that define the communication between them.
Contract – Policy definition; defines how an EPG communicates with other EPGs.
Contracts contain one or more subjects.
Subject – Something being discussed; used to build definitions of communication
between EPGs. Contains: filter, action, and optional label.
Filter – Identifier for a subject, i.e. the traffic do you want to take action on. Required
within a subject.
Action – Action to be taken on the filtered traffic with a subject; required within a
subject.
Label – Optional advanced identifier, when used labels allow for more complex
definition of relationships within the policy model.
Application Network Profile (ANP)
ANP defines which EPGs can communicate and how they communicate
• Provider–Consumer relationships define application connectivity in application terms
• Contracts define one-way or two-way conversations between EPGs

APPLICATION NETWORK PROFILE

Users WEB Farm APP Servers DB Farm
C C C

One-way Contract Two-way Contract One-way Contract

 Users consume WEB  WEB & APP provide & consume services  DB provides services
services between each other to APP
 Contracts
Contracts are groups of Subjects which define communication between source and
destination EPGs
• Subjects are a combination of a filter, an action, and optional label

Users Contract 1 WEB Farm

SUBJECT
Filter TCP Port 80

Action Permit Optional Label

Label Web Access • Allows greater complexity in
relationship definitions
SUBJECT . .  Ex: internal, guest
Policy Contract Actions
Current policy options supported:
 Permit traffic Deny
Permit
 Deny (block) traffic (Taboo)
 Redirect traffic
Redirect Mark DSCP
 Mark traffic (DSCP/CoS)

Future options supported: Copy Log …

 Copy traffic Packet …
 Log traffic

Policy encompasses traffic handling, quality of service,

security monitoring and logging.
Shared Services
Route leaking, etc. configured automatically

Provide DNS
MGMT
APP Servers AD Contract
Consume

Providers are abstracted away

Subject
WEB Farm by the contract
Consume Subject DHCP
MGMT

Provide

Providing/Consuming EPGs can be in different VRF

Taboos – Explicit Deny
Explicit Denies never allow stated traffic for any EP in the group

Web Services App Services

Taboos enforced
Subject Subject
before all other
Subject Subject contracts
Taboos
Provide Consume Filter
WEB Farm
Filter
Communication within a Tenant/EPGs
Communication between EPGs are dependent upon several factors
Bridge Domain – Subnet Scope
Private to VRF: Local to VRF
Advertised Externally: Global (including external L2/L3)
Shared between VRFs: Any VRF
Contract Scope
Application Profile: Within an Application Profile
VRF: Within a VRF
Global: Global (including external L2/L3)
Tenant: Within a Tenant
Contact Visibility. Where was the contract created
Tenant: Only originating tenant
Export/Consumed Contract Interface: The originating Tenant and the
exported to Tenant
Common-Tenant: Available to all Tenants
Contract Scope
VRF – Within a VRF
Application Profile – Within an Application Profile
Tenant – Within a Tenant
Global - All
Contract and Subnet Scope Between Tenant/EPGs
For Tenant-A/EPG to Tenant-B/EPG communication
Subnets of the respective Tenant EPGs must known to each other
Shared Between VRFs – Network scope is shared to all VRFs in ACI Domain
Contract Scope: Communication within a Tenant/EPGs
For EPG-A to EPG-B communication in the same tenant
Contracts created in a tenant are available by any VRF/EPG in the tenant. However, they
are still governed by the Contract Scope rules
EPG to EPG within a VRF require as a minimum of a single Consume-Provide relationship
All routing is already established in the VRF
In this case, the contract is only providing the traffic filter requirements
EPG to EPG in different VRFs require both a Consume-Provide and Provide-Consume
relationship
This provides the traffic filter in at least one direction
The contract relationships provide the routes in both directions between the 2 VRFs
Contract Scope Communication Between Tenant/EPGs
For Tenant-A/EPG to Tenant-B/EPG communication
Contracts defined by a Tenant are only visible to that tenant
• By Default a Tenant’s defined contacts are not known by other Tenants
• In order to be known by other Tenants, the contract must be exported
from one tenant to a specific tenant.
• The export step only exposes the contract to that specific tenant
• The provide-consume relationship rules must be followed
• The destination Tenant-B/EPG must “consume” the exported contract
• The initiator Tenant-A/EPG must also “provide” the exported contract
• Route leaking is established based on the contract relationship formed
between Tenant-A to Tenant-B
• In order for the reverse route to be established you preform the process in
reverse starting with Tenant-B
Contract Scope Communication Between Tenant/EPGs
Export Contract from Tenant 01 to Tenant 02
Contract Scope Communication Between Tenant/EPGs
The export step only exposes the contract to the specified destination tenant
The provide-consume relationship rules must be followed
The destination Tenant-B/EPG must “consume” the exported contract
“Add Consumed Contract Interface”
Contract Scope Communication Between Tenant/EPGs
The initiator Tenant-A/EPG must also “provide” the exported contract
This also creates the required routing from Tenant-A to Tenant-B
Contract Scope Communication Between Tenant/EPGs
If you need the route established for return traffic between Tenant-B to Tenant-A
you will need to perform the same steps in reverse between Tenant-B to Tenant-A
• Export a contract from Tenant-B to Tenant-A
• Provide the contract from the Tenant-B/EPG
• In the Tenant-A/EPG “Add Consumed Interface Contract” that was
exported from Tenant-B
• In this case, the contract itself may not be necessary for any filtering, it is
just to provide the reverse route
Communication to and from the Common Tenant/EPGs
For Tenant-A/EPG to Common-Tenant/EPG communication
The Common tenant route leaking is much easier to establish
Contracts created in the Common VRF are visible to all tenants. So, a separate “export”
step is not required
• The provide-consume relationship rules must be followed
• The Common/EPG must “provide” the Common-Tenant defined contract
• The recipient Tenant/EPG must “consume” the Common-Tenant defined contract
• This also forms the required routing from Common-Tenant to Recipent-Tenant
• Perform the reverse provide/consume with the Common-Tenant defined
Contract to provide routes in the reverse direction
• The Common/EPG must “consume” the Common-Tenant defined contract
• The recipient Tenant/EPG must “provide” the Common-Tenant defined contract
Communication to and from the Common Tenant/EPGs
From a Tenant – The list of all locally created and Common Tenant Contracts
ACI Quality of Service
ACI Overall QoS Process Flow
Ingress Leaf Spine Egress Leaf

Traffic Classification
Marking Buffering Buffering
Buffering Queuing Queuing
Queuing

APIC

ACI Fabric Application Policy

Infrastructure
Controller
ACI QoS Features
Three user configurable classes of  Marking
service • DSCP Marking
 Level 1
 Queuing and Scheduling
 Level 2
• Strict Priority queuing
 Level 3 (default)
• DWRR
Classification Options
• Class ID is carried in the dot1p of
• Contract VxLAN outer header
• DSCP and dot1p
• EPG
• No re-classification on spine or
egress leaf
ACI QoS Classification Examples
QoS Policy in Contract
• QoS configuration set by contract
• All traffic between EPG1 and EPG2 are assigned to
“Level1”
• Set priority for traffic between two EPGs

Contract
Configuration
 ACI QoS Classification Examples
Using Custom QoS
Set Custom QoS under EPG Detail Custom QoS Configuration

• Custom QoS. Applied at EPG level. Match EPG plus DSCP(or dot1p priority).
• DSCP takes precedence when both DSCP and dot1p matches
• Same policy supports DSCP marking
ACI QoS Classification Examples
EPG Default Class

• When no QoS policy configured in

contract and no “Custom QoS” enabled,
traffic for the EPG is assigned to class of
service based on the setting in “QoS
Class”.
• Traffic will be assigned to
“default/scavenger” when QoS class is
not specified.
Application Network Profile
EPG Svc
EP EP

DNS AD
infra shared services contract filter filter

consume consume consume

Outside EPG WEB consume

EPG APP consume
EPG DB
EP EP EP EP EP EP
web java sql
EPG Public consume contract contract contract
EP EP EP

provide

provide

provide
0.0.0.0/0
http java sql
filter filter filter

provide
EPG Dev provide provide
ssh
consume filter dev contract
10.19.21.0/24

bd bd bd
L3 context
Fabric Access Policies – Logical Model
Fabric and Tenant - Complete Logical Model
root/uni VLAN Pool

Tenant VMM / Physical

Domain

ANP
AEP
EPG Interface Policy Group
+
Policies
L2 Bridge Domain
Subnet/SVI - 10.1.1.1/24 Interface(s)

L3 VRF Leaf Switch(es)