Devnet 1086
Devnet 1086
Suhaib Ahmad
Technical Marketing Engineer
DEVNET-1086
#CiscoLive
Cisco Webex App
https://ciscolive.ciscoevents.com/
ciscolivebot/#DEVNET-1086
Questions?
Use Cisco Webex App to chat
with the speaker after the session
How
1 Find this session in the Cisco Live Mobile App
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 2
Agenda
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Introduction to
SONiC, Design
Goals, and Use
Cases
The SONiC Network Operating System
• SONiC – Software for Open Networking in the Cloud is an open-source
network operating system based on Debian Linux.
• Initially created by Microsoft for use in their Azure datacenters. Open-
sourced by Microsoft in 2016.
• It became part of the Linux Foundation in 2022 which focuses on the
software components of SONiC and partners with Open Compute
Platform (OCP) on aligning hardware and specifications like the Switch
Abstraction Interface.
• https://github.com/sonic-net/SONiC/wiki
• https://sonicfoundation.dev/
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
SONiC Design Goals
• Open-Source Software: Open-source approach to network
operating systems.
• Hardware Agnostic: Creating a Network Operating System can be
integrated with different ASICs and NPUs.
• Modularity: Network Operating System functions separated as
docker containers.
• Operational Ease: Simplify network operations with support for APIs
and configuration management tools.
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Use Cases
• Hyperscaler datacenters
• IP/BGP Datacenter fabric
• Overlay fabric with static VXLAN
• ToR, Leaf, and Fixed-Chassis (or Multi Instance) Spine roles.
• AI/ML Frontend and backend networks
• Backend clusters
• RDMA over Converged Ethernet v2 (RoCEv2)
• QoS with Priority Flow Control (PFC) and Explicit Congestion Notification (ECN).
• BGP Routing, ECMP load balancing
• Frontend clusters
• IP/BGP Routing and VXLAN
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Architecture and
Components
Deep-dive
SONiC Architecture
User space lldp container bgp container
dhcp-relay container pmon container snmp container teamd container
lldpd bgpd
portsyncd
CLI
intfsyncd
sonic-
neighsyncd cfggen
9
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public
SONiC Architecture
• Networking Operating System
(NOS) running on Debian Linux
• NOS components implemented
as separate docker containers.
• In-memory Redis databases
used by all containers for of
application state, configuration
state, operational state, ASIC
state and counters.
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
SONiC Components: Switch State Service
• Switch State Service (SwSS) container: This container exists to
provide communication services between all SONiC modules.
• Orchagent: Reads relevant state from daemons in other containers that
need to program the ASIC (fpmsyncd, lldp_syncd, teamsyncd), process it
and writes it to the ASIC_DB database in the Redis container.
portsyncd
intfsyncd
neighsyncd
orchagent
intfmgrd
vlanmgrd
swss container
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
SONiC Components: syncd
• syncd container: Responsible for synchronizing the state present at
the SONiC NOS layer and the actual forwarding ASIC in the
platform.
• SAI API: Switch Abstraction Interface API defines a vendor-agnostic set
of endpoints for managing the forwarding ASIC in the platform.
• syncd: Subscribes to the ASIC_DB database (which is updated by the
SWSS container) and programs the ASIC using the ASIC SDK.
syncd
sai api
asic sdk
syncd container
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
SONiC Components: bgp
• bgp container: The SONiC routing stack is implemented in this
container. It is based on the open-source Free Range Routing
(FRR) protocol suite.
• bgpd: Collects routing updates updates from BGP neighbors.
• zebra: Provides kernel routing table updates, interface lookups, and
redistribution of routes between different routing protocols.
• fpmsyncd: Writes FIB state calculated by zebra to the Redis container.
bgpd
zebra
fpmsyncd
bgp container
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
SONiC Components: Other containers
• Platform Monitor (pmon) container: Collects sensor readings from
platform sensors. Raises alarms (if any) and controls fan speeds on
the basis of sensor readings.
• SNMP (snmp) container: Implements an SNMP server for SONiC.
• Teamd (teamd) container: Provides link aggregation functionality in
SONiC. Based on teamd, an open-source LAG protocol.
• LLDP (lldp) container: Implements Link Layer Discovery Protocol for
SONiC.
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
SONiC
Operational
Considerations
SONiC Install
• Open Network Install Environment (ONIE) is SONiC’s supported
install environment.
• It combines a boot loader with a Linux kernel and Busybox and
Provides boot options such as iPXE.
• The busybox environment allows user-configurable install actions
as well as password recovery
• SONiC CLI also supports install upgrade and rollback actions.
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
SONiC Install: ONIE
GNU GRUB version 2.02~beta2+e4a1fe391
+----------------------------------------------------------------------------+
|*ONIE: Install OS |
| ONIE: Rescue |
| ONIE: Uninstall OS |
| ONIE: Update ONIE |
| ONIE: Embed ONIE |
+----------------------------------------------------------------------------+
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
SONiC Install: sonic-installer
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
SONiC Install: sonic-installer
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
SONiC Install: sonic-installer
root@sonic:/home/admin#
Sets the image to be loaded on
the next router reload
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
SONiC Configuration
• The device configuration is stored as a json • Configuration actions can be made by
file and can be accessed from either:
/etc/sonic/config_db.json • Modifying the config_db.json file using the
sonic-cfggen utility
• Contents of /etc/sonic/config_db.json
are loaded into the Redis database on • CLI commands in the Linux shell
startup or by issuing a config load or • vty shell in FRR
config reload command
• SONiC configuration management can be
• SONiC also has a simple Python Click simplified by using open-source tooling
based CLI interface. such as Ansible and Terraform.
• “show” commands can be executed by any • gRPC/gNMI also supported, however, gNMI
user whereas configuration commands implementation is recent and does not
require sudo privileges cover all endpoints
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
SONiC show CLI examples
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
SONiC show CLI
examples
<...>
Docker images:
REPOSITORY TAG IMAGE ID SIZE
docker-macsec latest f7a9a7d4ff58 340MB
docker-dhcp-relay latest 4bc1d2cc1192 328MB
docker-ipxeserver-cisco 202305.1.2-6514cfa26 6d594511b9ea 352MB
docker-ipxeserver-cisco latest 6d594511b9ea 352MB
docker-gbsyncd-cisco 202305.1.2-6514cfa26 29a320c79090 386MB
docker-gbsyncd-cisco latest 29a320c79090 386MB
docker-sonic-telemetry 202305.1.2-6514cfa26 0974c2d9654e 407MB
admin@sonic:~$ show version docker-sonic-telemetry latest 0974c2d9654e 407MB
docker-teamd 202305.1.2-6514cfa26 e8e364924112 338MB
SONiC Software Version: SONiC.202305.1.2-6514cfa26 docker-teamd latest e8e364924112 338MB
SONiC OS Version: 11 docker-snmp 202305.1.2-6514cfa26 a498e1e14b1c 359MB
Distribution: Debian 11.9 docker-snmp latest a498e1e14b1c 359MB
Kernel: 5.10.0-23-2-amd64 docker-sflow 202305.1.2-6514cfa26 bfd6484d7a6d 339MB
Build commit: 6514cfa26 docker-sflow latest bfd6484d7a6d 339MB
Build date: Wed Mar 13 09:33:29 UTC 2024 docker-router-advertiser 202305.1.2-6514cfa26 a4ab1e96403f 320MB
Built by: sonicci@sonic-ci-8-lnx docker-router-advertiser latest a4ab1e96403f 320MB
docker-platform-monitor 202305.1.2-6514cfa26 087a1cb73d85 443MB
Platform: x86_64-8201_32fh_o-r0 docker-platform-monitor latest 087a1cb73d85 443MB
HwSKU: 32x400Gb docker-orchagent 202305.1.2-6514cfa26 40b5f419bfd3 350MB
ASIC: cisco-8000 docker-orchagent latest 40b5f419bfd3 350MB
ASIC Count: 1 docker-nat 202305.1.2-6514cfa26 3fb6329dc917 341MB
Serial Number: FLM27200ADU docker-nat latest 3fb6329dc917 341MB
Model Number: 8201-32FH-O docker-mux 202305.1.2-6514cfa26 fcaa4514b7af 369MB
Hardware Revision: 1.0 docker-mux latest fcaa4514b7af 369MB
Uptime: 22:25:57 up 31 days, 4:27, 1 user, load average: 1.32, 1.14, 1.04 docker-lldp 202305.1.2-6514cfa26 da9e0a1f94c9 363MB
Date: Fri 31 May 2024 22:25:57 docker-lldp latest da9e0a1f94c9 363MB
docker-fpm-frr 202305.1.2-6514cfa26 bf63f51bab81 369MB
<...> docker-fpm-frr latest bf63f51bab81 369MB
docker-eventd 202305.1.2-6514cfa26 6fc2fd9a7bc4 320MB
docker-eventd latest 6fc2fd9a7bc4 320MB
docker-database 202305.1.2-6514cfa26 52c1c0481fdf 320MB
docker-database latest 52c1c0481fdf 320MB
docker-syncd-cisco 202305.1.2-6514cfa26 fa836211e367 812MB
docker-syncd-cisco latest fa836211e367 812MB
docker-sonic-mgmt-framework 202305.1.2-6514cfa26 45c7465d9c8b 415MB
docker-sonic-mgmt-framework latest 45c7465d9c8b 415MB
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
SONiC Containers
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
SONiC show CLI examples
admin@sonic:~$ show interfaces status
Interface Lanes Speed MTU FEC Alias Vlan Oper Admin Type Asym PFC
----------- --------------------------------------- ------- ----- ----- ------- ------ ------ ------- ------ ----------
Ethernet0 2304,2305,2306,2307,2308,2309,2310,2311 400G 9100 N/A etp0 routed down up N/A N/A
Ethernet8 2320,2321,2322,2323,2324,2325,2326,2327 400G 9100 N/A etp1 routed down up N/A N/A
Ethernet16 2312,2313,2314,2315,2316,2317,2318,2319 400G 9100 N/A etp2 routed down up N/A N/A
Ethernet24 2056,2057,2058,2059,2060,2061,2062,2063 400G 9100 N/A etp3 routed down up N/A N/A
Ethernet32 1792,1793,1794,1795,1796,1797,1798,1799 400G 9100 N/A etp4 routed down up N/A N/A
Ethernet40 2048,2049,2050,2051,2052,2053,2054,2055 400G 9100 N/A etp5 routed down up N/A N/A
Ethernet48 2560,2561,2562,2563,2564,2565,2566,2567 400G 9100 N/A etp6 routed down up N/A N/A
Ethernet56 2824,2825,2826,2827,2828,2829,2830,2831 400G 9100 N/A etp7 routed down up N/A N/A
Ethernet64 2832,2833,2834,2835,2836,2837,2838,2839 400G 9100 N/A etp8 routed down up N/A N/A
Ethernet72 2816,2817,2818,2819,2820,2821,2822,2823 400G 9100 N/A etp9 routed down up N/A N/A
Ethernet80 2568,2569,2570,2571,2572,2573,2574,2575 400G 9100 N/A etp10 routed down up N/A N/A
Ethernet88 2576,2577,2578,2579,2580,2581,2582,2583 400G 9100 N/A etp11 routed down up N/A N/A
Ethernet96 1536,1537,1538,1539,1540,1541,1542,1543 400G 9100 N/A etp12 routed down up N/A N/A
Ethernet104 1800,1801,1802,1803,1804,1805,1806,1807 400G 9100 N/A etp13 routed down up N/A N/A
Ethernet112 1552,1553,1554,1555,1556,1557,1558,1559 400G 9100 N/A etp14 routed down up N/A N/A
Ethernet120 1544,1545,1546,1547,1548,1549,1550,1551 400G 9100 N/A etp15 routed down up N/A N/A
Ethernet128 1296,1297,1298,1299,1300,1301,1302,1303 400G 9100 N/A etp16 routed down up N/A N/A
Ethernet136 1288,1289,1290,1291,1292,1293,1294,1295 400G 9100 N/A etp17 routed down up N/A N/A
Ethernet144 1280,1281,1282,1283,1284,1285,1286,1287 400G 9100 N/A etp18 routed down up N/A N/A
Ethernet152 1032,1033,1034,1035,1036,1037,1038,1039 400G 9100 N/A etp19 routed down up N/A N/A
Ethernet160 264,265,266,267,268,269,270,271 400G 9100 N/A etp20 routed down up N/A N/A
Ethernet168 272,273,274,275,276,277,278,279 400G 9100 N/A etp21 routed down up N/A N/A
Ethernet176 16,17,18,19,20,21,22,23 400G 9100 N/A etp22 routed down up N/A N/A
Ethernet184 0,1,2,3,4,5,6,7 400G 9100 N/A etp23 routed down up N/A N/A
Ethernet192 256,257,258,259,260,261,262,263 400G 9100 N/A etp24 routed down up N/A N/A
Ethernet200 8,9,10,11,12,13,14,15 400G 9100 N/A etp25 routed down up N/A N/A
Ethernet208 1024,1025,1026,1027,1028,1029,1030,1031 400G 9100 N/A etp26 routed down up N/A N/A
Ethernet216 768,769,770,771,772,773,774,775 400G 9100 N/A etp27 routed down up N/A N/A
Ethernet224 520,521,522,523,524,525,526,527 400G 9100 N/A etp28 routed down up N/A N/A
Ethernet232 776,777,778,779,780,781,782,783 400G 9100 N/A etp29 routed down up N/A N/A
Ethernet240 512,513,514,515,516,517,518,519 400G 9100 N/A etp30 routed down up N/A N/A
Ethernet248 528,529,530,531,532,533,534,535 400G 9100 N/A etp31 routed down up N/A N/A
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
Configuring SONiC: CLI
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
Configuring SONiC: sonic-cfggen utility
• The sonic-cfggen is a utility to read SONiC config from various
input types and then write the config to the config database in
Redis, print as a json or render a jinja2 config template.
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
Configuring SONiC: sonic-cfggen utility
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
Configuring SONiC: Ansible
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
SONiC support
on Cisco 8000
routers
SONiC on Cisco 8000 routers
• Cisco 8000 is a family of routers based on Cisco’s Silicon One ASICs
designed for deployment in service provider or hyperscaler
environments.
• SONiC is supported on certain routers in the Cisco 8000 family.
• Cisco 8000 routers supporting SONiC come with device drivers and a
Base Support Package (BSP) which Controls board, fans, drivers, FPD,
LED, etc.
• These routers ship with SONiC installed on them. New images and
upgrades are available at www.software.cisco.com
• Cisco is an active participant in SONiC’s open-source community and
upstreams code into the SONiC open-source project.
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
Cisco Silicon One ASICs
Enterprise
Core
Large
One Architecture, SDK, and P4 Forwarding Code
Provider/Web Scale
Aggregation
Provider
Service
Core
running IOS-XR Silicon One ™️ Silicon One ™️ Silicon One ™️
P100 Q200 Q100 Peering
Cisco© 2019 Cisco© 2019 Cisco© 2019
DCI
Routing
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco 8000 Routers supporting SONiC
Power Optimized
Single ASIC
Systems
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
Conclusion
Conclusion
• SONiC is an open-source, hardware agnostic, modular Network Operating System.
• It was created by hyperscale cloud providers for their datacenter and AI/ML
environments and the feature support on SONiC is still dominated by these use-
cases.
• SONiC’s open-source approach extends into its components which are based on
other popular open-source projects such as FRR, teamd, etc.
• SONiC has support for a variety of management tools. However, advanced
configuration may need multiple configuration touchpoints (e.g., ConfigDB, FRR
CLI).
• Cisco supports SONiC on certain routers in the 8000 family and ships these routers
with the SONiC image.
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
Further Reading
• SONiC on Cisco 8000 routers:
• https://www.cisco.com/c/en/us/products/routers/8000-series-
routers/sonic.html
• SONiC on Cisco 8000 at DevNet:
https://developer.cisco.com/docs/sonic/
• SONiC Learning Lab:
https://developer.cisco.com/learning/labs/sonic-notebooks/getting-
started-with-notebooks-for-sonic-on-the-cisco-8000-emulator/
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Complete Your Session Evaluations
Earn 100 points per survey completed and compete on the Cisco Live
Challenge leaderboard.
#CiscoLive DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
• Visit the Cisco Showcase
for related demos
DEVNET-1086 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Thank you
#CiscoLive