Configuration convertor from Cumulus to SONiC

This Configuration convertor is intended for network administrators with CUMULUS background. This guide will help network administrators to migrate their current CUMULUS deployment for various Fabric networks to SONiC . CUMULUS User guide https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-42/ can be used as a checklist to get started with the migration plan from CUMULUS to SONiC. Following document provides example configuration commands for comparison.

Switch Management

Operator has to login to CUMULUS and SONiC switch as super user using sudo su

CUMULUS	SONiC
Management IP
# OOB Management IP # Syntax nv set interface eth0 ip address / nv set interface eth0 ip gateway # Configure edit /etc/network/interfaces file # Example sudo nano /etc/network/interfaces # Management interface auto eth0 iface eth0 address 192.0.2.42/24 gateway 192.0.2.1 # OOB Management IP with VRF # Syntax #Management VRF is enabled by default in Cumulus Linux so logins to the switch are set into the management VRF context. To disable management VRF, following are the options- Run NCLU command - net del vrf mgmt command Remove the auto mgmt and auto eth0 stanzas from the /etc/network/interfaces file and reboot the switch #Example command brings down the management VRF, then brings it back up with the ifup --with-depends mgmt command: sudo ifdown mgmt sudo ifup --with-depends mgmt	# OOB Management IP # Syntax config interface ip add <mgmt-if> <Ipv4_address> / <Ipv4_subnet> <gateway_ipv4_address> # Example config interface ip add eth0 192.168.1.1/24 192.168.1.254 # OOB Management IP with VRF # Syntax config VRF add mgmt config interface ip add mgmt <VRF-NAME> <Ipv4_address>/<Ipv4_subnet> <gateway_IPV4_address> # Example config VRF add mgmt config interface ip add mgmt VRF-1 192.168.1.1/24 192.168.1.254 # Command to verify management IP address configured show management_interface address Management IP address = 192.168.1.1/24 Management Network Default Gateway = 192.168.1.254
Switch Reboot
# Linux Command in CUMULUS to reboot the system but it will cause traffic disruption sudo reboot # Linux Cold restart - cold restarts the system and resets all the hardware devices on the switch sudo csmgrctl -c # Linux Fast restart - fast restarts the system more efficiently with minimal impact to traffic by reloading the kernel and software stack without a hard reset of the hardware sudo csmgrctl -f # Linux Warm restart system with no interruption to traffic for existing route entries sudo csmgrctl -w	# Command to perform a system reboot which may cause some disruption of data traffic reboot # Command to define the cause of reboot of a Sonic device show reboot-cause show reboot-cause history # Command to enable a switch to reboot quickly with minimum disruption to the data plane fast-reboot # Warm reboot commands perform in-service NOS upgrade without impacting the data plane traffic warm-reboot -v config warm_restart enable/disable config warm_restart enable # Command to show the configuration of warm restart settings and whether the service is enabled or disabled show warm_restart config show warm_restart state # Command to view syslogs tail -f /var/log/syslog
Upgrade NOS
# Command to check the version in CUMULUS nv show platform software installed # ONIE install Cumulus image via FTP ONIE#onie-nos-install ftp://local-ftp-server/cumulus-install-[PLATFORM].bin # ONIE install Cumulus image via TFTP ONIE#onie-nos-install tftp://local-tftp-server/cumulus-install-[PLATFORM].bin # Syntax to Upgrade Cumulus Switch sudo onie-install -a -i http://10.x.x.x/cumulus-linux-4.1.0-mlx-amd64.bin sudo reboot # Cumulus Install command using the installer sudo -E apt-get update # Command to see the additional package dependencies that will be installed or upgraded sudo -E apt-get upgrade --dry-run # Upgrade all the packages to the latest distribution sudo -E apt-get upgrade # Reboot the switch sudo reboot	# Command to check the version in SONiC Show version # Command to upgrade the version in SONiC sonic-installer sonic-installer install sonic-installer install [OPTIONS] <image_file_path> sonic-installer list # Command to set which image will be used for default boot image after any system reboot sonic-installer set-default sonic-installer set-default <image_name> sonic-installer set-next-boot <image_name> # Operator can use the following command to remove a saved SONiC image in device flash/disk sonic-installer remove sonic-installer remove [y\|-yes] <image_name>
Configuration Save
# Command to save the configuration on Cumulus sudo config-backup sudo config-backup -d -D <CONFIG_FILE> sudo config-backup -q -X .~$ sudo config-backup -pv # Command to restore configuration* sudo config-restore -b config_backup-2019-04-23-21.30.47_leaf01 sudo config-restore -n 10 sudo config-restore -N sudo config-restore -L -N # CLI to delete and re-add a new saved config net add <config_file> net del <config_file> # Use the net pending command to review staged changes net pending <config_file> # Command to commit the changes in config net commit # Command to revert the last config change net abort	# Command to save the configuration on SONiC config save -y # Command to delete and re-add a new saved config config reload <config_db.json/SONiCYang> # Command to load the configuration from a JSON file config load <config_json_file> # Replace a new configuration on top of the existing running configuration config replace <config_db.json/SONiCYang>
Platform Information
# Show system platform information sudo decode-syseeprom # Command to show the platform type sudo decode-syseeprom	# Command to verify platform details in SONiC # Syntax show system status show clock show boot show environment show system status show reboot-cause show uptime show logging show users show platform fan show platform firmware status show platform firmware version show platform pcieinfo show platform psustatus show platform ssdhealth show platform summary show platform syseeprom show platform temperature show interfaces transceiver

Management Services

CUMULUS

SONiC

SYSLOG

# Configure syslog server
# Syntax

net add syslog host ipv4 <IP_ADDRESS> port udp <PORT_NUMBER>

net pending

net commit

# Example

net add syslog host ipv4 192.168.0.254 port udp 514

net pending

net commit

# Command to delete syslog server

net del syslog host ipv4 <IP_ADDRESS> port udp <PORT_NUMBER>

# Syslog commands in SONiC
# Syntax

config syslog add

config syslog delete

# Command to add or delete a specific syslog server IP

config syslog add <ipv4-address> --source <source_ipv4_address>

config syslog del <ipv4-address>

# View syslog for a particular protocol in SONiC

show logging

show logging <any_protocol>

# Command to show syslog server IP and port configuration

show syslog

# Location of syslog configuration file
Configuration file for syslog available at: /etc/rsyslog.conf

# Example Configuration

config syslog add 1.1.1.1 --source 192.168.8.231

config syslog del 1.1.1.1

# Command to view syslog file location
Path: /var/log/syslog*

ZTP

# Configuration to enable Zero Touch Provisioning
# Syntax
# ZTP Over DHCP command
# Example

Edit /etc/dhcp/dhcpd.conf file for an ISC DHCP server

option cumulus-provision-url code 239 = text;

subnet 192.0.2.0 netmask 255.255.255.0 {

range 192.0.2.100 192.168.0.200;

option cumulus-provision-url "http://192.0.2.1/demo.";

# Command to specify the hostname of the switch in
ZTP script
# Example

subnet 192.168.0.0 netmask 255.255.255.0 {

range 192.168.0.100 192.168.0.200;

option cumulus-provision-url "http://192.0.2.1/demo.";

host dc1-tor-sw1 { hardware ethernet 44:38:39:00:1a:6b;
fixed-address 192.168.0.101; option host-name "dc1-tor-sw1"; }

# Command function to demonstrate the ZTP function through
a Linux call

function init_ztp(){

CUMULUS_TARGET_RELEASE=5.0.0

CUMULUS_CURRENT_RELEASE=$(cat /etc/lsb-release | grep RELEASE
| cut -d "=" -f2)

IMAGE_SERVER_HOSTNAME=webserver.example.com

IMAGE_SERVER="http://"$IMAGE_SERVER_HOSTNAME"/"
$CUMULUS_TARGET_RELEASE".bin"

ZTP_URL="http://"$IMAGE_SERVER_HOSTNAME"/ztp."

if [ "$CUMULUS_TARGET_RELEASE" != "$CUMULUS_CURRENT_RELEASE" ];
then

ping_until_reachable $IMAGE_SERVER_HOSTNAME

/usr/cumulus/bin/onie-install -fa -i $IMAGE_SERVER -z $ZTP_URL
&& reboot

else

init_ztp && reboot

exit 0

}

# Command to test the ZTP Scripts
# Validate and debug your ZTP scripts

sudo ztp -v -r http://192.x.x.x/script.

# Verify ZTP status

sudo systemctl -l status ztp.service

# Configuration
# Syntax
# Enable the ZTP services

admin@sonic:~$ config ztp enable

# Running the ZTP Services

admin@sonic:~$ config ztp run -y

# Check the ZTP Status

admin@sonic:~$ show ztp status

# Check the /etc/sonic, user will be able to see config_db.json

admin@sonic:~$ ls /etc/sonic/ | grep config_db.json

Config_db.json

# Server where ZTP server is hosted, the operator can edit in a customized way various parameters like URL, source path location, destination path location during ZTP automated discovery process
# Example

Example for ztp.json.

{ "ztp": { "01-configdb-json": { "url": { "source": "tftp://188.188.36.36/7326_56X_config_db.json", "destination": "/etc/sonic/config_db.json" } }, "02-firmware": { "install": { "url": "http://188.188.36.36:8000/sonic-broadcom.bin", "skip-reboot": true } } } }

SNMP

# Add SNMP Community and Agent Address
# Command to start SNMP service

sudo systemctl start snmpd.service

# Enable snmpd daemon to start automatically after reboot

sudo systemctl enable snmpd.service

# Command to reload

sudo systemctl daemon-reload

# Configure the snmp daemon to listen on the localhost IPv4 and IPv6 interfaces

net add snmp-server listening-address localhost

net add snmp-server listening-address localhost-v6

# Configure SNMP listening address on the loopback interface

net add snmp-server listening-address localhost

# Configure snmpd daemon to listen on all interfaces for either IPv4 or IPv6

net add snmp-server listening-address all

net add snmp-server listening-address all-v6

# Configure snmpd to listen to a specific IPv4 or IPv6 address

net add snmp-server listening-address <SNMP_LISTENING_ADDRESS>

# Configure SNMPv3 username

net add snmp-server username <SNMP_USERNAME> auth-none

# Configure SNMP server username with password options
# Example

net add snmp-server username user1 auth-none

net add snmp-server username user2 auth-md5 user2password

net add snmp-server username user1 auth-none oid 1.3.6.1.2.1

net add snmp-server username user1 auth-none oid system

net add snmp-server username user3 auth-sha testshax encrypt-aes testaesx oid 1.3.6.1.2.1

# SONiC - Add SNMP Community and Agent Address
# Syntax to add SNMP community

config snmp community add <snmp_community_name> <Mode_Readonly or read Write>

# Example to add SNMP community

config snmp community add testcomm ro

# Command to add SNMP Agent IP address

config snmpagent add <Agent_IPV4_Address> -v <VRF-NAME>

# Command to add SNMP user
# Syntax

# Example to add SNMP user

config snmp user add testuser3 priv rw md5 testuser3_auth_pass aes testuser3_encrypt_pass

# Add SNMP traps and SNMP server target address

config snmptrap modify 2 <Server_IP_Address>

show snmptrap

show snmp agentaddress

show running configuration snmp

AAA/Radius

# Configure Radius Server IP and Port
# Local Fallback Authentication
sudo useradd -u 1002 -g 1001 -o -s /sbin/radius_shell johnadmin

# Enable the local privileged user to run sudo
and NCLU commands
sudo adduser <USERNAME> netedit
sudo adduser <USERNAME> sudo
sudo systemctl restart netd

# Modify /etc/passwd file to move the local
user line before the radius_priv_user
sudo vi /etc/passwd
johnadmin:x:1002:1001::/home/johnadmin:/sbin/
radius_shell
radius_priv_user:x:1002:1001::/home/radius_priv
_user:/sbin/radius_shell

# Set the local password for the local user
sudo passwd johnadmin

# Verify radius client configuration
net add interface <INTERFACE_NAME>
source /etc/network/interfaces.d/*.intf

# SONiC - Configure Radius Server IP and Port
# Syntax to configure AAA authentication login

config aaa authentication login {radius | tacacs+ | local} [radius | tacacs+ | local]

config radius add <Radius_server_ip>

# Show Radius commands

show aaa

show radius

# AAA authentication options
# Syntax

aaa authentication login tacacs+

# If one AAA server fails, go to the backup AAA server for authentication

aaa authentication failthrough <enable/disable/default>

aaa authentication fallback <enable/disable/default>

# AAA accounting enable commands in SONiC
# Syntax

config aaa accounting local

config aaa accounting tacacs+

# Command to add AAA accounting server IP and bind it to a data interface

config radius add <accounting_server_ip>

config radius add <accounting_server_ip> --s <source_interface>

sFlow

# sFlow Commands
# Start sFlow process

sudo systemctl start hsflowd.service

# Validate which sFlow agent IP was selected

grep agentIP /etc/hsflowd.auto

# Command to send sFlow to an in-band collector on the default VRF:

sudo systemctl enable hsflowd.service

sudo systemctl start hsflowd.service

# Command to add sFlow collector

config sflow collector add <collector_name1> <sflow_collector_ipv4> <port_number>

config sflow collector add <collector_name2> <sflow_collector_ipv6> <port_number>

# Command to delete sFlow collector

config sflow collector del <collector-name1>

config sflow collector del <collector-name2>

# Command to add and delete sFlow agent

config sflow agent-id add

config sflow agent-id del

# Command to bind sFlow agent to an interface

config sflow agent-id add <Ethernet_interface_number>

config sflow agent-id add <loop_interface_number>

# Command to enable/disable sFlow

config sflow enable

config sflow disable

config sflow interface

config sflow interface <enable/disable>

config sflow interface enable <Ethernet_interface>

# Configure sFlow sample rate and interval

config sflow interface sample-rate <interface_name> <sample_rate>

config sflow polling-interval <time_interval_seconds>

# Command to show sFlow configuration

show sflow

show sflow interface

NTP

# Configuring the NTP Server IP Address

net add time ntp server .cumulusnetworks.pool.ntp.org iburst

# Commands to add the NTP server to the list of servers in /etc/ntp.conf

ps -ef | grep ntp

# Verify that ntpd is running on the system

net show time ntp servers

# Verify the NTP peer status

net del time ntp server 0.cumulusnetworks.pool.ntp.org

net del time ntp server 1.cumulusnetworks.pool.ntp.org

# Command to change the NTP source interface

net add time ntp source <interface_name>

# Validate NTP configuration

sudo systemctl restart ntp

sudo systemctl status -n0 ntp.service

# Edit the /etc/ntp.conf file to specify the server you want to use

sudo nano /etc/ntp.conf

server ntp.your-provider.example

# Command to configure NTP Server IP

config ntp add <NTP_SERVER_IP>

# Example

config ntp add 100.100.10.10

# Command to delete a configured NTP Server IP

config ntp del <ip_address>

# Restart NTP-config daemon after applying NTP server config through config_db.json

systemctl restart ntp-config

# Command to list system timezone

timedatectl list-timezones

# Command to modify the time zone

timedatectl set-timezone <TIME_ZONE>

# Command to show the NTP server information

show ntp

Port Mirroring

CUMULUS

SONiC

# Configuring Port Mirroring
# Syntax to configure SPAN with NCLU

net add port-mirror session <session-id> (ingress|egress) span src-port <interface> dst-port <interface>

# Syntax to configure ERSPAN with NCLU

net add port-mirror session <session-id> (ingress|egress) erspan src-port <interface> src-ip <interface> dst-ip <ip-address>

# Command to mirror all packets received on swp1, and copy and transmit the packets to swp2 for monitoring

net add port-mirror session 1 ingress span src-port <Source_interface> dst-port <dest_interface>

# Command to mirror all packets that are sent out of swp1, and copy and transmit the packets to swp2 for monitoring

net add port-mirror session 1 egress span src-port <Source_interface> dst-port <dest_interface>

# Show Session Configuration

net show port-mirror session 1

# Show SPAN and ERSPAN configuration for all sessions

net show port-mirror session all

# Delete a SPAN or ERSPAN session

net del port-mirror session 1

# Delete all SPAN or ERSPAN sessions

net del port-mirror session all

# ERSPAN Command Example

net add port-mirror session 1 ingress erspan src-port swp1 src-ip 10.10.10.1 dst-ip 10.10.10.234

# Create a Mirror Session

config mirror_session add ts1_everflow <Source_Ip_address> <destination_Ip_Address> <dscp_number> <queue_number>

# Command to create ACL table

config acl add table ACL_Mirror MIRROR --description 'mirror' --stage ingress --ports Ethernet0

# Command to create an ACL JSON file and load it to the configuration database for everflow

cat acl.json

{ "ACL_RULE": { "ACL_Mirror|ACE_Mirror": { "PRIORITY": "55", "IP_TYPE": "ipv4any", "MIRROR_ACTION": "ts1_everflow" } } }

# Command to load the acl.json with new config related to ACL applied

config load acl.json -y

# Command to verify the mirror status

show mirror_session

# Command to create a mirror session for SPAN

config mirror_session span add <session_name> <Destination_interface_Analyzer> <Source_intertface_switch>

# Command to create a mirror session for Remote SPAN

config mirror_session erspan add <session_name> <src_ip> <dst_ip> <dscp> <ttl> [gre_type] [queue] [src_port] [direction]

# Command to create a mirror session and ACL table

config mirror_session span add <session_name> <Destination_port> <Source_port> <Direction>

# Example

config acl add table Test MIRROR -p Ethernet8 -s ingress

# Command to verify the mirror table

show mirror_session

# Create ACL JSON file and load it to the configuration database for Mirror

cat acl.json

{ "ACL_RULE": { "Test|Forward": { "PRIORITY": "2", "MIRROR_ACTION": "test", "VLAN_ID": "20" } } }

# Command to load the JSON file with ACL config applied

config load acl.json -y

config save -y

# Command to check the status of ACL table and mirror session

show mirror_session

show acl table

Layer 2 Switching

Interface and Port VLAN

Layer2 Switching

CUMULUS	SONiC
Port VLAN
# Create VLANs net add bridge bridge ports swp1-2 net add bridge bridge vids 100,200 # Configure an Ethernet port to override the bridge VIDs net add bridge bridge ports swp1-3 net add bridge bridge vids 100,200 net add bridge bridge pvid 1 net add interface swp3 bridge vids 200 # Command to add a primary VLAN nativeVLAN net add bridge bridge PVID 1 # Configure the new VLAN reserved range sudo cat /etc/cumulus/switchd.conf Resv_vlan_range 1 to 100 # Command to restart switch services sudo systemctl restart switchd.service	# Configure Interface Speed config interface speed Ethernet<interface> <speed> config interface advertised-speeds Ethernet<interface> <speed> # Set Auto Negotiation for an Ethernet Interface config interface autoneg Ethernet<interface> enable # Show Auto Negotiation Status for an Ethernet Interface show interface autoneg status Ethernet0 # Show Operational Status of Interfaces show interface status # Configure 4x10GE Breakout for a 40GE Port config interface breakout Ethernet1 '4x10G' # Show Interface Breakout Options show interface breakout # Configure FEC Mode of an Ethernet Interface config interface fec Ethernet<interface> <FEC_MODE> # Create VLANs config vlan add <vlan_value1> config vlan add <vlan_value2> # Show VLAN Configuration show vlan config # Add Interface to VLAN in Tagged (Trunk) Mode config vlan member add <vlan_value1> Ethernet<interface1> config vlan member add <vlan_value2> Ethernet<interface2> # Add Interface to VLAN in Untagged (Access) Mode config vlan member add -u <vlan_value1> Ethernet<interface1> config vlan member add -u <vlan_value2> Ethernet<interface2> # Show VLAN Information show vlan brief
LAG- IEEE 802.3ad link aggregation mode that allows one or more links to be aggregated together to form a link aggregation group (LAG) so that a media access control (MAC) client can treat the group as if it were a single link. IEEE 802.3ad link aggregation is the default mode.
# Create Dynamic LACP in an Aggregated Interface # Syntax The bond is configured by default in IEEE 802.3ad link aggregation mode - LACP net add bond [bond-name] bond slaves [slaves] net pending # Example # Command to create a bond called bond0 with 4 member link ports swp1, swp2, swp3, and swp4 net add bond bond0 bond slaves swp1-4 net pending net commit # Change LACP Mode to balance-xor net add bond bond1 bond mode balance-xor # Change LACP Mode to 802.3ad net add bond bond1 bond mode 802.3ad # Command to Verify LACP LAG Information net show interface bond1	# Create Port Channel # Syntax config portchannel add PortChannel<Channel1> # Add Members to Port Channel config portchannel add PortChannel<Channel1> Ethernet<interface> # Command to Verify Port Channel Interface show interface portchannel # Command to Show VLAN Status show vlan brief # Command to Show IP Interface Status show ip interfaces show interfaces status # Command to Create a PortChannel Interface and Set the Specific LACP Key config portchannel add PortChannel<Channel1> --lacp-key <Key-number> config portchannel member add PortChannel<Channel1> Ethernet<interface> # Command to Create a PortChannel Interface in Fast Rate Mode config portchannel add PortChannel<number> --fast-rate true # Command to Create a PortChannel Interface in Static Mode config portchannel add PortChannel<interface> --static true # Command to Add Member Ports to PortChannel Interface config portchannel member add PortChannel<number> Ethernet<interface1> config portchannel member add PortChannel<number> Ethernet<interface2> # Save the Setting to config_db.json config save -y # Add Member Ports to PortChannel Interface config portchannel member add PortChannel<interface> Ethernet<interface1> config portchannel member add PortChannel<interface> Ethernet<interface2> # Command to Show Interface PortChannel show interfaces portchannel

CUMULUS

SONiC

FDB/MAC

# MAC Learning Configurations

# Command to Show MAC Addresses of Bridge

net show bridge macs

# Command to Set MAC Aging Address

net add bridge bridge ageing 600

# Command to Show MAC Entries Learned and Filtered Based on Hostname, MAC Address, etc.

# Display the MAC (FDB) Entries

show mac

# Display the MACs Learned on a Particular VLAN ID

show mac -v <vlan_value>

# Display the MACs Learned on a Particular Port

show mac -p Ethernet<interface>

# Clear the MAC (FBD) Table

sonic-clear FDB all

CUMULUS

SONiC

# Syntax for Displaying MAC Entries

netq show macs <mac> [vlan <1-4096>] [origin] [around <text-time>] [json]

# Syntax for Displaying MAC Entries on a Specific Host

netq <hostname> show macs <mac> [vlan <1-4096>] [origin | count] [around <text-time>] [json]

# Syntax for Displaying MAC Entries on a Specific Egress Port

netq <hostname> show macs egress-port <egress-port> <mac> [vlan <1-4096>] [origin] [around <text-time>] [json]

# Syntax for Displaying MAC History

netq [<hostname>] show mac-history <mac> [vlan <1-4096>] [diff] [between <text-time> and <text-endtime>] [listby <text-list-by>] [json]

# Syntax for Displaying MAC Commentary

netq [<hostname>] show mac-commentary <mac> vlan <1-4096> [between <text-time> and <text-endtime>] [json]

# Check MAC Aging Time

show mac aging-time

Multi-Chassis Link Aggregation Group (MC-LAG)

This is a pair of links that terminates on two cooperating switches and appears as an ordinary link aggregation group (LAG).

Layer 2 Multi Chassis LAG

Layer2 MultiChasis LAG

CUMULUS

SONiC

PortChannel (LACP) and Member

# Enable LACP

net add bond bond1 bond mode 802.3ad

net add bond bond2 bond mode 802.3ad

# Add Members

net add bond bond1 bond slaves swp1-4

net add bond bond2 bond slaves swp5-8

# Add Port Channel

config port channel add <PCH ID>

# Add Members

config port channel member add <PCH-ID> <member-port>

MC-LAG

# Command to add unique MLAG ID (clag-id) to each bond.

net add bond bond1 clag id 1

net add bond bond2 clag id 2

# Command to add the bonds to a bridge

net add bridge bridge ports bond1,bond2

# Command to set peer link IP address

net add clag peer sys-mac <MAC_ADDRESS_SYSTEM> interface <interface_name>1-4 primary backup-ip <IP_ADDRESS>

# Validate status of MC LAG config

net show clag

# Verify all MCLAG settings

clagctl params

# Monitor MCLAG services

systemctl status clagd.service

# MCLAG Domain & Peer Configuration

config interface ip add <VLAN ID> <SVI-IP>

config mclag add <mclag-id> <local-ip> <remote-ip> <peer-pch>

config mclag unique-ip add <peer-vlan>

# MCLAG Members

config mclag member add <mclag-id> <member-pch>

# MCLAG Show

show mclag brief

Show mac

VLAN

# Add VLAN members to bridge

net add bridge bridge ports <INTERFACE_NAME>1-2

net add bridge bridge vids <vlan-id1>,<vlan-id2>

# VLAN Configuration

config vlan add <id>

config vlan member add <vid> <pch-id>

SONiC Port Channel Configuration

Creating port channel on the MCLAG pair switches running SONiC

config portchannel add PortChannel01
config portchannel add PortChannel02
config portchannel add PortChannel03
config portchannel member add PortChannel01 Ethernet0
config portchannel member add PortChannel02 Ethernet1
config portchannel member add PortChannel03 Ethernet56
config portchannel member add PortChannel03 Ethernet60

Creating VLAN interface on MC LAG pair switches running SONiC

config vlan add 10
config vlan add 100
config vlan member add 10 PortChannel03
config vlan member add -u 100 PortChannel01
config vlan member add 100 PortChannel02
config vlan member add 100 PortChannel03

Configure MCLAG pair switches with domain ID and child member links

config mclag add 1 192.168.10.1 192.168.10.2 PortChannel03
config mclag unique-ip add Vlan10
config mclag member add 1 PortChannel01
config mclag member add 1 PortChannel02

SONiC configuration for MC LAG peer health check

config interface ip add Vlan10 192.168.10.1/24
config interface ip add Vlan10 192.168.10.2/24

SONiC Command to Display MC LAG operationalstatus

 show mclag brief

SONiC command to show MAC address learned for host traffic through member link interfaces

show mac
No.    Vlan  MacAddress         Port           Type
-----  ------  -----------------  -------------  -------
    1      10  68:21:5F:29:C0:D2  PortChannel03  Static
    2     100  B8:6A:97:19:BA:12  PortChannel01  Dynamic
    3     100  80:A2:35:5A:22:50  PortChannel02  Dynamic
Total number of entries 3

Layer 3 Multi Chassis LAG

Layer3 MultiChasis LAG

Command to create PortChannel on MC LAG Pair switches

config portchannel add PortChannel01
config portchannel add PortChannel02
config portchannel add PortChannel03
config portchannel member add PortChannel01 Ethernet0
config portchannel member add PortChannel02 Ethernet1
config portchannel member add PortChannel03 Ethernet56
config portchannel member add PortChannel03 Ethernet60

Commands to Create Port Channel IPs on MC LAG pair switches

config interface ip add PortChannel01 192.168.11.1/24
config interface ip add PortChannel02 192.168.12.1/24
config interface ip add PortChannel03 192.168.10.1/24

config interface ip add PortChannel01 192.168.11.1/24
config interface ip add PortChannel02 192.168.12.1/24
config interface ip add PortChannel03 192.168.10.2/24

Command to configure MCLAG on MC LAG pair switches (Domain ID, VLANs and MLAG members)

config mclag add 1 192.168.10.2 192.168.10.1
config mclag member add 1 PortChannel01
config mclag member add 1 PortChannel02

config mclag add 1 192.168.10.1 192.168.10.2
config mclag member add 1 PortChannel01
config mclag member add 1 PortChannel02

SONiC command to Configure IP for MCLAG Peer health check on MC LAG peers

config interface ip add Vlan10 192.168.10.1/24
config interface ip add Vlan10 192.168.10.2/24

Command to show MCLAG Status

MC1 switch configuration -

show interfaces portchannel
Flags: A - active, I - inactive, Up - up, Dw - Down, N/A - not available,
       S - selected, D - deselected, * - not synced
  No.  Team Dev       Protocol     Ports                          Oper Key  Admin Key    Fast Rate
-----  -------------  -----------  ---------------------------  ----------  -----------  -----------
  01  PortChannel01  LACP(A)(Up)  Ethernet0(S)                       101  auto         false
  02  PortChannel02  LACP(A)(Up)  Ethernet1(S)                       102  auto         false
  03  PortChannel03  LACP(A)(Up)  Ethernet60(S) Ethernet56(S)        103  auto         false

show mclag brief  
       Domain ID                    : 1
        Role                         : Active
        Session Status               : Up
       Peer Link Status             :
        Source Address               : 192.168.10.1
        Peer Address                 : 192.168.10.2
        Peer Link                    :
       Keepalive Interval           : 1 secs
        Session Timeout              : 15 secs
        System MAC                   : 00:a0:c9:00:00:00
       Number of MCLAG Interfaces   : 2
        MCLAG Interface              Local/Remote Status
        PortChannel01                Up/Up
        PortChannel02                Up/Up

MC2 switch configuration -

admin@sonic:~$ show interfaces portchannel
Flags: A - active, I - inactive, Up - up, Dw - Down, N/A - not available,
       S - selected, D - deselected, * - not synced
  No.  Team Dev       Protocol     Ports                          Oper Key  Admin Key    Fast Rate
-----  -------------  -----------  ---------------------------  ----------  -----------  -----------
  01  PortChannel01  LACP(A)(Up)  Ethernet0(S)                       101  auto         false
  02  PortChannel02  LACP(A)(Up)  Ethernet1(S)                       102  auto         false
  03  PortChannel03  LACP(A)(Up)  Ethernet60(S) Ethernet56(S)        103  auto         false
admin@sonic:~$ show mclag brief  
       Domain ID                    : 1
        Role                         : Standby
        Session Status               : Up
        Peer Link Status             :
        Source Address               : 192.168.10.2
        Peer Address                 : 192.168.10.1
        Peer Link                    :
        Keepalive Interval           : 1 secs
        Session Timeout              : 15 secs
        System MAC                   : 00:a0:c9:00:00:00
        Number of MCLAG Interfaces   : 2
        MCLAG Interface              Local/Remote Status
        PortChannel01                Up/Up
        PortChannel02                Up/Up

SONiC Command to verify ARP synchronization

mclagdctl dump arp -i 1
No.   IP                  MAC                 DEV                 Flag
1     192.168.12.2        80:a2:35:5a:22:50   PortChannel02       R
2     192.168.11.2        b8:6a:97:19:ba:12   PortChannel01       L

Link Layer Discovery protocol

LLDP is a standard link-layer discovery protocol which can broadcast its capability, IP address, ID, and interface name as TLVs (Type/Length/Value) in LLDP PDUs (Link Layer Discovery Protocol Data Units).

LLDP

CUMULUS

SONiC

# Command to Configure LLDP

sudo cat /etc/lldpd.conf

configure lldp tx-interval 40

configure lldp tx-hold 3

configure system interface pattern *,!eth0,swp*

# Command to Show All Neighbors on All Ports and Interfaces

sudo lldpcli show neighbors

# Command to Show LLDP Statistics for All Ports

sudo lldpcli show statistics

# Command to Show LLDP Running Configuration

sudo lldpcli show running-configuration

# Command to Enable / Disable LLDP globally

config feature state lldp enabled

config feature state lldp disabled

# Command to Configure LLDP information

config lldp global hello_timer <timer_value>

config lldp global management_ip <switch_mgmt_ip>

config lldp global system_description AS5835-Leaf1

config lldp global system_name <LEAF1>

# Command to validate LLDP status

show feature status lldp

show lldp table

show lldp neighbors

show lldp global

# Command to enable/disable LLDP over local interfaces

docker exec -i lldp lldpcli

configure ports Ethernet<interface> lldp status disable

configure ports Ethernet<interface> lldp status enable

< br />

Layer 3 Routing

Routed Interface

CUMULUS

SONiC

# Command to configure IP addresses for interface swp1

net add interface swp1 ip address <IP_ADDRESS>/<SUBNET>

# Command to bring up an interface or apply changes to an existing interface

sudo ifup <ifname>

# Command to bring down a single interface

sudo ifdown <ifname>

# Checking the Configuration

net show interface <INTERFACE_NAME>

net show interface alias

# Command to add a static route

net add routing route <NETWORK_ROUTE> <NEXT_HOP>

# Command to delete a static route

net delete routing route <NETWORK_ROUTE>

# Command to verify static routes

net show route static

# Command to add a Layer 3 Interface address on a physical interface

config interface ip add Ethernet<Number1> <IP_ADDRESS>

config interface ip add <vlan_number> <IP_ADDRESS>

# Example

config interface ip add Loopback<Number> 10.0.2.1/32

config interface ip add Ethernet0 172.16.10.1/31

config interface ip add Vlan100 18.0.0.1/24

# Command to create a sub-interface

config interface ip add Ethernet<interface_number>.<vlan-id> <IP_ADDRESS>

# Example

config interface ip add Ethernet0.10 192.168.10.2/24

# Validate sub-interface operational status

show subinterfaces status

# Command to add static routes

ip route <Network_IP_ADDRESS> <SUBNET_MASK> <NEXTHOP>

# Command to delete a static route

no ip route <Network_IP_ADDRESS> <SUBNET_MASK>

# Command to verify static routes

show ip route

SVI & DHCP Relay

CUMULUS	SONiC
SVI
# Create VLAN ID net add bridge bridge ports <interface_name>1-2 net add bridge bridge vids <vlan-id1>,<vlan-id2> # Create an interface binded to Layer3 VLAN net add vlan <VLAN-ID> vrf <VRF_NAME> # Commands configure an SVI using swp1, swp2 ports and VLAN ID net add bridge bridge ports<interface_name>1-2 net add vlan <VLAN-ID> ip address <IP_ADDRESS>/<SUBNET> # Command to Bring up and Bring down Layer 3 interface net add interface swp1 link down net del interface swp1 link down # Verify Layer3 interface net show interface swp1 # Verify IP routes ip route show	# Create VLANs config vlan add <vlan_value1> config vlan add <vlan_value2> # Show VLAN configuration show vlan config # Add Interface to VLAN in Tagged (Trunk) mode: config vlan member add <vlan_value1> Ethernet<interface1> config vlan member add <vlan_value2> Ethernet<interface2> # Inter-VLAN routing # Configure IP addresses on VLAN1 and VLAN2 config interface ip add Vlan <IP_ADDRESS1> config interface ip add Vlan <IP_ADDRESS2> # Example config interface ip add Vlan1 192.168.1.2/24 config interface ip add Vlan2 192.168.2.1/24 # Validate IP Interface show ip interface # Verify the Subinterface and VLAN status show vlan brief
DHCP Relay
# DHCP Relay Command # Command to set DHCP server IP address, layer 3 VLAN, and relay interfaces net add dhcp relay interface <relay_interface1> net add dhcp relay interface <relay_interface2> net add dhcp relay interface vlan<VLAN-ID> net add dhcp relay server <SERVER_IP> net pending net commit # Configure the IP address of the DHCP relay agent net add dhcp relay giaddr-interface <AGENT_INTERFACE> # Command to allocate IP to relay Agent net add dhcp relay giaddr-interface <interface_name> <IP_ADDRESS>	# SONiC Command to enable DHCP relay config feature state dhcp_relay enabled # Enable DHCP relay on VLAN number config vlan dhcp_relay add <vlan_number> <IP_ADDRESS> # Enable DHCP relay on Loopback interface config vlan dhcp_relay src_intf add <vlan_number> Loopback0 # Example config vlan dhcp_relay add 10 192.168.20.100 config vlan dhcp_relay src_intf add 10 Loopback0

BGP Routing

CUMULUS

SONiC

# Command to configure BGP routing # Command to set BGP node by assigning an ASN

net add bgp autonomous-system <ASN_NUMBER>

# Command to set auto BGP to assign an ASN automatically

net add bgp auto leaf

# Command to assign Router-ID

net add bgp router-id <SYSTEM_LOOPBACK_IP>

# Command to specify BGP neighbor

net add bgp neighbor <NEIGHBOR_IP_ADDRESS> remote-as external

# Command to advertise specifics using network

net add bgp ipv4 unicast network <LOOPBACK_IP>

net add bgp ipv4 unicast network <NETWORK_IP_ADDRESS>

# Example

net add bgp autonomous-system 65101

net add bgp router-id 10.10.10.1

net add bgp neighbor 10.0.1.0 remote-as external

net add bgp ipv4 unicast network 10.10.10.1/32

net add bgp ipv4 unicast network 10.1.10.0/24

# Command to show BGP routes summary # Syntax

net show bgp summary

net show bgp ipv4 unicast summary

net show bgp ipv4 unicast

net show bgp ipv4 unicast <network_address>

net show bgp neighbor <interface_name>

# vtysh Sonic command to configure BGP routing

router bgp <ASN_NUMBER>

bgp router-id <System_loopback_IP>

no bgp ebgp-requires-policy

bgp bestpath as-path multipath-relax

neighbor FABRIC peer-group

neighbor FABRIC capability extended-nexthop

neighbor <Neighbor_IP> remote-as <REMOTE_ASN_NUMBER>

neighbor <Neighbor_IP> peer-group FABRIC

# Example BGP routing configuration

router bgp 65001

bgp router-id 10.0.2.1

no bgp ebgp-requires-policy

bgp bestpath as-path multipath-relax

neighbor FABRIC peer-group

neighbor FABRIC capability extended-nexthop

neighbor 172.16.10.0 remote-as 2001

neighbor 172.16.10.0 peer-group FABRIC

neighbor 172.16.10.8 remote-as 2002

neighbor 172.16.10.8 peer-group FABRIC

neighbor 192.168.3.1 remote-as 2003

neighbor 192.168.3.1 peer-group FABRIC

# Command to show BGP routes summary

show ip bgp summary

show ip bgp neighbors

show ip bgp network

show ipv6 bgp summary

show ipv6 bgp neighbors

show ipv6 bgp network

OSPF Routing

CUMULUS

SONiC

OSPF Routing

# Configure OSPF routing # Configure the unnumbered interface

net add loopback lo ip address <system_loopback_ip>

net add interface <interface_number> ip address <ip_address>

net add ospf router-id <system_loopback_ip>

net add ospf network <network_address> area <AREA_NUMBER>

net add ospf passive-interface <interface_name1>

net add ospf passive-interface <interface_name2>

# Command to configure OSPF passive interface

net add ospf passive-interface default

net del ospf passive-interface <interface_name>

# Configure to set network type to point-to-point

net add interface <interface_name> ospf network point-to-point

net add interface <interface_name> ospf hello-interval <hello-interval-time-secs>

net add interface <interface_name> ospf dead-interval <dead-interval-time-secs>

# Configure OSPF interface with priority

net add interface <interface_name> ospf priority <priority_number>

net add interface <interface_name> ospf message-digest-key 1 md5 <KEY_VALUE>

net add interface <interface_name> ospf authentication message-digest

# Command to create a summary route for all the routes in a network address range in a specific area <Area_number>

sudo vtysh

router ospf

area <area_number> range <network_address>

# Command to verify OSPF neighbor

net show ospf neighbor

net show route ospf

# Example Configuration - OSPF Routing

net add loopback lo ip address 10.10.10.1/32

net add interface <interface_name> ip address <address_ip>

net add ospf router-id 10.10.10.1

net add ospf network 10.10.10.1/32 area 0

net add ospf network 10.0.1.0/31 area 0

net add ospf passive-interface swp1

net add ospf passive-interface swp2

# Configure the unnumbered interface

net add loopback lo ip address 10.10.10.1/32

net add interface swp51 ip address 10.10.10.1/32

# Configure OSPF routing # Syntax

router ospf

ospf router-id <router-id>

network <Network_address> area <Area_number>

network <Network_address1> area <Area_number1>

network <Network_address2> area <Area_number2>

# Command to set OSPF time intervals

interface Ethernet<interface>

ip ospf hello-interval <hello-interval-time-secs>

ip ospf dead-interval <dead-interval-time-secs>

router ospf

area <aread_number> authentication

# Command to set OSPF authentication key

interface Ethernet<interface>

ip ospf authentication

ip ospf authentication-key <key>

# Command to set OSPF MD5 Authentication

router ospf

area 0 authentication message-digest

interface Ethernet<interface>

ip ospf message-digest-key <key> md5 <key>

# Command to configure OSPF Virtual links

router ospf

area <area_number> virtual-link <System_loopback>

# Command to verify OSPF IP routes learned

show ip route

# Configuration OSPF Routing - Example

router ospf

ospf router-id 1.1.1.1

network 10.0.0.0/31 area 0

network 192.168.10.0/24 area 0

network 192.168.20.0/24 area 0

network 192.168.30.0/24 area 0

# Enable OSPF hello timers under the interface - Example

interface Ethernet56

ip ospf hello-interval 20

ip ospf dead-interval 20

# Enable OSPF Authentication globally - Example

router ospf

area 0 authentication

# Enable OSPF Authentication over the interface - Example

interface Ethernet56

ip ospf authentication

ip ospf authentication-key 123

# Enable OSPF MD5 Key - Example

router ospf

area 0 authentication message-digest

interface Ethernet56

ip ospf message-digest-key 1 md5 123

# Verify IP routing Table - Example

show ip route

# Configure OSPF virtual links - Example

router ospf

area 1 virtual-link 3.3.3.3

router ospf

area 1 virtual-link 2.2.2.2

OSPF Routing Image 2

AS7326-56X-OS1 Configuration VLAN and IP Configuration

config interface ip add Loopback0 1.1.1.1/32
config vlan member add 10 Ethernet0
config vlan member add 20 Ethernet0
config vlan member add 30 Ethernet0
config interface ip add Ethernet0.10 192.168.10.1/24
config interface ip add Ethernet0.20 192.168.20.1/24
config interface ip add Ethernet0.30 192.168.30.1/24
config interface ip add Ethernet56 10.0.0.0/31

OSPF Configuration

admin@sonic:~$ vtysh
sonic(config)# router ospf
sonic(config-router)# network 10.0.0.0/31 area 0
sonic(config-router)# network 192.168.10.0/24 area 0
sonic(config-router)# network 192.168.20.0/24 area 0
sonic(config-router)# network 192.168.30.0/24 area 0

OSPF Routing Verification Command

sonic# show ip ospf neighbor
Neighbor ID     Pri State           Dead Time Address         Interface            RXmtL RqstL DBsmL
192.168.25.1      1 Full/DR           31.440s 10.0.0.1        Ethernet56:10.0.0.0      0     0     0

sonic# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route
K>* 0.0.0.0/0 [0/202] via 188.188.1.1, eth0, 00:07:45
C>* 1.1.1.1/32 is directly connected, Loopback0, 00:07:25
O   10.0.0.0/31 [110/10] is directly connected, Ethernet56, 00:06:42
C>* 10.0.0.0/31 is directly connected, Ethernet56, 00:07:25
C>* 188.188.0.0/16 is directly connected, eth0, 00:07:46
O>* 192.168.5.0/24 [110/20] via 10.0.0.1, Ethernet56, 00:06:32
O   192.168.10.0/24 [110/10] is directly connected, Vlan10, 00:04:54
C>* 192.168.10.0/24 is directly connected, Vlan10, 00:07:24
O>* 192.168.15.0/24 [110/20] via 10.0.0.1, Ethernet56, 00:06:32
O   192.168.20.0/24 [110/10] is directly connected, Vlan20, 00:04:50
C>* 192.168.20.0/24 is directly connected, Vlan20, 00:07:24
O>* 192.168.25.0/24 [110/20] via 10.0.0.1, Ethernet56, 00:06:32
O   192.168.30.0/24 [110/10] is directly connected, Vlan30, 00:04:47
C>* 192.168.30.0/24 is directly connected, Vlan30, 00:07:24

AS7326-56X-OS2 Configuration VLAN and IP Configuration

config interface ip add Loopback0 2.2.2.2/32
config vlan member add 5 Ethernet0
config vlan member add 15 Ethernet0
config vlan member add 25 Ethernet0
config interface ip add Ethernet0.5 192.168.51/24
config interface ip add Ethernet0.15 192.168.15.1/24
config interface ip add Ethernet0.25 192.168.25.1/24
config interface ip add Ethernet56 10.0.0.1/31

OSPF Configuration

admin@sonic:~$ vtysh
sonic(config)# router ospf
sonic(config-router)# network 10.0.0.0/31 area 0
sonic(config-router)# network 192.168.5.0/24 area 0
sonic(config-router)# network 192.168.15.0/24 area 0
sonic(config-router)# network 192.168.25.0/24 area 0

OSPF Routing Verification Command

OS2:
sonic# show ip ospf neighbor
Neighbor ID     Pri State           Dead Time Address         Interface            RXmtL RqstL DBsmL
188.188.98.39     1 Full/Backup       33.721s 10.0.0.0        Ethernet56:10.0.0.1      0     0     0

sonic# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued route, r - rejected route
K>* 0.0.0.0/0 [0/202] via 188.188.1.1, eth0, 02:15:38
C>* 2.2.2.2/32 is directly connected, Loopback0, 02:15:18
O   10.0.0.0/31 [110/10] is directly connected, Ethernet56, 00:08:47
C>* 10.0.0.0/31 is directly connected, Ethernet56, 00:08:47
C>* 188.188.0.0/16 is directly connected, eth0, 02:15:39
O   192.168.5.0/24 [110/10] is directly connected, Vlan5, 00:35:34
C>* 192.168.5.0/24 is directly connected, Vlan5, 00:35:34
O>* 192.168.10.0/24 [110/20] via 10.0.0.0, Ethernet56, 00:06:14
O   192.168.15.0/24 [110/10] is directly connected, Vlan15, 00:35:34
C>* 192.168.15.0/24 is directly connected, Vlan15, 00:35:34
O>* 192.168.20.0/24 [110/20] via 10.0.0.0, Ethernet56, 00:06:10
O   192.168.25.0/24 [110/10] is directly connected, Vlan25, 00:35:34
C>* 192.168.25.0/24 is directly connected, Vlan25, 00:35:34
O>* 192.168.30.0/24 [110/20] via 10.0.0.0, Ethernet56, 00:06:07

VRF Routing

CUMULUS

SONiC

# Command to configure VRRP

set ip routing enable true

set ip vrf <VRF-ID> description “Description_value”

# Command to bind the Layer 3 VLAN interface to the VRF.

set vlan-interface interface vlan<vlan-id> vrf <VRF-ID>

# Command to add a static route entry into the VRF.

set protocols static vrf <VRF-ID> route <IPV4_address> next-hop <IPV4_address>

set protocols static vrf <VRF-ID> route <IPV6_address> next-hop <IPV6_address>

# Command to validate and show VRF instances created

run show vrf

# SONiC command to create a VRF

config vrf add

config vrf add <vrf-name>

config vrf del <vrf-name>

config vrf add_vrf_vni_map <vrf-name> <vni>

# Command to bind Layer 3 VLAN interface to the VRF

config vrf add <VRF-ID>

config vxlan add vtep <VTEP_ENDPOINT_IP>

config vxlan evpn_nvo add evpnnvo vtep

config vrf add_vrf_vni_map <VRF-ID> <VNI_VALUE>

# Command to unbind the Layer 3 VLAN interface from the VRF

config vrf del_vrf_vni_map <vrf-name>

# Command to configure a static route entry into the VRF

ip route <A.B.C.D/M> <A.B.C.D> nexthop-vrf <vrf-name>

# Command to import VRF table into the default routing table

import vrf default

# Command to add BGP routing entry with VRF and import route leaking policy into VRF routing table

router bgp <AS_NUMBER> vrf <VRF-ID>

address-family ipv4 unicast

router bgp <AS_NUMBER> vrf <VRF-ID>

address-family ipv4 unicast

SONIC - VRF Routing Create VRF instance

admin@sonic:~$ config vrf add Vrf_01

Binding the Ethernet0 to VRF instance.

admin@sonic:~$ config interface vrf bind Ethernet0 Vrf_01

Checking the VRF

admin@sonic:~$ show vrf
VRF     Interfaces
------  ------------
Vrf_01  Ethernet0
admin@sonic:~$ show ip interfaces
Interface   Master  IPv4 address/mask   Admin/Oper  BGP Neighbor    Neighbor IP
-----------  --------  -------------------  ------------  --------------  -------------
Ethernet0   Vrf_01  192.168.1.1/24      up/up       N/A             N/A
Loopback0           10.1.0.1/32         up/up       N/A             N/A
docker0                240.127.1.1/24       up/down     N/A             N/A
eth0                188.188.97.31/16    up/up       N/A             N/A
lo                  127.0.0.1/8         up/up       N/A             N/A

Checking the routing table.

admin@sonic:~$ show ip route vrf Vrf_01
Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route
VRF Vrf_01:
C>* 192.168.1.0/24 is directly connected, Ethernet0, 00:02:37
admin@sonic:~$ show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route
VRF Vrf_01:
C>* 192.168.1.0/24 is directly connected, Ethernet0, 00:00:31
Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route
K>* 0.0.0.0/0 [0/202] via 188.188.1.1, eth0, 00:15:16
C>* 10.1.0.1/32 is directly connected, Loopback0, 00:15:16
C>* 188.188.0.0/16 is directly connected, eth0, 00:15:16

Management VRF Create Management VRF

admin@sonic:~$ config vrf add mgmt

Checking the Management VRF

admin@sonic:~$ show mgmt-vrf
ManagementVRF : Enabled
Management VRF interfaces in Linux:
128: mgmt: <NOARP,MASTER,UP,LOWER_UP> mtu 65536 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 52:2f:cc:b8:28:b5 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 1500
    vrf table 5000 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master mgmt state UP mode DEFAULT group default qlen 1000
    link/ether 80:a2:35:4f:4f:40 brd ff:ff:ff:ff:ff:ff
129: lo-m: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue master mgmt state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 0a:25:2e:1f:32:90 brd ff:ff:ff:ff:ff:ff
admin@sonic:~$ show ip interfaces
Interface   Master  IPv4 address/mask   Admin/Oper  BGP Neighbor    Neighbor IP
-----------  --------  -------------------  ------------  --------------  -------------
Ethernet0   Vrf_01  192.168.1.1/24      up/up       N/A             N/A
Loopback0           10.1.0.1/32         up/up       N/A             N/A
docker0             240.127.1.1/24      up/down     N/A             N/A
eth0        mgmt    188.188.97.31/16    up/up       N/A             N/A
lo                  127.0.0.1/8         up/up       N/A             N/A
lo-m        mgmt    127.0.0.1/8         up/up       N/A             N/A

Checking the routing table.

admin@sonic:~$ show ip route vrf mgmt
Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route
VRF mgmt:
K>* 0.0.0.0/0 [0/0] via 188.188.1.1, eth0, 00:12:12
C>* 188.188.0.0/16 is directly connected, eth0, 00:12:12
admin@sonic:~$ show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route
VRF Vrf_01:
C>* 192.168.1.0/24 is directly connected, Ethernet0, 00:01:04
Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route
C>* 10.1.0.1/32 is directly connected, Loopback0, 00:01:05
Codes: K - kernel route, C - connected, S - static, R - RIP,
    O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
    T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
    F - PBR, f - OpenFabric,
    > - selected route, * - FIB route, q - queued route, r - rejected route
VRF mgmt:
K>* 0.0.0.0/0 [0/0] via 188.188.1.1, eth0, 00:01:21
C>* 188.188.0.0/16 is directly connected, eth0, 00:01:21

L2-VXLAN Asymmetric IRB Configuration

CUMULUS

SONiC

# Configure VLAN ID, L3 VLAN interfaces, loopback interfaces, and IP addressing. # Syntax

net add vlan <VLAN-ID> vrf <VRF_NAME>

# Command to create two unique VXLAN devices-

net add interface <interface_name1> bridge access <vlan1>

net add interface <interface_name2> bridge access <vlan2>

net add vxlan vni<number1> vxlan id <vlan1>

net add vxlan vni<number2> vxlan id <vlan2>

# Configure VXLAN VNI and map VNI IDs to VLAN IDs.

net add bridge bridge ports vni<number1>,vni<number2>

net add bridge bridge vids <vlan1>,<vlan2>

net add vxlan vni<number1> bridge access <vlan1>

net add vxlan vni<number2> bridge access <vlan2>

# Command to configure VXLAN tunnels with local and remote VTEP tunnel IP

net add loopback lo vxlan local-tunnelip <local_ip>

net add vxlan vni-<vni_value> vxlan remoteip <remote_ip>

# Configure and advertise BGP L2 EVPN Routes

net add bgp autonomous-system <ASN_NUMBER>

net add bgp l2vpn evpn neighbor <NEIGHBOR_IP> remote-as internal

net add bgp l2vpn evpn neighbor <NEIGHBOR_IP> activate

net add bgp l2vpn evpn advertise-all-vni

# Command to show VXLAN traffic stats

net show bgp l2vpn evpn summary

net show bgp l2vpn evpn vni

net show evpn vni

net show evpn mac vni <VNI_VALUE>

net show evpn mac vni all

net show evpn next-hops vni all

nv show nve vxlan

# Configure LoopBack, VLAN IDs, and IP addressing # Syntax

config vlan add <VLAN-ID>

config vlan member add <VLAN-ID> Ethernet<interface1>

config interface ip add Loopback0 <SYSTEM_LOOPBACK>

# Configure BGP routing

router bgp <LOCAL_AS_NUMBER>

bgp router-id <SYSTEM_LOOPBACK>

neighbor <ebgp_neighbor_ip> remote-as <REMOTE_AS_NUMBER>

address-family ipv4

network <Network_prefix_advertised>

# Configure VXLAN VNI and map VNI IDs to VLAN IDs

config vxlan add vtep <SOURCE_VTEP_IP>

config vxlan evpn_nvo add nvo vtep

config vxlan map add vtep <VLAN-ID> <VNI_VALUE>

config vxlan add vtep <DEST_VTEP_IP>

config vxlan evpn_nvo add nvo vtep

config vxlan map add vtep <VLAN-ID> <VNI_VALUE>

# Configure and advertise BGP L2 EVPN Routes

router bgp <LOCAL_AS_NUMBER>

address-family l2vpn evpn

neighbor <ebgp_neighbor_ip> activate

Advertise-all-vni

# Show VXLAN tunnels, interfaces, and EVPN route details

show ip route

show vxlan interface

show vxlan vlanvnimap

show vxlan tunnel

show vxlan remotevtep

show evpn vni detail

L2-VXLAN EVPN topology between edge core SONiC switches

Sample SONiC- L2VXLAN EVPN Asymmetric IRB Step 1: Configure IP address to Loopback0 of both switches.

AS7326-56X:

admin@AS7326-56X:~$ config interface ip remove Loopback0 10.1.0.1/32   
admin@AS7326-56X:~$ config interface ip add Loopback0 1.1.1.1/32

AS5835-54X:

admin@AS5835-54X:~$ config interface ip remove Loopback0 10.1.0.1/32   
admin@AS5835-54X:~$ config interface ip add Loopback0 2.2.2.2/32

Step 2: Establish BGP Session between Ethernet52 and announce the network. AS7326-56X:

admin@AS7326-56X:~$ vtysh
Hello, this is FRRouting (version 7.2.1-sonic).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
AS7326-56X# configure terminal
AS7326-56X(config)# router bgp 65100
AS7326-56X(config-router)# bgp router-id 1.1.1.1
AS7326-56X(config-router)# neighbor 10.0.0.1 remote-as 65100
AS7326-56X(config-router)# address-family ipv4
AS7326-56X(config-router-af)# network 1.1.1.1/32
AS7326-56X(config-router-af)# end
AS7326-56X# exit
AS5835-54X:
admin@AS5835-54X:~$ vtysh
Hello, this is FRRouting (version 7.2.1-sonic).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
AS5835-54X# configure terminal
AS5835-54X(config)# router bgp 65100
AS5835-54X(config-router)# bgp router-id 2.2.2.2
AS5835-54X(config-router)# neighbor 10.0.0.0 remote-as 65100
AS5835-54X(config-router)# address-family ipv4
AS5835-54X(config-router-af)# network 2.2.2.2/32
AS5835-54X(config-router-af)# end
AS5835-54X# exit

Step 3. Create Vxlan AS7326-56X:

admin@AS7326-56X:~$ config vxlan add vtep 1.1.1.1
admin@AS7326-56X:~$ config vxlan evpn_nvo add nvo vtep
admin@AS7326-56X:~$ config vxlan map add vtep 30 3000

AS5835-54X:

admin@AS5835-54X:~$ config vxlan add vtep 2.2.2.2
admin@AS5835-54X:~$ config vxlan evpn_nvo add nvo vtep
admin@AS5835-54X:~$ config vxlan map add vtep 30 3000
Note :
VNI (VxLAN Network Identifier) : virtual extension of VLAN over IP network.
VTEP (VXLAN Tunnel End Point) : an entity that originates and/or terminates VXLAN tunnels which is specified by a source IP address.
Only one VTEP is allowed on one device. Please use loopback IP address for VTEP's IP address.
NVO (Network Virtualization Overlay)
Only one NVO is allowed on one device.
VNI (VxLAN Network Identifier) : virtual extension of VLAN over IP network.

Step 4: Advertise L2VPN EVPN routes. AS7326-56X:

admin@AS7326-56X:~$ vtysh
Hello, this is FRRouting (version 7.2.1-sonic).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
AS7326-56X#
AS7326-56X# configure terminal
AS7326-56X(config)# router bgp 65100
AS7326-56X(config-router)# address-family l2vpn evpn
AS7326-56X(config-router-af)# neighbor 10.0.0.1 activate
AS7326-56X(config-router-af)# advertise-all-vni

AS5835-54X:

admin@AS5835-54X:~$ vtysh
Hello, this is FRRouting (version 7.2.1-sonic).
AS5835-54X# 
AS5835-54X# configure terminal
AS5835-54X(config)# router bgp 65100
AS5835-54X(config-router)# address-family l2vpn evpn
AS5835-54X(config-router-af)# neighbor 10.0.0.0 activate
AS5835-54X(config-router-af)# advertise-all-vni

Check VxLAN interface configuration.AS7326-56X:

admin@AS7326-56X:~$ show vxlan interface 
VTEP Information:
VTEP Name : vtep, SIP : 1.1.1.1
Source interface : Loopback0
AS5835-54X:
admin@AS5835-54X:~$ show vxlan interface 
VTEP Information:
VTEP Name : vtep, SIP : 2.2.2.2
Source interface : Loopback0

Check vxlan and VLAN mapping.AS7326-56X:

admin@AS7326-56X:~$ show vxlan vlanvnimap
+--------+-------+
| VLAN   |   VNI |
+========+=======+
| Vlan30 |  3000 |
+--------+-------+
Total count : 1

AS5835-54X:

admin@AS5835-54X:~$ show vxlan vlanvnimap
+--------+-------+
| VLAN   |   VNI |
+========+=======+
| Vlan30 |  3000 |
+--------+-------+
Total count : 1

Check the status for Vxlan tunneling. AS7326-56X:(202111.3)

admin@AS7326-56X:~$ show vxlan tunnel
vxlan tunnel name    source ip    destination ip    tunnel map name    tunnel map mapping(vni -> vlan)
-------------------  -----------  ----------------  -----------------  ---------------------------------
vtep                 1.1.1.1                       map_3000_Vlan30    3000 -> Vlan30
Total count : 1

AS7326-56X:(202111.3)

admin@AS7326-56X:~$ show vxlan remotevtep
+---------+---------+-------------------+--------------+
| SIP   | DIP   | Creation Source   | OperStatus   |
+=========+=========+===================+==============+
| 1.1.1.1 | 2.2.2.2 | EVPN              | oper_up   |
+---------+---------+-------------------+--------------+
Total count : 1

AS5835-54X:(202111.3)

admin@AS5835-54X:~$ show vxlan tunnel
vxlan tunnel name    source ip    destination ip    tunnel map name    tunnel map mapping(vni -> vlan)
-------------------  -----------  ----------------  -----------------  ---------------------------------
vtep                 2.2.2.2                      map_3000_Vlan30    3000 -> Vlan30
Total count : 1

AS5835-54X:(202111.3)

admin@AS5835-54X:~$ show vxlan remotevtep
| SIP   | DIP   | Creation Source   | OperStatus   |
+=========+=========+===================+==============+
| 2.2.2.2 | 1.1.1.1 | EVPN              | oper_up   |
+---------+---------+-------------------+--------------+
Total count : 1

Check the Mac learning. AS7326-56X:(202111.3)

admin@AS7326-56X:~$ show mac
  No.   Vlan  MacAddress        Port                Type
-----  ------  -----------------  ------------------  -------
    1   30  8C:EA:1B:30:DA:50  VxLAN DIP: 2.2.2.2  Static
    2   30  8C:EA:1B:30:DA:4F  Ethernet0        Dynamic
Total number of entries 2

AS7326-56X(202111.3)

admin@AS7326-56X:~$ show mac
  No.   Vlan  MacAddress        Port    Type
-----  ------  -----------------  ---------  -------
    1   30  8C:EA:1B:30:DA:4F  Ethernet0  Dynamic
Total number of entries 1
admin@AS7326-56X:~$ show vxlan remotemac all
+--------+-------------------+--------------+-------+-------+---------+
| VLAN   | MAC              | RemoteVTEP   | ESI   |   VNI | Type   |
+========+===================+==============+=======+=======+=========+
| Vlan30 | 8c:ea:1b:30:da:50 | 2.2.2.2      |       |  3000 | dynamic |
+--------+-------------------+--------------+-------+-------+---------+
Total count : 1
Note.
"8C:EA:1B:30:DA:50" is synced from remote vtep(2.2.2.2).
"8C:EA:1B:30:DA:4F" is learned locally.

AS5835-54X:(202111.3)

admin@AS5835-54X:~$ show mac
  No.   Vlan  MacAddress        Port                Type
-----  ------  -----------------  ------------------  -------
    1   30  8C:EA:1B:30:DA:50  Ethernet0        Dynamic
    2   30  8C:EA:1B:30:DA:4F  VxLAN DIP: 1.1.1.1  Static
Total number of entries 2

AS5835-54X:(202111.3)

admin@AS5835-54X:~$ show mac
  No.   Vlan  MacAddress        Port    Type
-----  ------  -----------------  ---------  -------
    1   30  8C:EA:1B:30:DA:50  Ethernet0  Dynamic
Total number of entries 1
admin@AS5835-54X:~$ show vxlan remotemac all
+--------+-------------------+--------------+-------+-------+---------+
| VLAN   | MAC              | RemoteVTEP   | ESI   |   VNI | Type   |
+========+===================+==============+=======+=======+=========+
| Vlan30 | 8c:ea:1b:30:da:4f | 1.1.1.1      |       |  3000 | dynamic |
+--------+-------------------+--------------+-------+-------+---------+
Total count : 1

Check IPv4 BGP session AS7326-56X:

AS7326-56X# show bgp ipv4 summary
IPv4 Unicast Summary:
BGP router identifier 1.1.1.1, local AS number 65100 vrf-id 0
BGP table version 6
RIB entries 3, using 552 bytes of memory
Peers 1, using 20 KiB of memory
Neighbor        V       AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
10.0.0.1        4   65100   80      85      0   0   0 01:01:28          1
Total number of neighbors 1

AS5835-54X:

AS5835-54X# show bgp ipv4 summary
IPv4 Unicast Summary:
BGP router identifier 2.2.2.2, local AS number 65100 vrf-id 0
BGP table version 6
RIB entries 3, using 552 bytes of memory
Peers 1, using 20 KiB of memory
Neighbor        V       AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
10.0.0.0        4   65100   79      79      0   0   0 01:01:28          1
Total number of neighbors 1

Check L2EVPN BGP session AS7326-56X:

AS7326-56X# show bgp l2vpn evpn summary
BGP router identifier 1.1.1.1, local AS number 65100 vrf-id 0
BGP table version 0
RIB entries 3, using 552 bytes of memory
Peers 1, using 20 KiB of memory
Neighbor        V       AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
10.0.0.1        4   65100   82      87      0   0   0 01:03:43          3
Total number of neighbors 1
AS5835-54X:
AS5835-54X# show bgp l2vpn evpn summary
BGP router identifier 2.2.2.2, local AS number 65100 vrf-id 0
BGP table version 0
RIB entries 3, using 552 bytes of memory
Peers 1, using 20 KiB of memory
Neighbor        V       AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
10.0.0.0        4   65100   81      81      0   0   0 01:03:43          3
Total number of neighbors 1

Check underlay routing AS7326-56X:

AS7326-56X# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
K>* 0.0.0.0/0 [0/202] via 188.188.1.1, eth0, 00:49:45
C>* 1.1.1.1/32 is directly connected, Loopback0, 00:49:14
B>* 2.2.2.2/32 [200/0] via 10.0.0.1, Ethernet52, 00:42:04
C>* 10.0.0.0/31 is directly connected, Ethernet52, 00:49:13
C>* 188.188.0.0/16 is directly connected, eth0, 00:49:45

AS5835-54X:

AS5835-54X# show ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
K>* 0.0.0.0/0 [0/0] via 188.188.1.1, eth0, 00:49:57
B>* 1.1.1.1/32 [200/0] via 10.0.0.0, Ethernet52, 00:42:25
C>* 2.2.2.2/32 is directly connected, Loopback0, 00:46:34
C>* 10.0.0.0/31 is directly connected, Ethernet52, 00:46:33
C>* 188.188.0.0/16 is directly connected, eth0, 00:49:57

Check Vxlan VNI status AS7326-56X:

AS7326-56X# show evpn vni detail 
VNI: 3000
Type: L2
Tenant VRF: default
VxLAN interface: vtep-30
VxLAN ifIndex: 68
Local VTEP IP: 1.1.1.1
Mcast group: 0.0.0.0
Remote VTEPs for this VNI:
2.2.2.2 flood: HER
Number of MACs (local and remote) known for this VNI: 3
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 3
Advertise-gw-macip: No

AS5835-54X:

AS5835-54X# show evpn vni detail 
VNI: 3000
Type: L2
Tenant VRF: default
VxLAN interface: vtep-30
VxLAN ifIndex: 66
Local VTEP IP: 2.2.2.2
Mcast group: 0.0.0.0
Remote VTEPs for this VNI:
1.1.1.1 flood: HER
Number of MACs (local and remote) known for this VNI: 3
Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 3
Advertise-gw-macip: No

Check the evpn mac learning AS7326-56X:

AS7326-56X# show evpn mac vni all
VNI 3000 #MACs (local and remote) 3
MAC             Type   Intf/Remote VTEP     VLAN  Seq #'s
8c:ea:1b:30:da:50 remote 2.2.2.2                    1/0
8c:ea:1b:30:da:4f local  Ethernet0          30  0/0

AS5835-54X:

AS5835-54X# show evpn mac vni all
VNI 3000 #MACs (local and remote) 3MAC              Type   Intf/Remote VTEP     VLAN  Seq #'s
8c:ea:1b:30:da:50 local  Ethernet0          30  0/0
8c:ea:1b:30:da:4f remote 1.1.1.1                    1/0

Check the type 2 EVPN route AS7326-56X:

AS7326-56X# show bgp l2vpn evpn route type macip 
BGP table version is 2, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
                   Network          Next Hop            Metric LocPrf Weight Path
            Extended Community
Route Distinguisher: 1.1.1.1:2
*> [2]:[0]:[48]:[8c:ea:1b:cc:10:a4]
                    1.1.1.1                            32768 i
                    ET:8 RT:65100:3000
Route Distinguisher: 2.2.2.2:2
*>i[2]:[0]:[48]:[80:a2:35:5a:22:50]
                    2.2.2.2                       100      0 i
                    RT:65100:3000 ET:8
Displayed 2 prefixes (2 paths) (of requested type)

AS5835-54X:

AS5835-54X# show bgp l2vpn evpn route type macip 
BGP table version is 2, local router ID is 2.2.2.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
   Network          Next Hop            Metric LocPrf Weight Path
                    Extended Community
Route Distinguisher: 1.1.1.1:2
*>i[2]:[0]:[48]:[8c:ea:1b:cc:10:a4]
                    1.1.1.1                       100      0 i
                    RT:65100:3000 ET:8
Route Distinguisher: 2.2.2.2:2
*> [2]:[0]:[48]:[80:a2:35:5a:22:50]
                    2.2.2.2                            32768 i
                    ET:8 RT:65100:3000
Displayed 2 prefixes (2 paths) (of requested type)

Check the type 3 EVPN route AS7326-56X:

AS7326-56X# show bgp l2vpn evpn route type multicast 
BGP table version is 3, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
Extended Community
Route Distinguisher: 1.1.1.1:2
*> [3]:[0]:[32]:[1.1.1.1]
1.1.1.1 32768 i
ET:8 RT:65100:3000
Route Distinguisher: 2.2.2.2:2
*>i[3]:[0]:[32]:[2.2.2.2]
2.2.2.2 100 0 i
RT:65100:3000 ET:8
Displayed 2 prefixes (2 paths) (of requested type)

AS5835-54X:

AS5835-54X# show bgp l2vpn evpn route type multicast 
BGP table version is 3, local router ID is 2.2.2.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
Network Next Hop Metric LocPrf Weight Path
Extended Community
Route Distinguisher: 1.1.1.1:2
*>i[3]:[0]:[32]:[1.1.1.1]
1.1.1.1 100 0 i
RT:65100:3000 ET:8
Route Distinguisher: 2.2.2.2:2
*> [3]:[0]:[32]:[2.2.2.2]
2.2.2.2 32768 i
ET:8 RT:65100:3000

L3-VXLAN Symmetric IRB Configuration

CUMULUS

SONiC

# Configure physical interfaces, VLAN interfaces, and assign VLAN IDs and IP addresses

net add vlan <VLAN-ID> vrf <VRF_NAME>

# Create an L3 VNI in vrf1. # Configure a Per-tenant VXLAN Interface

net add vxlan vni-<VNI_NUMBER> vxlan id <VLAN-ID>

net add vxlan vni-<VNI_NUMBER> bridge access <VLAN-ID>

net add vxlan vni-<VNI_NUMBER> vxlan local-tunnelip <LOCAL_IP>

net add vxlan vni-<VNI_NUMBER> vxlan remoteip <REMOTE_IP>

net add bridge bridge ports vni<VNI_NUMBER>

# Configure an SVI for the Layer 3 VNI

net add vlan <VLAN-ID> vrf <VRF_NAME>

# Configure the VRF to Layer 3 VNI Mapping

net add vrf <VRF_NAME> vni<VNI_NUMBER>

# Configure and BGP EVPN routes

net add bgp vrf <VRF_NAME> l2vpn evpn advertise ipv4 unicast

# Configure and advertise BGP L2 EVPN Routes

net add bgp autonomous-system <ASN_NUMBER>

net add bgp l2vpn evpn neighbor <NEIGHBOR_IP> remote-as internal

net add bgp l2vpn evpn neighbor <NEIGHBOR_IP> activate

net add bgp l2vpn evpn advertise-all-vni

# VTYSH command for BGP L2VPN EVPN command

router bgp <ASN_NUMBER> vrf <VRF_NAME>

address-family l2vpn evpn

advertise ipv4 unicast

# Command to show BGP L2VPN EVPN VNI routes # NCLU Command

net show bgp l2vpn evpn vni <VNI_NUMBER>

# VTYSH shell command

sudo vtysh

show bgp l2vpn evpn route

net show bgp vrf <VRF_NAME> ipv4 unicast

# Command to show VXLAN traffic stats

net show bgp l2vpn evpn summary

net show bgp l2vpn evpn vni

net show evpn vni

net show evpn mac vni <VNI_VALUE>

net show evpn mac vni all

net show evpn next-hops vni all

nv show nve vxlan

# Configure physical interfaces, VLAN interfaces, and assign VLAN IDs and IP addresses

config interface ip add Loopback0 <SYSTEM_LOOPBACK>

# Configure VRF Setting

config vrf add <VRF-NAME>

config interface vrf bind VLAN<VLAN_NUMBER> <VRF-NAME>

config interface ip add VLAN<VLAN_NUMBER> <IP_ADDRESS>

# Create VxLAN and map VNI to VLAN

config vxlan add vtep <SOURCE_VTEP_IP>

config vxlan evpn_nvo add nvo vtep

config vxlan map add vtep <VLAN-ID> <VNI_VALUE>

config save -y

# Configure layer3 VNI and map it to VRF value

config vrf add_vrf_vni_map <VRF-NAME> <VNI_VALUE>

config save -y

# Establish a BGP environment for EVPN # vtysh command

router bgp <LOCAL_AS_NUMBER>

neighbor <ebgp_neighbor_ip> remote-as <REMOTE_AS_NUMBER>

address-family ipv4 unicast

network <PREFIX_ADVERTISED>

exit

address-family l2vpn evpn

neighbor <ebgp_neighbor_ip> activate

advertise-all-vni

end

# Configure VRF and VNI values

configure terminal

vrf <VRF-NAME>

vni <VNI_VALUE>

# Configure BGP routing and advertise EVPN routes

router bgp <LOCAL_AS_NUMBER> vrf <VRF-NAME>

address-family ipv4 unicast

redistribute connected

address-family l2vpn evpn

advertise ipv4 unicast

write

# Commands to verify VXLAN tunnels

show vxlan interface

show vxlan vlanvnimap

show vxlan tunnel

show vxlan remotevtep

# Commands to verify EVPN routes and BGP routes

show evpn vni detail

show bgp summary

show ip route vrf all

L3-VXLAN EVPN Symmetric IRB

Sample SONiC L3-VXLAN EVPN Symmetric IRB Example

Configure IP address and Loopback IPs of both switches.

AS5835-54X
admin@SONIC01:~$ config interface ip add Loopback0 1.1.1.1/32
admin@SONIC01:~$ config interface ip add Ethernet48 10.0.0.4/31

A4630-54PE
admin@SONIC02:~$ config interface ip add Loopback0 2.2.2.2/32
admin@SONIC02:~$ config interface ip add Ethernet52 10.0.0.5/31

Configure VRF Setting

AS5835-54X
admin@SONIC01:~$ config vrf add Vrf01                                                           
admin@SONIC01:~$ config interface vrf bind Vlan30 Vrf01                                          
admin@SONIC01:~$ config interface vrf bind Vlan10 Vrf01                                          
admin@SONIC01:~$ config interface ip add Vlan10 192.168.1.254/24                            

A4630-54PE
admin@SONIC02:~$ config vrf add Vrf01                                                             
admin@SONIC02:~$ config interface vrf bind Vlan30 Vrf01                                         
admin@SONIC02:~$ config interface vrf bind Vlan20 Vrf01                                           
admin@SONIC02:~$ config interface ip add Vlan20 192.168.2.254/24

Establish BGP Session between Ethernet48 and Ethernet52

AS5835-54X
admin@SONIC01:~$ vtysh                                                                                 
sonic# configure terminal
sonic(config)# router bgp 65100                                                                        
sonic(config-router)# neighbor 10.0.0.5 remote-as 65100                                      
sonic(config-router)# address-family ipv4 unicast                                       
sonic(config-router-af)# network 1.1.1.1/32                                                 
sonic(config-router-af)# exit
sonic(config-router)# address-family l2vpn evpn                                          
sonic(config-router-af)# neighbor 10.0.0.5 activate                                      
sonic(config-router-af)# advertise-all-vni                                                      
sonic(config-router-af)# end

sonic# configure terminal                                                                              
sonic(config)# vrf Vrf01                                                                                 
sonic(config-vrf)# vni 3000                                                                            
sonic(config-vrf)# end
sonic# configure terminal 
sonic(config)# router bgp 65100 vrf Vrf01                                                      
sonic(config-router)# address-family ipv4 unicast                                        
sonic(config-router-af)# redistribute connected                                           
sonic(config-router-af)# exit
sonic(config-router)# address-family l2vpn evpn                                          
sonic(config-router-af)# advertise ipv4 unicast                                              
sonic(config-router-af)# end
sonic# write

A4630-54PE
admin@SONIC02:~$ vtysh                                                                                 
sonic# configure terminal
sonic(config)# router bgp 65100                                                                   
sonic(config-router)# neighbor 10.0.0.4 remote-as 65100             
sonic(config-router)# address-family ipv4 unicast                                       
sonic(config-router-af)# network 2.2.2.2/32                                                 
sonic(config-router-af)# exit
sonic(config-router)# address-family l2vpn evpn                                          
sonic(config-router-af)# neighbor 10.0.0.4 activate                                      
sonic(config-router-af)# advertise-all-vni                                                      
sonic(config-router-af)# end
sonic# configure terminal                                                                              
sonic(config)# vrf Vrf01                                                                                 
sonic(config-vrf)# vni 3000                                                                            
sonic(config-vrf)# end
sonic# configure terminal 
sonic(config)# router bgp 65100 vrf Vrf01                                                     
sonic(config-router)# address-family ipv4 unicast                                        
sonic(config-router-af)# redistribute connected                                           
sonic(config-router-af)# exit
sonic(config-router)# address-family l2vpn evpn                                          
sonic(config-router-af)# advertise ipv4 unicast                                              
sonic(config-router-af)# end
sonic# write

Create Vxlan

AS5835-54X

configuring VTEP_name (vtep) and its IP address

admin@SONIC01:~$ config vxlan add vtep 1.1.1.1

create nvo_name (nvo) and bind it to VTEP_name (vtep)

admin@SONIC01:~$ config vxlan evpn_nvo add nvo vtep

Command to map VXLAN VNI to VLAN

admin@SONIC01:~$ config vxlan map add vtep 10 1000                                                
admin@SONIC01:~$ config vxlan map add vtep 30 3000                                               
admin@SONIC01:~$ config save -y

A4630-54PE configuring VTEP_name (vtep) and its IP address

admin@SONIC02:~$ config vxlan add vtep 2.2.2.2

create nvo_name (nvo) and bind it to VTEP_name (vtep)

admin@SONIC02:~$ config vxlan evpn_nvo add nvo vtep

Command to map VXLAN VNI to VLAN

admin@SONIC02:~$ config vxlan map add vtep 20 2000                                                
admin@SONIC02:~$ config vxlan map add vtep 30 3000                                               
admin@SONIC02:~$ config save -y

Configure the layer3 VNI on both switches. AS5835-54X

admin@SONIC01:~$ config vrf add_vrf_vni_map Vrf01 3000

A4630-54PE

admin@SONIC01:~$ config vrf add_vrf_vni_map Vrf01 3000

Verify EVPN-VNI Route Status

AS5835-54X
sonic# show evpn vni detail
VNI: 1000
 Type: L2
 Tenant VRF: Vrf01
 VxLAN interface: vtep-10
 VxLAN ifIndex: 67
 SVI interface: Vlan10
 SVI ifIndex: 9
 Local VTEP IP: 1.1.1.1
 Mcast group: 0.0.0.0
 No remote VTEPs known for this VNI
 Number of MACs (local and remote) known for this VNI: 1
 Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 1
 Advertise-gw-macip: No
 Advertise-svi-macip: No
VNI: 3000
  Type: L3
  Tenant VRF: Vrf01
  Local Vtep Ip: 1.1.1.1
  Vxlan-Intf: vtep-30
  SVI-If: Vlan30
  State: Up
  VNI Filter: none
  System MAC: 00:a0:c9:00:00:00
  Router MAC: 00:a0:c9:00:00:00
  L2 VNIs: 1000

A4630-54PE

sonic# show evpn vni detail
VNI: 2000
 Type: L2
 Tenant VRF: Vrf01
 VxLAN interface: vtep-20
 VxLAN ifIndex: 78
 SVI interface: Vlan20
 SVI ifIndex: 76
 Local VTEP IP: 2.2.2.2
 Mcast group: 0.0.0.0
 No remote VTEPs known for this VNI
 Number of MACs (local and remote) known for this VNI: 1
 Number of ARPs (IPv4 and IPv6, local and remote) known for this VNI: 1
 Advertise-gw-macip: No
 Advertise-svi-macip: No
VNI: 3000
  Type: L3
  Tenant VRF: Vrf01
  Local Vtep Ip: 2.2.2.2
  Vxlan-Intf: vtep-30
  SVI-If: Vlan30
  State: Up
  VNI Filter: none
  System MAC: 68:21:5f:29:c0:d2
  Router MAC: 68:21:5f:29:c0:d2
  L2 VNIs: 2000

Verify BGP Route Summary

AS5835-54X

sonic# show bgp summary
IPv4 Unicast Summary (VRF default):
BGP router identifier 188.188.9.14, local AS number 65100 vrf-id 0
BGP table version 17
RIB entries 3, using 552 bytes of memory
Peers 1, using 723 KiB of memory
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
10.0.0.5        4      65100      1436      1449        0    0    0 03:02:18            1        1 N/A
Total number of neighbors 1
L2VPN EVPN Summary (VRF default):
BGP router identifier 188.188.9.14, local AS number 65100 vrf-id 0
BGP table version 0
RIB entries 27, using 4968 bytes of memory
Peers 1, using 723 KiB of memory
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
10.0.0.5        4      65100      1436      1449        0    0    0 03:02:18            4        4 N/A
Total number of neighbors 1

A4630-54PE

sonic# show bgp summary
IPv4 Unicast Summary (VRF default):
BGP router identifier 188.188.9.6, local AS number 65100 vrf-id 0
BGP table version 8
RIB entries 3, using 552 bytes of memory
Peers 1, using 723 KiB of memory
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
10.0.0.4        4      65100       220       221        0    0    0 03:02:18            1        1 N/A
Total number of neighbors 1
L2VPN EVPN Summary (VRF default):
BGP router identifier 188.188.9.6, local AS number 65100 vrf-id 0
BGP table version 0
RIB entries 11, using 2024 bytes of memory
Peers 1, using 723 KiB of memory
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
10.0.0.4        4      65100       220       221        0    0    0 03:02:18            4        4 N/A
Total number of neighbors 1

Validate EVPN route learning AS5835-54X

sonic# show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
VRF Vrf01:
C>* 192.168.1.0/24 is directly connected, Vlan10, 03:18:41
K>* 192.168.1.254/32 [0/0] is directly connected, Vlan10, 03:18:41
B>* 192.168.2.0/24 [200/0] via 2.2.2.2, Vlan30 onlink, weight 1, 03:04:24
B>* 192.168.2.2/32 [200/0] via 2.2.2.2, Vlan30 onlink, weight 1, 02:21:18
VRF default:
K>* 0.0.0.0/0 [0/202] via 188.188.1.1, eth0, 22:59:15
K * 1.1.1.1/32 [0/0] is directly connected, Loopback0, 22:54:06
C>* 1.1.1.1/32 is directly connected, Loopback0, 22:54:06
B>* 2.2.2.2/32 [200/0] via 10.0.0.5, Ethernet48, weight 1, 03:04:24
C>* 10.0.0.4/31 is directly connected, Ethernet48, 03:07:18
K>* 10.0.0.4/32 [0/0] is directly connected, Ethernet48, 22:45:24
C>* 188.188.0.0/16 is directly connected, eth0, 22:59:15

sonic# show bgp l2vpn evpn
BGP table version is 14, local router ID is 188.188.9.14
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 188.188.9.6:2
*>i[2]:[0]:[48]:[80:a2:35:5a:22:50]
                    2.2.2.2                       100      0 i
                    RT:65100:2000 ET:8
*>i[2]:[0]:[48]:[80:a2:35:5a:22:50]:[32]:[192.168.2.2]
                    2.2.2.2                       100      0 i
                    RT:65100:2000 RT:65100:3000 ET:8 Rmac:68:21:5f:29:c0:d2
*>i[3]:[0]:[32]:[2.2.2.2]
                    2.2.2.2                       100      0 i
                    RT:65100:2000 ET:8
Route Distinguisher: 188.188.9.14:2
*> [2]:[0]:[48]:[b8:6a:97:19:ba:12]
                    1.1.1.1                            32768 i
                    ET:8 RT:65100:1000
*> [2]:[0]:[48]:[b8:6a:97:19:ba:12]:[32]:[192.168.1.1]
                    1.1.1.1                            32768 i
                    ET:8 RT:65100:1000 RT:65100:3000 Rmac:00:a0:c9:00:00:00
*> [3]:[0]:[32]:[1.1.1.1]
                    1.1.1.1                            32768 i
                    ET:8 RT:65100:1000
Route Distinguisher: 192.168.1.254:3
*> [5]:[0]:[24]:[192.168.1.0]
                    1.1.1.1                  0         32768 ?
                    ET:8 RT:65100:3000 Rmac:00:a0:c9:00:00:00
Route Distinguisher: 192.168.2.254:3
*>i[5]:[0]:[24]:[192.168.2.0]
                    2.2.2.2                  0    100      0 ?
                    RT:65100:3000 ET:8 Rmac:68:21:5f:29:c0:d2
Displayed 8 out of 8 total prefixes

#A4630-54PE
sonic# show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure
VRF Vrf01:
B>* 192.168.1.0/24 [200/0] via 1.1.1.1, Vlan30 onlink, weight 1, 03:04:23
B>* 192.168.1.1/32 [200/0] via 1.1.1.1, Vlan30 onlink, weight 1, 02:20:51
C>* 192.168.2.0/24 is directly connected, Vlan20, 03:07:28
K>* 192.168.2.254/32 [0/0] is directly connected, Vlan20, 03:07:28
VRF default:
K>* 0.0.0.0/0 [0/202] via 188.188.1.1, eth0, 03:17:24
B>* 1.1.1.1/32 [200/0] via 10.0.0.4, Ethernet52, weight 1, 03:04:23
K * 2.2.2.2/32 [0/0] is directly connected, Loopback0, 03:07:29
C>* 2.2.2.2/32 is directly connected, Loopback0, 03:07:29
C>* 10.0.0.4/31 is directly connected, Ethernet52, 03:07:17
K>* 10.0.0.5/32 [0/0] is directly connected, Ethernet52, 03:07:18
C>* 188.188.0.0/16 is directly connected, eth0, 03:17:24

sonic# show bgp l2vpn evpn
BGP table version is 12, local router ID is 188.188.9.6
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[EthTag]:[ESI]:[IPlen]:[VTEP-IP]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 188.188.9.6:2
*> [2]:[0]:[48]:[80:a2:35:5a:22:50]
                    2.2.2.2                            32768 i
                    ET:8 RT:65100:2000
*> [2]:[0]:[48]:[80:a2:35:5a:22:50]:[32]:[192.168.2.2]
                    2.2.2.2                            32768 i
                    ET:8 RT:65100:2000 RT:65100:3000 Rmac:68:21:5f:29:c0:d2
*> [3]:[0]:[32]:[2.2.2.2]
                    2.2.2.2                            32768 i
                    ET:8 RT:65100:2000
Route Distinguisher: 188.188.9.14:2
*>i[2]:[0]:[48]:[b8:6a:97:19:ba:12]
                    1.1.1.1                       100      0 i
                    RT:65100:1000 ET:8
*>i[2]:[0]:[48]:[b8:6a:97:19:ba:12]:[32]:[192.168.1.1]
                    1.1.1.1                       100      0 i
                    RT:65100:1000 RT:65100:3000 ET:8 Rmac:00:a0:c9:00:00:00
*>i[3]:[0]:[32]:[1.1.1.1]
                    1.1.1.1                       100      0 i
                    RT:65100:1000 ET:8
Route Distinguisher: 192.168.1.254:3
*>i[5]:[0]:[24]:[192.168.1.0]
                    1.1.1.1                  0    100      0 ?
                    RT:65100:3000 ET:8 Rmac:00:a0:c9:00:00:00
Route Distinguisher: 192.168.2.254:3
*> [5]:[0]:[24]:[192.168.2.0]
                    2.2.2.2                  0         32768 ?
                    ET:8 RT:65100:3000 Rmac:68:21:5f:29:c0:d2
Displayed 8 out of 8 total prefixes

QoS Configuration

CUMULUS(SN2700)

SONiC

# Commands to create QoS classifiers # To change the default profile to map PCP 0 to switch priority 4

nv set qos mapping default-global trust l2

nv set qos mapping default-global pcp 0 switch-priority 4

# Configuration to change the default profile to map ingress DSCP 22 to switch priority 4

nv set qos mapping default-global trust l3

nv set qos mapping default-global dscp 22 switch-priority 4

nv show qos mapping default-global dscp 22

# Command to assign all traffic to switch priority 3

nv set qos mapping default-global trust port

nv set qos mapping default-global port-default-sp 3

nv show qos mapping default-global

# Configuration to remark switch priority 0 to egress DSCP 22

nv set qos remark default-global rewrite l3

nv set qos remark default-global switch-priority 0 dscp 22

# Configure PFC

nv set qos pfc default-global switch-priority 0

nv set qos pfc default-global tx enable

nv set qos pfc default-global rx disable

nv set qos pfc default-global cable-length 50

# Assign switch priority 2 to egress queue 7

nv set qos egress-queue-mapping default-global switch-priority 2 traffic-class 7

# Show the egress queue mapping configuration for the default profile

nv show qos egress-queue-mapping default-global

# Applies the traffic shaping configuration to swp1, swp2, swp3, and swp5.

nv set qos egress-shaper shaper1 traffic-class 2 min-rate 100

nv set qos egress-shaper shaper1 traffic-class 2 max-rate 500

nv set qos egress-shaper shaper1 port-max-rate 200000

nv set interface swp1-swp3,swp5 qos egress-shaper profile shaper1

# Remarking configuration

nv set qos remark remark_port_group1 rewrite l3

nv set interface swp1 qos remark profile remark_port_group1

nv set qos remark remark_port_group2 switch-priority 0 dscp 37

nv set qos remark remark_port_group2 switch-priority 1 dscp 37

nv set interface swp2 qos remark profile remark_port_group2

# Egress scheduling

nv set qos egress-scheduler list2 traffic-class 2,5,6 mode dwrr

nv set qos egress-scheduler list2 traffic-class 2,5 bw-percent 50

nv set qos egress-scheduler list2 traffic-class 6 mode strict

nv set interface swp1,swp3,swp18 qos egress-scheduler profile list2

nv set interface swp2 qos egress-scheduler profile list1

# Commands to create QoS classifiers (EdgeCore SONiC Platform - AS9716-32D) # Create a profile for DOT1P/DSCP mapped to TC (Traffic Class). # Example for DSCP:

config qos dscp-tc add DSCP_TC --dscp 7 --tc 1

# Modify the existing Dot1p/DSCP to TC profile. # Example for DOT1P:

config qos dot1p-tc update 1p_tc --dot1p 1 --tc 2

# Validate the profile for DOT1P/DSCP to Traffic class. # DOT1P to TC:

show qos dot1p-tc

# Validate Queue mapping from DSCP queue to Traffic class.

show qos dscp-tc

# Create a profile for traffic class and map it to Queue.

config qos tc-queue add TC_Q --tc 1 --queue 2

# Validate the profile of Traffic class to Queue.

show qos tc-queue

# Binding the mapping table to the specified interface.

config interface qos dscp-tc bind Ethernet0 DSCP_TC

# Validate the binding table.

show interfaces qos

# Clear the queue counter

sonic-clear queue counters

# Check Specific Ethernet port (egress port) queue counters.

show queue counters Ethernet8

# Marking configuration # Create a profile for DOT1P remarking. # Example for DOT1P:

config qos remark dot1p add remark_dot1p --tc 0 --dot1p 1

# Validate the remark profile.

show qos remark dot1p

# Bind the remark table to the egress interface.

config interface qos remark dot1p bind Ethernet8 remark_dot1p

# Validate the binding table.

show interfaces qos

# Scheduler Configuration (EdgeCore SONiC platform - AS7326-56X) # Set the scheduler mode.

config scheduler add strict_mode --sched_type STRICT

# Validate scheduler status.

show scheduler

# Bind the scheduler to Ethernet sub-interface. # Example:

config interface scheduler bind queue Ethernet 5.3 strict_mode

# Command to unbind the scheduler from the Ethernet interface.

config interface scheduler unbind queue Ethernet 5.3

# Validate scheduler status.

show interfaces scheduler

# Set the scheduler mode # Example:

config scheduler add wrr_7 --sched_type WRR --weight 7

config scheduler add wrr_3 --sched_type WRR --weight 3

# Validate scheduler status.

show scheduler

# Bind the scheduler to Ethernet interface. # Example:

config interface scheduler bind queue Ethernet5 3 wrr_7

config interface scheduler bind queue Ethernet5 4 wrr_3

# Validate scheduler status.

show interfaces scheduler

ACL Configuration

CUMULUS(SN2700)

SONiC

# Command to create ACL rules # Install and Manage ACL Rules with NCLU command

-A FORWARD -i <interface_name> -s <source_ip> -d <destination_ip> -p tcp -j ACCEPT

# Create ACL rule with NCLU command

net add acl ipv4 <ACL_NAME> accept tcp source-ip <source_ip> source-port any dest-ip <destination_ip> dest-port any

# Apply ACL rule to inbound or outbound interface

net add int <interface_name> acl ipv4 <ACL_NAME> inbound

# Verify the ACL rule

net show configuration acl

# Command to apply ACL rule to a control plane interface

net add control-plane acl ipv4 <ACL_NAME> inbound

# To remove an ACL rule

net del acl ipv4 <ACL_NAME>

# Command to examine the current state of ACLs and list all installed ACL rules

sudo cl-acltool -L all

# Command to create ACL Tables # Syntax

config acl add table <ACL_table_name> L3 --description 'ACL_Test1' --stage 'ingress' --ports 'Ethernet<number>'

# Example

config acl add table ACL_Test1 L3V6 --description 'ACL_Test1' --stage 'egress' --ports 'Ethernet16'

# Command to delete ACL tables

config acl remove table <ACL_Table_Name>

# Command to create ACL Rule with source_ip_address # Example

config acl add rule --src-ip4 100.0.0.1 --priority 3 ACL_Test1 deny

# Commands to verify ACL table and rule created

show acl table

show acl rule

References

Cumulus References

https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-42/Quick-Start-Guide/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-41/Layer-3/Management-VRF/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-42/Monitoring-and-Troubleshooting/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-42/Installation-Management/Back-up-and-Restore/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-50/System-Configuration/Smart-System-Manager/
https://docs.nvidia.com/networking-ethernet-software/knowledge-base/Setup-and-Getting-Started/Cumulus-Linux-Command-Reference-Guide/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-43/Installation-Management/Upgrading-Cumulus-Linux/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-41/Layer-3/Management-VRF/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-42/Layer-2/Link-Layer-Discovery-Protocol/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-44/System-Configuration/Netfilter-ACLs/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-54/Layer-1-and-Switch-Ports/Quality-of-Service/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-41/Network-Virtualization/Ethernet-Virtual-Private-Network-EVPN/Inter-subnet-Routing/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-42/Network-Virtualization/Ethernet-Virtual-Private-Network-EVPN/
https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-42/Network-Virtualization/Static-VXLAN-Tunnels/

Edgecore SONIC References

https://support.edge-core.com/hc/en-us/articles/900004369066--Enterprise-SONiC-OSPF-Open-Shortest-Path-First-
https://support.edge-core.com/hc/en-us/articles/900000789566--Enterprise-SONiC-BGP-Step-1-Establish-BGP-Session
https://support.edge-core.com/hc/en-us/articles/900000809363--Enterprise-SONiC-BGP-Step-2-1-Redistribute-routes-to-BGP-process-Optional-
https://support.edge-core.com/hc/en-us/articles/900002377366--Enterprise-SONiC-BGP-Unnumbered
https://support.edge-core.com/hc/en-us/articles/900004277226--Enterprise-SONiC-VRF-Virtual-routing-and-forwarding-
https://support.edge-core.com/hc/en-us/articles/900002380706--Enterprise-SONiC-MC-LAG
https://support.edge-core.com/hc/en-us/articles/900002741223--Enterprise-SONiC-Symmetric-EVPN-IRB
https://support.edge-core.com/hc/en-us/articles/900002720523--Enterprise-SONiC-EVPN-L2-VxLAN
https://support.edge-core.com/hc/en-us/articles/900000214926--Enterprise-SONiC-ACL-Access-Control-List-
https://support.edge-core.com/hc/en-us/articles/9391323739417--Enterprise-SONiC-ZTP-Zero-Touch-Provisioning-
https://support.edge-core.com/hc/en-us/articles/900000240066--Enterprise-SONiC-QoS-Quality-of-Service-
https://support.edge-core.com/hc/en-us/articles/900000198943--Enterprise-SONiC-DHCP-Relay
https://support.edge-core.com/hc/en-us/articles/900000307246--Enterprise-SONiC-SONiC-overview
https://support.edge-core.com/hc/en-us/articles/900004277226--Enterprise-SONiC-VRF-Virtual-routing-and-forwarding-
https://support.edge-core.com/hc/en-us/articles/900000214266--Enterprise-SONiC-LLDP-Link-Layer-Discovery-Protocol-
https://support.edge-core.com/hc/en-us/articles/900007074363--Enterprise-SONiC-Sub-Interface