Hello everyone,
I am using a CRS312-4C+8XG as my main switch.
It is connected to two Proxmox servers via LACP.
On my Proxmox servers I run an OPNsense appliance for firewalling and intervlan routing.
I recently stumbled accross a strange behaviour regarding switching traffic: If I run iperf3 within the same VLAN I get near wire speed (~ 9.5 Gbit).
If I do VLAN routing via OPNsense I get around 5 Gbit but after some time the switch CPU gets to 100 % mostly at bridging and networking.
On CRS312 everything is HW offloaded.
The current config would be:
For comparsion I did also run some tests on my CCR2004 router which is in general in front of my OPNsense firewall.
With that routing config CRS312 is not getting to 100 % but CCR2004 does (which is kinda expected).
Anyone got ideas why my switch is having troubles with handling L3 traffic which is not being routed by itself?
I am using a CRS312-4C+8XG as my main switch.
It is connected to two Proxmox servers via LACP.
On my Proxmox servers I run an OPNsense appliance for firewalling and intervlan routing.
I recently stumbled accross a strange behaviour regarding switching traffic: If I run iperf3 within the same VLAN I get near wire speed (~ 9.5 Gbit).
If I do VLAN routing via OPNsense I get around 5 Gbit but after some time the switch CPU gets to 100 % mostly at bridging and networking.
On CRS312 everything is HW offloaded.
The current config would be:
Code:
# 2023-12-20 12:27:50 by RouterOS 7.13# software id = BD85-SIEN## model = CRS312-4C+8XG# serial number = abc123/interface bridgeadd admin-mac=74:4D:28:B7:7C:A4 auto-mac=no fast-forward=no ingress-filtering=no name=bridge port-cost-mode=short priority=0x2000 vlan-filtering=yes/interface ethernetset [ find default-name=ether1 ] l2mtu=1592 name=Desktop rx-flow-control=auto tx-flow-control=autoset [ find default-name=ether7 ] l2mtu=1592 name="Mikrotik R1" rx-flow-control=auto tx-flow-control=autoset [ find default-name=combo3 ] l2mtu=1592 name=ccr2004set [ find default-name=combo1 ] l2mtu=1592 rx-flow-control=auto tx-flow-control=autoset [ find default-name=combo2 ] l2mtu=1592 rx-flow-control=auto tx-flow-control=autoset [ find default-name=combo4 ] l2mtu=1592 name=css326set [ find default-name=ether2 ] l2mtu=1592 rx-flow-control=auto tx-flow-control=autoset [ find default-name=ether3 ] l2mtu=1592 rx-flow-control=auto tx-flow-control=autoset [ find default-name=ether4 ] l2mtu=1592 rx-flow-control=auto tx-flow-control=autoset [ find default-name=ether5 ] l2mtu=1592 rx-flow-control=auto tx-flow-control=autoset [ find default-name=ether6 ] l2mtu=1592 rx-flow-control=auto tx-flow-control=autoset [ find default-name=ether8 ] l2mtu=1592 rx-flow-control=auto tx-flow-control=autoset [ find default-name=ether9 ] l2mtu=1592 rx-flow-control=auto tx-flow-control=auto/interface vlanadd interface=bridge name=vlan10 vlan-id=10/interface bondingadd mode=802.3ad name=Proxmox1 slaves=ether5,ether6 transmit-hash-policy=layer-2-and-3add mode=802.3ad name=Proxmox2 slaves=combo1,combo2 transmit-hash-policy=layer-2-and-3/interface ethernet switchset 0 l3-hw-offloading=yes/interface ethernet switch portset 1 l3-hw-offloading=noset 2 l3-hw-offloading=noset 3 l3-hw-offloading=noset 4 l3-hw-offloading=noset 5 l3-hw-offloading=noset 6 l3-hw-offloading=noset 7 l3-hw-offloading=noset 8 l3-hw-offloading=noset 9 l3-hw-offloading=noset 10 l3-hw-offloading=noset 11 l3-hw-offloading=noset 12 l3-hw-offloading=no/interface listadd name=WANadd name=LAN/interface lte apnset [ find default=yes ] ip-type=ipv4 use-network-apn=no/interface wireless security-profilesset [ find default=yes ] supplicant-identity=MikroTik/ip hotspot profileadd hotspot-address=192.168.10.248 html-directory=flash/hotspot name=hsprof1/portset 0 name=serial0/routing ospf instanceadd disabled=no name=default-v2/routing ospf areaadd disabled=yes instance=default-v2 name=backbone-v2/caps-man managerset enabled=yes upgrade-policy=suggest-same-version/interface bridge portadd bridge=bridge interface=Mathias-Desktop internal-path-cost=10 path-cost=10 pvid=10add bridge=bridge interface=ccr2004 internal-path-cost=10 path-cost=10add bridge=bridge interface=css326 internal-path-cost=10 path-cost=10add bridge=bridge frame-types=admit-only-vlan-tagged interface=Proxmox1 internal-path-cost=10 path-cost=10add bridge=bridge frame-types=admit-only-vlan-tagged interface=Proxmox2 internal-path-cost=10 path-cost=10add bridge=bridge interface=vlan10 pvid=10add bridge=bridge interface="Mikrotik R1" internal-path-cost=10 path-cost=10/interface ethernet switch l3hw-settingsset ipv6-hw=yes/ip neighbor discovery-settingsset discover-interface-list=none/ip settingsset max-neighbor-entries=8192/ipv6 settingsset max-neighbor-entries=8192/interface bridge vlanadd bridge=bridge tagged=bridge,vlan10,Proxmox1,Proxmox2,ccr2004,css326 vlan-ids=10add bridge=bridge tagged="bridge,Proxmox1,Proxmox2,ccr2004,css326,Mikrotik R1" vlan-ids=11add bridge=bridge tagged="bridge,Proxmox1,Proxmox2,ccr2004,css326,Mikrotik R1" vlan-ids=20add bridge=bridge tagged="bridge,Proxmox1,Proxmox2,ccr2004,css326,Mikrotik R1" vlan-ids=30add bridge=bridge tagged="bridge,Proxmox1,Proxmox2,ccr2004,css326,Mikrotik R1" vlan-ids=31add bridge=bridge tagged="bridge,Proxmox1,Proxmox2,ccr2004,css326,Mikrotik R1" vlan-ids=90add bridge=bridge tagged="bridge,Proxmox1,Proxmox2,ccr2004,css326,Mikrotik R1" vlan-ids=500add bridge=bridge tagged="bridge,Proxmox1,Proxmox2,ccr2004,css326,Mikrotik R1" vlan-ids=450/interface list memberadd interface=ether9 list=LANadd interface=Mathias-Desktop list=LANadd interface=ether2 list=LANadd interface=ether3 list=LANadd interface=ether4 list=LANadd interface=ether5 list=LANadd interface=ether6 list=LANadd interface="Mikrotik R1" list=LANadd interface=ether8 list=LANadd interface=combo1 list=LANadd interface=combo2 list=LANadd interface=ccr2004 list=LANadd interface=css326 list=LAN/interface ovpn-server serverset auth=sha1,md5/ip addressadd address=192.168.10.248/24 interface=vlan10 network=192.168.10.0/ip cloudset update-time=no/ip dhcp-clientadd comment=defconf disabled=yes interface=bridge/ip dnsset servers=192.168.10.103,192.168.10.104,192.168.101.170/ip hotspot useradd name=user1/ip routeadd disabled=no dst-address=0.0.0.0/0 gateway=192.168.10.254/ip serviceset telnet disabled=yesset ftp disabled=yesset www disabled=yesset www-ssl tls-version=only-1.2set api address=192.168.20.50/32set api-ssl disabled=yes/ip sshset host-key-size=4096 strong-crypto=yes/ipv6 ndset [ find default=yes ] advertise-dns=no interface=vlan10add advertise-dns=no/system clockset time-zone-name=Europe/Vienna/system identityset name=Switch-CRS312/system noteset show-at-login=no/system ntp clientset enabled=yes/system ntp client serversadd address=pool.ntp.org/system routerboard settingsset auto-upgrade=yes boot-os=router-os/system scheduleradd interval=1d name="schedule update" on-event="/system package update\r\ \ncheck-for-updates once\r\ \n:delay 3s;\r\ \n:if ( [get status] = \"New version is available\") do={ install }" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive,romon start-date=2022-04-18 start-time=03:00:00/tool bandwidth-serverset authenticate=noFor comparsion I did also run some tests on my CCR2004 router which is in general in front of my OPNsense firewall.
With that routing config CRS312 is not getting to 100 % but CCR2004 does (which is kinda expected).
Anyone got ideas why my switch is having troubles with handling L3 traffic which is not being routed by itself?
Statistics: Posted by An5teifo — Wed Dec 20, 2023 1:33 pm