General • CRS510 Problems setting up 100GBe trunk

Hi, since a week I have the CRS510 switch, pretty happy with its capabilities, but I'm a bit stuck, so decided to put my question here.
I have a Mellanox 100GBe NIC, which seems to correctly connect to my switch(port qsfp28-1-1), after I changed FEC auto to FEC91, the status of the connection is OK 100GBe great!, microtik optical module.
Similarly I have a 25GBe NIC which also connects ok to the switch (port sfp28-1), on another port I have SFP+ optical module also connects fine(port sfp28-4), status is ok, which connects to a small zyxel switch on my desk, for each port (all optical) I selected FEC91 and used auto-negotiation
What I need is a trunk connection(qsfp28-1-1) (with vlans 20,30,40,45,50,60) from my server to the zyxel sfp+ (via sfp28-4) switch on my desk.
So I first tried to create a trunk using the 25GBe port on my server connected to port sfp28-1 on one end and port sfp28-4 running a 10GBe at the other end, all vlans only specified in the tagged section.
When I add the vlans 20..60 and activate vlan filtering, everything works !!, which is great of course, see printout of the bridge and ports below,

I disabled the qsfp28 ports 1-4 at first

[admin@MikroTikCRS510] /interface/bridge> print
Flags: X - disabled, R - running
0 R ;;; default config with management port
name="BridgeMain" mtu=auto actual-mtu=1500 l2mtu=1584 arp=enabled arp-timeout=auto mac-address=78:9A:18:01:8F:37 protocol-mode=rstp
fast-forward=yes igmp-snooping=no auto-mac=no admin-mac=78:9A:18:01:8F:37 ageing-time=5m priority=0x8000 max-message-age=20s forward-delay=15s
transmit-hold-count=6 vlan-filtering=yes ether-type=0x8100 pvid=1 frame-types=admit-all ingress-filtering=yes dhcp-snooping=no

print
Flags: X - DISABLED, I - INACTIVE; H - HW-OFFLOAD
Columns: INTERFACE, BRIDGE, HW, PVID, PRIORITY, PATH-COST, INTERNAL-PATH-COST, HORIZON
# INTERFACE BRIDGE HW PVID PRIORITY PATH-COST INTERNAL-PATH-COST HORIZON
;;; defconf
0 I ether1 BridgeMain yes 1 0x80 10 10 none
;;; defconf
1 IH qsfp28-1-1 BridgeMain yes 1 0x80 10 10 none
;;; defconf
2 IH qsfp28-1-2 BridgeMain yes 1 0x80 10 10 none
;;; defconf
3 IH qsfp28-1-3 BridgeMain yes 1 0x80 10 10 none
;;; defconf
4 IH qsfp28-1-4 BridgeMain yes 1 0x80 10 10 none
;;; defconf
5 X qsfp28-2-1 BridgeMain 1 0x80 10 10 none
;;; defconf
6 X qsfp28-2-2 BridgeMain 1 0x80 10 10 none
;;; defconf
7 X qsfp28-2-3 BridgeMain 1 0x80 10 10 none
;;; defconf
8 X qsfp28-2-4 BridgeMain 1 0x80 10 10 none
;;; defconf
9 H sfp28-1 BridgeMain yes 1 0x80 10 10 none
;;; defconf
10 X sfp28-2 BridgeMain 1 0x80 10 10 none
;;; defconf
11 X sfp28-3 BridgeMain 1 0x80 10 10 none
;;; defconf
12 H sfp28-4 BridgeMain yes 1 0x80 10 10 none
;;; defconf
13 X sfp28-5 BridgeMain 1 0x80 10 10 none
;;; defconf
14 X sfp28-6 BridgeMain 1 0x80 10 10 none
;;; defconf
15 X sfp28-7 BridgeMain 1 0x80 10 10 none
;;; defconf
16 X sfp28-8 BridgeMain 1 0x80 10 10 none

vlan table:
print
Flags: D - DYNAMIC
Columns: BRIDGE, VLAN-IDS, CURRENT-TAGGED, CURRENT-UNTAGGED
# BRIDGE VLAN-IDS CURRENT-TAGGED CURRENT-UNTAGGED
;;; CAMERAVLAN
0 BridgeMain 20 sfp28-1 sfp28-4
;;; PRIVATEWIFIVLAN
1 BridgeMain 30 sfp28-1 sfp28-4
;;; GUESTWIFIVLAN
2 BridgeMain 40 sfp28-1 sfp28-4
;;; IOTVLAN
3 BridgeMain 45 sfp28-1 sfp28-4
;;; SERVERSVLAN
4 BridgeMain 50 sfp28-1 sfp28-4
;;; VIRTUALMACHINESVLAN
5 BridgeMain 60 sfp28-1 sfp28-4
6 D BridgeMain 1 BridgeMain
sfp28-1
sfp28-4
Like I said the above works perfect, but only via my 25GBe nic, I want to use my 100GBe NIC. The problem with that it has 4 ports, what I did is add 4 ports (qfsp28-1-1 to qfsp28-1-4) to each of the aforementioned VLANs, on the print only the first qsfp28-1-1 gets mentioned. I also added only the first qsfp28-1-1 port, in both cases nothing is routed through the bridge, If I deactivate the sfp28-1 port (25GBe) on one side of the trunk, I also deleted the sfp28-1 from the config, because I maybe thought it would interfere, but still nothing was forwarded from the bridge to my 100GBe NIC. see vlan output:
[admin@MikroTikCRS510] /interface/bridge/vlan> print
Flags: D - DYNAMIC
Columns: BRIDGE, VLAN-IDS, CURRENT-TAGGED, CURRENT-UNTAGGED
# BRIDGE VLAN-IDS CURRENT-TAGGED CURRENT-UNTAGGED
;;; CAMERAVLAN
0 BridgeMain 20 sfp28-1 sfp28-4 qsfp28-1-1
;;; PRIVATEWIFIVLAN
1 BridgeMain 30 sfp28-1 sfp28-4 qsfp28-1-1
;;; GUESTWIFIVLAN
2 BridgeMain 40 sfp28-1 sfp28-4 qsfp28-1-1
;;; IOTVLAN
3 BridgeMain 45 sfp28-1 sfp28-4 qsfp28-1-1
;;; SERVERSVLAN
4 BridgeMain 50 sfp28-1 sfp28-4 qsfp28-1-1
;;; VIRTUALMACHINESVLAN
5 BridgeMain 60 sfp28-1 sfp28-4 qsfp28-1-1
6 D BridgeMain 1 BridgeMain
sfp28-1
sfp28-4
qsfp28-1-1
What am I doing wrong with regard to the qsfp28 port ? It seems to use the hardware correctly, the links itself are all reporting OK at the correct speeds. It seems to have something todo with the fact that 100GBe uses 4 ports, like I mentioned before, I tried adding qsfp28-1-1..qsfp28-1-4 to each vlan, I also tried only specifying only the first port qsfp28-1-1 in the tagged section of each vlan, this won"t solve my problem, but which method should be used ?

Statistics: Posted by Rup — Tue Dec 12, 2023 5:57 am

---