Cloudstack creates VLAN sub-interface on wrong interface on KVM

[cisco-abrandel]

problem

I have a fresh cloudstack setup using advanced networking (without security groups) in a zone, where the VLAN sub-interface is being created on the wrong interface on the KVM hosts.

On the KVM host, I have a single physical interface attached to a bridge, cloudbr0. I then have a VLAN sub-interface for management configured on the bridge, as the port is a trunk and all VLANs we want to be tagged.

cloudbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:50:56:92:98:5b  txqueuelen 1000  (Ethernet)
        RX packets 5479511  bytes 7977472947 (7.4 GiB)
        RX errors 0  dropped 1976  overruns 0  frame 0
        TX packets 192836  bytes 1912563651 (1.7 GiB)
        TX errors 0  dropped 1 overruns 0  carrier 0  collisions 0

cloudbr0.3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet y.y.y.181  netmask 255.255.255.0  broadcast y.y.y.255
        inet6 x:x:x:x:x:x  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::c551:98a9:1cbf:9df  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:92:98:5b  txqueuelen 1000  (Ethernet)
        RX packets 320144  bytes 1258105503 (1.1 GiB)
        RX errors 0  dropped 18  overruns 0  frame 0
        TX packets 192837  bytes 1912563993 (1.7 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:50:56:92:98:5b  txqueuelen 1000  (Ethernet)
        RX packets 6535565  bytes 8146940376 (7.5 GiB)
        RX errors 0  dropped 66  overruns 0  frame 0
        TX packets 195594  bytes 1912679835 (1.7 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

This works fine, cloudstack can add the host, communication works as expected.

The problem is when I go and create either a guest network, or assign public IPs with a VLAN tag, the VLAN sub-interface along with a new bridge is getting created on ens33 instead of cloudbr0.

brens33-30: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:50:56:92:98:5b  txqueuelen 1000  (Ethernet)
        RX packets 3603  bytes 275466 (269.0 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2  bytes 108 (108.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:50:56:92:98:5b  txqueuelen 1000  (Ethernet)
        RX packets 6535565  bytes 8146940376 (7.5 GiB)
        RX errors 0  dropped 66  overruns 0  frame 0
        TX packets 195594  bytes 1912679835 (1.7 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens33.30: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:50:56:92:98:5b  txqueuelen 1000  (Ethernet)
        RX packets 3615  bytes 276276 (269.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 7  bytes 306 (306.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I can't figure out why this is happening. The labels seem to be configured properly in the agent:

private.network.device=cloudbr0
guest.network.device=cloudbr0
public.network.device=cloudbr0

To describe a bit more about our use case, we're primarily looking to leverage L2 networks in cloudstack. Each host will have a single interface that is trunked, and we will be dividing traffic based on VLANs. I would have expected that cloudstack creates the VLAN sub-interfaces on the cloudbr0 bridge, and not the parent of this bridge.

versions

Cloudstack version: 4.20.2.0
Rocky Linux 9.6

The steps to reproduce the bug

1. Install cloudstack
2. Configure a zone with advanced networking, no security groups
3. Create a single physical network, with all traffic types
4. Assign KVM network label as "cloudbr0"
5. Wait for system VMs to spin up (or spin your own) and watch cloudstack create the VLAN sub-interface on the wrong interface

What to do about it?

Cloudstack should use the right bridge

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[rajujith]

@cisco-abrandel could you try this bridge configuration on the KVM:

cloudbr0 -> ens33.3
cloudbr1 -> ens33

[weizhouapache]

this is intended I think.
cloudstack does create a virtual nic for vlan <physical nic name>.<vlan>, not <bridge name>.<vlan>

have you encountered any problem caused by it ? @cisco-abrandel