Ubuntu 20.04 Mellanox MT27800 40Gbit Capacity

Server Specifications

• Platform: HPE ProLiant DL380 Gen9 BIOS v2.90
• Processors: Dual Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz
• Network Interface: HPE InfiniBand EDR/Ethernet 100Gb 2-Port 841QSFP28 Mellanox MT27800 Firmware 16.31.1200

When I run a lshw -class network I get the output below:

*-network:1 description: Ethernet interface product: MT27800 Family [ConnectX-5] vendor: Mellanox Technologies physical id: 0.1 bus info: pci@0000:84:00.1 logical name: ens5f0 version: 00 serial: 94:40:XX:XX:XX:XX capacity: 40Gbit/s width: 64 bits clock: 33MHz capabilities: pciexpress vpd msix pm bus_master cap_list ethernet physical 1000bt-fd 10000bt-fd 25000bt-fd 40000bt-fd autonegotiation configuration: autonegotiation=on broadcast=yes driver=mlx5_core driverversion=5.4-1.0.3 duplex=full firmware=16.31.1200 (HPE0000000009) ip=10.X.X.X latency=0 link=yes multicast=yes resources: irq:17 memory:96000000-97ffffff memory:39ffae00000-39ffcdfffff

You can see that the Capacity is 40Gbit/s, however when I run ethtool I do see the Speed as 100000Mb/s. The Cisco Nexus Switch also shows the link speed at 100Gbps.

The HPE iLO Device inventory shows that the 100Gbps card is running at x16 can anyone help me figure out why I am still be bottle necked at 40Gbit/s?

Is there an issue with the PCI configuration or the auto negotiation?

Know someone who can answer? Share a link to this question via email, Twitter, or Facebook.