Overview
400Gigabit Ethernet (400GbE) is the latest generation of the IEEE 802.3 Ethernet family. Defined in the IEEE802.3bs2022 amendment, it delivers up to 400Gbit/s of raw bandwidth over a single logical link. The standard was introduced to meet the explosive growth of datacenter traffic driven by artificialintelligence workloads, highperformance computing (HPC), and the continued migration of enterprise applications to the cloud.
Why 400GbE?
Modern data centers are hitting the limits of 25GbE, 50GbE, and 100GbE. Four key forces push the need for 400GbE:
- Dataintensive AI/ML workloads Large model training and inference generate petabytes of traffic per day.
- Scaleout storage NVMeoverFabric and distributed file systems demand higher perlink throughput.
- Network consolidation Fewer higherspeed links reduce cabling complexity and power consumption.
- Futureproofing Planning for the next 510 years avoids premature hardware refresh cycles.
Key Specifications
The 400GbE standard defines several variants, but the common technical foundations are:
- Line rate: 400Gbit/s gross, 394.24Gbit/s usable payload after encoding.
- Encoding: 64b/66b with PAM4 signaling for 28Gb/s per lane.
- Lanes: 428Gb/s or 814Gb/s depending on the physical medium.
- FEC: RS544 (ReedSolomon) to correct up to 10 BER.
- Latency: Typically <50ns per hop in a wellengineered fabric.
- Power: Approximately 4W per 10Gbps lane, so a 400GbE transceiver consumes around 1215W.
Physical Layer Options
IEEE 802.3bs defines three main PHY families:
400GBASESR8
Eight parallel multimode fiber (MMF) lanes of 50m reach using OM4 or OM5 fiber. Suited for shorthaul intrarack connections.
400GBASELR8
Eight singlemode fiber (SMF) lanes with a typical reach of 10km (extendable to 40km with optical amplifiers). Ideal for interdatacenter links.
400GBASECR4
Four twinaxial copper cables (CAUI4) offering up to 7m of reach. Used for topofrack (ToR) or servertoswitch connections where cost and power are critical.
Each PHY uses the CAUI4 (four 25Gb/s lane) or CAUI8 (eight 25Gb/s lane) electrical interface. The choice depends on the existing cabling ecosystem and the required reach.
Primary Use Cases
- AI clusters: Largescale GPU farms need 400GbE to move tensors between nodes without bottlenecks.
- HPC interconnects: Exascale supercomputers adopt 400GbE to replace proprietary InfiniBand solutions.
- Datacenter spineleaf fabrics: A 400GbE spine can serve dozens of 100GbE leaf switches, simplifying the architecture.
- NVMeoverFabric: Highperformance storage arrays achieve linerate I/O with fewer links.
- Interdatacenter transport: 400GbE over SMF provides a costeffective alternative to DWDM for up to 40km.
Deployment Challenges
Despite its advantages, moving to 400GbE is not without hurdles:
- Cabling density: 8lane fiber cables are thicker and require careful routing.
- Signal integrity: PAM4 is more susceptible to jitter and loss; highquality transceivers and careful PCB design are essential.
- Power & cooling: Higher transceiver power can increase rack heat density.
- Cost: Initial investment in 400GbE NICs, switches, and optics is higher than 100GbE.
- Compatibility: Legacy equipment must be upgraded or bridged via aggregation switches.
Many vendors mitigate these issues with hybrid solutions (e.g., 100GbE aggregation to 400GbE uplinks) and softwaredefined networking that dynamically balances traffic.
Future Outlook
Research is already progressing toward 800GbE and 1.6TbE, driven by opticalmodule advancements such as 56GBd PAM4 and the adoption of coherent modulation for shortreach links. In the meantime, 400GbE is expected to become the default spine speed in all hyperscale data centers by 2028, while many enterprise environments will adopt it selectively for AI workloads and highperformance storage.
Further Reading
- IEEE 802.3bs2022 Standard (official specification)
- Understanding 400GbE Whitepaper from the Ethernet Alliance
- Broadcom and Mellanox product briefs on 400GbE adapters
- PAM4 Signaling for Ethernet IEEE Communications Magazine, 2023
- Open Compute Project 400GbE reference designs
