AI and HPC are in the process of changing the data center network. Modern infrastructure typically consists of a front-end network and a back-end network: the front-end network connects users, stores, and manages systems, while the back-end network interconnects GPU and computing nodes. At the largest scale, AI clusters are deployed across multiple locations and rely on cross-domain network expansion (scale-across networking) to build a unified infrastructure.

Although most modern data centers adopt the leaf-spine architecture, the specific implementation is not determined solely by technical requirements. Supplier relationships, product availability, supply chain factors, and total cost of ownership all play important roles. As a result, organizations with similar goals often implement their networks based on their own business priorities, operational constraints, and upgrade strategies.

AI and HPC infrastructure handle high-value data, which also represents a substantial investment of funds. To protect these environments, control measures need to be taken in the following areas:

At the same time, the security mechanisms must support the basic objectives of the AI infrastructure:

• Full-line speed throughput and minimal latency impact
• Extensible to a large number of ports and nodes
• Power consumption, silicon area, and deployment costs are acceptable.

The industry has accumulated over a decade of operational experience with technologies such as MACsec and IPsec in high-speed networks. The main challenge at present is to determine where these capabilities should be implemented and how to do so.

The modern AI network has reached a transmission speed of hundreds of Gbps, and is rapidly moving towards multi-Tbps. At this level of performance, network security must be implemented in the following ways:

The security data plane is responsible for real-time packet processing and protection. These functions are integrated into switches ASIC, SmartNIC, DPU, optical DSP, or interface devices.

The control plane is responsible for security policy management, session establishment, and periodic refresh of encryption keys, among other runtime operations.

Figure 1: Hardware-accelerated network security architecture with a data plane and a control plane.

One of the most important architectural decisions is where to place security features such as MACsec, IPsec, or the emerging Ultra Ethernet security mechanisms.

Integrating security directly into the exchange ASIC allows encryption and authentication to become part of the data packet forwarding pipeline.

Advantages:

• Natively integrated with packet classification and forwarding
• Greater flexibility in the selection of optical modules and pluggable modules

Challenge:

• It will occupy silicon area that could otherwise be used to increase port density, packet caching, and switching capacity.
• Security capabilities are limited by the ASIC lifecycle, with limited room for future upgrades.

Another approach is to place the security functions in external PHY devices (such as line card timers/ gearbox), or in pluggable modules that support security functions.

Advantages:

• Easier to introduce new security capabilities
• Reserved switch ASIC resources are used for network functions.
• Move the additional power consumption away from the main ASIC and use technologies that are more suitable for power optimization.

Challenge:

• Depends on the availability of external suppliers and modules
• The security boundary has shifted from the chip level to the front panel level.

AI and computing systems rely on dedicated transmission protocols, including InfiniBand, RoCEv2, and Ultra Ethernet Transport ( UET ). Unlike the security at the fabric layer, transmission security is usually associated with specific users, tasks, or workloads, and therefore operates as an end-to-end service. Security must be maintained transparently without affecting the underlying fabric, while also preserving the low latency and high throughput required for accelerated computing.

Therefore, transmission security is integrated into ( Smart ) NIC and is closely coupled with hardware-accelerated transmission processing. It usually supports the simultaneous operation of multiple transmission and security protocols to accommodate different communication requirements.

Figure 2: Cybersecurity in AI NIC.

Modern infrastructure deploys multiple layers of security mechanisms simultaneously based on topology, trust boundaries, and workload requirements:

• Used for two-layer protection: MACsec
• Used for three-layer security IPsec
• Used for Ultra Ethernet transmission security of UET-TSS
• Proprietary security protocol tailored for specific fabric and deployment optimization

These mechanisms are generally complementary rather than competitive. MACsec is usually used for link-level protection, IPsec is used to protect cross-network routing traffic, while transport layer security protects the communication flows of AI and HPC.

Advances in optical technology are expanding the options for network security deployment. Nowadays, MACsec has begun to be directly integrated into 800G and 1.6T optical modules, thanks to silicon technology optimized for low-power packet processing.

Looking to the future, chiplet-based exchanges and the NIC architecture, as well as near-packaging optics and co-packaging optics, may create new opportunities for secure integration. Dedicated I/O chiplets are expected to become the natural destination for encryption and authentication functions, while preserving valuable computing and exchange resources.

Network security has now become a basic requirement for AI, HPC, and cloud-scale data centers. To protect traffic at Tbps rates, hardware implementation is necessary, while also meeting cost and power consumption targets and maintaining line-speed performance.

The industry offers a variety of deployment options, including exchanges ASIC, external PHY, optical modules, SmartNIC, as well as future chiplet architectures. This flexibility enables data center designers to strike a balance between security, performance, energy efficiency, and system cost based on their own infrastructure requirements.

Rambus addresses these challenges through its extensive and industry-verified combination of cybersecurity solutions IP. It currently supports up to 1.6T and can be scaled up to 3.2T in the future. Customers can make trade-offs between security features, throughput, power consumption, and integration complexity to deploy cybersecurity capabilities that meet their requirements for silicon device performance, cost, and time-to-market. For more information, please refer to the white paper available in the link below.

Link

White Paper – Enhancing Cybersecurity for AI, HPC and Data Centers

Rambus Secure Networking IP

Maxim Demchenko