High Speed Interconnects: Navigating Your High-Speed Options

In most markets, Category 6A/ Class E_A copper twisted-pair cabling has dominated in the data center for 10 gigabit Ethernet links. Now as the demand for high-speed, low-latency access to information continues to grow, server interconnect speeds are increasing beyond 10 Gig and the feasibility of twisted-pair copper cabling. That means network professionals like yourself are faced with choosing between Direct Attach Copper Cables (DAC), Active Optical Cables (AOC), or structured cabling using fiber assemblies and separate optical transceivers. Understanding the pros, cons, and capabilities of each of these options can help you make an informed decision and achieve the best outcome for your specific application and environment.

DACs

Factory-terminated, fixed-length DACs made from twinax copper with small form-factor pluggable modules provide a cost-effective, low-latency direct connection between active equipment transceiver ports. The modules come in various single- or multi-lane technologies to support from 10 to 400 Gig. The most common DACs used in data center switch-to-server links include:

• 10 Gig SFP+ single-lane
• 25 Gig SFP28 single-lane
• 40 Gig QSFP+ four-lane
• 100 Gig QSFP28 four-lane

It’s important to note that SFP28 DACs share the same mating interface as SFP+ DACs, and QSFP28 DACs share the same mating interface as QSFP+ DACs, making them backwards compatible to ease migration. To help you reduce cost via lower switch port counts and space savings, these DACs also come in hybrid breakout assemblies, including QSFP28 to 4 x SFP28 for 4 x 25 Gig applications and QSFP+ to 4 SFP+ for 4 x 10 Gig applications.

While DACs offer a cost-effective and efficient way to connect equipment, they are limited in length. Passive DACs are limited to about 5-meter lengths, while active DACs that contain electronics for signal conditioning can reach to 10 meters. The distance limitations of DACs make them suited for a Top-of-Rack (ToR) data center topology.

AOCs

AOCs are similar to DACs in that they are factory terminated and provide an easy, high-performance direct connection between equipment. However, they leverage the distance capabilities of fiber optic cable and embedded transceivers to form a complete cable assembly that can support up to 100-meter lengths. Like DACs, they are available in SFP+, SFP28, QSFP+ or QSFP28 form factors for 10 to 100 Gig applications, as well as hybrid breakout assemblies for cost-effective 4 x 25 Gig and 4 x 10 Gig deployments. The greater distance capability means that AOCs can be used in ToR, End-of-Row (EoR) and Middle-of-Row (MoR) topologies.

Considered a cost-effective alternative to structured cabling and optical transceivers due to their lower material cost and power consumption, AOCs also benefit from the smaller diameter of fiber that allows for tighter bend radius, easy installation and increased airflow for cooling.

Fiber Cabling with Transceivers

The third option for high-speed data center links is to use two optical transceivers and connectorized fiber, either factory or field terminated to duplex or multi-fiber MPO connectors. Fiber cabling with transceivers can support much greater distances of up to 10 kilometers depending on the application and fiber type. That makes them suitable for virtually any architecture (e.g., 3-tier, full mesh, leaf-spine) and topology (e.g., ToR, EoR, MoR) including large hyperscale data centers where switches may be centrally located well over 100 meters away from servers.

Fiber cabling with transceivers also supports the use of standards-based structured cabling that offers the most flexible deployment and the use of cross-connects or interconnects for superior manageability. This can be useful in multi-tenant colocation data centers where connections between customer and service provider equipment needs to be accomplished outside of the meet-me room or customer space. While implementing cross-connects or interconnects adds additional points of failure, reliability is improved by ensuring that quality fiber cable and connectivity (including jumpers) are purchased from reputable manufacturers with third-party verified standards-based performance.

The other main benefit of using fiber cabling with transceivers is that the fiber can support multiple generations of applications and does not need to be replaced—only the transceivers need to be replaced. Multimode fiber currently supports 10 to 400 Gig, while Singlemode will support up to 800 Gig and beyond.

While the cabling lasts generations, the use of structured cabling with transceivers can ultimately be as high as 74% more expensive than DACs and AOCs due to several factors. First, the material cost of cabling and separate transceivers is higher than DACs and AOCs. Second, the power consumption is higher per port on the equipment side. Initial installation is also more complex and can cost up to three times more due to more components and the need to ensure proper polarity as explained here in one or our previous blogs.

How Do You Choose?

When it comes to selecting an option for high-speed data center links, first and foremost you need to know your application, architecture and topology to understand your speed and distance requirements. For example, if you’re deploying an EoR or MoR topology with link distances greater than 7 to 10 meters, that pretty much rules out DACs. You also need to consider your plans for the future. If you plan to eventually migrate to a 200 Gig application like 200BASE-SR4, Multimode fiber with MPO connectivity deployed today for 100 Gig will support it.

While application and topology may dictate your choice, we also know that budget is one of your biggest decision factors—including material, labor and even cooling. We put together the following comparison table for a snap shot .