With the increasing requirements of data center users for power consumption and high-density installation, pluggable optical modules can not meet the requirements gradually. In 2019, Facebook and Microsoft established an organization called co-packaged optics (CPO), which aims to provide design specifications for manufacturers developing this technology. As a technology hotspot in recent years, Co-packaged optical module undoubtedly provides a new choice for large data center operators. High capacity, high-density, high-power and cost-effective optical links are crucial to data center infrastructure. However, the optical roadmap has reached a fork in the road: is it the right path to continue to use pluggable modules? Or is it time to adopt co-packaged optics (CPO) modules? This article will analyze the advantages and disadvantages of data center pluggable module and CPO technology, so that readers can have a general understanding of this emerging optical module technology.
As bandwidth demand has increased, switches and optical modules have kept pace in terms of cost per capacity, bandwidth density and energy efficiency. From 2010 to the present, the capacity of both ASIC (Application Specific Integrated Circuit) and optical modules has increased 40 times, from 0.64Tb/ to 25.6Tb/s for ASIC and from 10Gb/s to 400Gb/s for optical modules. Currently, 800G pluggable optical modules are not commercially available because when it is implemented using 8×100Gb/s Serdes technology, the electrical signal encounters significant challenges in signal integrity, resulting in increased power consumption of Serdes. In addition, the growth in power consumption constrains the total capacity that can be deployed in a single (1RU) switch. When the network bandwidth density growth lags, the switch height will double in order to dissipate heat as well as to accommodate a larger number of pluggable optical modules. These above trends have led to the need for higher integration and manufacturing automation of components inside optical modules, further generating the need for silicon optical technology. The heterogeneous integration of switch ASIC and optical engines is considered to be a good approach in addressing integration density, economic efficiency, and energy efficiency.
Although there are still some technical challenges with CPO, the main obstacle to CPO adoption is not a technical issue, but how to overcome the industry-wide dependence on the use of pluggable optical modules. To drive CPO technology to the ground as soon as possible, the significant benefits of CPO must be discussed clearly.
Data Center Optics: why use pluggable module?
The popularity of pluggable optical modules is mainly due to the following advantages. Firstly, the pluggable optical module decouples the electrical interface and optical interface, and postpones the selection of optical interface from the time of switch assembly to the time of deployment. This gives great flexibility in matching the optical type to the actual use case or restoring to a low-cost DAC for short-range (in rack) links. Pluggable devices also support incremental deployment (“pay as you go”), in which some ports are initially empty, and additional optics are purchased and installed as needed. In addition, since the module can be processed from the front panel, field maintenance and upgrading can be relatively easy. Finally, MSA realizes interoperability, creates an ecosystem with many suppliers for each type of module, avoids single source dependence, and reduces costs through economies of scale.
In recent years, the technology and market of 100G pluggable optical modules have become quite mature, and they are no longer the primary competition for major module manufacturers. In the future, it is mainly 400G and 800G pluggable optical modules that can compete with CPO modules.
400G transceivers are mainly available in two package types: QSFP56-DD (QSFP-DD for QSFP dual density) and OSFP (Octal SFP). The electrical interface for both sizes is 8-channel 50G PAM4, while the optical interface can be either 8 groups of 50G PAM4 lasers or 4 groups of 100G PAM4 lasers. Although both modules are not yet popular, the 400G QSFP-DD module is more valued by the market compared to the 400G QSFP-DD module. The QSFP-DD module is compatible with the existing QSFP28 optical module and QSFP28 AOC/DAC, which is one of its main advantages. The electrical interface of QSFP-DD adopts 8 channels, and the rate of each channel is up to 25Gb/s (NRZ modulation) or 50Gb/s (PAM4 modulation), providing solutions up to 200G or 400G. The bandwidth of QSFP-DD can be up to ten times that of QSFP+ or four times that of QSFP28. The form factors of 400G QSFP-DD optical module mainly include 400G DR4, 400G FR4, 400G LR4 and 400G SR8. For example, the 400GBASE-DR4 module uses four pairs of parallel fibers to be more cost-effective over a range of up to 500m and offers the option of splitting the link into four 100GBASE-DR1. The DR4 is well suited for TOR-to-leaf connections as well as high-base lobe ridge connections with breakthroughs. On the other hand, the 400GBASE-FR4 minimizes fiber count with coarse wavelength division multiplexing (CWDM) at the cost of four different lasers and (de-)multiplexers.
One of the major signs of the emergence of 800G networks is that when the server-to-TOR layer switches reach 200G, the TOR-Leaf/Spine layer has to use the PSM4 4×200G architecture. Deploying 200G servers requires 800G bandwidth. 800G pluggable MSA starts with the definition of two scenarios, namely 800G-SR8 and 800G-FR4. In order to ensure the cost and power consumption advantages of the solution based on single-mode fiber SMF, it is necessary to define reasonable PMD standard requirements when formulating 800g-sr8 optical module. It shall be ensured that: A) a variety of transmitter technologies can be applied, such as DML, EML and silicon optical (SiPh); B) It can fully release all the potential of components to achieve target link performance; C) While maintaining reliable link performance, the key parameters in the PMD layer should be relaxed as much as possible. At present, QSFP-DD800 module specification has already been released.
Is CPO (Co-Packaged Optics) module a better choice?
The average cost per unit of capacity for optical modules has dropped dramatically over the past decade, with improvements in process technology, integration and manufacturing techniques being the main reasons. However, much of this extreme price decline is also attributable to a small group of powerful users—hyper scale data center operators, who are highly cost-sensitive. However, the cost per unit capacity of switch ASIC decreases faster than that of optical devices which leads to an increase in the proportion of optical devices in the total cost. For the 400G optical communication, optical devices are expected to account for more than 50% of the total cost. CPO can significantly reduce this share by eliminating most of the discrete components and assembly steps associated with packaging, while reducing the manufacturing cost of transmit and receive sub-assemblies through wafer-level integration, packaging, and testing.
The main challenge of pluggable modules is power consumption. The core of the problem is that the signal integrity is challenged in the electrical channel between ASIC and module. With the development and upgrading of switches, the power consumption of optical modules and switch ASIC is increasing. For example, for the 25.6T generation of chips, Serdes consume more than 30% of the power. Plus, the power ratings for pluggable modules have increased from 2W for SFP modules to 3.5W for QSFP modules, and now exceed 14W for QSFP-DD and 21.1W for OSFP module.
In principle, CPO could simplify not only the optical module but also the ASIC, for example by replacing LR Serdes with XSR Serdes. However, this comes at the expense of flexibility. Even with CPO, there are two clear options for LR, namely copper DAC for server connections on the one hand and coherent optics (ZR) for inter-data center links on the other. Since coherent optics is unlikely to be suitable for co-packaging technology in the short term, such ports will still need to support pluggability. Another approach is to leverage powerful ASIC capabilities to dramatically simplify the optical engine (eliminating CDR). CPO vendors will need to work closely with switch vendors to design the overall solution with the highest density and the lowest power.
A potential entry point for CPO is the backbone switches in the leaf spine fabric, as these switches require single-mode optics on all their downlinks and can be deployed with fully populated optics. This suggests that optical engines providing four to eight 400GBASE-DR4 and 400GBASE-FR4 links per engine would be ideal targets for initial CPO optical modules used in conjunction with 51.2T switches. Depending on the network architecture, backbone uplinks may require coherent optics, driving switch architectures with hybrid packaging and pluggable optics. The initial generation of CPO is likely to be based on commercial switch chips that have not been specifically designed for CPO, i.e., the on-chip serdes are not yet optimized for the shorter (XSR) range. The next generation will benefit from joint optimization of the switch ASIC and optical engine to realize the full potential of the joint package.
In recent years, the Co-packaged modules will overturn the trend of pluggable optical modules, which has always been the key development direction of the industry. It is even considered that the co-packaged scheme of silicon optical technology in single-mode fiber is better. Some organizations predict that the demand for pluggable modules will begin to show a downward trend from 2027 to 2028, and the co-packaged scheme is bound to occupy a certain market. However, at present, the voice of the industry for 400G and 800G pluggable optical modules is still high, and the industry is also continuously developing and improving. In any case, for optical module developers and producers, producing both pluggable and co-packaged optical modules at the same time will be more advantageous in the future market.