IBM Bets on Co-Packaged Optics to Handle AI WorkloadsIBM Bets on Co-Packaged Optics to Handle AI Workloads

The idea of leveraging photonics technology to meet future computing demands is not new. Companies such as Intel have dabbled with optical technology for applications such as satellite communications.  But the trick has been finding potential large-scale applications that can justify the cost and challenges in adopting optical technology. The rapid rise of generative AI has created the need for high-capacity, high-speed data handling, leading IBM to bet on adopting optics technology.

IBM researchers designed and assembled a polymer optical waveguide (PWG) to enable the development of co-packaged optics (CPO) to enable light-speed connectivity within data centers. The researchers believe these optics will redefine how the computing industry transmits high-bandwidth data between chips, circuit boards, and servers.

“Co-packaged optics enables high-speed optical connectivity through fibers assembled in close proximity to accelerators, reducing the communications gap between AI models,” said Mukesh Khare, General Manager of IBM’s Semiconductors division and Vice President of Hybrid Cloud Research,” during a recent press conference. Khare has been leading a global team of researchers looking at next-generation computing solutions.

Copper wires slow data

Related:Intelligent Interconnect Eases Connection of Optical Modules

The need for alternative optical technology in data centers has become painfully apparent as while data centers use fiber optics for their external communications networks, the racks in these data centers still usually run communications on copper-based electrical wires. These wires connect GPU accelerators that may spend more than half of their time idle, waiting for data from other devices in a large, distributed training process which can incur significant expense and energy.

To mitigate these issues, IBM set out to develop a CPO prototype module that enables high-speed optical connectivity to increase the bandwidth of data center communications, minimizing GPU downtime while accelerating AI processing. IBM’s research paper outlined how these new high bandwidth density optical structures, coupled with transmitting multiple wavelengths per optical channel, can potentially boost bandwidth between chips as much as 80 times compared to electrical connections. This CPO technology would help meet AI’s increasing performance demands.

Boosting optical density six-fold

IBM’s module enables chipmakers to add six times as many optical fibers at the edge of a silicon photonics chip, called “beachfront density,” compared to the current state-of-the-art CPO technology. Each fiber, about three times the width of a human hair, could span centimeters to hundreds of meters in length and transmit terabits of data per second. Scientists assembled a high-density PWG at 50 micrometer pitch optical channels, adiabatically coupled to silicon photonics waveguides, using standard assembly packaging. According to IBM, the optical module would reduce energy consumption by a factor of five compared to mid-range electrical interconnects, while allowing data center interconnect cables to be lengthened. This would then in turn speed up AI model training.

Related:Intel to Supply Optical Tech for DARPA Satellite Communications

The research paper also indicates that these CPO modules with PWG at 50 micrometer pitch are the first to pass all stress tests required for manufacturing. Components are subjected to high-humidity environments and temperatures ranging from -40°C to 125°C, as well as mechanical durability testing to confirm that optical interconnects can bend without breaking or losing data. Moreover, researchers have demonstrated PWG technology to an 18-micrometer pitch. Stacking four PWGs would allow for up to 128 channels for connectivity at that pitch.

The design, modeling, and simulation work for the technology was performed at the National Semiconductor Technology Center (NTSC) in Albany, NY, with the researchers assembling the prototypes and testing modules at IBM’s chip assembly and test site in Bromont, Quebec.