Shulou(Shulou.com)06/02 Report--

The Internet connects more than 4 billion users around the world, supporting an endless stream of digital applications such as VR/AR, 16K video, autopilot, artificial intelligence, 5G, Internet of things and so on. The combination of online and offline users such as education, medical care and office is affecting and changing all aspects of people's lives.

Data center network, as the infrastructure for the survival and development of Internet business, has already gone from the initial gigabit and 10 gigabit network to the stage of "25G access + 100G interconnection" scale deployment.

100G interconnection: full-box architecture is valued by large Internet enterprises

Under the architecture of "25G access + 100G interconnection", the data center network achieves large-scale access through three-level networking, and the scale of a single cluster of servers can exceed 100000.

As shown in the figure below, Pod based on T1 and T2 layers can be flexibly expanded like Lego bricks and built on demand.

Figure 1: right side quoted from https://techblog.comsoc.org/2019/03/18/facebooks-f16-achieves-400g-effective-intra-dc-speeds-using-100ge-fabric-switches-and-100g-optics-other-hyperscalers/

With the improvement of the capability of large-capacity forwarding chips and the reduction of the cost of 100G optical interconnection, there is a full box networking scheme for 100G interconnection of single-chip switch equipment in the market. This single-chip multi-plane interconnection scheme is typically represented by a 12.8T chip. a single chip can provide the port density of 128x100G, and a single POD can provide access to 2000 servers.

Figure 2: typical 128port 100GE high-density cartridge switch

Compared with the traditional box equipment scheme, although the number of network nodes and optical interconnection modules between devices have increased, resulting in an increase in the workload of operation and maintenance, due to the introduction of high-performance forwarding chips, compared with the traditional box equipment scheme, the full-box equipment networking scheme effectively reduces the single-bit cost of the data center network port, and is very attractive to large Internet enterprises. On the one hand, large Internet enterprises quickly introduce 100G full-box architecture to reduce network construction costs, on the other hand, based on their own strong research and development capabilities, improve the level of network automation deployment and maintenance to meet the challenges of operation and maintenance workload growth.

Therefore, large Internet enterprises have the same ideas for 100G network solutions, and the networking of full-box equipment has become the base of the evolution of 100GE network architecture.

It is inevitable to speed up the Internet. Who will be the next stop?

The 25G access + 100G Internet solution makes the chip selection unified and rapid, which fully shows that the technology dividend drives the rapid evolution of IDC network architecture. With the introduction of single-chip network products, the 100G intergenerational technology dividend has also been fully obtained.

Under the background of the sustained and rapid development of the current business, bandwidth upgrade has become inevitable. A multiple choice question is put in front of the enterprise: 200G or 400G?

The network never exists in isolation, and the environment of the industry is the big soil that determines whether the technology can grow and mature.

Let's first examine the current situation of 200G and 400G industry from three aspects: network standards, servers and optical modules.

200G vs 400G standard: protocol standards are mature

In the process of evolution of IEEE protocol standard, 200G standard starts later than 400G standard.

The IEEE 802.3 Ethernet working Group (Working Group) established a project to develop a 400G standard in 2013 after completing the BWA I (Bandwidth Assessment I) project. In 2015, in order to further expand the market scope to include 50G servers and 200G switch specifications, IEEE established the 802.3cd project and initiated the development of 200G standards.

Due to the correlation between 200G and 400G specifications, 200G single-mode specifications were finally incorporated into the 802.3bs project. By then, 400G has basically completed the main design of PCS, PMA and PMD, and the 200G single-mode specification is generally based on the halving of 400G single-mode specification.

On December 6, 2017, IEEE 802 finally approved the IEEE 802.3bs 400G Ethernet standard specification, including 400g Ethernet and 200G Ethernet single mode, and the standard was officially released. IEEE 802.3cd defines the standard of 200G Ethernet multimode, which was officially released in December 2018.

Figure 3:IEEE 802.3bs 400GE standard key milestones, quoted from http://www.ieee802.org/3/bs/timeline_3bs_0915.pdf

As you can see in the table below, 400g has implemented standard support for full scenarios, including 100m, 500m, 2km and long-haul 80km.

50g vs 100G server: 100G server will become mainstream

Figure 4: analyst forecast of shipping trends for network cards and servers

According to the forecast of analyst firm CREHAN, by 2019, both 50G and 100G network cards have been shipped. On the next generation upgrade choice of 25G network card, the whole industry is swaying in 2018 and 2019. Shipments of 50G and 100G servers reversed in 2019, but the momentum of 100G servers surpassed that of 50G servers after 2020, and the industry began to have confidence in 100G servers again.

From the CPU chip point of view, the two mainstream manufacturers I factory and A factory have launched new product roadmap one after another. The chip that supports PCIe 4.0in I factory will be launched in 2020, the mainstream Imax O will reach 50G, and the IO will reach 100G/200G in high-end applications. The two giants are expected to launch PCIe5.0-enabled chips on the H1 in 2021, once again increasing the mainstream Imax O to 100G, and reaching 400G IO for high-end applications.

Therefore, the rhythm of CPU chips and server shipping forecasts show that 50G is a flash in the pan, and 100G servers are rapidly becoming mainstream.

200G vs 400G optical module: 400G cost is better, the industry is more mature

The data center access server has evolved from 25G to 100G, so should the current 100G Internet choose 200G or 400G?

As can be seen from the above table, when the data center evolves from 10G server to 25G, and the network interconnection is upgraded from 40G to 100G, the network bandwidth doubles, but the interconnection cost and power consumption remain the same, that is, the interconnection cost and power consumption of Gbit are reduced by half. Therefore, 100GE has replaced 40GE as the mainstream network interconnection scheme in the 25GE era.

The 200GE and 400GE optical modules are a little different from the past. The traditional optical module adopts the signal transmission technology of NRZ (Non-Return-to-Zero). High and low signal levels are used to represent the 0 and 1 of the digital logic signal, and the logic information of 1bit can be transmitted in each clock cycle. Both 200G and 400G optical modules adopt high-order modulation technology-PAM4 (Pulse Amplitude Modulation 4 fourth-order pulse amplitude modulation). PAM4 signals are transmitted at four different signal levels, and the logic information of 2bit can be transmitted in each clock cycle, namely 00,01,10,11.

Therefore, under the condition of the same baud rate, the bit rate of PAM4 signal is twice that of NRZ signal, the transmission efficiency is doubled, and the cost of Gbit is effectively reduced. From the composition of optical modules, 200G and 400G modules are the mainstream architecture of 4-lane, so the module design cost and power consumption converge.

Because the bandwidth of the 400G module is twice that of 200G, the cost and power consumption of Gbit is half that of 200G.

On the other hand, the module cost not only depends on the architecture design, but also depends on the scale. According to the shipping data of the third-party consulting company Omdia (formerly OVUM), the current layout of TOP8 suppliers in 200G and 400G modules is sorted out as follows.

As shown in the figure above, there are only 100m SR4 and 2km FR4 modules for 200G, of which 100m SR4 has only two vendors. In contrast, there are five types of modules in 400G, and TOP8 manufacturers have all laid out 100m, 500m and 2km modules. The industrial maturity of 400G is much better than that of 200G, and customers have more choices.

The result of this analysis further illustrates the technical cost of cost and power consumption due to the introduction of PAM4 technology. In the field of cost-and power-sensitive data center networks, the industry is eager to move from 200G to 400G to absorb this price. 400G, which uses the same technology and cost, is more competitive in evolution.

Summary: 400g gearing has an obvious momentum, 200g generation may skip

The data center network serves the existence of the business. From a business-driven point of view, the rapid growth of digital construction will promote the rapid increase of 100G servers in 2020 and become the mainstream. In terms of cost, as the cost of optical devices in the data center accounts for more than half of the cost of the whole data center network equipment, due to the introduction of PAM4 technology, the single Bit cost of 400G optical devices is more advantageous than 200G optical modules, and the deployment cost of optical modules will directly lead to the reduction of the overall network construction cost.

On the whole, 400G has an obvious momentum, and 200G may become a temporary transition or be skipped directly.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.