Shulou(Shulou.com)12/24 Report--

The popularity of AI has boosted Nvidia's market capitalization and successfully joined the trillion-dollar club. As the soul of the company, what stories has Huang Renxun experienced along the way? How did the leather traveler build his graphics card empire step by step?

From the neural network AlexNet to ChatGPT, and then to the explosion of generative AI, Nvidia's GPU plays an important role.

In this AI gold rush, Nvidia's market capitalization rose, successfully entered the trillion-dollar club, becoming the sixth largest company in the world by market capitalization.

If it is said that behind Nvidia's success, it must be inseparable from this soul-Huang Renxun.

Everyone knows the story of Steve Jobs, Gates and other technology giants, but Lao Huang, who has been reluctant to appear in public, has little known about his experience except in leather clothes.

This time, the latest interview from the New Yorker delves into Lao Huang's entrepreneurial process, management style, and how to lead Nvidia to success.

Huang Renxun was born in Taiwan in 1963. At the age of nine, he and his brother were sent to the United States to study at Oneida Baptist College (Oneida Baptist Institute, in Kentucky) in Kentucky.

Huang Renxun lives with a 17-year-old roommate who can read. In exchange, his roommate teaches him to push. Every night before going to bed, Huang Renxun does 100 push-ups.

Huang Renxun could not take classes in this school because he was too young, so he went to a nearby public school.

At that time, the headmaster introduced the short Asian immigrant with long hair and strong accent. However, it is precisely because of these characteristics that Huang Renxun is bullied by his classmates.

A few years later, Huang Renxun's parents were allowed to enter the United States and settled in Oregon, where the brothers were reunited with their parents.

Huang Renxun did well in high school and was one of the top table tennis players in the country. He joined the school's math, computer and science club, skipped two grades and graduated at the age of 16. But he also said, "I don't have a girlfriend." "

Later, Huang Renxun entered Oregon State University, majoring in electrical engineering.

In the introductory class, his lab partner was Lori Mills, who was serious, cute and had curly brown hair.

According to Huang Renxun, there were 250 students majoring in electrical engineering at that time, of whom there were only about three girls. Boys scramble to attract the attention of Mills, and Huang Renxun feels at a disadvantage. I am the youngest child in the class. I look only about 12 years old.

However, every weekend, Huang Renxun would call Mills and haunt her to do her homework.

"I want to impress her, not because of my appearance, but because of my ability to finish my homework. "

After six months of homework, Huang Renxun plucked up the courage to ask her out. She accepted the invitation.

After graduation, Huang Renxun and Mills got a job as a microchip designer in Silicon Valley-"she actually earns more than I do."

The couple got married, and a few years later, Mills left his job to raise their children. By that time, Huang Renxun had already started running his own department and was a graduate student at Stanford University in the evening.

Three people started their business because of a restaurant. In 1993, he co-founded Nvidia with two senior microchip designers, Chris Malachowsky and Curtis Priem.

Malachowsky and Priem hope to design a graphics chip. They initially named the company NVision, but later learned that the name had been used by a toilet paper maker.

Huang Renxun suggests using Nvidia, which is taken from the Latin I Invida, which means "jealousy". He chose Denny's as a place to organize his business because it was quieter than at home and had cheap coffee.

He worked at a restaurant chain in Oregon in the 1980s. "I find that my mind is the most active in adversity. "

Huang Renxun likes video games, and he thinks the market needs better graphics chips. At that time, artists began to assemble three-dimensional polygons in shapes called "primitives" instead of drawing pixels by hand, saving time and effort, but requiring new chips.

Nvidia's competitors used triangles as primitives, but Huang Renxun and his companions decided to switch to quadrilaterals. But it turned out to be a mistake and almost destroyed the company. Because shortly after Nvidia released its first product, Microsoft announced that its graphics software would only support triangles.

Due to the shortage of funds, Huang Renxun decided to return to the traditional triangular method. In 1996, he laid off half of Nvidia's more than 100 employees and bet the rest of the company's money on untested microchip production, which he was not sure would succeed.

The odds of success and failure are fifty-fifty, but we are going to close down anyway.

When the product, called RIVA 128, went on sale, Nvidia had only enough money to pay a month's salary. But the gamble paid off, with Nvidia selling 1 million RIVA in four months.

Huang Renxun encouraged his employees to continue to ship goods with despair, and in the days to come, whenever he gave a speech to his employees, he began by saying, "our company is going out of business in 30 days." This is still the unofficial motto of the company.

In the center of Nvidia headquarters in Santa Clara, there are two huge buildings, each triangular. The interior of the building is a microcosm of this shape, from sofas and carpets to the splash shield of urinals.

There is a bar on the top floor of each building, and employees are encouraged to use the office as a flexible space for dining, coding and socializing. If the employee is eating at the conference table, AI can send a cleaner to clean up within an hour. Nvidia was named one of the best workplaces in the United States before the share price rose.

In the standard computer architecture, most of the work is done by microchips called central processing units (CPU). For decades, the main manufacturer of CPU has been Intel, which has repeatedly tried to force Nvidia out of the market.

Huang Renxun described the relationship between Nvidia and Intel as "Tom and Jerry relationship"-whenever they approached, we picked up the chip and ran away.

In response, Nvidia takes a different approach. In 1999, the company launched a graphics card called GeForce shortly after it went public.

Different from the general CPU, GPU decomposes complex mathematical tasks into small computations, and then processes them all at once by parallel computing. The CPU functions like a delivery truck, delivering packages one at a time, while the GPU is more like a motorcade, shuttling through the city.

The GeForce series is a success. The popularity of the Quake video game series, which uses parallel computing to render monsters that players can shoot with grenade launchers, has been boosted by a series of video games.

"Quake" series also launched a multiplayer mode, PC players in order to gain an advantage, each upgrade will buy a new GeForce graphics card.

In 2000, Ian Buck, a graduate student in computer graphics at Stanford University, connected 32 GeForce graphics cards together and used eight projectors to play Quake. This is the first 8K resolution game console, which occupies the whole wall. "It was beautiful."

GeForce graphics cards come with a primitive programming tool called shaders. Funded by the American research institute darpa, Ian Buck hacked into the shader and accessed the following parallel computing circuits, turning GeForce into a low-cost supercomputer.

Soon after that, Ian Buck began to work at Nvidia.

Since 2004, Buck has been responsible for the development of Nvidia supercomputing software package (CUDA). Huang Renxun's vision is to enable CUDA to run on every GeForce graphics card.

While Buck was developing software, Nvidia's hardware team began allocating space for supercomputing on microchips. Arjun Prabhu, Nvidia's chief chip engineer, likens microchip design to urban planning, with different areas of the chip dedicated to different tasks.

When CUDA was released at the end of 2006, Wall Street reacted with consternation. Huang Renxun brought supercomputing to the public, but Volkswagen did not show that they needed such a thing.

Ben Gilbert, host of Acquired, a popular Silicon Valley podcast, said Nvidia spent billions of dollars targeting an obscure corner of academic and scientific computing, which was not a big market at the time.

By the end of 2008, Nvidia's share price had fallen by 70%.

Huang Renxun believes that the existence of CUDA will expand the field of supercomputing. However, this view has not been widely shared.

2 Zhang Yingwei graphics card, 1 CUDA architecture, detonated neural network at the beginning of the 20th century, mentioned AI, is a completely unpopular discipline. The progress of artificial intelligence in image recognition, speech recognition and other fields has been stagnant.

In this unpopular academic field, the use of "neural networks" (brain-inspired computing structures) to solve problems has not been favored by many computer scientists.

At that time, deep learning researcher Bryan Catanzaro dissuaded Lao Huang, "Don't study neural networks." Because at that time, people thought that it was out of date and didn't work.

Catanzaro will also continue to study neural network researchers, collectively known as "prophets in the wilderness".

One of the prophets refers to Geoffrey Hinton, who retired from the University of Toronto and is known as the godfather of AI.

In 2009, Hinton's team used Nvidia's CUDA platform to train a neural network to recognize speech.

Unexpectedly, Hinton himself was surprised by the quality of the results of the study and reported the results at a meeting that year. Then he took the initiative to contact Nvidia.

"I sent an email saying," listen, I just told thousands of machine learning researchers that they should buy Nvidia graphics cards. Can you give me a piece for free? "".

However, Nvidia's final reply was only one word "No".

Despite being left out in the cold, Hinton encouraged his students to use CUDA, including his proud apprentice Alex Krizhevsky.

In 2012, Krizhevsky and research partner Ilya Sutskever bought two GeForce graphics cards, the GTX 580GPU, under tight budgets.

Then Krizhevsky began to train the visual recognition neural network, AlexNet, on Nvidia's parallel computing platform, inputting tens of millions of images into it in a week.

"the two GPU pieces in his bedroom were buzzing all the time," Hinton recalls. "it's conceivable that his parents must have paid a considerable amount of electricity."

Subsequently, Krizhevsky and his partner took AlexNet to the annual ImageNet competition and won the championship, and the first deep convolution network model was born.

The ability of GeForce graphics card surprised both Ilya and Krizhevsky.

In fact, in early 2012, Google researchers Enda Wu and Jeff Dean trained a neural network that could "identify cats."

Google's work uses about 16000 CPU, while Sutskever and Krizhevsky use just two Nvidia circuit boards to produce "world-class" results.

AlexNet correctly identified pictures of items such as scooters, leopards and container ships. However, AlexNet scored so high in the race that organizers initially wondered whether Krizhevsky cheated in some way. Because neural networks were not popular at the time, Ilya and Krizhevsky were the only teams that used the technology.

"it was a big bang moment," Hinton said. This is the paradigm shift.

This 9-page masterpiece of "ImageNet Classification with Deep Convolutional Neural Networks" has been cited 140000 times since its birth in 2021, making it an important milestone in the history of computers.

Krizhevsky pioneered many important programming techniques, but his main discovery was that "dedicated GPU can train neural networks 100 times faster than general-purpose CPU."

"without CUDA, machine learning would be very troublesome," Hinton added.

Over the next few years, every contestant in the ImageNet competition used a "neural network". By the mid-1920s, the neural network trained on GPU had achieved 96% accuracy in image recognition, far exceeding that of humans.

Over the past decade, Huang Renxun has achieved great success in promoting the popularity of supercomputing and GPU.

He said, "in fact, they can now solve completely unstructured computer vision problems, so what else can you teach it to do next? "

Lao Huang bet again: Nvidia upgraded "AI Company" from the graphics company.

The answer seems to be: anything!

Huang Renxun concluded that neural networks will revolutionize society, and he can use CUDA to capture the necessary hardware market.

At the time, he announced another bet on the company.

"everything will turn to deep learning," he sent an email on Friday night. "We are no longer a graphics company. From next Monday morning, we are an artificial intelligence company.

Nvidia's transformation, literally, is so fast.

Just as Huang Renxun sent the email, he went to Catanzaro, Nvidia's chief artificial intelligence researcher, to conduct a thought experiment.

"he asked me to imagine bringing 8000 Nvidia employees into the parking lot, and then I was free to choose anyone from the parking lot to join my team," Catanzaro said.

After the success of the Nuggets shovel in AlexNet, venture capitalists began to invest a lot of money in AI.

"We have been investing in many startups that apply deep learning to many areas, each of which is effectively built on Nvidia's platform," Andreessen Horowitz's Marc Andreessen said in 2016.

Around that time, Nvidia delivered the first dedicated artificial intelligence supercomputer, DGX-1, to OpenAI's research team.

Huang Renxun personally took DGX-1 to OpenAI's office, where Musk, the then chairman, opened the box.

In 2017, Google researchers proposed the neural network architecture of Transformer. The following year, OpenAI researchers used Google's framework to build the first "generative pre-training Transformer."

The GPT model is trained on the Nvidia supercomputer, uses a large text corpus, and learns how to make human-like connections.

At the end of 2022, after years of iteration, Deep-Fried Chicken ChatGPT was finally released to the public.

Since then, the demand for Nvidia graphics cards has exploded.

Among them, the most powerful DGX H100, a metal box weighing more than 160kg, which costs as much as $500000, has been out of stock for months.

The DGX H100 runs five times faster than the hardware that trains the ChatGPT, and can train the AlexNet in less than a minute.

Nvidia expects to sell 500000 DGX H100s by the end of 23.

The stronger the processing power applied to the neural network, the more complex its output. For state-of-the-art AI systems, dozens of Nvidia DGX H100s may be required.

If that's not enough, Nvidia will arrange the computers like a library stack and fill the data center with tens of millions of dollars worth of supercomputing devices.

Obviously, there is no obvious limit to the ability of artificial intelligence.

Over the next few years, Nvidia's hardware will accelerate to the speed of computer clock cycles, training a variety of similar artificial intelligence models.

According to reports, Nvidia sells equipment with a gross profit margin of close to 70%.

The huge profits make Google, Tesla, and startups, which develop AI training hardware, salivate.

Nvidia's fiercest competitor is AMD.

AMD has been run by another talented engineer, Lisa Su, since 2014. In the years since she became head of the company, AMD's share price has risen 30-fold, making her the most successful semiconductor CEO of her time, second only to Huang Renxun.

It is worth mentioning that Lao Huang and Su Zifeng are still related.

Huang's management law, Huang himself, rarely gives interviews. "I didn't do anything special, mainly the efforts of my team, and I'm not sure why I was chosen as chief executive," he said. "I don't have any particular driving force."

When Lao Huang made up his mind to run a business at the age of 30, his co-founder Chris Malachowsky said, "you are really not a good speaker because you are more introverted."

Lao Huang said, "I only have one super ability-to do homework." Dwight Diercks, director of Nvidia Software, says Lao Huang can master any subject in one weekend.

Huang Renxun prefers an agile corporate structure with no fixed department or hierarchy. Instead, employees submit a weekly list of the five most important things they are doing.

He himself, on the other hand, writes hundreds of emails and chats with employees every day, usually with only a few words. One executive compared the emails to a haiku and another to a ransom note.

Lao Huang has also developed a set of management maxims that he often quotes.

When arranging tasks, Lao Huang will ask employees to consider the "speed of light". This doesn't just mean acting quickly; instead, employees should consider the absolute speed at which a task can be accomplished, and then work in reverse toward an achievable goal.

Perhaps Lao Huang's most radical belief is that "failure must be shared."

In early 2000, Nvidia shipped a faulty graphics card with a loud and overactive fan.

However, instead of firing the product manager of the graphics card, Huang Renxun arranged a meeting for the managers to introduce to hundreds of people every decision they made that led to a fiasco.

Nvidia employees sometimes complain that Lao Huang is capricious.

Huang Renxun said, "in fact, what I think in my head is inconsistent with what I say in my mouth." When the dislocation is very serious, it will show anger.

Even when he is calm, Huang Renxun's strength may be overwhelming. "communicating with him is like sticking your finger into an electrical outlet," one employee described it.

Nevertheless, Nvidia's staff turnover rate is very low.

The soaring sales of GPU have also enabled Yingwei to successfully enter the trillion-dollar club as the world's dominant computing power. This is inseparable from the "crazy" management strategy of the leader Huang Renxun.

Lao Huang once said that when you start a company, it's natural to start with the first principle.

Reference:

Https://www.newyorker.com/magazine/2023/12/04/how-jensen-huangs-nvidia-is-powering-the-ai-revolution

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.