Reuters: Distillation of US models in China spreads from research to industry
Reuters reviewed more than 300 Chinese documents: Chinese organisations share out the work of training small models on answers from US models, and Beijing supports the practice. Distillation makes up for a shortage of computing power but does not replace chips.
Reuters reviewed more than 300 Chinese research papers, patents, company announcements, job advertisements and government documents. Germany’s heise online and The Decoder reported on the investigation on 17 November. In China, distillation is increasingly moving from research into industrial use, Reuters found.
In distillation, a powerful teacher model produces answers or solution steps that are used to train a smaller student model. In China, the work is divided up, Reuters reported. Researchers have US models generate answers, code and problem solutions. Companies develop programs that automate the collection, and specialist workers prepare the data for training. The expensive query is made once, and the answer can be reused as a training example again and again.
In November 2024, for example, researchers at Shanghai Jiao Tong University selected about 5,000 answers from OpenAI’s o1. They translated the answers with GPT-4o mini, turned them into 10,750 bilingual examples and used these to further train Alibaba’s Qwen2.5-72B. The researchers themselves warned that relying too heavily on distillation can hold back independent innovation.
Distillation does not transfer all of a large model’s capabilities, nor does it remove the need for powerful hardware. It can, however, teach small models selected capabilities. A US official quoted by Reuters said the US administration sees distillation as the main reason China has narrowed the gap in the AI race. Two people close to Chinese developers, by contrast, see it as only one factor alongside other developments.
Beijing does not order companies to use distillation but supports the practice, Reuters reported. In January, eight government agencies published the ‘AI+Manufacturing’ programme, which explicitly names distillation as a method that supports industrial applications.
Distillation is not illegal, but it violates the terms of service of companies such as OpenAI and Anthropic. Treasury Secretary Scott Bessent is threatening sanctions and placement on the US export-control Entity List if covert distillation turns into intellectual property theft, The Decoder reported.
China may need distillation even more in future, heise online wrote. This week the US announced a rule that restricts exports of AI chips to four Southeast Asian countries and Brazil, and the rental of computing power to Chinese companies outside the US. In the spring, ChinaTalk estimated that about two-fifths of China’s AI computing runs remotely on capacity rented abroad.