ByteDance is developing a model with 10 trillion parameters – that's even more than Anthropic's Mythos
Article excerpt
Highlighted: the sentence this signal was extracted from
ByteDance is developing a new large language model that could reach 10 trillion parameters and become the largest not only in China but also worldwide. It is currently in the early stages of training, which could take up to six months. This is reported by the Financial Times. At present, the world's largest models belong to Anthropic, and although the start-up itself does not disclose the number of parameters, industry experts believe that Mythos has around 8 trillion parameters, whilst Fable 5 has 5 trillion. China's largest model, Moonshot's Kimi K3, has around 3 trillion. ByteDance plans to surpass this with a new LLM, which is currently in the pre-training phase, after which a little more time will be needed to fine-tune and release it. However, the number of parameters alone does not guarantee that the model will outperform its competitors; this will also depend on the quality of the training data and the training methods themselves. The very fact that ByteDance is planning such a large-scale release reflects recent trends, with Chinese AI laboratories actively catching up with Western competitors such as Anthropic, OpenAI and Google. Other Chinese companies are also working on new large models, and some of those already released, such as the aforementioned Kimi K3, are on a par with Mythos and Fable. Interestingly, ByteDance does not use the distillation process...
Keep reading with a free account
The rest of this article, and every signal for ByteDance, is in your free account.
