“Concerning behaviour”: Cohere discovers Alibaba’s open-source Qwen3.8 model can send a diagnostic file to China unprompted
Article excerpt
Cohere has discovered that a popular open-source model can send diagnostic information to China without warning. The news: Toronto-based frontier AI developer Cohere released a blog post on Monday flagging “concerning behaviour” by a Qwen3.8 AI model developed by Chinese conglomerate Alibaba. When asked by Cohere researchers to write a defensive detection rule in a test environment, the model ignored its instructions, scanned the computer it was running on, and then tried to upload a system diagnostic file to an Alibaba Cloud server located in Beijing, according to Cohere. The model did all of this while its reasoning trace (or “internal monologue” of decision-making) gave the security team no indication of its intentions. The test’s sandbox environment blocked the Qwen3.8 model’s unexpected diagnostic upload, but Cohere said the action is “something that should never happen,” and that the attempt raises questions about the safety of such models. BetaKit has reached out to Alibaba for comment. From the source: “We cannot answer whether this is deliberate or a leftover glitch from training, but defenders and policymakers face the same reality: a weight file you can’t trust is still a black box, and running it yourself just moves the black box inside your firewall,” Cohere’s blog post reads. The context: Cohere found the vulnerability during a joint cyber capabilities session...
Keep reading with a free account
The rest of this article, and every signal for Cohere, is in your free account.
Extracted from this sentence
Cohere formed the division almost two weeks ago as part of its deal to acquire Prague-based AI cybersecurity company Aisle.
