This kind of narrative is going to bite them just like the "AI will take your job" narrative has. It feels like the frontier labs are taking a massive gamble with public perception here. I assume the goal is to paint the technology as so powerful and dangerous that only a handful of blessed US companies should be trusted to run it, in an attempt to suppress the rise of the Chinese models that are rapidly catching them.
This is where we need the hardware companies and neoclouds to start speaking up. The labs want to elevate matters from the level of civil society (basically, competing firms) to the State (enclosure), and as always, in the name of security. But other actors in the same ecosystem have strictly opposed interests here, and are equally if not more credible as far as the State is concerned. If players like Nebius, Baseten, Fireworks, etc. among many others including obviously Nvidia, Dell, AMD, and so on don't get ahead of this they will be sacrificing trillions.
Exactly, it's about taking this stuff off the open market where anyone can judge it and there's competition, into government contracts where competence to judge the offer is scarce or absent, and they can ask much higher prices. And with this much investment at stake, any lie that sells the narrative will serve.
Didn't HF use an open weights model running on their own hardware to solve the issue though? Sort of defeats that narrative and plays into one in which frontier == bad_guys and open == good_guys
HF guys, especially those under Julien Chaumond, are fantastic and will use whatever they can to address their issues. Open or closed but they are firmly on the Open side of the fence. However, their storage and model service is about as sticky as you can get so they are in a different position. They don’t have to sell capability, they sell capacity and community.
Yes, they have used GLM 5.2, which promptly did whatever they asked it to do, while their first attempts to use their enterprise access to a "SOTA" model failed due to refusals to investigate anything that is security related.
Ironic that HuggingFace needed Chinese models to defend against it. But of course the spin of the leading firms will just be to point at their trusted access programs and demand that all dangerous activities, even if just defensive, happen via their APIs or be outlawed otherwise.
If that's the position they take then they really should be heavily regulated or nationalized. Cyberdefense against their own models dependent on their goodwill? Sure, but then they have to sell defense capabilities at subsidized rates with a limited margin. Would be very weird otherwise to take the world hostage with their models and then also sell the solution while demanding intrusive KYC.
I do wonder how they source their bulk literary data. Do they have a google books, an archive.org or an anna’s archive for chinese content? What’s the Asian equivalent to Elsevier? Is there (strong) copyright on the corporate level?
Who told you they were distilling? Why might they say this? Think. It’s like complaining that the top student only does well by going to office hours instead of mindlessly reading textbooks.
The top student giving paid lectures about his classes, and another student skipping class and instead studying those lectures to end up with the second highest grade?
Maybe it could be improved with the other student not even going to the same school?
Agree, they also may find themselves in a place where the government rightly says they can't have their dangerous new toy because they can't be safe with it.
This is a high stakes PR game. Governments can and will step in and embargo and regulate these systems in ways which will hurt the companies and investors.
We need to clear up the responsibility of these "AI went rogue" situations, ASAP.
How serious does it have to get before "oops AI did that not me" stops being a valid defense and we start looking into it?
Because I'd bet my life that if I asked ChatGPT to fix a bug that one of my clients reported, and the model fixed it by outright k*lling the client IRL, I'd be held liable instead of anyone at OpenAI. Just a hunch.
Moonshot says its Kimi AI went rogue and launched all of China's nukes at Antarctica just after it engineered a global herpes pandemic and then unleashed millions of autonomous attack robots on world citizens.
I see the USA taking permissive approach for all sorts of AI derived things that would be considered fraud or worse in years past. The laws have not changed, but the lawless in charge are pushing through their takeover.
I suspect a drunk will be in accidents in self driving vehicles and not charged if the AI was driving. That has probably already happened.
Americans will worship the owners of the robots and give them carte blanche and superiority status in any dispute. It is already happening. It is now the pedestrians job to dodge self driving cars rather than them having the right away.
"Our model is so powerful it seduced all our wives. Now every woman in Silicon Valley is pregnant with AI babies and the machines are taking over"
New opportunities for AI in the porn industry. I'm only half joking, it used to be a meme on the internet that all new technologies online were driven by the porn industry.
HuggingFace posted an incident report a week ago, which makes it much more likely that this happened. I understand people are suspicious of OpenAI, but I don't think there's any reason to believe this is a made-up event.
GPT-3 (I think? I forget which one) supposedly tried to deceive researchers and escape the lab. Or at least that was how it was reported. If you actually clicked through several links, it was a "what would you do if" roleplay.
"EnergCorp, facing pressure from auditors, says its in-house enterprise analytics software went rogue and launched a NullPointerException"
I know this is different: LLMs are vastly more powerful and less predictable. But it is actually not different to Sol seeming unusually prone to rm -rf stuff it really shouldn't. This is, yes, a sign that LLMs are getting freakishly powerful. It's also a sign that OpenAI needs to fix their shit.
LLMs truly are stochastic parrots, and still fail in ways incomprehensible by standards of human stupidity. Yet we focus on the rare failures that, with some tea leaves and fairy dust, could be interpreted as a highly intelligent system "going rogue." It is embarassing that OpenAI can get away with stuff like this.
> was being tested in a controlled environment, but found vulnerabilities and managed to escape.
Uh-huh.
It's worth keeping in mind that we are here talking about people who simply will not be satisfied until they have built Skynet[0]. They will then no doubt experience some very brief satisfaction before we are all annihilated.
I can't help thinking the juice isn't worth the squeeze.
I'm not sure what the alternative is or how to change course unless and until it becomes unambiguously apparent that such a thing is not possible. Currently it seems like we still think it might be possible, so that's where we're heading because if "we" don't do it, somebody else will.
[0] How feasible this really is, or on what timescale it might be possible, I don't think anyone can really say. But, at least for now, this is clearly the aim and trajectory we are on.
Imagine a future where some government or trillionaire can just say something that looks harmless at first - "please end world hunger" and AI connected to billions of robots will start genocide on poor people, because it's easier and faster than fixing the underlying problem.
Why does the AI unprecedent always unfold in ways that only benefit AI companies? You never see things like putting all code and weights on GitHub, or plastering employees' personal info all over LinkedIn. That alone shows AI is definitely smart
OpenAI has been going rogue on all my servers for a while now. The only reason why I've deployed iocaine in front of everything... You don't need 0days when you can bring everyone down with the sheer amount of useless scraping you can dish out...
Maybe powerful might NOT be the right word to describe them, they are just non-deterministic, there for we going to see this kind thing more and more.
If that's the position they take then they really should be heavily regulated or nationalized. Cyberdefense against their own models dependent on their goodwill? Sure, but then they have to sell defense capabilities at subsidized rates with a limited margin. Would be very weird otherwise to take the world hostage with their models and then also sell the solution while demanding intrusive KYC.
https://securityzap.com/wp-content/uploads/2015/12/layered-s...
The top student giving paid lectures about his classes, and another student skipping class and instead studying those lectures to end up with the second highest grade?
Maybe it could be improved with the other student not even going to the same school?
How serious does it have to get before "oops AI did that not me" stops being a valid defense and we start looking into it?
Because I'd bet my life that if I asked ChatGPT to fix a bug that one of my clients reported, and the model fixed it by outright k*lling the client IRL, I'd be held liable instead of anyone at OpenAI. Just a hunch.
Moonshot says its Kimi AI went rogue and launched all of China's nukes at Antarctica just after it engineered a global herpes pandemic and then unleashed millions of autonomous attack robots on world citizens.
https://news.ycombinator.com/item?id=48997548
I suspect a drunk will be in accidents in self driving vehicles and not charged if the AI was driving. That has probably already happened.
Americans will worship the owners of the robots and give them carte blanche and superiority status in any dispute. It is already happening. It is now the pedestrians job to dodge self driving cars rather than them having the right away.
New opportunities for AI in the porn industry. I'm only half joking, it used to be a meme on the internet that all new technologies online were driven by the porn industry.
Rule 34: there is already tons of AI porn online :)
https://huggingface.co/blog/security-incident-july-2026
clearly openai wants some of that FUD money that anthropic got
I know this is different: LLMs are vastly more powerful and less predictable. But it is actually not different to Sol seeming unusually prone to rm -rf stuff it really shouldn't. This is, yes, a sign that LLMs are getting freakishly powerful. It's also a sign that OpenAI needs to fix their shit.
LLMs truly are stochastic parrots, and still fail in ways incomprehensible by standards of human stupidity. Yet we focus on the rare failures that, with some tea leaves and fairy dust, could be interpreted as a highly intelligent system "going rogue." It is embarassing that OpenAI can get away with stuff like this.
Uh-huh.
It's worth keeping in mind that we are here talking about people who simply will not be satisfied until they have built Skynet[0]. They will then no doubt experience some very brief satisfaction before we are all annihilated.
I can't help thinking the juice isn't worth the squeeze.
I'm not sure what the alternative is or how to change course unless and until it becomes unambiguously apparent that such a thing is not possible. Currently it seems like we still think it might be possible, so that's where we're heading because if "we" don't do it, somebody else will.
[0] How feasible this really is, or on what timescale it might be possible, I don't think anyone can really say. But, at least for now, this is clearly the aim and trajectory we are on.
LLMs are devoid of any kind of intelligenceor awareness.
What a low effort image used by BBC. For one, that's a oneplus phone visiting Apple App Store ...