> it runs up to 1.7× faster than comparable MoEs on-device, with 184 tokens/s on an Apple M5 Max and up to 21K output tokens/s on a single NVIDIA B300 GPU
Wow, those are seriously impressive numbers for how intelligent this model seems to be!
One thing that this article doesn't clarify, and that I guess I really should look into, is what exactly AutomationBench tests for as compared to other benchmarks. The standout performance on AutomationBench is interesting to see, but I don't know what it really means because the model seems to score relatively close to other local models on every other benchmark.
I'd guess it's mostly just an artifact of the strength of their Agentic RL environments for those specific domains. If they had the same kind of access to massive amounts of institutional software knowledge, compute infrastructure, and hundreds of millions of users dropping them endless highly technical agent traces, there may have been similar disparities on the agentic benchmarks in other fields.
It's possible that they may have generally capped out for some reason... but, I'd rather be bullish on a new American AI lab making it to first place among western open-weight models that can run on a common consumer device. (Closest in size that's better across that range of benchmarks is Inkling-Small I think, at 276B params, unless I'm forgetting someone.)
Maybe someone knows what order of cost a release like this requires in real terms of hardware and energy spend?
The cost over the diff from the moment of beginning training to the end. It's iterative from a prior model so it starts there and becomes this release after spending occurs I presume.
For example, is liquid spending millions in energy in this case and the cost is amortized over successive training runs over the initial datacenter spend? What if it's just eating cloud costs and has no dedicated hardware, that sort of thing. I don't particularly mean this model and liquid specifically. Just for this class of smaller fast on-device range of model what is the order of the training cost in real terms?
Maybe its short sighted to ask it as if it's a singular training run and is perhaps the end state of many variations of testing thus all of the R&D training runs would have to be included to to inform what became the final release. Seems like this should be taken into consideration. If anyone happens to know it'd be appreciated!
Training a mid-size model like this can take anywhere between $500k to like $5m. Liquid is spending money on GPU-hours (not sure if you'd call that "energy"). The amortization question is harder to answer without knowing their financials.
The license doesn't really make much sense to me, because if you're already at $10m revenue, you can probably train your own (better, more tailor-suited) model, or at the very least raise money to do so.
Very cool. I’ve been on Gemma 4 12B because of my limited RAM. There’s already an Unsloth quantization of this model so I’m downloading that and will check its performance in the morning. Someone else in this thread recommended Ternary-Bonsai-27B, which I’ll also try.
Just also wanted to note that this model has a questionably-enforceable non-commercial use clause in its license. I really wish they had consulted with an IP lawyer, it’s very confusing.
Don't waste time on bonsai unless you have like 16gb or less ram (ram, not vram). Although I'm pretty sure you could still run this model well on 16+8, and qwen3.6 35b-a3b runs well on ~32gb combined ram at q4. Just offload most of the experts to cpu/ram if you have limited vram budget. With the moe models it doesn't hurt performance too much.
But bonsai is basically trash on real agentic coding work. I've tried both bonsai models, all the quants, messed with the hyperparameters, tried to fine tune (didn't succeed), and it just kept looping or failing to finish tasks. Qwen3.6 35b-a3b is my current go to for agentic work for any systems under 24gb vram, although honestly I'm not sure the dense qwen3.8 27b is meaningfully better. The main difference I see is it does a lot more testing attempts and burns a ton of tokens + compactions, so every task it takes 1.2-2x as long as 35b-a3b does. But the quality of the results is comparable.
If Qwen had released a Qwen3.8 MoE of similar size, with a naive extrapolation its capabilities would roughly be similar to this model. So basically someone has caught up to Qwen (albeit Qwen from 1-2 months ago) in parameter efficiency.
I don't agree with that conclusion. I'm looking at the benchmarks page for LFM2.5-24B-A2B, and it appears to get a worse score on SWE-Bench-Verified (by a lot) when compared to Qwen 3.6 35B-A3B. It only clearly beats Qwen 3.6 35B on AutomationBench, ToolSandbox, and LiveClaw.
At least for my use cases, Qwen 3.6 35B-A3B appears better than this new release. An MoE for Qwen 3.8 would be even better than that!
Would you call Qwen 3.6 35B "similar size" with ~50% more parameters (both active and total) though? I believe what GP meant that if there was a Qwen 3.8 24B-A2B, it would roughly be as intelligent as this one.
yeah it's kind of the point. they're pedagogical tools. they give you the full recipe for building it from scratch or fine-tuning it beyond what it is.
but i'm prepared to be surprised, maybe they'll conjure up something standalone useful this time
I think their entire pitch about local AI is best demonstrated with their own model that they can optimise for it, and the shape of this model seems optimised for the smaller RTX chips. Indeed, this model appears to be part of their launch picture:
I asked it to implement the simplest todo in ghostty codebase and it failed badly - kept going in circles fixing one compile error and introducing another. Qwen 3.8 was slower (40 tok/s compared to 75) but did get it done.
in my experience it's stronger for terminal use, like chaining bash calls, monitoring background tasks, etc. rather than for straight up coding/swe work. Good with web search and file search rather than just editing software.
In a private benchmark I have, it performs much, much worse than Qwen 35B A3B for general knowledge. Still, probably beats Qwen 9B at much higher speed.
>How many tokens/s do you need for a coding agent to feel like Claude or codex?
it's kind of a loaded question, this model gets a lot of tok/s but reasoning is forced on; the time-to-answer is really a combination of things like "how much does this thing want to over-reason" and "how much does this thing like tool use?" .
You are right, they are not exactly the same thing, just wanted to have a reference point
When running locally usually I have to trade off speed for a better model. I can run decently big models at about 2tk/s, or a pretty dumb one at 40-100tk/s
AFAIU prefill speed makes a bigger difference in agentic coding workflows as the harness likely spends more time reading than creating new code. Qwen in particular spends a lot of time reasoning so it does make a larger impact there
Wow, those are seriously impressive numbers for how intelligent this model seems to be!
One thing that this article doesn't clarify, and that I guess I really should look into, is what exactly AutomationBench tests for as compared to other benchmarks. The standout performance on AutomationBench is interesting to see, but I don't know what it really means because the model seems to score relatively close to other local models on every other benchmark.
It's possible that they may have generally capped out for some reason... but, I'd rather be bullish on a new American AI lab making it to first place among western open-weight models that can run on a common consumer device. (Closest in size that's better across that range of benchmarks is Inkling-Small I think, at 276B params, unless I'm forgetting someone.)
I get ~45 tokens/s generation and ~680 tokens/s prompt processing.
The problem is ollama is not caching the prompt, I get -> forcing full prompt re-processing due to lack of cache data
So this wouldn't work in its current state in an agentic loop.
The cost over the diff from the moment of beginning training to the end. It's iterative from a prior model so it starts there and becomes this release after spending occurs I presume.
For example, is liquid spending millions in energy in this case and the cost is amortized over successive training runs over the initial datacenter spend? What if it's just eating cloud costs and has no dedicated hardware, that sort of thing. I don't particularly mean this model and liquid specifically. Just for this class of smaller fast on-device range of model what is the order of the training cost in real terms?
Maybe its short sighted to ask it as if it's a singular training run and is perhaps the end state of many variations of testing thus all of the R&D training runs would have to be included to to inform what became the final release. Seems like this should be taken into consideration. If anyone happens to know it'd be appreciated!
The license doesn't really make much sense to me, because if you're already at $10m revenue, you can probably train your own (better, more tailor-suited) model, or at the very least raise money to do so.
Just also wanted to note that this model has a questionably-enforceable non-commercial use clause in its license. I really wish they had consulted with an IP lawyer, it’s very confusing.
But bonsai is basically trash on real agentic coding work. I've tried both bonsai models, all the quants, messed with the hyperparameters, tried to fine tune (didn't succeed), and it just kept looping or failing to finish tasks. Qwen3.6 35b-a3b is my current go to for agentic work for any systems under 24gb vram, although honestly I'm not sure the dense qwen3.8 27b is meaningfully better. The main difference I see is it does a lot more testing attempts and burns a ton of tokens + compactions, so every task it takes 1.2-2x as long as 35b-a3b does. But the quality of the results is comparable.
If Qwen had released a Qwen3.8 MoE of similar size, with a naive extrapolation its capabilities would roughly be similar to this model. So basically someone has caught up to Qwen (albeit Qwen from 1-2 months ago) in parameter efficiency.
At least for my use cases, Qwen 3.6 35B-A3B appears better than this new release. An MoE for Qwen 3.8 would be even better than that!
but i'm prepared to be surprised, maybe they'll conjure up something standalone useful this time
https://blogs.nvidia.com/blog/local-ai-rtx-spark-laptops-sof...
Is that Qwen 3.8 27B? Because yes, you'd expect a plain 27B model to be better than a 24B-A2B MoE model all else being equal.
(All else isn't equal though of course - Qwen models are very, very good at coding)
Gemma-4 26BA4B would be a more similar comparison.
fast is fast (18tk/s on a Intel Core 7 155H laptop), but quality is something else
very nice.
I can't speak to the intelligence of it yet.
these small agentic-focused models are a lot of fun to play with if you haven't yet.
That seems pretty decent. How many tokens/s do you need for a coding agent to feel like Claude or codex?
it's kind of a loaded question, this model gets a lot of tok/s but reasoning is forced on; the time-to-answer is really a combination of things like "how much does this thing want to over-reason" and "how much does this thing like tool use?" .
When running locally usually I have to trade off speed for a better model. I can run decently big models at about 2tk/s, or a pretty dumb one at 40-100tk/s
the results i've gotten so far have been a little braindead though. still playing around. bad luck with traversing filesystems remotely so far.
it seems to want to make concluding decisions very fast regardless of reasoning levels.