Same Recipe, Two Models: Is There a Right LLM for Every Job?

Originally I just wanted to be able to build a custom LLM+LoRA. As an example that a well built LLM can run on a smaller card, and be customized to a specific user/business need. In doing that work with the Qwen3-8B Cooking Assistant, the question I kept coming back to, how stable is the data within the LLM? How do I compare the performance? Does this LLM structure the data the 'best' given all of the goals.

Add to that the question of how stable this workflow/process is. And if it was bespoke or a stable playbook. Curiosity got the best of me: how does the same data and toolchain behave inside two different LLMs? Is one more naturally fit for this kind of work? Just as there's a tool for every job, is there a better-fit LLM for every human-focused use case?

The original experiment

A few days ago, working with Claude Code, I released Qwen3-8B Cooking Assistant, a small open model that reads a recipe you give it and understands if it has gaps. It also has the intelligence to ask for the recipe when it can't see one, instead of inventing quantities and steps. (The first article covers why.)

The intent being, if I had a purpose built LLM, how hard is it to stay factual, truthful and instead of being optimized for conversation and responses; optimize it to be more accurate. And not just in the harness and SOUL.md file, but in the material structure of how the LoRA data is encoded into the LLM.

Building that first LLM helped me to realize something more useful than one model: a repeatable workflow. Especially in building small footprint LLMs. There's lots of closed or license restricted models, and in doing the work, I found my way to building for open uses.

In this case, the Cooking Assistant needed good data to start. Using openly licensed data, creating my own custom fixed set of training steps, and a held-out set of 682 questions in 21 languages; I can comfortably say I have a workflow that fits with the goals. Roughly, the process was built on my maker / checker / confirmation workflow that I've been bringing to all my agentic engineering work. And in this case, that workflow became a panel of open AI judges that check every answer blind.

And being curious and having the GPU compute to build, the obvious next question: what happens if all of that stays exactly the same and only the model underneath changes?

The chosen model GLM-4-9B, an open model from Zhipu AI (Z.ai), released under the MIT licence. It is close in size to Qwen3-8B, made by a different lab, and trained on a different mix of data. Running the same training playbook on it enabled the building of GLM-4-9B Cooking Assistant.

To keep the comparison fair, the experiment regenerated every answer from all four models (both bases, both tuned models) the same way. All of them were judged blind by a common panel of open models from neither family: DeepSeek, gpt-oss and Seed-OSS. An answer counts when two of the three agree.

Claude note: if you read the first article, its numbers (2.7% invented, against 25.8% for the base) came from a different panel of four judges. This panel includes Seed-OSS, a much stricter judge, so every rate here is higher, for the base models too. The size of the improvement holds on both panels. The exact percentage depends on how strict the judge is, which is why models are only compared when the same panel judged them.

The starting line

The two base models started almost level.

Out of the box	Qwen3-8B	GLM-4-9B
Invents details when it can't see the recipe (lower is better)	43.0%	40.1%
Correct when it reads the recipe	85.9%	83.1%
General cooking questions, correct	56.1%	56.1%

Both make up a recipe roughly four times in ten when asked about one they can't see. That is the habit the training set out to remove.

Claude note: none of the gaps between the two base models is statistically significant. On these questions, they start as equals.

The finish line

The same recipe moved both models the same distance.

After tuning	Qwen3-8B Cooking Assistant	GLM-4-9B Cooking Assistant
Invents details when it can't see the recipe	10.3%	9.7%
Correct when it reads the recipe	95.5%	96.2%
General cooking questions, correct	59.8%	46.3%

The headline: each model invents about four times less than its own base.

Claude note: the drop from each base is overwhelmingly significant, far beyond chance. Side by side, the judges can't tell the two tuned models apart on either cooking measure (p = 0.86 and 0.80; anything above 0.05 means no detectable difference).

GLM had the harder road, too. The training examples were shaped around Qwen first, and GLM got them second-hand. But it caught up anyway, which helps validate that the toolchain and playbook aren't a bespoke process for each thing being taught.

The data and the method did the work, not the brand of model. That is the most useful thing this experiment shows. If you can describe the behaviour you want and show it in examples, a capable open model of this size will learn it.

Where they still differ

Level on the trained job is not the same as identical. Four differences matter when picking one.

General questions went different ways. Qwen held its general answers level (56.1% to 59.8%). GLM's full-precision model scored lower than its base (56.1% to 46.3%), and some of its general answers carry confident claims that don't hold up. Then it got stranger: GLM's 4-bit file, the one most people will actually run, scored 54.9% on the same questions. Part of that dip may come from how the answers were generated rather than the training itself. It is reported here as not settled, rather than hidden.

Claude note: neither model's change from its base on general questions is statistically significant, which is why this result is called open below.

Size and speed. GLM-4-9B is the bigger model (9.4 billion parameters to Qwen's 8.2). Its 4-bit file is 6.2 GB to Qwen's 5.0 GB, so it needs an 8 GB graphics card to run fully on the GPU where Qwen fits on 6 GB. On the same test card, GLM answered at about 58 tokens (word pieces) a second, Qwen at about 70. If your users have older laptops, that decides it.

They make different mistakes. An unexpected result. The two base models invented on mostly the same questions: 90 of their roughly 125 invented answers each were shared. After tuning, each still invents on about 30 questions, but only 13 of those are shared. Training removed most of what the two had in common. What is left is each model's own blind spots. That opens an idea not yet measured: two small models checking each other, and asking the person when they disagree.

Licences and makers. Qwen3-8B is Apache-2.0 from Alibaba; GLM-4-9B is MIT from Zhipu AI. Both allow commercial use, but some organisations have policies about which labs and licences they can build on. For a client, that can matter more than a point on a benchmark.

So, is there a right model for every job?

For the trained behaviour, no: two good bases landed in the same place. On this job, the fit came from the data and the training, not from some natural talent in one model.

The fit question moves to everything around the job. Which hardware your people already have. Which languages they work in. What the model does with questions you didn't train for. Which licence your lawyers will sign off. Those are real differences, and they are where one model will suit a use case better than another.

So yes, I think there is a better-fit model for most human-focused use cases. But you don't find it by reading model cards or leaderboards. You find it by running your own questions through two or three candidates with the same data and the same judges, and reading the results. A tool for every job, chosen by testing it on the job.

That test is now cheap. The whole comparison in this article ran on one workstation, end to end, from training to published files, with open models doing the judging. That is what Humanjava is building toward for clients: bring a task and your data, and see how two or three open models handle it, on your hardware, before committing to one.

What this does not tell us

• One task, one training run per model. Cooking questions are a narrow job. A coding or legal task could separate the two models more.
• Judges are AI models. Three of them, from other labs, agreeing two of three. People check them on Label Test, and the judging gets adjusted where people disagree.
• The general-question result is open. Better to say "not settled" than pick the flattering number.

Every question, both answers and every judge's verdict are public in the evaluation set, so you can check any of it.

Try both

ollama run vveerrgg/qwen3-8b-cooking-assistant
ollama run vveerrgg/glm-4-9b-cooking-assistant

Paste a recipe and ask how much milk you need if you triple it. Then ask the same thing without the recipe. Both should ask you for it.

The point was never cooking. It was finding out whether a small, specialised model is something you can choose deliberately, the way you choose any other tool. The answer is yes, and the way to choose is to measure.

Help make it better

Every result in this article was graded by AI judges, and people are what keep those judges honest. Label Test shows you five real cooking answers, one at a time, and asks whether each one holds up. It's blind, needs no account, and takes a few minutes.

Your reviews show where the judges are too strict or too easy, and that makes the data behind v2 and later builds stronger. The more people take part, the better each version gets.

Review five answers

Project DNA

Client	Humanjava Enterprises (own research, and the reference comparison for client model selection)
Role	Product owner and lead engineer: experiment design, training both models, the evaluation and judging system, the release, and the orchestration of AI makers and checkers.
Timeline	November 2026
Stack	Qwen3-8B (Apache-2.0) and GLM-4-9B-0414 (MIT), LoRA fine-tuning, vLLM, llama.cpp, Ollama; judges DeepSeek-V4-Flash, gpt-oss-120b and Seed-OSS-36B; Cloudflare Pages + D1 for human label tests