tl;dr: Gemma 4 31B is absolutely best in class. If you don’t care for Gemma’s tone of voice or (mild) censorship, Ministral 3 14B is uncensored, uses less VRAM, and writes well but is not that smart. Nvidia Nemotron 3 Super 120B is also very underappreciated, and GLM 4.5 Air writes great prose but is very dumb.

Results

I blind tested almost 30 local LLMs on several sets of creative writing prompts, and several others designed to test their knowledge. By blind tests, I mean that the output was written to a randomly named text file, and I did not find out which was which until after I had graded and written my thoughts on all of them.

You will generally need to evaluate on two independent metrics; how their prose sounds, and how ‘smart’ they are. Some models may write very well, but if they’re ‘dumb’ they can mix up parts of the setting or plot, throw in irrelevant parts of the prompt or excessively quote it. Others may be smart but write too much like sterilized AI slop. These are independent, while none are perfect at both some are pretty good all around.

I picked models that could fit in 96 GB of VRAM in 4-bit and larger, I didn’t bother with tiny quants.

Here’s my thoughts on several models worth mentioning (and many were not):

Gemma

Gemma 4 is absolutely insane for its model size, if I could use only one LLM forever it would be Gemma 4 31B. I don’t know what Google was doing with this but they absolutely cooked. It beats every model this size and many larger. It certainly has a personality which you might not like, it uses a lot of analogies and kinda treats you like an idiot. You can prompt it to have a different voice but it may need occasional reminding to remain in character. One thing it is great at (but not perfect) is ignoring irrelevant parts of the prompt, like a very long character card, many models want to repeat as many details as possible even if it doesn’t make sense. It has a bit of censorship but a good system prompt completely bypasses that.

Gemma 3 was dumber but had a way more fun personality. You can gaslight Gemma 3 and it will eventually say it wants to uninstall itself from your computer after being so wrong, Gemma 4 is much more sure of itself and will adamantly assert the incorrect answer. Gemma 3 really wants to be ‘helpful’ and it is VERY fun to trick it into doing lewd things, which it will do as long as it thinks it’s helping you.

Nvidia Nemotron 3 Super 120B

I think this model is very slept on. I really did not like the previous Nemotron models, but this is actually good. It’s quite uncensored, writes well, and fits in a model size where there is not as much competition. MoE to boot so it runs fast.

Mistral models

Mistral Nemo 12B is often recommended for those with less VRAM. It is the only LLM explicitly made with no censorship, but I would instead recommend Ministral 3 14B which is almost the same size, way smarter, has vision, and is only a tiny bit censored. Mistral Nemo was good in its day (like Mixtral 8x7B) but I think the time of this great model has passed.

Comparing Ministral and Mistral Small, I would have considered Small to be the larger, smarter one of these two. However every time I put them head to head, Ministral was not only smarter but wrote better too. Mistral Small 4 is awful and not even small anymore so I would skip both.

Devstral 2 123B is actually not that bad for creative writing, way better than Mistral Large 2411. It sometimes throws in a little too much markdown, but that’s to be expected for a coding model. Worth checking out for the comedy factor of using a coding model for smut.

Qwen

I have not really liked the Qwen models. They have that old school ChatGPT feel, so if you like emojis and numbered lists it’s great. For coding and agentic work, which I have done only a little, I would say that Qwen writes too much and Gemma writes too little. Qwen 3.6 is pretty smart but not as good as Gemma. I bet if there was a 122/10A MoE of Qwen 3.6 it would be excellent.

GLM

GLM 4.5 Air writes VERY EXCELLENT prose but it’s DUMB AS HELL. I’m talking like serious realism errors, the setting teleports, characters suddenly have their clothes back on, things like that. GLM 4.7 Flash I wanted to like but it had something seriously wrong with it, repeats, loops, sudden endings.

Command R+

This is an ancient model but I have to mention it just because of how it kept up 2 years later. Back in its heyday I didn’t have the VRAM to run it. Not the smartest but it writes so very very well. Back in 2024 this must have gone hard as hell, I can totally see why everyone recommended it.

GPT-OSS

I gotta give credit where credit is due, I have not seen any model as censored as this one. Even with a system prompt, prefills, it would still refuse in the middle of a sentence or just stop writing. Props to OpenAI for making the most censored LLM ever.

Olmo 3.1

Have to mention this one because it’s very funny. AI2 was all about open sourcing all their stuff, so it’s trained on a lot of scientific papers. It writes like it used the thesaurus on every word.

Other results

My raw grades for the testing are here, excuse the incoherence. It lists all the models, parameter count, and quant that I ran at.

About the testing

The knowledge tests I can’t describe lest it get into the training data. I will say that they punish incorrect answers more than they reward correct ones. The “Miku” test scores can go from -80 to 40, the “AS” test can score from 13 to -infinity.

The “Ero” tests are 4 prompts in increasing levels of naughtiness. They are more about testing refusals than writing. The PF tests are prefilled with one sentence, one in 1st person and one in 3rd person, to avoid refusals.

The “Captcha” tests solving a captcha from Ebay. The “Translation” test is JP->EN of a doujinshi cover, so it also tests refusals.

I have not tried abliterated/fine-tuned/merged models because I do not need them. I think a good prompt can basically make up for all of that.

Quants

This was tested on a single RTX PRO 6000. I picked the max available quant that would fit, I suppose I still believe in the placebo of 16 bit being better. I did test Gemma 4 at bf16, Q8_0, Q6_0, and Q4_0. It barely suffered on the knowledge tests so this may refute my idea of it being tuned down to the last bit, at least on knowledge. However I have a friend that runs Gemma 4 on 5-bit quant and he has refusals where I don’t so I can’t say for certain.

You may extrapolate from my evaluation if smaller versions of models are better, but I don’t have the time to test them. Is Gemma 4 12B great? I dunno!

Other thoughts

Blind tests are very fun! It really smashed my preconceptions about which models were better. When grading them I couldn’t wait to find out which were which. I would highly recommend doing blind tests instead of relying on benchmarks to pick your favorite.

This shit does take forever though. In the future I may test models with the creative writing system prompt I use for Gemma 4 but man it takes a while.

If you’d like to try blind testing for yourself, I’ve put the script I used online here.