Italian in local models: what to expect

The right question to ask

"How well does my model handle Italian?" has no single answer: it depends on the model, its size, and what you ask it. The good news is that in 2026 almost every major open-weight model handles Italian solidly. The bad news is that the difference between "handles it" and "handles it properly" only shows when you actually test.

In this article I'll tell you which models lean into Italian, where they break down, and how to measure them yourself in half an hour, with your own data.

Why Italian is a sore spot

There are two technical reasons why many local AIs do better in English than in Italian, and understanding them helps you choose.

The tokenizer. Models don't read words, they read tokens: chunks of text. The tokenizers of top models were built on massive corpora where English dominates. For Italian, more tokens are often needed to say the same thing: words like "confermazione" or "immediatamente" break into three or four pieces. The result is twofold:

• more computation for equivalent sentences, hence a bit more latency;
• context that drains faster: a long document in Italian "weighs" more tokens than the same document in English.

The training data. How much a model "knows" Italian depends on how much Italian text it has seen. Models trained on balanced multilingual corpora are more at home in Italian than those trained almost exclusively on English. That's why a smaller but multilingual model can beat a larger but English-only one, for an Italian task.

The models, one by one

Here's what to expect from the main players, with the numbers the vendors declare.

Llama 3.1 (Meta). Italian is one of the 8 officially supported languages (alongside English, German, French, Portuguese, Hindi, Spanish and Thai). In the official model card, the multilingual MMLU evaluation (5-shot) in Italian is 61.63 for the 8B Instruct, 80.40 for the 70B, and 85.04 for the 405B. The message is clear: Italian holds up, but the quality jump is mostly visible with the large models. On a 16 GB GPU at Q4 quantization, the 8B is what you'll actually run, and there Italian is "decent" more than "excellent". Source: Llama 3.1 model card.

Qwen3 (Alibaba/Qwen). Here the story changes: the technical report declares 119 languages and dialects that Qwen3 is pre-trained on across 36 trillion tokens, up from 29 (Qwen2.5) to 119. Italian is explicitly in the list of the 10 best-represented, alongside English, Chinese, Portuguese, Arabic, Japanese, Korean, Indonesian, Russian, Vietnamese, German and Thai. For anyone working in Italian, Qwen3 is often the most "at home" choice of all, and the family includes small sizes (8B) that run on 16 GB.

Gemma 3 (Google DeepMind). The official announcement leans on a new multilingual tokenizer covering more than 140 languages. Fewer "official" languages but a wider vocabulary, and that has a direct practical effect: Italian is encoded more efficiently than with an English-only tokenizer, i.e. fewer tokens per sentence and context that lasts longer.

Mistral 7B. The base model is designed primarily for English (the model card lists "en"); in Italian it works but with lower quality, and in French it does better than in Italian. I use it as a reference for "what an English-first model looks like in Italian" rather than as a main choice for Italian content.

LLaMAntino (community). A special case: a family of fine-tunes based on Llama, adapted specifically for Italian by the University of Bari (UNIBA) and then refined on Italian datasets. The paper describes the fine-tuning of Llama 2 on extended Italian text. It's the kind of model that, on modest hardware (16 GB, a Mac with an M-series chip), often beats larger but non-adapted models, for Italian tasks.

Where they fail

How "real" is a local model's Italian? Three typical failure zones:

1. Idioms, register and irony. "Fare il diavolo a quattro", "avere la penna veloce", the tone difference between an email to a client and a social post: here, even large models tend to normalize toward neutral, slightly translated-sounding Italian.
2. Long sentences and complex syntax. The more complex the sentence, the higher the risk of an agreement error, a missing object, a misplaced participle.
3. English tail. In technical contexts, models tend to insert anglicisms even when established Italian terms exist. In consumer content it's a tone problem; in technical content it's sometimes just a style choice.

Small models (7-8B) fail more often; large ones (70B, 405B) rarely break but cost in hardware.

How to evaluate a model in Italian, yourself

No benchmark covers your use case. Here's the method I use:

1. Prepare a test set. 30-50 examples taken from your real cases: a support email, a document summary, a reply to a customer, a rewrite in a formal tone. In Italian, as you'd write it yourself.
2. Run every model on the same test. Ollama does the work in one line: ollama run <model>.
3. Score with explicit criteria. Three scores from 0 to 5: grammatical correctness, appropriate tone, relevance to the content.
4. Look at the failures, not the averages. An average of 4.2 tells you little. The three worst examples tell you everything: if they all fail on register or agreement, the model isn't for you.

In 30 minutes with two models you have a concrete answer.

The practical recommendation

• If Italian is central to your work: start with models that declare Italian in their support — Qwen3 (119 languages), Llama 3.1 (8 languages, Italian included), Gemma 3 (multilingual tokenizer 140+). The largest model your hardware can run at Q4 is almost always the right choice.
• If your hardware is modest (16 GB or less): an Italian fine-tune like LLaMAntino, or a Qwen3 8B, often beats a Llama 3.1 8B for Italian tasks.
• Before trusting the numbers: a test with 30 of your own examples is worth more than any table. Vendors declare what they measured; you measure what you need.

Italian in local models is no longer a limit: it's a variable you can measure. Choose based on what you need, not on what you're told "works".

Article generated with AI