Hugging Face Launches SmolLM3 Small Language Model

Hugging Face has introduced SmolLM3, a flagship 3-billion-parameter small language model designed to run efficiently on consumer-grade hardware. Released under an Apache 2.0 license, the model was trained on 11.2 trillion tokens across a staged curriculum of web, code, math, and reasoning data, supplemented by 140 billion reasoning tokens during post-training. It supports a 128k context window, native tool calling, and six languages, including English, French, Spanish, German, Italian, and Portuguese.

Despite its compact size, SmolLM3 outperforms several larger competitors on zero-shot benchmarks. It scores 76.7 on the IFEval instruction-following benchmark, beating Qwen3-4B's score of 68.9. On the Global MMLU multilingual evaluation, it scores 53.5, surpassing Llama-3.1-3B's 46.8, and it ties Llama's tool-call fine-tune on the BFCL benchmark with a score of 92.3. To achieve this efficiency, the architecture employs Grouped Query Attention to reduce key-value cache memory by roughly 25 percent, alongside a three-to-one ratio of rotary positional encoding to no positional encoding, known as NoPE, to improve long-context generalization.

For practitioners, SmolLM3 represents a shift toward cost-effective, local deployment. It features dual-mode reasoning, allowing developers to toggle between a fast direct answer and a structured chain-of-thought process using simple system prompts. This makes it ideal for focused pipelines like multilingual customer support routers. Additionally, the model is small enough to fine-tune locally. Using QLoRA with the TRL and PEFT libraries, developers can train just 13 million parameters, or about 0.4 percent of the model, on a single consumer GPU in under 12 minutes.

The release is part of a broader family that includes SmolLM3-3B, SmolLM3-3B-Base, SmolLM2-1.7B, and the SmolVLM vision-language variant. To run the new model, developers will need version 4.53.0 or higher of the Hugging Face transformers library. By offering high-performance reasoning without the high operating costs or privacy concerns of API-dependent 70-billion-parameter models, SmolLM3 makes local, specialized AI pipelines highly practical.