Skip to main content
← Back to news

Co-Design Bench: 273 tasks from real user requests to evaluate multimodal coding agents

Benchmarks
Source:Hugging Face (paper)
A new reference-free Benchmark: 273 tasks and 2,331 expert-authored acceptance criteria, curated from 37,463 deduplicated queries across 26,314 real user coding sessions; two tracks (100 multimodal editing tasks, 173 repository editing tasks) and a judge-then-refute protocol that evaluates the running application with browser evidence.
Read original article

Related articles

On-device phone agents: what open mobile agents actually bring (after OpenMobile-2)

OpenMobile-2 opens up mobile agent training environments and recipes, and LFM3-1.7B shows a 1.7GB agent already runs on your phone: numbers, minimum requirements and honest limits.

ML agents that run locally: MLExpert and EpochGo on Ollama, zero API costs

MLExpert raises the bar to 35B and a 72.7% medal rate; EpochGo runs three AutoML agents on 16 GB at zero API cost: what works today, the math and the limits.

Ollama vs llama.cpp: under the hood

Ollama runs llama.cpp under the hood, but the two are not the same thing: one is the engine, the other the car. What's really underneath, when the difference matters, and how to choose without slogans.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.

Related resources

Tools and resources from the iaLocale archive relevant to this content.

🎨

FLUX 3 Image Klein

Models

A 5.6B open-weights image model for text-to-image, single-reference editing and multi-reference composition in one model, with placement boxes in the prompt. Qwen3-VL-4B text encoder. In BF16 (11.1GB), FP8 and NVFP4 (5.3GB). FLUX Kommunity license (non-commercial + qualified-user exception).

💎

Gemma 3

Models

Google's open model family in 1B, 4B, 12B and 27B sizes. 128K-token context, 140+ supported languages and multimedia input (text and images). An excellent choice for RAG and long documents on consumer hardware.

💎

Gemma 3 4B

Models

La taglia compatta della famiglia Gemma 3 di Google: 4,32B di parametri, contesto di 128K token e supporto nativo a 140+ lingue (incluso l'italiano), con input testo+immagini. Licenza Gemma (uso commerciale con condizioni). In Q4_K_M occupa circa 2,9GB: gira su GPU da 8GB e su CPU moderne.

💎

Gemma 4 12B

Models

La taglia 12B della famiglia Gemma 4 di Google: 11,95B dense, contesto 256K e multimodalità integrata senza encoder separati (testo+immagini+audio). Licenza Apache 2.0. In Q4_K_M occupa circa 7,4GB: il punto di equilibrio tra qualità e requisiti hardware per l'uso locale su 8-16GB.

Browse all resources