← All Articles

AI benchmarks

We test each model inside NotesXML on three kinds of work, weighted equally: document processing, reading images (OCR), and chat knowledge and reasoning. Speed counts too. We tested 52 models; these 10 make up the NotesXML catalog.

Scores

ModelDocumentsOCRChatTokens/sOverall
LFM3 VL 2B (free)0.861.000.843290.89
Gemma 4 E2B0.891.000.822260.90
Ministral 3 3B (free)0.951.000.751940.88
Gemma 4 E4B0.921.000.891710.93
Gemma 4 12B0.931.000.871060.89
GPT-OSS 20B0.95—0.811480.90
Devstral Small 2 24B0.941.000.86470.78
Gemma 4 26B-A4B0.951.000.931220.93
Gemma 4 31B0.971.000.92440.79
Laguna XS 2.1 33B-A3B0.89—0.861330.89

Scores run from 0 to 1. Pass marks: documents 0.85, OCR 0.90, chat 0.70. — = text-only model. Tokens/s on the desktop below. Overall is the harmonic mean of documents, chat and a speed rating, 1 − e−tokens/s ÷ 50 (50 tokens/s rates 0.63, 150 rates 0.95), so a model must do well on every count. OCR is left out of Overall because every vision model scored 1.00.

Speed on other hardware

ModelDesktop, CPU onlyLaptop, Radeon 780MLaptop, Intel 155HGalaxy S24 Ultra
LFM3 VL 2B—611214
Gemma 4 E2B3275108.7
Ministral 3 3B15308.55.3
Gemma 4 E4B21506.14.5
Gemma 4 12B8.7182.0desktop only
GPT-OSS 20B—213.9desktop only
Devstral Small 2 24B2.65.02.4desktop only
Gemma 4 26B-A4B148.93.8desktop only
Gemma 4 31B4.03.30.9desktop only
Laguna XS 2.1 33B-A3B—9.03.5desktop only

Tokens per second. — = not measured.

Which model?

How we test

Each test scores 0 to 1, and each of the three scores is the average of its tests. The tests send the same prompts NotesXML itself sends.

Test machines

Desktop: AMD Ryzen 7 7800X3D, 32 GB RAM, NVIDIA RTX 5070 Ti 16 GB + RTX 5060 Ti 16 GB. Laptops: AMD Ryzen 9 7940HS with Radeon 780M graphics; Intel Core Ultra 7 155H with its built-in graphics. Phone: Samsung Galaxy S24 Ultra. Results as of October 2026.

Model sizes and memory needs →