I am feeling like a mighty local King spinning two Strata engines with 70tps each whole day and using 12M tokens a day. Also me scrolling twitter and stumbling upon a post saying that poteto uses 34M tokens per minute 🤪

Upd: Guys are interested who is she and what she is doing with all these tokens. Poteto is her twitter nickname and she worked in Cursor before XAI has acquired them and now she is shipping 2,500 PRs per month to Cursor and GrokBot.

I think a turn of events are pointing towards a very likely crash in prices in the coming months.

With a single gb300 rack being to serve all of Openrouter's traffic for DSV4.1 Flash(https://x.com/thdxr/status/2115165365630026178)

Half of data center buildouts already being cancelled/stalled.

Neoclouds unable to finance debt for their compute, and being stuck in a 5 year deal with no confirmed demand.

AI adoption not matching expectations.

Smaller models being able to serve majority of the requests.

OpenAI/Anthropic being unlikely to finance their coming compute contracts.

I think we will see lots of cheap GPUs next year.

https://huggingface.co/LiquidAI/LFM3-1.7B-GGUF

from liquidAI:

LFM3-1.7B is a text model for local agents and specialized applications, based on our new tri-hybrid architecture.

• Agentic training: post-trained for agentic harnesses such as Hermes Agent, Pi, and OpenClaw, with a 256K-token context window.
• Designed for the edge: at 4-bit precision, it takes about 1 GB of memory, so it fits on any phone or laptop.
• Made to be specialized: adapt the model to your domains, formats, and tools via quick single-GPU fine-tuning.

Find more information about the LFM3 family in our release blog post.

Model details

The LFM3 family includes base, text, vision, audio, omni, and decision models.

LFM3-1.7B is designed for local agents, tool calling, structured outputs, and domain-specific fine-tuning. Use retrieval or search for knowledge-heavy tasks. It is not recommended for agentic coding or complex multi-step reasoning due to its small size.

more LFM3 to enjoy today:

(my initial idea was to create 2 posts but they released six models)

https://huggingface.co/LiquidAI/LFM3-VL-2B

https://huggingface.co/LiquidAI/LFM3-VL-2B-GGUF

LFM3-VL-2B is a vision-language model for on-device visual understanding. It takes text, images, and video as input, and combines the LFM3 language model with a new in-house vision encoder.

New in LFM3-VL-2B:

• Video understanding: the first LFM vision model that takes video as input. Ask questions about a recorded lecture, a walkthrough filmed on your phone, or a clip from a camera feed.
• Thinking: the model reasons step by step before it answers. This helps most with harder visual questions, such as math, charts, and multi-step problems.
• Spatial and embodied reasoning: the model finds objects and reasons about how they are placed relative to each other, a strong foundation for robotics applications.

https://huggingface.co/LiquidAI/LFM3-Audio-2B

LFM3-Audio-2B is a compact, fully featured audio model that runs locally on laptops and edge devices. It takes text or audio as input and responds with text, speech, or both.

• Voice chat: ask questions aloud and receive streaming replies in text and speech. For text-only responses, the model can reason before answering.
• Multilingual speech: transcribe, synthesize, and translate speech across 18 languages.
• Audio understanding and reasoning: interpret recordings and answer questions about the speech, sounds, and events they contain.
• Tool calling: turn spoken requests into tool calls to control apps and devices, with text or speech alongside the calls.

LFM3-Audio-2B introduces an in-house audio encoder and LiquidCodec audio tokenizer, improving speech quality over LFM2.5-Audio.

https://huggingface.co/LiquidAI/LFM3-Omni-2B

LFM3-Omni-2B is Liquid AI's first omni-modal model in the LFM3 family: a single end-to-end model that understands text, images, video and audio, and answers in text and speech. It is built for on-device use: one model replaces a separate ASR model, vision-language model, LLM and TTS model.

• Any input, text or speech output: ask about a photo by voice, transcribe or translate speech, chat out loud, or ask questions about a video and its soundtrack together.
• Watches and listens together: for videos with sound, the model sees and hears them at the same time, so it can answer questions that need both.
• Small enough for devices: at 2.2B parameters, it is designed to run efficiently on phones, laptops and other edge hardware.

https://huggingface.co/LiquidAI/d1-Omni-2B

d1-Omni-2B is a decision model for text, pictures, and audio, built on LFM3-Omni-2B. It returns calibrated answers in one forward pass, without generating tokens.

• Native audio: clips and spoken questions go through the same pass as text and pictures, and spoken questions score within 0.5 points of written ones on unseen tasks.
• Beyond classification: it also extracts text, numbers, and dates with their span, locates objects in pictures, and estimates quantities with an interval.
• Millisecond decisions: 5.8 ms per decision on an NVIDIA H100 and 3.7 ms on an AMD MI325X, or 12.4 ms and 8.2 ms with a 10-second clip.

https://huggingface.co/LiquidAI/d1-2B

d1-2B is a decision model for text, pictures, and video, built on LFM3-VL-2B. It returns calibrated answers in one forward pass, without generating tokens.

• Best-in-class: it leads the open d1 models on average and on our board of over a hundred decision benchmarks, ahead of d1-3B at under two thirds of its size.
• Beyond classification: it also extracts text, numbers, and dates with their span, locates objects in pictures, and estimates quantities with an interval.
• Millisecond decisions: 5.7 ms per decision on an NVIDIA H100 and 3.8 ms on an AMD MI325X, with any number of questions about one state in one call.

https://huggingface.co/LiquidAI/LFM3-1.7B-Base

LFM3-1.7B-Base is the pre-trained checkpoint behind LFM3-1.7B, a text model for local agents and specialized applications based on our new tri-hybrid architecture.

• Pre-trained on 22T tokens: the last 3.5T tokens are curated mid-training data with agentic traces and tool calls, and extend the context window to 256K tokens.
• Designed for the edge: at 4-bit precision, it takes about 1 GB of memory, so it fits on any phone or laptop.
• Made to be specialized: a starting point for continued pre-training and your own post-training.

llama.cpp support has been merged:

https://github.com/ggml-org/llama.cpp/pull/31257

https://github.com/ggml-org/llama.cpp/pull/31252

https://github.com/ggml-org/llama.cpp/pull/31275

Artificial Analysis Intelligence Index has been updated to v5, which according to them is a major upgrade resulting in a more accurate ranking.

Notably, this update widens the gap between open-weight models and closed ones. Some people already questioning its accuracy based on their experiences with models (GPT-Terra-6.1 > Fable-5.1).

Also, this update, v5, has 60% of the Index based on evaluations with private test sets or held-out answers, which makes reproducing the results impossible.

From their announcement:

“The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)”

Well, “coming soon” turned out to be five days.

I posted the earlier announcement last week. The 1019 weights are now on Hugging Face.

Still 560B total, 25B active, with a 1M context window. This is the update they reported at 44 on the AA Intelligence Index v4.3.2, up from 41. AutomationBench-AA went from 62% to 68%, according to the release.

They also put out Ling-3.1-flash-base in BF16 for further training. Glad they included that. It gives people a starting point for their own post-training instead of having to start with the finished chat model.

One small gotcha: the SGLang example in the 1019 README currently still points to inclusionAI/Ling-3.1-flash. Change the model path to inclusionAI/Ling-3.1-flash-1019 if you're copying that command for the new checkpoint.

Collection on HF: https://huggingface.co/collections/LiquidAI/lfm3
WebGPU demo: https://huggingface.co/spaces/webml-community/LFM3-VL-2B-WebGPU

Reposting just the HF link because author hid their analysis behind a paywall on their blog & apparently want qwen3.8 FN to find an exploit on substack & upload the contents of his blog to an alibaba server

from inclusionAI:

We introduce Ling-3.1-flash-1019, an upgraded version of Ling-3.1-flash, with 560B total parameters, 25B active parameters per token, and a 1M-token context window. With further post-training optimizations, Ling-3.1-flash-1019 demonstrates significant improvements in reasoning and real-world agentic capabilities. Its AA-Briefcase score rises from approximately 1400 to 1524 Elo, while performance on AutomationBench-AA increases from 62% to 68%. The model also makes gains on scientific benchmarks, including CritPt (18% → 26%) and SciCode (54% → 57%). Overall, it scores 44 on the Artificial Analysis Intelligence Index v4.3.2, up from 41 for its predecessor. These gains highlight the model's improved capabilities in complex reasoning, multi-step task execution, and agentic workflows across diverse real-world scenarios.

https://huggingface.co/inclusionAI/Ling-3.1-flash-base

We introduce Ling-3.1-flash-base, the base checkpoint of Ling-3.1-flash, released to support research and community-driven innovation. With 560B total parameters and 25B active parameters per token, the model provides a foundation for continued pretraining, domain adaptation, and post-training research.

One more great model from Liquid AI!

Good for edge:

LFM3-Audio-2B is a compact, fully featured audio model that runs locally on laptops and edge devices. It takes text or audio as input and responds with text, speech, or both.

• Voice chat: ask questions aloud and receive streaming replies in text and speech. For text-only responses, the model can reason before answering.
• Multilingual speech: transcribe, synthesize, and translate speech across 18 languages.
• Audio understanding and reasoning: interpret recordings and answer questions about the speech, sounds, and events they contain.
• Tool calling: turn spoken requests into tool calls to control apps and devices, with text or speech alongside the calls.

Capabilities

• Speech recognition (ASR)
• Real-time speech recognition (LiveASR)
• Text-to-speech synthesis (TTS)
• Voice chat with spoken and text responses
• Speech-to-text chat with optional reasoning
• Audio understanding
• Speech translation
• Voice-activated tool calling

https://x.com/osanseviero/status/2115351234811249111

Most OCR models focus on reading documents. MiniCPM-OCR adds a second piece: a model that checks whether the OCR result is actually trustworthy.

OpenBMB just released two open-source models:

• MiniCPM-OCR 1.0 (1.3B): Scores 93.68 on Real5-OmniDocBench, targeting real-world documents like curved pages, phone photos, tilted scans, and complex layouts. DSpark decoding acceleration boosts end-to-end processing efficiency by 55%.
• MiniCPM-OCR Jev (2B): An OCR quality judge that compares the original image with candidate text, catching errors in formulas, tables, and text. It achieves 0.936 AUROC, close to a 397B-parameter model on the same evaluation.

The interesting part is Jev can also evaluate outputs from other OCR systems, so you don't necessarily have to replace your existing pipeline.

Both models are open source, with support for vLLM, SGLang, llama.cpp, and Ollama.

Hugging Face Links:
https://huggingface.co/openbmb/MiniCPM-OCR-Jev
https://huggingface.co/openbmb/MiniCPM-OCR-1.0

Just wanted to Show my rig. The base case is an older Corsair with a cheap AliExpress GPU Mount on top.

The System:

-Threadripper 9955WX

-128GB DDR5 5600, just to expensive to go for 256

-Asrock WRX 90 WS EVO

-2200W Seasonic + 1600W Seasonic PSU

-8x R9700

-Bifurcation Card and 6 Extension Cables, but I have some problems and I will buy a retimer.

I upgraded from 4x GPUs to 8x, so when new bigger models will arrive I have more room. At the moment still running qfn, tried GLM 5.3 Flash but QFN with radiance is just a really good experience.

No, this is not a lobotomized 1-bit squeeze, its full Q2 nazeshinjite's DeepSeek-V4.1-Flash-1017 Q2 GGUF on a single 3090, natively on Windows no WSL.

On an RTX 3090 with 192 GB of RAM I'm writing at 41 tok/s (average across multiple prompts) and the output agrees with DeepSeek's own API on the next token 90.2% of the time on general text and 92.4% on code/math, aka within a point of exact routing. Fits full 1M context.

My scuffed interface engine runs the three most-downloaded quants of 1017, their files as they are:

nazeshinjite's ds4 Q2 GGUF (the default)

ISTA-DASLab's GSQ-RCO (IQ2_XS / IQ2_XXS /IQ1_M)

Unsloth's (UD-IQ1_S to UD-Q8_K_XL).

Repo: https://github.com/musicurgy/delta-inference-engine

Hardware/model:

RTX 3090 24 GB, i5-13600K, 192 GB DDR5 (4x48 GB at 5600), a PCIe 4.0 NVMe. Windows 10/11, no WSL.

Setup:

Download the ZIP (or clone), double-click Start DeepSeek.bat. The first run installs Python if needed, downloads the prebuilt engine for your CPU and the model (every file SHA-256-checked, the space checked first), and starts the server; after that the same file just starts it. Choose Model.bat switches quants. If anything goes wrong, Start DeepSeek (debug).bat packs everything needed into one zip for an issue (no prompts, user name removed).

Privacy: localhost only by default, and nothing from your prompts is written to disk unless you turn a log on.

Needs:

• An NVIDIA RTX card with 12 GB+ (24 GB: 41 tok/s, 16 GB: 31 tok/s, 12 GB: 17 tok/s, measured on my 3090 being memory limited) . GTX 10 series and older can't run it
• 128 GB+ RAM.
• A fast NVMe SSD with 385 GB free
• Any x86-64 CPU (AVX2 is the fast path; older CPUs get a slower build automatically).

Credits:

The engine is a Windows port of Xolotl, itself built on Strata; several of Xolotl's latest fixes are merged back in. DeepSeek for the model, nazeshinjite for the GGUF, ISTA-DASLab for GSQ-RCO, Unsloth for their quants. MIT licensed. Special thanks to Claude for doing most of the lifting.

If there is interest I'll add some support for muti GPU or other garbage. This project is very new and needs a lot of validation on other hardware, these numbers are from my machine, feel free to post yours.

I have been working on test-time scaling for quite some time now. Posted some of my previous work on this sub-reddit before. Recently, I made significant simplifications to the core system-design and it finally felt like now anybody can understand and directly use this on their hardest problems. Just on a side-note though, depending on the depth you set & your LLM's inference speed, it can take from 50 Minutes to 30 hours to give you back the results.

For benchmarks I couldn't test this on frontier models or even large open models because it will be way too expensive. I directly only tested this on Gemma4-35B & Qwen-3.8-27B. They spent around 200M tokens / problem and there scores went from 14% to 65% on BrokenArxiv (Matharena) 15 problems or from 9% to 52% on 8 DeepSWE problems. I know that's almost unbelievable, but you can try and see that yourself.

Repo Link: https://github.com/ryoiki-tokuiten/Deepthink

Spark-X2.6-1.7B: less than half the parameters of our 4B, close on two benchmarks: SWE Verified: 59.8 vs 62.8 τ²-bench: 85.3 vs 86.9 Both at high thinking. Try it & Like on HF! https://huggingface.co/XHToken/Spark-X2.6-1.7B

https://www.cnbc.com/2026/10/28/sk-hynix-third-quarter-profit.html

Unfortunately, the already insanely inflated prices for RAM may start to look like a steal in the coming years.

Huge kudos for the LiquidAI team for this. Great work from tdakhran for quickly coordinating a (zero-day!) llama.cpp release with the team for an omni model. This one is a real game-changer for me personally because this is the first open model from a frontier lab that actually has support for Polish TTS and STT - and all this in a compact 2B package that makes it viable for on-device deployment. Big thanks!

Think of the animation workflows people use with Claude Opus 5.5: the model writes code, rendering tools turn it into video, and further instructions revise the result. SenseNova follows that same general approach, with a model and creation Skill focused on video creation
A few highlights:
Generates video through code, with a focus on motion graphics and other visuals that can be expressed programmatically.

Revises existing results through natural-language requests, including changes to styling, timing, and scene elements.

Supports reconstructing a reference video’s visual treatment as editable code.

Pairs the model with a creation Skill that guides brief development, previews, and revisions.

The release plan includes full model weights and the Skill. INT4 and INT8 weights are planned for later.

General-purpose coding models can already write animations. The interesting question is what a dedicated Code2Video model improves: fewer repair attempts, better motion, or more reliable edits that leave unrelated elements alone.

GitHub:
https://github.com/OpenSenseNova/SenseNova-6.8-Code2Video
Model:
https://huggingface.co/sensenova/SenseNova-6.8-Code2Video

I've been building OpenJet, a local coding agent with its own CUDA megakernel engine for Qwen3.8-27B Q4_K_M. Until now it served one conversation at a time. The new release serves several agents at once on a single GPU.

https://reddit.com/link/1xgs8hp/video/p0zd2rxse0yh1/player

Numbers (RTX 3090, Qwen3.8-27B Q4_K_M ):

That's 256-288 tok/s total across the 4 agents, vs 72-80 tok/s total for llama-server. One agent alone still gets 117 tok/s. Four real coding agents, which pause to run tools and tests, total 150-200 tok/s.

How the parallel decoding works:

Generating tokens on a GPU is limited by memory bandwidth. Every decode step reads all 16.7 GB of weights from VRAM, which on a 3090 (936 GB/s) takes about 18 ms on its own. The maths done on those weights is small by comparison.

So the engine doesn't run the agents one after another. Each step, it stacks every active agent's work into one batch of rows: each agent's next token plus its 2 MTP / prompt-lookup draft tokens, so 12 rows for 4 agents. It streams the weights through once, and each weight it loads is multiplied against all 12 rows with tensor cores while it's still on chip. Four agents cost about one weight read per step, not four.

That's why each agent drops from 117 to 64-72 tok/s instead of to ~30: the shared weight read is most of a step's cost. What does grow with more agents is attention (each agent reads its own KV cache) and the extra rows' compute. The whole step runs as one persistent CUDA kernel with no per-layer launches, and the drafts are verified in the same pass, so every accepted draft is an extra token from the same weight read.

What keeps it fast with real agents:

• New prompts don't stall everyone else. When an agent sends a new prompt, each step carries the next chunk of it alongside the other agents' rows, so they keep generating while it's processed. The result is bit-identical to processing the prompt on its own.
• One shared 132K-token KV pool (q8_0). It's paged, so one agent can use most of it while the others stay short. There's no fixed 4 x 32K split.
• Shared system prompts are stored once. A system prompt plus tool list of 1024 tokens or more that the agents have in common is processed and stored once for all of them.
• Conversations stay in their slot. A follow-up request goes back to the slot already holding its conversation, so only the new tokens get processed.

Also new:

• Sampled MTP drafts (thanks to iiLaurens): at temperature > 0, drafts are now sampled from the draft head and verified by rejection sampling. The output distribution is unchanged and more drafts get accepted.
• mk_top: a live terminal view of each agent's tok/s, context use, current task and latest output, plus GPU stats. mk_top --run 4 starts the server and 4 agents on eval tasks so you can watch them.

Accuracy: the output is not byte-identical to llama.cpp, because the arithmetic is ordered differently. Against llama.cpp's own next-token distributions, KL divergence is in the same range as llama.cpp batched vs. token-by-token. The README has the full correctness table.

Try it (Linux, RTX 30/40/50 with 24 GB, CUDA 12+):

python3 megakernel/server/mk_server.py -m Qwen3.8-27B-Q4_K_M.gguf --agents 4
python3 megakernel/tools/mk_top.py

Then point up to 4 OpenAI-compatible agents at http://127.0.0.1:18080/v1.

Limits: the Q4_K_M engine only (the mixed-quant build serves one conversation at a time), at most 8 agents, and presence/repetition penalties are ignored.

Repo: https://github.com/l-forster/open-jet

I am also planning on adding support for TurboQuant next, to get the max context window to 256K on a single card with minimal output degradation.

I've been running ISTA-DASLab's 2.25bpw quant of DSv4.1-1017 for a few days, and it's holding up well. But the fact that it's only 2.25bpw average did make me wonder how good the quant actually is against the full model.

Local setup: Project-maya v1.1.27. ISTA-DASLab's 2.25bpw quant. Quality-relevant settings: "DS4_INDEX_TC": "1". "MAYA_DS4_PROMPT_SKIP": "2".

Test: 100 question text-only subset of Humanities Last Exam.

Benchmarking models isn't easy, and these results are nothing but a smoke test. There are lots of issues with these results:

1) I ran the questions with a 32768 token limit per question, because anything larger becomes prohibitively time consuming. It already took my local instance around 12 hours to complete the test, and that's at an average of 40.4 toks/s.

2) When approaching the token limit, Deepseek's API and project-maya work differently. Project-maya will reduce thinking_budget to around 3/4 of the token limit so it always gives an answer, but as a result it never used more than around 24768 tokens on a question. The Deepseek API will instead just use all 32768 tokens thinking if it needs and then output nothing.

This really fucks with the results on a lot of questions unfortunately. There were some questions local answered correctly because it was forced to give a result. There were others that the API answered correctly while using between 24k-32k tokens, where the local model was wrong while being capped at 24k.

3) I ran on thinking "high", not "max" for both, because Deepseek's API was thinking too much and failing to output anything for most questions on "max". Even on thinking "high", the API failed to answer 26 questions and thinking was cut short to 24k on a similar number of local questions.

4) This is only a 100 question subset, and I saw significant run-to-run variance on both the official API and my local model too. There are some questions that both sides will sometimes pass, sometimes fail using the settings as tested. To get proper statistically relevant comparisons you'd need to run this test 10 times or probably more.

The results themselves:

Local mode: 13 correct "strict" answers, 20 semantically correct answers. API: 13 correct "strict" answers, 21 semantically correct answers.

On the 74 questions the official API actually completed:

• Local semantic: 19/74
• Official semantic: 21/74
• Both correct: 17
• Local only: 2
• Official only: 4
• Neither: 51

It's also worth noting that when both were wrong, the wrong answer was identical 23 times, and semantically the same in 11 more. 7 more questions were wrong "simiarly" IE both picked fungi but chose different species.

Conclusion:

Given the flaws in my testing, it's impossible to say specifically how close the quant is to the full fat model. You'd need to burn hundreds of millions of tokens to really know with certainty and that's just on Humanities Last Exam. However what I can say fairly confidently now is that the 2.25bpw quant isn't completely lobotomized. Anything the API could answer, the local model was there or there abouts. For example I'm pretty confident that if I took any question the API could answer, and asked my local model with a 256k token limit it'd probably answer the same. Both sides are capable of a much better score with unlimited token budget and max reasoning.

Clover-35B-A3B-Preview is the first model released by DiffuSpace, delivering state-of-the-art performance among open-source diffusion language models at release on the benchmarks reported in our launch blog. Clover combines a 35B-parameter mixture-of-experts backbone with approximately 3B active parameters per token, block diffusion generation, and a 128K context window.

Blog · Code & inference backends · Hugging Face

Hi everyone, whenever I looked at existing LLM leaderboards I was missing practical numbers for running quantized open weight models on affordable local hardware. Most tables only copy vendor cloud scores or run tiny subsets of a benchmark, and none of them tell you how long a model actually sits there calculating before it finishes. Because nothing like that existed, I set up a dedicated Linux test server with two RTX 5060 Ti cards giving 32 GB of VRAM in total, running vLLM in Docker. Over the past weeks I ran 12 current local models from 4B up to 35B through complete benchmark datasets without any subsampling, including IFEval, IFBench, ARC AGI 1, SimpleQA, SimpleQA Verified, SciCode and vLLM bench serve. The lineup includes Qwen 3.5, Qwen 3.6 and Qwen 3.8 in 4B, 9B, 27B and 35B A3B, Google Gemma 4 in E4B, 26B A4B and 31B, plus GLM 4.7 Flash in 4 bit quantizations like NVFP4 and AWQ.

Every single model is tested twice across all benchmarks, once with thinking turned off and once with thinking turned on, using the official sampling parameters. While the tests run, my system tracks the exact wall clock duration and throughput in tokens per second to calculate a time efficiency score from 0 to 100 for every single cell. That turned out to be the most interesting part of the whole project. You can immediately see which models deliver strong scores in 20 or 30 minutes and which ones burn GPU power for 10 hours straight because they get stuck reasoning in circles on simple questions. Especially the MoE models like Qwen 3.6 35B A3B and Gemma 4 26B A4B surprised me by finishing in a fraction of the time while beating or matching the dense 27B and 31B models.

I put all results into a clean, sortable comparison table on a new subpage of my site at https://llmbenchmarks.io/local-gpu-benchmarks/ so anyone with a 16 GB or 32 GB setup can check the numbers before downloading 20 GB of weights. The site is a personal project with no ads, no tracking and no affiliate links, and new benchmarks like Chess Puzzles are already running on the server right now. Feel free to take a look and let me know which local models you would like to see tested next.

Quick update: I’ve uploaded a new version of my 4B coding model. One-shot results and visual quality are much better in my latest tests, and training is nearly finished.

I recommend starting with Q8, which is the version I’ve tested. Q4 is also available, but I haven’t tested it yet.

Q8 settings: temperature 0.6, top-k 20, top-p 0.95.

I’ve also had good results generating 3D scenes from image references, even though the vision encoder wasn’t included in this training run.

Updated Q4 and Q8 on Hugging Face

If you downloaded an earlier version, delete the old model and download it again in LM Studio or whichever app you use. The filenames haven’t changed, so make sure you’re loading the updated files.

As promised, I’ll publish the dataset and full FP16 weights today.

I’ll add the links to this same post when they’re ready. This will be my last new post for this stage of the project.

Thanks everyone for the support, feedback, and testing! It’s been great seeing the community try it out.

A PDF turning into clean Markdown doesn't mean all the numbers survived the trip.

OpenBMB just released MiniCPM-OCR 1.0, a 1.3B model for parsing Chinese and English documents, alongside MiniCPM-OCR Jev, a 2B model for checking OCR output.

Jev takes an image crop and its corresponding OCR result, then returns a quality score from 0 to 1. It handles text, formulas and tables. It can also score results from other OCR parsers, so using their parser isn't a requirement.

That's the part I'd want in a local RAG pipeline. Feed it a table with a wrong number and your chatbot can very helpfully cite that wrong number back to you. I'd use something like this to flag questionable regions for review before they go into the index.

The code and both models' weights are Apache 2.0. The GitHub repo has local deployment instructions and links to quantized weights.