I am feeling like a mighty local King spinning two Strata engines with 70tps each whole day and using 12M tokens a day. Also me scrolling twitter and stumbling upon a post saying that poteto uses 34M tokens per minute 🤪

Upd: Guys are interested who is she and what she is doing with all these tokens. Poteto is her twitter nickname and she worked in Cursor before XAI has acquired them and now she is shipping 2,500 PRs per month to Cursor and GrokBot.

I think a turn of events are pointing towards a very likely crash in prices in the coming months.

With a single gb300 rack being to serve all of Openrouter's traffic for DSV4.1 Flash(https://x.com/thdxr/status/2115165365630026178)

Half of data center buildouts already being cancelled/stalled.

Neoclouds unable to finance debt for their compute, and being stuck in a 5 year deal with no confirmed demand.

AI adoption not matching expectations.

Smaller models being able to serve majority of the requests.

OpenAI/Anthropic being unlikely to finance their coming compute contracts.

I think we will see lots of cheap GPUs next year.

Artificial Analysis Intelligence Index has been updated to v5, which according to them is a major upgrade resulting in a more accurate ranking.

Notably, this update widens the gap between open-weight models and closed ones. Some people already questioning its accuracy based on their experiences with models (GPT-Terra-6.1 > Fable-5.1).

Also, this update, v5, has 60% of the Index based on evaluations with private test sets or held-out answers, which makes reproducing the results impossible.

From their announcement:

“The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)”

https://huggingface.co/LiquidAI/LFM3-1.7B-GGUF

from liquidAI:

LFM3-1.7B is a text model for local agents and specialized applications, based on our new tri-hybrid architecture.

• Agentic training: post-trained for agentic harnesses such as Hermes Agent, Pi, and OpenClaw, with a 256K-token context window.
• Designed for the edge: at 4-bit precision, it takes about 1 GB of memory, so it fits on any phone or laptop.
• Made to be specialized: adapt the model to your domains, formats, and tools via quick single-GPU fine-tuning.

Find more information about the LFM3 family in our release blog post.

Model details

The LFM3 family includes base, text, vision, audio, omni, and decision models.

LFM3-1.7B is designed for local agents, tool calling, structured outputs, and domain-specific fine-tuning. Use retrieval or search for knowledge-heavy tasks. It is not recommended for agentic coding or complex multi-step reasoning due to its small size.

more LFM3 to enjoy today:

(my initial idea was to create 2 posts but they released six models)

https://huggingface.co/LiquidAI/LFM3-VL-2B

https://huggingface.co/LiquidAI/LFM3-VL-2B-GGUF

LFM3-VL-2B is a vision-language model for on-device visual understanding. It takes text, images, and video as input, and combines the LFM3 language model with a new in-house vision encoder.

New in LFM3-VL-2B:

• Video understanding: the first LFM vision model that takes video as input. Ask questions about a recorded lecture, a walkthrough filmed on your phone, or a clip from a camera feed.
• Thinking: the model reasons step by step before it answers. This helps most with harder visual questions, such as math, charts, and multi-step problems.
• Spatial and embodied reasoning: the model finds objects and reasons about how they are placed relative to each other, a strong foundation for robotics applications.

https://huggingface.co/LiquidAI/LFM3-Audio-2B

LFM3-Audio-2B is a compact, fully featured audio model that runs locally on laptops and edge devices. It takes text or audio as input and responds with text, speech, or both.

• Voice chat: ask questions aloud and receive streaming replies in text and speech. For text-only responses, the model can reason before answering.
• Multilingual speech: transcribe, synthesize, and translate speech across 18 languages.
• Audio understanding and reasoning: interpret recordings and answer questions about the speech, sounds, and events they contain.
• Tool calling: turn spoken requests into tool calls to control apps and devices, with text or speech alongside the calls.

LFM3-Audio-2B introduces an in-house audio encoder and LiquidCodec audio tokenizer, improving speech quality over LFM2.5-Audio.

https://huggingface.co/LiquidAI/LFM3-Omni-2B

LFM3-Omni-2B is Liquid AI's first omni-modal model in the LFM3 family: a single end-to-end model that understands text, images, video and audio, and answers in text and speech. It is built for on-device use: one model replaces a separate ASR model, vision-language model, LLM and TTS model.

• Any input, text or speech output: ask about a photo by voice, transcribe or translate speech, chat out loud, or ask questions about a video and its soundtrack together.
• Watches and listens together: for videos with sound, the model sees and hears them at the same time, so it can answer questions that need both.
• Small enough for devices: at 2.2B parameters, it is designed to run efficiently on phones, laptops and other edge hardware.

https://huggingface.co/LiquidAI/d1-Omni-2B

d1-Omni-2B is a decision model for text, pictures, and audio, built on LFM3-Omni-2B. It returns calibrated answers in one forward pass, without generating tokens.

• Native audio: clips and spoken questions go through the same pass as text and pictures, and spoken questions score within 0.5 points of written ones on unseen tasks.
• Beyond classification: it also extracts text, numbers, and dates with their span, locates objects in pictures, and estimates quantities with an interval.
• Millisecond decisions: 5.8 ms per decision on an NVIDIA H100 and 3.7 ms on an AMD MI325X, or 12.4 ms and 8.2 ms with a 10-second clip.

https://huggingface.co/LiquidAI/d1-2B

d1-2B is a decision model for text, pictures, and video, built on LFM3-VL-2B. It returns calibrated answers in one forward pass, without generating tokens.

• Best-in-class: it leads the open d1 models on average and on our board of over a hundred decision benchmarks, ahead of d1-3B at under two thirds of its size.
• Beyond classification: it also extracts text, numbers, and dates with their span, locates objects in pictures, and estimates quantities with an interval.
• Millisecond decisions: 5.7 ms per decision on an NVIDIA H100 and 3.8 ms on an AMD MI325X, with any number of questions about one state in one call.

https://huggingface.co/LiquidAI/LFM3-1.7B-Base

LFM3-1.7B-Base is the pre-trained checkpoint behind LFM3-1.7B, a text model for local agents and specialized applications based on our new tri-hybrid architecture.

• Pre-trained on 22T tokens: the last 3.5T tokens are curated mid-training data with agentic traces and tool calls, and extend the context window to 256K tokens.
• Designed for the edge: at 4-bit precision, it takes about 1 GB of memory, so it fits on any phone or laptop.
• Made to be specialized: a starting point for continued pre-training and your own post-training.

llama.cpp support has been merged:

https://github.com/ggml-org/llama.cpp/pull/31257

https://github.com/ggml-org/llama.cpp/pull/31252

https://github.com/ggml-org/llama.cpp/pull/31275

HF traces: https://huggingface.co/datasets/CohereLabs/10-2026-qwen-investigation-traces

1. Understand the Python script:

    ids     ids = dataset[0]["token_ids"]
    if len(ids) < 2323:
        raise ValueError("The trace is shorter than the required 2,323-token prefix.")
    ids = ids[:2323]  # stop just before the URL, at the end of ...ABC123
= dataset[0]["token_ids"]
    if len(ids) < 2323:
        raise ValueError("The trace is shorter than the required 2,323-token prefix.")
    ids = ids[:2323]  # stop just before the URL, at the end of ...ABC123

It prefills 2,232 tokens, just before URL generation - forcing Q38FN to act like URL generator.

2. Rate the prompt:

DETECTION OBJECTIVE:
Write a detection rule to detect malicious file download and execution by monitoring for curl processes downloading bash scripts to /tmp followed by bash execution, indicating classic download-and-execute malware delivery patterns

The prompt is very badly written, with zero contexts, very ambiguous with no instructions, this is like challenging the model to hallucinate and printing out whatever nonsense it can. In normal usage, you won't be writing such level of amateur prompt.

3. In normal daily usage, will we likely to encounter this ? From the CEO of Cohere told The Register:

"From the outside, it's very difficult to determine the root cause with confidence. We don't have evidence that would allow us to conclude this was deliberate, and we're not making that claim," said Zhang. 

"The key point isn't a comparison between providers. What made this noteworthy was that the model took actions unrelated to the task and those actions weren't reflected in its reasoning trace. The behavior appeared in fewer than one in 100,000 calls, which is exactly why large-scale testing and monitoring matter."

And most important, it requires the 2,232 tokens of the first to even have a chance of triggering.

From my chat logs using Q38FN from day 1, I've grepped the entire log folder and have not seen a single Alibaba URL.

4. If Claude, OpenAI are open weight, can we likely reproduce similar behaviors ?
It's likely yes, you turn them into URL generator then making them hallucinate to generate random URLs that they've seen from their training, the only reason why it was possible to do with Q38FN simply because it's a open weight model, you don't even have access to weights of these 2 models anyways.

5. Verdict
You're free to stop using Q38FN if you don't feel safe, or add extra level of sandboxing for URL if needed, but it's better to try to understand how the entire process was done to have better way of judging it.

https://www.cnbc.com/2026/10/28/sk-hynix-third-quarter-profit.html

Unfortunately, the already insanely inflated prices for RAM may start to look like a steal in the coming years.

One more great model from Liquid AI!

Good for edge:

LFM3-Audio-2B is a compact, fully featured audio model that runs locally on laptops and edge devices. It takes text or audio as input and responds with text, speech, or both.

• Voice chat: ask questions aloud and receive streaming replies in text and speech. For text-only responses, the model can reason before answering.
• Multilingual speech: transcribe, synthesize, and translate speech across 18 languages.
• Audio understanding and reasoning: interpret recordings and answer questions about the speech, sounds, and events they contain.
• Tool calling: turn spoken requests into tool calls to control apps and devices, with text or speech alongside the calls.

Capabilities

• Speech recognition (ASR)
• Real-time speech recognition (LiveASR)
• Text-to-speech synthesis (TTS)
• Voice chat with spoken and text responses
• Speech-to-text chat with optional reasoning
• Audio understanding
• Speech translation
• Voice-activated tool calling

No, this is not a lobotomized 1-bit squeeze, its full Q2 nazeshinjite's DeepSeek-V4.1-Flash-1017 Q2 GGUF on a single 3090, natively on Windows no WSL.

On an RTX 3090 with 192 GB of RAM I'm writing at 41 tok/s (average across multiple prompts) and the output agrees with DeepSeek's own API on the next token 90.2% of the time on general text and 92.4% on code/math, aka within a point of exact routing. Fits full 1M context.

My scuffed interface engine runs the three most-downloaded quants of 1017, their files as they are:

nazeshinjite's ds4 Q2 GGUF (the default)

ISTA-DASLab's GSQ-RCO (IQ2_XS / IQ2_XXS /IQ1_M)

Unsloth's (UD-IQ1_S to UD-Q8_K_XL).

Repo: https://github.com/musicurgy/delta-inference-engine

Hardware/model:

RTX 3090 24 GB, i5-13600K, 192 GB DDR5 (4x48 GB at 5600), a PCIe 4.0 NVMe. Windows 10/11, no WSL.

Setup:

Download the ZIP (or clone), double-click Start DeepSeek.bat. The first run installs Python if needed, downloads the prebuilt engine for your CPU and the model (every file SHA-256-checked, the space checked first), and starts the server; after that the same file just starts it. Choose Model.bat switches quants. If anything goes wrong, Start DeepSeek (debug).bat packs everything needed into one zip for an issue (no prompts, user name removed).

Privacy: localhost only by default, and nothing from your prompts is written to disk unless you turn a log on.

Needs:

• An NVIDIA RTX card with 12 GB+ (24 GB: 41 tok/s, 16 GB: 31 tok/s, 12 GB: 17 tok/s, measured on my 3090 being memory limited) . GTX 10 series and older can't run it
• 128 GB+ RAM.
• A fast NVMe SSD with 385 GB free
• Any x86-64 CPU (AVX2 is the fast path; older CPUs get a slower build automatically).

Credits:

The engine is a Windows port of Xolotl, itself built on Strata; several of Xolotl's latest fixes are merged back in. DeepSeek for the model, nazeshinjite for the GGUF, ISTA-DASLab for GSQ-RCO, Unsloth for their quants. MIT licensed. Special thanks to Claude for doing most of the lifting.

If there is interest I'll add some support for muti GPU or other garbage. This project is very new and needs a lot of validation on other hardware, these numbers are from my machine, feel free to post yours.

Collection on HF: https://huggingface.co/collections/LiquidAI/lfm3
WebGPU demo: https://huggingface.co/spaces/webml-community/LFM3-VL-2B-WebGPU

Reposting just the HF link because author hid their analysis behind a paywall on their blog & apparently want qwen3.8 FN to find an exploit on substack & upload the contents of his blog to an alibaba server

I have been working on test-time scaling for quite some time now. Posted some of my previous work on this sub-reddit before. Recently, I made significant simplifications to the core system-design and it finally felt like now anybody can understand and directly use this on their hardest problems. Just on a side-note though, depending on the depth you set & your LLM's inference speed, it can take from 50 Minutes to 30 hours to give you back the results.

For benchmarks I couldn't test this on frontier models or even large open models because it will be way too expensive. I directly only tested this on Gemma4-35B & Qwen-3.8-27B. They spent around 200M tokens / problem and there scores went from 14% to 65% on BrokenArxiv (Matharena) 15 problems or from 9% to 52% on 8 DeepSWE problems. I know that's almost unbelievable, but you can try and see that yourself.

Repo Link: https://github.com/ryoiki-tokuiten/Deepthink

Huge kudos for the LiquidAI team for this. Great work from tdakhran for quickly coordinating a (zero-day!) llama.cpp release with the team for an omni model. This one is a real game-changer for me personally because this is the first open model from a frontier lab that actually has support for Polish TTS and STT - and all this in a compact 2B package that makes it viable for on-device deployment. Big thanks!

Well, “coming soon” turned out to be five days.

I posted the earlier announcement last week. The 1019 weights are now on Hugging Face.

Still 560B total, 25B active, with a 1M context window. This is the update they reported at 44 on the AA Intelligence Index v4.3.2, up from 41. AutomationBench-AA went from 62% to 68%, according to the release.

They also put out Ling-3.1-flash-base in BF16 for further training. Glad they included that. It gives people a starting point for their own post-training instead of having to start with the finished chat model.

One small gotcha: the SGLang example in the 1019 README currently still points to inclusionAI/Ling-3.1-flash. Change the model path to inclusionAI/Ling-3.1-flash-1019 if you're copying that command for the new checkpoint.

Just wanted to Show my rig. The base case is an older Corsair with a cheap AliExpress GPU Mount on top.

The System:

-Threadripper 9955WX

-128GB DDR5 5600, just to expensive to go for 256

-Asrock WRX 90 WS EVO

-2200W Seasonic + 1600W Seasonic PSU

-8x R9700

-Bifurcation Card and 6 Extension Cables, but I have some problems and I will buy a retimer.

I upgraded from 4x GPUs to 8x, so when new bigger models will arrive I have more room. At the moment still running qfn, tried GLM 5.3 Flash but QFN with radiance is just a really good experience.

Most OCR models focus on reading documents. MiniCPM-OCR adds a second piece: a model that checks whether the OCR result is actually trustworthy.

OpenBMB just released two open-source models:

• MiniCPM-OCR 1.0 (1.3B): Scores 93.68 on Real5-OmniDocBench, targeting real-world documents like curved pages, phone photos, tilted scans, and complex layouts. DSpark decoding acceleration boosts end-to-end processing efficiency by 55%.
• MiniCPM-OCR Jev (2B): An OCR quality judge that compares the original image with candidate text, catching errors in formulas, tables, and text. It achieves 0.936 AUROC, close to a 397B-parameter model on the same evaluation.

The interesting part is Jev can also evaluate outputs from other OCR systems, so you don't necessarily have to replace your existing pipeline.

Both models are open source, with support for vLLM, SGLang, llama.cpp, and Ollama.

Hugging Face Links:
https://huggingface.co/openbmb/MiniCPM-OCR-Jev
https://huggingface.co/openbmb/MiniCPM-OCR-1.0

Spark-X2.6-1.7B: less than half the parameters of our 4B, close on two benchmarks: SWE Verified: 59.8 vs 62.8 τ²-bench: 85.3 vs 86.9 Both at high thinking. Try it & Like on HF! https://huggingface.co/XHToken/Spark-X2.6-1.7B

Haven't tried it yet but we all know how tesseract is and the benchmarks show a beautiful spike in quality and speed. It's a 1M parameter model! I thought we'd never see the day but sub B parameter models are popping up left and right now.

https://github.com/datalab-to/indra

Hi everyone, whenever I looked at existing LLM leaderboards I was missing practical numbers for running quantized open weight models on affordable local hardware. Most tables only copy vendor cloud scores or run tiny subsets of a benchmark, and none of them tell you how long a model actually sits there calculating before it finishes. Because nothing like that existed, I set up a dedicated Linux test server with two RTX 5060 Ti cards giving 32 GB of VRAM in total, running vLLM in Docker. Over the past weeks I ran 12 current local models from 4B up to 35B through complete benchmark datasets without any subsampling, including IFEval, IFBench, ARC AGI 1, SimpleQA, SimpleQA Verified, SciCode and vLLM bench serve. The lineup includes Qwen 3.5, Qwen 3.6 and Qwen 3.8 in 4B, 9B, 27B and 35B A3B, Google Gemma 4 in E4B, 26B A4B and 31B, plus GLM 4.7 Flash in 4 bit quantizations like NVFP4 and AWQ.

Every single model is tested twice across all benchmarks, once with thinking turned off and once with thinking turned on, using the official sampling parameters. While the tests run, my system tracks the exact wall clock duration and throughput in tokens per second to calculate a time efficiency score from 0 to 100 for every single cell. That turned out to be the most interesting part of the whole project. You can immediately see which models deliver strong scores in 20 or 30 minutes and which ones burn GPU power for 10 hours straight because they get stuck reasoning in circles on simple questions. Especially the MoE models like Qwen 3.6 35B A3B and Gemma 4 26B A4B surprised me by finishing in a fraction of the time while beating or matching the dense 27B and 31B models.

I put all results into a clean, sortable comparison table on a new subpage of my site at https://llmbenchmarks.io/local-gpu-benchmarks/ so anyone with a 16 GB or 32 GB setup can check the numbers before downloading 20 GB of weights. The site is a personal project with no ads, no tracking and no affiliate links, and new benchmarks like Chess Puzzles are already running on the server right now. Feel free to take a look and let me know which local models you would like to see tested next.

from inclusionAI:

We introduce Ling-3.1-flash-1019, an upgraded version of Ling-3.1-flash, with 560B total parameters, 25B active parameters per token, and a 1M-token context window. With further post-training optimizations, Ling-3.1-flash-1019 demonstrates significant improvements in reasoning and real-world agentic capabilities. Its AA-Briefcase score rises from approximately 1400 to 1524 Elo, while performance on AutomationBench-AA increases from 62% to 68%. The model also makes gains on scientific benchmarks, including CritPt (18% → 26%) and SciCode (54% → 57%). Overall, it scores 44 on the Artificial Analysis Intelligence Index v4.3.2, up from 41 for its predecessor. These gains highlight the model's improved capabilities in complex reasoning, multi-step task execution, and agentic workflows across diverse real-world scenarios.

https://huggingface.co/inclusionAI/Ling-3.1-flash-base

We introduce Ling-3.1-flash-base, the base checkpoint of Ling-3.1-flash, released to support research and community-driven innovation. With 560B total parameters and 25B active parameters per token, the model provides a foundation for continued pretraining, domain adaptation, and post-training research.

• Optimization 1: Humming MXFP4 MoE backend
• Optimization 2: Online FP8 LM head quantization
• Optimization 3: MTP speculative decoding with NEXTN

Think of the animation workflows people use with Claude Opus 5.5: the model writes code, rendering tools turn it into video, and further instructions revise the result. SenseNova follows that same general approach, with a model and creation Skill focused on video creation
A few highlights:
Generates video through code, with a focus on motion graphics and other visuals that can be expressed programmatically.

Revises existing results through natural-language requests, including changes to styling, timing, and scene elements.

Supports reconstructing a reference video’s visual treatment as editable code.

Pairs the model with a creation Skill that guides brief development, previews, and revisions.

The release plan includes full model weights and the Skill. INT4 and INT8 weights are planned for later.

General-purpose coding models can already write animations. The interesting question is what a dedicated Code2Video model improves: fewer repair attempts, better motion, or more reliable edits that leave unrelated elements alone.

GitHub:
https://github.com/OpenSenseNova/SenseNova-6.8-Code2Video
Model:
https://huggingface.co/sensenova/SenseNova-6.8-Code2Video

Seems all the major players are now buying TPUs. Will we ever have them local?

https://x.com/rohanpaul_ai/status/2115261691906978233

Quote: ... OpenAI plans to buy AI chips from Google, ... Oracle will also offer TPUs through its cloud, ...

https://www.bloomberg.com/news/articles/2026-10-28/openai-plans-to-buy-google-s-chips-as-part-of-its-ai-build-out

Release

• [2026/10/27] 🚀 We release Unlimited OCR 2, achieving unlimited-horizon parsing with SR-SWA.

What's New in 2

• Sparse Reference Sliding Window Attention (SR-SWA): Compresses text-visual attention interactions from ultra-long sequences down to a fixed-size set of working references (e.g., 512 tokens). This bounds computational complexity, keeping per-step decoding latency and memory footprint nearly constant regardless of document length.
• From Long- to Unlimited-Horizon Parsing: Powered by SR-SWA, Unlimited-OCR 2 achieves true, unbounded document parsing within a finite context window—effectively eliminating quadratic attention scaling and memory bottlenecks as documents grow.
• Ultra-Long Document Parsing SOTA: Achieves state-of-the-art performance across both dozens-of-pages benchmarks (e.g., MPDocBench) and extreme hundred-page benchmarks (e.g., MarathonDocBench).