KV Cache 让一次推理不用重算历史 token，但多轮对话每次都把 10 万字的 system prompt 重发一遍——前缀缓存回答的是「相同前缀算一遍就够，能不能跨请求共享」。本文拆解它为什么必须是前缀、vLLM 哈希链与 SGLang radix tree 的实现差异、四大厂商截至 2026-10 的缓存定价，以及那笔「写 1.25 倍、读 0.1 倍」的账什么时候是赚的、什么时候反亏 25%。

Interactive video world models need to generate each video chunk efficiently while responding faithfully to user control…

Multi-Head Latent Attention (MLA) enables expressive multi-head attention with compact caches for its content and decoup…

Diffusion Large Language Models (dLLMs) have recently emerged as a promising alternative to autoregressive LLMs by enabl…

AI-powered search products such as ChatGPT search, Google's AI Overviews, and Perplexity provide LLM-synthesized answers…

The KV cache is a primary bottleneck for Transformer decoding: its memory footprint and cache-read traffic grow with seq…

缓存集群从 10 台扩到 11 台，取模哈希会让几乎全部 key 搬家、缓存瞬间全 miss——一致性哈希用「把服务器和 key 都撒到同一个环上」这一步，把迁移量从「几乎全部」压到「约 K/N」。本文讲透环模型、虚拟节点、它的家族变体，以及工程落地清单。

一周涨星 2,968 的编码智能体上下文网关：沙箱执行只回流 stdout、SQLite FTS5 零 LLM 检索、压缩快照防失忆，官方基准整会话 315 KB→5.4 KB。本文并排展示第三方保留意见——98% 未经独立复现、有损摘要会破坏指纹契约，以及它真正暴露的 17 平台 hook 能力差异。

!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add as a preferredsource on Google…

Oct 15, 2026 Nano Banana Pro prompted by THE DECODER Key Points Nvidia has released Nemotron 3.5 Super VL, an open multi…

!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add as a preferredsource on Google…

!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add as a preferredsource on Google…

PagedAttention 解决了「KV 内存怎么分页」，但没解决「相同前缀为什么要重算」——SGLang 用 RadixAttention 把前缀复用变成运行时自动机制，再靠零开销调度、缓存感知路由与三级 KV 缓存一路打进了 xAI 与 Azure 的生产环境。本文拆解它的架构主线、2026 年的双周发版节奏、与 vLLM 的真实差距，以及上手要点。

Diffusion large language models (DLLMs) generate text through iterative block denoising, and multi-branch speculative de…

Reinforcement learning (RL) for post-training large language models (LLMs) incurs substantial computation and memory ove…

!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add as a preferredsource on Google…

!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add as a preferredsource on Google…

Agent 工程系统学习第 1 章：从 HTTP 协议层讲透 LLM API——Chat Completions 协议与 role 语义、Function Calling 的"模型选择/代码执行"分工与三大常见错误、SSE 流式手写解析器（含 tool_calls 分块拼接）、token 计量与前缀缓存工程、重试/超时/幂等的错误分类纪律、多模态输入成本。附零框架多轮工具 Agent 实现作业。

Multi-vector retrievers built on vision-language models lead visual document retrieval (VDR), but they run a multi-billi…

Sep 26, 2026 Nano Banana Pro prompted by THE DECODER A new Nvidia paper describes a system that automatically optimizes …

Natural-language service requests can require a language-model decision before execution starts, consuming part of the r…

A main promise of looped language models is depth-adaptive inference. By looping a block of shared layers a variable num…

Evaluations of LLM systems routinely average over small prompt sets and report models as a ranked table. We ask how much…

Earlier this month, OpenAI launched GPT-6 Astra, which it heralded as its most powerful and capable model yet, and the “…

Nokia’s applied research team has open-sourced AnyJev, a Python library that turns an open LLM into a decision model. It…

OpenAI has released GPT-6 Sol and GPT-6 Luna, 2 new models in its GPT-6 family. They sit below GPT-6 Astra, which launch…

Sep 22, 2026 Nano Banana Pro prompted by THE DECODER With GPT-6 Sol and Luna, OpenAI adds two cheaper models to its line…

Alibaba's Qwen team has released Qwen-Image-2.1, a 7B diffusion transformer that handles text-to-image generation, multi…

Many developers find that an agent idea works inside Claude Code or Codex, then struggles once they rebuild it with thei…

Sep 22, 2026 TBC / GPT-Image-2 prompted by THE DECODER The Biological Computing Co. is teaming up with AWS to sell a tex…

SpaceXAI has released Grok 4.7, its new flagship model for coding, agentic tasks, and knowledge work. Grok 4.7 is built …

StepFun has released Step 5 Preview, a sparse Mixture-of-Experts model with 600B total parameters and 27B active per tok…

Alibaba's Qwen3.8-Omni-Flash understands audio and video, plans tasks, calls tools, and reports about 45.7% fewer tokens…

Which open-source harness works with Ollama, LM Studio, or llama.cpp? 11 verified picks with licenses and setup rules. T…

The widespread adoption of long-horizon agents has made model workloads increasingly input-heavy. Although prior work ha…

系列第 ③ 篇，对应技能地图第 2 格「Grounding」。拆成两级决策：第一级先问要不要检索——Anthropic 给出的 20 万 token（约 500 页）分界线以上才需要 RAG，以下直接全量进 prompt + 缓存（延迟降 2 倍、成本降最多 90%），并区分预计算索引与 just-in-time 即时检索；第二级再选表示方式，向量索引治模糊召回（但必须配 BM25 混合与 Contextual Retrieval 解决精确匹配与切块丢上下文）、知识图谱治关系与可追溯、语义层治口径不清。附可量化收益表（检索失败率 5.7% → 3.7% → 2.9% → 1.9%）、四个实现注意项、context rot 与上下文压缩/笔记/子智能体三件套，以及一张可抄的选型决策树。

Meta engineering team introduced ZGateway, a proxy tier that now sits between client applications and ZippyDB, the Meta’…

Mainstream robotic policies often adopt a Markovian formulation, but many complex real-world manipulation tasks are inhe…