一篇读懂 PD 分离：Prefill 与 Decode 为什么不再挤在一个 GPU 里
同一块 GPU 上，prefill 是计算受限的短跑冲刺，decode 是带宽受限的长跑——挤在一起互相拖累，一个长 prefill 能把解码速度拖慢最多 30 倍。本文讲透 2026 年成为默认架构的 PD 分离：干扰从哪来、KV cache 怎么搬、DistServe 与 Mooncake 交出的成绩单，以及什么时候该拆、什么时候千万别拆。
阅读全文 →
全部文章
ARCHIVE 共 363 条Steering Geometry: Validating Human Value Geometry in LLM Steering Space
As large language models (LLMs) are increasingly deployed in alignment-sensitive contexts, activation steering has emerg…
Online Learning with LLM Experts from Limited Feedback
We study adaptive routing of prompts to large language model (LLM) experts to maximize response quality in an online set…
Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization
Large language models (LLMs) are increasingly used to formulate optimization models from natural-language problem descri…
When Models Edit Too Much: On the Fidelity of Minimal Code Edits
Large language models (LLMs) are increasingly used to edit existing code, but correctness alone is not enough: useful re…
Select, Compress, Reinvest: A Controlled Study of Visual-Token Allocation in Long-Video MLLMs
Long-video language models cannot look at every frame: an hour sampled once per second is 3,600 images, and a system kee…
Privacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoys
We present a systems-security case study of a two-node split-LLM training system whose privacy evaluation passed while l…
What Else Needs Fixing? Exploring Cost-Effective Test-Time Compute for Revision Propagation in Artifacts Generated Through Conversation
Large Language Models (LLMs) often help users generate artifacts through iterative cycles of generation and revision in …
Enoki: Efficient Multi-Level Hallucination Detection
Ensuring factuality remains a critical challenge for deploying LLMs in high-stakes settings. Existing hallucination dete…
Recognition-Refusal Misalignment in LLMs: Why Models Answer Structurally Unanswerable Questions
Large language models often answer structurally unanswerable questions, such as computing cot(-540°) or evaluating (1).s…
HyQuant: Hybrid-Precision Quantization for LLM Attention
Quantization has been widely adopted in LLM training and inference to reduce cost and improve efficiency. However, low-b…
A Three-Layer Caching Architecture for Low-Latency LLM Web Search on Commodity CPU Hardware
AI-powered search products such as ChatGPT search, Google's AI Overviews, and Perplexity provide LLM-synthesized answers…
GPT-5 发布：统一系统路由下一代旗舰
OpenAI 发布 GPT-5，将快速响应与深度推理统一到一个系统内自动路由，在编码、数学、多语言与幻觉抑制上全面提升，并大幅加强 Agentic 任务能力。