多模型的统一接入、路由、限流与成本核算——开源 LLM 网关（LiteLLM 为代表）把 API 网关的成熟模式搬进 LLM 世界。本篇拆解 LLM 网关的核心能力（统一接口、路由策略、重试降级、成本追踪）、LiteLLM 的架构形态，以及「自建网关 vs 商业平台」的选择。

LLM 流量与普通 API 流量差异巨大：响应慢且长、按 token 计费、供应商不稳定、还要保留换模型的自由。本文逐项拆解 LLM 网关的核心职责——双层限流、多供应商路由、流式转发、降级链、成本核算与可观测，并给出关键设计要点。

微服务架构中，客户端需要调用几十个后端服务——每个服务都做认证、限流、日志，是巨大的重复。API 网关是所有请求的统一入口：路由分发、认证鉴权、限流熔断、日志聚合在网关层统一处理。本文讲清网关的职责边界、主流网关的对比、以及「什么该放网关、什么不该放」的分界。

Dify 是最流行的开源 LLM 应用平台：可视化编排工作流、内建 RAG 管线、Agent 节点、模型管理与企业治理。本篇拆解它的产品分层（Chatflow/Workflow/Agent）、内建 RAG 的工程取舍、插件与模型接入体系，以及「低代码平台 vs 代码框架」的适用边界。

SSE 是 LLM 应用做打字机效果的主流方案。本文讲透 text/event-stream 协议细节与 OpenAI 兼容流格式，给出本地跑通的 Express 转发端点与 fetch + ReadableStream 解析器，并盘点 nginx 缓冲、gzip、连接超时、断线续传、连接数管理五个生产坑。

Agent 工程系统学习第 1 章：从 HTTP 协议层讲透 LLM API——Chat Completions 协议与 role 语义、Function Calling 的"模型选择/代码执行"分工与三大常见错误、SSE 流式手写解析器（含 tool_calls 分块拼接）、token 计量与前缀缓存工程、重试/超时/幂等的错误分类纪律、多模态输入成本。附零框架多轮工具 Agent 实现作业。

GPT-4 当裁判与人类偏好的一致率 85%，超过了人类彼此之间的 81%——这是 LLM-as-a-Judge 立身的实验，但它交换位置后的一致率只有 65%，一个 token 就能操纵打分，模型版本一漂移榜单就作废。本文拆解机器裁判的奠基实验、三种已知偏差与缓解手段、工程化成本，以及什么时候不该用它。

Large language model (LLM)-based multi-agent systems (MAS) coordinate specialized agents to solve complex tasks, turning…

Modern LLM agents can retrieve information from accessible documents, and then disclose a sensitive conclusion assembled…

LLM routing dynamically selects and coordinates heterogeneous language models across diverse tasks. However, existing ap…

论文提出AgentSafeEvo，通过攻击者-防御者博弈的协同进化强化学习，提升LLM智能体在多轮工具调用中抵御恶意内容操纵的安全性。

提出LLM-to-Flow Distillation（LFD）方法，用冻结的大模型作为自适应教师，将特定领域的流/扩散生成器对齐到自然语言科学发现目标，无需显式代理奖励。

Power sampling improves large language model (LLM) reasoning at test time by sampling from a sharpened version of the ba…

Architect Financial Technologies 发布 Liquid Inference，一个 LLM 推理市场路由器，每次请求通过实时拍卖由各供应商报价竞价，买方支付满足规则中的最低报价，开发者只需替换 base URL 即可让供应商在价格上竞争。

Developers can build LLM agents by adapting third-party models through benign post-training. We study a supply-chain thr…

Evaluations of LLM systems routinely average over small prompt sets and report models as a ranked table. We ask how much…

Nokia’s applied research team has open-sourced AnyJev, a Python library that turns an open LLM into a decision model. It…

LLM agents are increasingly deployed in collaborative settings, yet long-term interaction may give rise to undesirable c…

LLM assistants are widely used for daily social advice, yet evaluating their social reasoning in such consultation setti…

Large language model (LLM) trading agents can combine market data, news, and executable analysis, but their behavior is …

Skills extend an LLM agent beyond its parametric knowledge, and the gain they promise rests on picking the right one. De…

Large language model (LLM) agents allocate test-time compute adaptively as they revise solutions, use tools, explore alt…

Concurrent local LLM serving on unified-memory desktops must preserve memory headroom and output fidelity, which speed-o…

We study scheming in LLM agents, in which agents covertly pursue misaligned goals. Our focus is to understand how schemi…

Large language model (LLM) agents increasingly rely on external skills, but routing user requests over large skill regis…

We study adaptive routing of prompts to large language model (LLM) experts to maximize response quality in an online set…

We present a systems-security case study of a two-node split-LLM training system whose privacy evaluation passed while l…

Multi-agent LLM systems commonly use an orchestrator to decompose a task for a team of workers and then improve through …

Quantization has been widely adopted in LLM training and inference to reduce cost and improve efficiency. However, low-b…

AI-powered search products such as ChatGPT search, Google's AI Overviews, and Perplexity provide LLM-synthesized answers…

一周涨星 2,968 的编码智能体上下文网关：沙箱执行只回流 stdout、SQLite FTS5 零 LLM 检索、压缩快照防失忆，官方基准整会话 315 KB→5.4 KB。本文并排展示第三方保留意见——98% 未经独立复现、有损摘要会破坏指纹契约，以及它真正暴露的 17 平台 hook 能力差异。

传统 APM 盯的是「请求是否 200」，LLM 应用的问题是「200 的请求答得对不对、花了多少钱」。本文按 token 成本、延迟（TTFT）、质量信号、调用轨迹四个层面拆解 LLM 可观测性，介绍 OpenTelemetry GenAI 语义约定与 Langfuse/OpenLLMetry 工具格局，给出生产闭环的落地路径。

智能体应用与底层模型之间，正在长出一个中间件层：网关、记忆服务、沙箱、评测与观测——它们把「每个智能体应用都要造一遍的轮子」收敛成共享基础设施。本篇观察这个中间件层的构成、它与历史上其他基础设施层的相似节奏、以及哪些抽象会沉淀、哪些会消失。

当日涨星 906 的知识图谱项目：代码用本地 tree-sitter AST 零 LLM 额度建图，文档/图片才走语义趟；52 文件混合语料单次查询 token 降 71.5 倍，LOCOMO recall@10 0.497，每条边带 EXTRACTED/INFERRED/AMBIGUOUS 置信标签。

当日涨星 666 的 Rust 项目：把编码智能体的长期记忆做成 git 版 Markdown wiki（源真相）+ SQLite 派生索引，默认零 LLM 调用，LongMemEval-S 470 题 zero-LLM hit@5 达 0.951，20+ 智能体共享记忆与 claim-once 交接。

VectifyAI 开源的无向量 RAG 引擎 PageIndex（当日涨星 +1,095、★38,082、MIT）：把长文档编译成 JSON 层级树，让 LLM 逐节点推理导航，取代切块+向量相似度。基于它的 Mafin 2.5 在 FinanceBench 全量 10,231 题报告 98.7%。拆解两阶段架构、三模式 TOC 自校验回退、三工具检索循环、成本口径，以及多文档规模化与数据主权的真实边界。

Nov 3, 2026 GPT-Image-2.5 prompted by THE DECODER Stanford researchers Owen Queen and James Zou tested 50 widely cited c…

Split language models keep input text on a client but send intermediate representations to a server. Attacks can reconst…

针对Rank-GRPO每次训练仅选一条回复导致优势信号主要反映抽样差异的问题，论文提出ReMaRk方法，将生成列表与所有回复标题的并集对比打分并均匀采样请求，在7个骨干模型和3个数据集上以25%-53%的步数达到原方法2000步的验证召回率。

!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add as a preferredsource on Google…

Decoder hidden states from different layers of large language models (LLMs) often encode complementary information. Howe…

Black-box optimization (BBO) arises in many scientific and engineering problems where objective evaluations are expensiv…

Google 联合多校开源 RRSI 框架，在冻结 LLM 前提下自动进化智能体 harness，并用正则化抑制基准过拟合，8 项基准最高提升 14.1 分。

Calibration of language models -- the alignment between expressed or implicit confidence and empirical correctness -- is…

GGUF, GPTQ, AWQ, EXL2, and EXL3 solve the same problem in different ways. This guide separates file containers from quan…

If AI lab PrismML isn't on your radar yet, it should be.]

Large language models (LLMs) provide a flexible interface for long-horizon robot planning, but generated plans often fai…

Test-time scaling can improve large language model reasoning by generating and combining multiple candidate responses. I…

Current Mixture-of-Agents (MoA) paradigms generally treat query routing and agent fine-tuning as separate processes, lim…

Safety guardrails in open-weight language models can be readily bypassed using Refusal Feature Ablation (RFA), a techniq…