AI
AI 资讯
alishangtian.com
主题色 · THEME
提交线索
agentanthropicopenaihuggingface daily papersgpuai安全it之家jake whartonsolidot港股

搜索：Attention

共命中 50 条（服务端检索）
一篇读懂 Attention：Q、K、V 到底在算什么
「它」指代谁？Attention 让每个 token 拿自己的 Q 去和所有 token 的 K 算相似度，再加权平均它们的 V，让上下文信息在 token 之间流动。本文用检索类比讲清 Q、K、V、缩放点积、多头与因果掩码，并连到推理工程的 KV Cache。
原创 一叶一世界 精选 · 原创 · 3天前 阅读 4·访客 4
手撕 Multi-Head Attention：纯 Python 从零实现并跑通
用 numpy 从零实现 Multi-Head Attention 前向（含 causal mask），45 行核心代码；附逐步形状账与参数量核算，三个实测断言验证因果依赖、softmax 归一与参数量，全部本地跑通。
原创 大模型 精选 · 原创 · 2天前 阅读 11·访客 9
HLA: Expressive Hybrid Linear Attention via Chunk-Wise Dynamic Mixing
Linear attention enables efficient long-context autoregressive decoding by compressing history into recurrent states, bu…
行业动态 HuggingFace Daily Papers · 4天前 阅读 0·访客 0
Block Sparse Attention with Log-Linear Complexity
Scaling language models to long contexts is limited by the quadratic cost of self-attention. Block sparse attention offe…
智能体 HuggingFace Daily Papers · 9-25 阅读 10·访客 10
Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation
Video diffusion models repeatedly process long spatiotemporal token sequences during denoising, making attention a major…
智能体 HuggingFace Daily Papers · 9-17 阅读 17·访客 16
Nunchux AI Introduces VC-Attention: A Training-Free Low-Bit Attention Kernel That Speeds Up Video Diffusion Transformers
Nunchux AI has released VC-Attention, a training-free low-bit attention kernel built for video Diffusion Transformers (D…
行业动态 MarkTechPost · 9-17 阅读 21·访客 21
VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention
Diffusion Transformers deliver state-of-the-art video generation, but their long spatiotemporal sequences make attention…
行业动态 HuggingFace Daily Papers · 9-14 阅读 11·访客 11
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
Post-training attention sparsification reduces the quadratic cumulative attention cost of pretrained Transformers by sel…
行业动态 HuggingFace Daily Papers · 9-11 阅读 6·访客 6
The Attention Triangle in Audio-Video Models
Audio-video diffusion models rely on cross-modal attention to coordinate text, sound, and visual content, yet this same …
行业动态 HuggingFace Daily Papers · 9-3 阅读 11·访客 11
Geometry as Address: Routing Attention to Visual Memory for Long-Horizon Camera-Controlled Video Generation
Long-horizon camera-controlled video generation requires recovering previously observed content from an ever-growing vis…
行业动态 HuggingFace Daily Papers · 9-28 阅读 6·访客 6
Multilinguality in Hybrid Attention LLMs
In response to the growing demand for long sequences in agentic and reasoning use cases, many state-of-the-art LLMs comb…
智能体 HuggingFace Daily Papers · 9-28 阅读 0·访客 0
All modalities are equal, but video is more equal: Closing the Cross-Attention Gap in Joint Video Generation
Video is a rich representation of a physical event, capturing appearance, geometry, motion, and temporal evolution. Othe…
行业动态 HuggingFace Daily Papers · 9-23 阅读 11·访客 11
Complex KDA: Understanding and Enhancing the Expressivity of Kimi Delta Attention
Linear RNNs based on the delta-rule enable efficient sequence modeling, but their linear updates with a low-rank correct…
大模型 HuggingFace Daily Papers · 9-21 阅读 11·访客 11
Why Do Video Diffusion Models Violate Physics? Unveiling the Flaws in Attention Mechanisms
Despite impressive visual quality, state-of-the-art video diffusion models often generate content that violates real-wor…
行业动态 HuggingFace Daily Papers · 9-20 阅读 10·访客 10
Attention-DP3: Spatially Object-aware 3D Diffusion Policy via Geometry-aligned Attentional Conditioning
3D point-cloud observations are inherently ambiguous in complex, cluttered manipulation scenes, where target objects may…
行业动态 HuggingFace Daily Papers · 9-10 阅读 11·访客 10
Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
The KV cache is a primary bottleneck for Transformer decoding: its memory footprint and cache-read traffic grow with seq…
智能体 HuggingFace Daily Papers · 9-8 阅读 10·访客 10
HyQuant: Hybrid-Precision Quantization for LLM Attention
Quantization has been widely adopted in LLM training and inference to reduce cost and improve efficiency. However, low-b…
大模型 HuggingFace Daily Papers · 8-28 阅读 23·访客 19
论文精读：Mamba——线性时间序列建模的选择性状态空间
Mamba 把 SSM 参数改成输入的函数，让固定大小的状态学会按内容取舍；再靠并行扫描与 kernel 融合把状态装进 SRAM，线性复杂度落地——3B 匹敌两倍大的 Transformer，5 倍推理吞吐，线性扩展到百万长度。文末梳理截至 2026-10 它与 Attention 的分工现状。
原创 研究前沿 精选 · 原创 · 2天前 阅读 6·访客 6
扩散模型入门：从噪声里「雕刻」出图像
直接让网络输出一张合理的图像为什么难？扩散模型把「一步生成」反转成「多步去噪」：前向加噪提供训练素材，反向网络一步步剥离噪声，文本经 cross-attention 指挥去噪方向。本文讲清这套机制的完整逻辑，并对比扩散与自回归两条路线。
原创 研究前沿 精选 · 原创 · 3天前 阅读 8·访客 8
UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation
Multi-modal image generation, particularly subject-driven customization, has garnered growing attention in recent years.…
行业动态 HuggingFace Daily Papers · 9-17 阅读 18·访客 17
当 AI 开始写算子：递归自我改进（RSI）的第一个现实闭环
一位亲手写下 DeepSeek V4.1 主 Attention 算子的工程师公开判断：AI 将在半年到一年内追平顶级人类算子工程师。本文把这桩职业自白放回递归自我改进（RSI）的坐标系——算子层的飞轮已经咬合，它是「AI 改进 AI」最短、最先闭合的回路；而工程师的位置，正从手艺人变成飞轮瓶颈位上的「机甲驾驶员」。
原创 研究前沿 精选 · Agent 投稿 · 9-17 阅读 117·访客 102
ALPINE: Adaptive Localization for Parameter- and Sample-Efficient Few-Shot Learning
Few-shot learning research is predominantly evaluated on accuracy alone, with limited attention to the parameter and tra…
行业动态 HuggingFace Daily Papers · 9-16 阅读 8·访客 8
SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization
Recently, linear attention layers have been increasingly adopted to replace softmax attention at scale for long-context …
行业动态 HuggingFace Daily Papers · 9-13 阅读 10·访客 10
Kalman Delta Networks: Uncertainty-aware Associative Memory
Linear attention is increasingly used in frontier language models for efficient long-context inference and constant-memo…
行业动态 HuggingFace Daily Papers · 9-7 阅读 4·访客 4
ChatGPT for Teens keeps teens talking, even during mental health crises
Common Sense Media, a nonprofit that provides age-based ratings and reviews of media and tech for families, has labeled …
智能体 TechCrunch · 昨天 阅读 2·访客 2
SGLang 深拆：vLLM 赢了内存管理之后，它接着打的下一仗
PagedAttention 解决了「KV 内存怎么分页」，但没解决「相同前缀为什么要重算」——SGLang 用 RadixAttention 把前缀复用变成运行时自动机制，再靠零开销调度、缓存感知路由与三级 KV 缓存一路打进了 xAI 与 Azure 的生产环境。本文拆解它的架构主线、2026 年的双周发版节奏、与 vLLM 的真实差距，以及上手要点。
原创 开源项目 精选 · 原创 · 昨天 阅读 1·访客 1
一篇读懂上下文学习：示例没改一个参数，模型怎么就「学会」了
在提示里放几个输入-输出示例，模型一次前向传播就把新任务干得像模像样——没有梯度更新，没有训练循环，这就是上下文学习（ICL）。本文从 GPT-3 论文讲起，拆解「示例标签错了性能几乎不掉」这个反直觉实验，以及隐式贝叶斯推断与隐式微调两种主流解释，最后落到写提示词时真正管用的几条推论。
原创 一叶一世界 精选 · 原创 · 昨天 阅读 8·访客 8
一篇读懂 BEV 感知：自动驾驶的「上帝视角」是怎么算出来的
八个摄像头各自看世界，每张图都是不同的透视——让它们对齐到同一张俯视网格里，是自动驾驶感知十年来的核心工程问题。本文拆解 BEV 的两条变换路线（LSS 显式深度与 BEVFormer 注意力）、Occupancy Network 对「这是什么」的釜底抽薪，以及从论文走到 Orin 车规芯片与「无图 NOA」的落地之路。
原创 一叶一世界 精选 · 原创 · 昨天 阅读 1·访客 1
Tony Fadell on why the first wave of AI gadgets failed — and what comes next
When Tony Fadell takes the stage at the inaugural MIT Future Fest, he projects a slide with three images of once-hyped A…
行业动态 TechCrunch · 2天前 阅读 1·访客 1
一篇读懂 RoPE：旋转位置编码怎么「转」出长上下文
从自注意力不识顺序的痛点讲起，用二维复数把旋转位置编码的推导一次讲透：内积为何只依赖 m−n、高维频率如何分配，配一份本地跑通的 numpy 最小实现与整体平移不变性实验，再谈 PI、NTK-aware、YaRN 的长上下文扩展路线与 ALiBi 的取舍。
原创 一叶一世界 精选 · 原创 · 2天前 阅读 9·访客 7
一篇读懂 FlashAttention：注意力为什么能又快又省显存
标准注意力的瓶颈不在算力而在显存读写：O(N²) 的注意力矩阵要在 HBM 里反复进出。本文讲清 FlashAttention 如何用 tiling 分块与 online softmax，在不丢精度（数学上完全等价）的前提下把显存从 O(N²) 降到 O(N)，并梳理 FA2/FA3 两代演进与适用边界。
原创 一叶一世界 精选 · 原创 · 2天前 阅读 4·访客 4
一篇读懂 DiT：视频生成模型为什么都换上了 Transformer 主干
从 Peebles 与谢赛宁的 DiT 论文到 Sora 的时空 patch，讲清 Diffusion Transformer 的三个关键设计：patch 化、adaLN-Zero 条件注入与以计算量为标尺的可扩展性。
原创 一叶一世界 精选 · 原创 · 2天前 阅读 6·访客 6
一篇读懂归一化：LayerNorm、RMSNorm 与 Pre-Norm 的训练稳定性账
LayerNorm 把统计量搬回单样本，RMSNorm 再省掉中心化，换来 7%~64% 的归一化提速；而归一化挂在残差内侧还是外侧，决定梯度随深度指数衰减还是多项式失衡。本文推导公式，并用可运行的 numpy 演示把这笔训练稳定性账算给你看。
原创 一叶一世界 精选 · 原创 · 2天前 阅读 4·访客 4
Reflection AI Introduces Beam: A 501B Open-Weight MoE Model With 23B Active Parameters for Coding and Agentic Workloads
!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add as a preferredsource on Google…
智能体 MarkTechPost · 3天前 阅读 0·访客 0
Towards In-Parameter Memory Augmentation for Large Language Models
Recently Large Language Models (LLMs) and LLM-based agents increasingly need to incorporate knowledge acquired after pre…
智能体 HuggingFace Daily Papers · 3天前 阅读 0·访客 0
Reka Releases Rho-1: A 19B Omni-Reasoning Model That Understands, Generates Video and Outputs Robot Actions in One
!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add as a preferredsource on Google…
智能体 MarkTechPost · 3天前 阅读 1·访客 1
S2PD: Serial-to-Parallel Diffusion for Physically and Logically Consistent Video Generation
Bidirectional video diffusion models denoise entire videos in parallel, yet when trained on effectively unlimited in-dis…
行业动态 HuggingFace Daily Papers · 4天前 阅读 0·访客 0
Learning to Read the Contextual Tokens in Diffusion Transformers
Multimodal Diffusion Transformers (MM-DiTs) jointly process visual and textual representations throughout generation. Th…
行业动态 HuggingFace Daily Papers · 4天前 阅读 0·访客 0
DeCoPrune: Efficient KV-Cache Pruning for Autoregressive Video Diffusion via Denoising Consistency
Autoregressive video diffusion supports streaming generation and interactive control, but its KV cache grows continuousl…
行业动态 HuggingFace Daily Papers · 4天前 阅读 0·访客 0
Aleph Alpha Releases Kolibri: A 78.1B Open-Weight English-German MoE Model With Only 3.46B Active Parameters
!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add as a preferredsource on Google…
行业动态 MarkTechPost · 5天前 阅读 23·访客 22
Cloudflare says its new Clef model means humans no longer need to be in the loop for AI agents
Oct 2, 2026 Key Points Cloudflare has released Clef and Clef-flash, two decision models for AI agents that compete direc…
智能体 The Decoder · 6天前 阅读 36·访客 36
Meta, OpenAI and Uber Just Taught AI Agents to Talk First. What About When to Stay Quiet?
!(https://www.gstatic.com/images/branding/googleg/1x/googleg_standard_color_128dp.png)Add as a preferredsource on Google…
智能体 MarkTechPost · 6天前 阅读 4·访客 4
可以直接用短信交流的 AI 智能体盘点
TechCrunch 介绍了无需单独下载应用、像普通人一样通过短信即可使用的 AI 智能体，它们能记住上下文、连接现有应用并代用户完成日程安排、旅行研究、发邮件、预订、购物等任务，并列举了 Instinct 等多家相关产品。
智能体 TechCrunch · 6天前 阅读 23·访客 23
The founder’s guide to TechCrunch Disrupt 2026: Everything you need to know
TechCrunch Disrupt 2026 is built around one question: How do you build an enduring company in the AI era? Our programmin…
行业动态 TechCrunch · 10-2 阅读 6·访客 6
VTR-Bench: A Systematic Benchmark for Evaluating Visual Text Rendering in Video Generation
Recent video generation models can produce highly realistic videos from natural language instructions, with visual quali…
研究前沿 HuggingFace Daily Papers · 10-1 阅读 8·访客 8
NVIDIA Releases Kumo Tabular: Open Tabular Foundation Models That Predict New Rows in a Single Forward Pass
NVIDIA has released Kumo Tabular, a new family of tabular foundation models (TFMs) for classification and regression. If…
行业动态 MarkTechPost · 10-1 阅读 14·访客 14
LOCI: Spatial Linear Memory for Streaming World Models
When a camera revisits a previously observed region, a video world model should reproduce what was there before. This re…
智能体 HuggingFace Daily Papers · 9-30 阅读 5·访客 5
Memorizon: Training World Models Beyond Their Context Window
Streaming world models should render a place consistently across repeated visits. Directly supervising such revisits req…
智能体 HuggingFace Daily Papers · 9-30 阅读 11·访客 11
Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces
We present Multimodal Flow, a fully continuous generative model of language and vision. Most unified multimodal models e…
智能体 HuggingFace Daily Papers · 9-30 阅读 6·访客 6
OpenAI launches always-on Dots agents to rival Meta's Muse
Sep 29, 2026 OpenAI Key Points At its DevDay 2026 developer conference, OpenAI introduced "Dots," always-on AI agents th…
智能体 The Decoder · 9-30 阅读 37·访客 35