SiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops

Papers

arxiv:2609.19169

Published on Sep 12

· Submitted by

Ranran Haoran Zhang on Sep 21

· Pennsylvania State University

Upvote

3

Authors:

,

,

,

,

Abstract

Concurrent local LLM serving on unified-memory desktops must preserve memory headroom and output fidelity, which speed-only rankings overlook. We introduce SiliconBench, which evaluates nine Apple Silicon serving engines through three lenses: speed, memory, and fidelity. We evaluate chat and agent serving on Qwen3, Qwen3.5, and Gemma 4. We use a classification task to check for quality regressions against an NVIDIA reference. DGX Spark provides a complementary serving-performance reference. Three desiderata guide interpretation: serving architecture readiness, memory discipline, and multi-node scaling. On Qwen3-0.6B, vllm-metal alone more than doubles throughput on both workloads from concurrency 1 to 16. CUDA vLLM and SGLang show stronger concurrency scaling on the same prompts. Explicit memory budgets do not guarantee memory headroom: two stacks complete every request while memory use approaches physical capacity and throughput declines. The newer model architectures have narrower engine support. Their evaluated implementations match the fidelity reference. Only three stacks satisfy the completion, fidelity, and model-coverage gates. Comparisons on larger dense and MoE models reinforce the importance of scheduling prompt processing alongside ongoing generation: vllm-metal's packed prefill-decode path maintains lower first-token latency than omlx under concurrent load. In the tested two-machine configurations, tensor parallelism over Thunderbolt RDMA scales while pipeline parallelism over TCP regresses. We release benchmark code, per-run results, and maintenance journals, supported by a workflow combining bounded agent fixes with human review.

View arXiv page View PDF Project pageGitHub 7 Add to collection

Community

windchimeran

Paper submitter about 21 hours ago

Benchmark desktop inference engines on Apple Silicon (9 engines) & DGX-Spark (3 engines), on Speed, Memory, and Fidelity

• Apple Silicon: vllm-metal, omlx, llama.cpp, ollama, mlx_lm, vllm-mlx, sglang, huggingface transformers, mistral.rs

• DGX-Spark: vllm, sglang, llama.cpp

librarian-bot

about 13 hours ago

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

• FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution (2026)

• OasisKV: Scaling In-Decode KV Cache Beyond HBM with Lookahead Sparse Prefetching (2026)

• Hydra: Phase-Aware Workload Characterization of LLM Inference across Edge SoC Generations, Backends, and Quantization Levels (2026)

• Budget-Aware Compression Pipeline for Single-GPU LLM Inference: Methods, Trade-offs, and Coupling Effects (2026)

• SSD-LLaMA: SSD-Native Inference for Trillion-Parameter MoE at 1+ Token/s on a Consumer PC (2026)

• GreenBench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source LLM Inference on Apple Silicon (2026)

• HiSparse: Scaling Sparse-Attention Decoding with Hierarchical KV Cache Management (2026)

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on HF Mirror checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment

Upvote

3

Get this paper in your agent:

hf papers read 2609.19169

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2609.19169 in a model README.md to link it from this page.

Datasets citing this paper 1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2609.19169 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.