Reproduce “Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference” on OpenResearch

About this paper

Kexin Chu, Zecheng Lin, Dawei Xiang, Zixu Shen, Jianchang Su, Cheng Chu, Yiwei Yang, Wenhui Zhang and 2 more

Kexin Chu et al. developed SafeKV, a system that selectively shares KV-cache blocks in multi-tenant LLM inference to mitigate timing side-channel attacks. It uses an asynchronous multi-tier privacy detection pipeline and a runtime safeguard, achieving over 94% defense success while improving throughput by up to 2.66 times and reducing Time-To-First-Token overhead by up to 40.58% compared to full cache isolation.

Key insights

• Asynchronous privacy classification allows for robust detection of sensitive content without introducing significant latency overheads on the critical inference path.
• Selective KV-cache sharing, guided by privacy labels, effectively balances performance optimization with strong privacy guarantees, outperforming strict per-user isolation.
• A dynamic runtime safeguard provides crucial adaptive protection against active probing attacks, enhancing robustness even when initial privacy classifications are imperfect.

Summary by alphaXiv.