Skip to main content
← Back to news

KIWI: sub-microsecond GPU-to-CPU invocation, up to 2.3x faster expert-parallel communication

Research
The KIWI paper (arXiv) flips the device-initiated communication hierarchy: reverse offload on host threads, exposed as a programmable interface, beats GPUDirect backends — up to 2.3x lower expert-parallel MoE communication latency on AMD GPUs and -29% time per token in vLLM.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.