Homa: The end of TCP for AI clusters [video]
John Ousterhouts Homa transport protocol replaces TCP with receiver-driven congestion control for RPC traffic in AI clusters, sparking technical debate over its architecture.
summary is enough The video link lacks a transcript, but the comment thread delivers a thorough technical breakdown of Homa and valid architectural critiques.
The article could not be read automatically; this analysis is based on the title and discussion.
summary
Homa is a transport protocol designed by John Ousterhout to bypass TCP bottlenecks in AI clusters by using receiver-driven congestion control for remote procedure calls. It sends an initial block of data blindly before requiring explicit grant packets from the receiver for the rest. While supporters praise its fit for predictable datacenter workloads, critics highlight flaws like silent RPC loss, lack of encryption, and poor multi-threaded throughput.
key points
- Homa splits messages into unscheduled initial bytes followed by scheduled bytes requested via receiver GRANT packets.
- Supporters note that modern hardware switches minimize small-packet overhead, making receiver-driven flow control efficient for predictable workloads.
- Critics highlight architectural weaknesses, including vulnerability to silent total RPC loss, missing built-in encryption, and low per-thread throughput.
who should read it
Systems engineers and network architects working on high-performance datacenter infrastructure and AI cluster networking.
discussion mixed
consensus
Commenters agree that TCP is increasingly a bottleneck for AI clusters, but they remain deeply split on whether Homa is an optimal replacement.
pushback
A prominent critic argues that Homa has poor benchmark performance, lacks encryption, and fails to handle complete RPC packet loss cleanly.
Homa is not a good design. [1]
1. No way to detect whole RPC loss. Since there is no outer connection state, if every packet in the send-side of a RPC is lost then there is no way for a server to detect that it should issue a resend. The RPC is just lost to the ether. This affects small messages, like messages that fit in a single packet, more since there are fewer packets in the send-side.
2. Related to the above, there is no builtin encryption support. So, if you want encryption then you need to layer it either above or below.
3. Benchmarked performance is awful. The 60 kB average …
It outlines four specific technical and performance criticisms of Homa's protocol design.
Generated by an LLM from the article and thread. Check the sources before relying on it.