Alan Kay: Shannon gave us a way of dealing with noisy channels [video]
A short video clip of Alan Kay discussing information theory prompts a technical Hacker News debate on Shannon limits and error-correcting codes.
summary is enough The video clip is brief, but the comment thread provides a concise, high-level debate on information theory and error correction.
summary
An excerpted video features Alan Kay talking about Claude Shannon's foundational work on noisy channels. The HN comment thread expands on this, clarifying the distinction between theoretical limits and practical implementation through codes like LDPC and Turbo coding. Commenters also share links to the full symposium talk and drift into discussions of alternative object-oriented models.
key points
- Shannon's theorem mathematically defines channel capacity limits rather than prescribing efficient, practical encoding algorithms.
- Modern error-correction algorithms like LDPC and Turbo codes are what actually allow transmissions to approach the theoretical Shannon limit.
- The comment thread surfaces links to the full Nygaard celebration symposium where Kay's talk was recorded.
who should read it
Engineers and computer scientists interested in information theory, communications, and computing history.
discussion positive
consensus
Commenters agree that Shannon established fundamental theoretical limits, while practical engineering solved the heavy computation required to approach them.
pushback
A minority debate focuses on whether Shannon's original proofs implicitly contained the optimal random coding strategy despite its initial computational infeasibility.
If I recall correctly, he invented the optimal way to do it and published it in the original paper: use an arbitrarily long random code. The difficulty is that decoding a random code in the obvious way (compare the received codeword against each codeword in the codebook and decode as the one with the lowest Hamming distance) requires an exponentially large amount of both memory and computation. As I understand it, the advances since then have all been about how to get closer to the Shannon limit with reasonable amounts of computation by using codewords that aren't truly random.
It details how Shannon's original random code concept was theoretically optimal but computationally intractable, framing decades of subsequent research.
Generated by an LLM from the article and thread. Check the sources before relying on it.