S3 Is the Future, S3 Is the Past

summary

Amazon S3 has become the bedrock of modern cloud software, yet its architecture is built around the physical limitations of mechanical hard disks, requiring complex caching, batching, and metadata workarounds. As SSD performance and datacenter network speeds have drastically improved, these architectural taxes are increasingly unnecessary. Although cloud offerings like S3 Express One Zone fall short due to high costs and limitations, the obstacle to true SSD-backed disaggregated storage is corporate inertia rather than technology. Commenters debate the true cost-effectiveness of S3, noting stagnant price drops, alternative zero-egress providers like Cloudflare R2, and the ongoing difficulty of replacing S3's sheer convenience.

key points

1. S3's design enforces an architectural tax—requiring caching layers, batching, and separate metadata stores—because it emulates a million hard disks behind an HTTP API.
2. Modern datacenter networks and SSDs offer microsecond latency and high IOPS, eliminating the physical constraints that originally necessitated S3-style design patterns.
3. AWS's S3 Express One Zone fails to serve as a general-purpose replacement due to single-AZ limitations, steep bandwidth pricing, and high per-gigabyte costs.

who should read it

Cloud architects, backend engineers, and systems designers working with large-scale data systems.

discussion positive