Neki – Sharded Postgres

summary

PlanetScale is launching Neki, a sharded Postgres solution designed to help databases scale beyond the limits of a single machine. It uses a router-based architecture that maintains real Postgres instances on every shard to ensure full compatibility with extensions and SQL support. The system allows users to start with a single instance and transition to a sharded environment through automated workflows. This approach avoids the pitfalls of application-level sharding or proprietary distributed databases.

key points

1. Neki employs a router that speaks the Postgres wire protocol, allowing existing drivers and ORMs to function without code changes while handling query planning and result aggregation.
2. The architecture utilizes a JSON data topology to map logical tables to physical shards, where each shard is a standard Postgres cluster with a primary and multiple replicas across availability zones.
3. Administrative tasks such as schema changes, version upgrades, and resharding are handled as online workflows, eliminating the need for maintenance windows and downtime.

who should read it

Database administrators and software engineers managing high-growth Postgres workloads that have reached the hardware limits of a single server. It is also suitable for teams seeking a scalable database without sacrificing Postgres compatibility.

discussion skeptical