KDSSDistributed StorageErasure coding
Profiles from 5+2 to 29+3, selected automatically for your cluster size. 29+3 delivers 90.6% usable capacity, and shards of a stripe never share a node.
Learn more →
Enterprise distributed storage
Petabyte-scale storage
Petabyte-scale storage
you can trust
Built on Reed-Solomon erasure coding and a raw-disk storage engine, KDSS stores more data on fewer drives. With 29+3, lose any three drives or nodes at once and every byte stays readable — then repairs itself automatically.
POSIX and S3 accessNon-disruptive rolling upgradesProven in long-running production
Swipe to see the full diagram
90.6%Usable capacity with EC 29+3, vs. 33% for 3-way replication
3Simultaneous drive or node failures tolerated (29+3), with data intact and readable
~190PBDeployed footprint (raw capacity)
11,000+Data drives across 300+ nodes, holding hundreds of millions of stripes
Core capabilities
Engineered for data that has to last
From encoding and on-disk layout to access and operations, every layer is built around three goals: fewer drives, data that stays intact through multiple failures, and simple operations.
Raw-disk engine
Writes straight to block devices, bypassing the local file system — sequential appends with CRC checks end to end. Freed space is reused by exact size and safely compacted.
Learn more →POSIX and S3
Mount the cluster like a local directory with the FUSE client, or use the S3 gateway, which supports the core S3 API and is verified with AWS CLI, boto3 and s3cmd.
Learn more →Self-healing operations
Four layers of drive health monitoring, automatic isolation and rebuild, and rolling upgrades without downtime. The console shows exactly how much you can still write.
Learn more →
Storage economics
Store 10 PB of data on less than half the drives
Three-way replication keeps three full copies. With 29+3 erasure coding, KDSS adds only about 10% parity overhead — and tolerates one more simultaneous failure.
- 90.6% usable capacity, vs. 33.3% for 3-way replication
- Survives 3 lost shards at once; replication survives 2
- Large 256 MB stripes keep metadata small and scaling simple
| For 10 PB of data | Raw capacity | Usable | Tolerates |
|---|---|---|---|
| 3-way replication | 30.0 PB | 33.3% | 2 |
| KDSS EC 9+2 | 12.2 PB | 81.8% | 2 |
| KDSS EC 18+3 | 11.7 PB | 85.7% | 3 |
| KDSS EC 29+3 | 11.0 PB | 90.6% | 3 |
Architecture
Metadata and data decoupled, every tier scales out
A group of Masters elects a leader through a MongoDB replica set and manages metadata. Storage nodes own their raw disks directly. Clients encode locally and write shards to different nodes in parallel.
Swipe to see the full diagram
Master cluster
Typically five nodes with automatic leader election and failover. Handles metadata, stripe allocation, scheduling of repair, rebalancing and garbage collection, and placement audits.
Storage nodes
Each node manages many raw disks and handles shard I/O, local rebuild and drive health monitoring. Can be co-located with Masters.
Access tier
The ksfs FUSE client and the S3 gateway. Writes are redirected to the leader automatically; every shard read is verified for identity and CRC.
Use cases
Built for write-once, keep-for-years data
Decentralized storage
Durable storage for sealed data on networks such as Filecoin — sequential writes of large sector files and steady reads for PoSt proofs.
Large-scale archive
Imaging, logs and raw research data kept for the long term, with high usable capacity driving cost per TB down.
Video and media assets
Sequential writes and range reads of large files, with S3 plugging directly into transcoding and distribution pipelines.
Backup and recovery
An S3 target for backup software, with a recycle bin and retention window that guard against accidental deletion.
Raw data lake
Centralize training corpora, sensor and experiment data; compute jobs read it directly through a POSIX mount.
Private cloud object storage
Runs in your own data center on standard servers, with capacity that grows as you need it.
Reliability and operations
Detect, isolate and repair — automatically
Detect early
SMART collection, kernel log scanning, a stuck-I/O watchdog and a sliding window of I/O errors surface failing drives before they cause trouble.
Isolate automatically
Faulty drives leave the read/write path without slowing the cluster, and return on their own once a transient fault clears.
Rebuild locally
Storage nodes fetch surviving shards and decode locally, repairing the most degraded stripes first; hundreds of thousands of stripes are queued in minutes.
Upgrade without downtime
By default, nodes enter maintenance mode and are replaced one at a time; at most parity shards − 1 nodes are ever in maintenance together. Upgrades are transparent to applications — no maintenance window required.
Swipe to see the full diagram
Let's talk about your storage plans
Tell us about your data volume, access patterns and data center environment. We'll recommend node counts, an erasure-coding profile and a hardware configuration.