LiteFS

lastmod 2023-03-04created 2022-09-296 edits

An open-source distributed file system for SQLite

Features

• A new project from Fly io, which is going all in on full-stack use of SQLite
• Extends the idea of Litestream through fine-grained transaction control
  • Litestream simply replicates the WAL file, while LiteFS inspects and applies each transaction individually
• There are two ways to intercept SQLite's file system API: using the VFS abstraction vs. writing a FUSE file system
  • LiteFS chose the harder approach of creating a FUSE file system
  • VFS has a preliminary step before it can be used, so anyone who wants to use the DB has to load the VFS first
  • FUSE solves most of these usability problems
• Leases the cluster's primary node to increase availability (using Hashicorp's Consul)
• Uses distributed replication
• Upcoming features
  • WAL mode support
  • Write Forwading
  • S3 Replication
  • Encryption

See also

• filesystem
• SQLite
• Distributed file system (DFS)
• HDFS

Favorite site

• superfly/litefs: FUSE-based file system for replicating SQLite databases across a cluster of machines
• Introducing LiteFS · Fly
• Introducing LiteFS - an open-source distributed file system for SQLite (fly.io)