F3
lastmod 2025-10-06created 2025-10-061 edits
An open-source data file format for the future
About
- F3 (Future-proof File Format) is a next-generation open-source columnar storage format built on the core design principles of interoperability, extensibility, and efficiency, removing the need to create a new format every time data processing and computing environments change
- Each F3 file has a self-describing structure that embeds not only data and metadata but also a WebAssembly (Wasm) binary decoder, guaranteeing compatibility on every platform even without a native decoder
- Provides an efficient storage layout including modern encoding techniques (cascading compression, vectorized decoding), and improves on the layout problems of Parquet and ORC by separating the I/O, encoding, and dictionary units
- A plugin-based decoding API and a Wasm embedding mechanism let developers easily add new encoding schemes, and every file can be read regardless of library version, solving Parquet's interoperability problems
- Evaluation demonstrated the efficiency of F3's storage layout and the benefits of Wasm-based decoding; storage overhead is negligible at the kilobyte level, and the Wasm decoding slowdown is an acceptable 10–30% compared to native