Skip to content
Search

RocksDB

lastmod 2025-08-12created 2019-05-294 edits

A library that provides an embeddable, persistent key-value store for fast storage.

How we replaced Elasticsearch and MongoDB with Rust and RocksDB

  • How we replaced Elasticsearch and MongoDB with Rust and RocksDB | GeekNews
  • Radar provides geospatial infrastructure handling over 1 billion API requests per day, and to solve performance and scalability problems it moved from its existing Elasticsearch and MongoDB to HorizonDB, developed in-house
  • HorizonDB is a high-performance geospatial database written in Rust that combines various open-source tools such as RocksDB, S2, Tantivy, FST, LightGBM, and FastText
  • In the previous architecture, the scaling cost and complexity of Elasticsearch and MongoDB were high, making operations difficult
  • HorizonDB runs as a single multithreaded process, achieving lower cost, better performance, and high reliability
  • Overall, development productivity and operational efficiency improved greatly, enabling new data and features to be applied quickly
  • Data is preprocessed with Apache Spark and then stored by version in AWS S3, and developers can easily run and test it locally
  • As a result, the Mongo and Elasticsearch clusters were shut down, cutting costs significantly while improving feature development speed and data processing efficiency

Introduction and background

  • Radar is a geolocation infrastructure platform that handles over 1 billion API calls per day from hundreds of millions of devices worldwide
    • Main APIs: Geocoding, Search, Routing, Geolocation compliance, etc.
  • As data scale and products grew, solving high-performance, scalability, and cost problems became urgent
  • To this end, it introduced HorizonDB, written in Rust, providing various location service features from a single high-performance binary
    • Handles 1,000 QPS per core
    • Median latency of 50ms for forward geocoding, <1ms for reverse geocoding
    • Scales linearly on commodity hardware

Limitations of the previous system

  • Previous architecture: forward geocoding handled by Elasticsearch, reverse by MongoDB
  • Problems:
    • Elasticsearch distributes queries across all shards and requires periodic batch updates
    • MongoDB struggles with large batch ingestion, requires excessive resource allocation, and lacks reliable rollback

HorizonDB architecture goals

  • Efficiency - runs on commodity hardware, predictable autoscaling, serves as the single source of data for all geo entities
  • Operability - build and process data assets multiple times a day, easy changes and rollbacks, simplified operations
  • Developer experience - can run in a local environment, easy to change and test

Technology stack used

Uses several open-source projects such as RocksDB, S2, Tantivy, FSTs, LightGBM, and FastText; data is preprocessed with Apache Spark, then stored from Rust as versioned files in S3

  • Rust
    • A systems programming language developed by Mozilla
    • Guarantees compile-time and memory safety, enabling predictable memory management of large indexes without garbage collection
    • Supports high-level abstractions such as null handling and pattern matching, making complex search ranking logic easy to express
    • Optimized for handling hundreds of GB of data on SSD in a single multithreaded process
  • RocksDB
    • High-performance in-process storage based on LSM trees
    • Microsecond-level responses, stable speed even with large data
  • S2
    • Google's spatial indexing library, which divides the Earth into quadrants to speed up point-in-polygon lookups
    • Radar developed its own Rust bindings for the C++ S2 library, to be open-sourced soon
  • FSTs (Finite State Transducers)
    • A data structure for efficient string compression and prefix search
    • Reflecting that 80% of queries follow a regular "happy path", millions of paths can be cached in a few MB of memory
  • Tantivy
    • An in-process inverted index library similar to Lucene
    • Reasons for adopting it instead of an external service like the previous Elasticsearch:
      • Search quality - handles advanced search processing such as dynamic keyword expansion without UML communication latency
      • Simplified operations - processing within a single process; even large indexes can scale easily via memory mapping
  • FastText
    • Uses FastText models trained on its own corpus and logs to generate vector representations of words, used in ML applications
    • Robust to typos and out-of-vocabulary words, and can understand search meaning by using the semantic similarity of neighboring vectors
  • LightGBM
    • Uses multiple LightGBM models for query intent classification, attribute tagging within queries, etc.
    • E.g., region queries such as "New York" skip address search, while cases like "841 Broadway" skip POI/region search
  • Apache Spark
    • Processes hundreds of millions of data points quickly in under an hour; jobs are continuously improved for better join/aggregation performance
    • Final data is stored in S3, where results can be explored with SQL using Amazon Athena, DuckDB, etc.

Results of adopting HorizonDB

  • Services became much faster, operations simpler, and reliability improved
  • The development team can apply and evaluate new features and data sources within a day
  • Shutting down large clusters such as Mongo and Elasticsearch, plus several microservices, saves tens of thousands of dollars per month
  • Radar is now ready to handle large-scale growth. The design process of specific features will be covered in future blog posts

See also

Favorite site

author Zer0 sitemap