ArkFlow

lastmod 2025-03-18created 2025-03-181 edits

High-performance Rust stream processing engine, providing powerful data stream processing capabilities, supporting multiple input/output sources and processors.

A high-performance Rust stream processing engine

Features

• Built on Rust and the Tokio asynchronous runtime, providing excellent performance and low latency. Can process large-scale data streams quickly
• Supports various data sources: Kafka, MQTT, HTTP, files, and other input/output sources
• Real-time data filtering and transformation: built-in SQL queries, JSON and Protobuf encoding/decoding support, and batch processing
• Modular design makes it easy to extend with new input, output, and processing components
  • Supports custom data processing by adding user-defined processors
• Provides high reliability through an asynchronous processing architecture
  • Automatic recovery and reprocessing when errors occur
• Easy to configure and set up using YAML configuration files
• Can be tested and debugged quickly locally

See also

• rust
• Tokio

Favorite site

• Github - chenquan/arkflow