Skip to content
Search

TinyKVM

lastmod 2025-03-18created 2025-03-181 edits

VMM for native-performance sandboxing.

A fast sandbox that runs on top of Varnish.

About

  • A KVM-based single-process sandbox
    • Can run ordinary Linux programs or programs that use a specific API in the sandbox
  • Uses hardware virtualization to deliver native performance
  • Uses only part of the KVM API → small and efficient codebase

Design of TinyKVM

  • Supports running static Linux ELF programs
    • Support for dynamic executables planned for later
    • Can also be extended with APIs to provide access to external HTTP servers or caches
    • Currently works on AMD64 (x86_64), with an AArch64 (64-bit ARM) port planned
  • Hugepage support
    • Can create hugepages for guest pages
    • Hugepages can also be used on the host → performance improvement
    • Example: 5% faster LLVM compilation observed when allocating 2MB pages
  • Fast function calls
    • Overhead of a function call into the guest is 2μs
    • When running without a timer, overhead drops to 1.2μs
  • Remote debugging support
    • Remote debugging with GDB
    • The program can resume normally after debugging
  • Copy-on-Write support
    • Supports its own fork feature → minimizes memory duplication
    • Example: cloning a 6GB model needs only 260MB of memory per instance
  • Fast state reset
    • Guest state can be reset quickly → improved security
    • Resetting on every request reduces the risk of state exposure
  • Simplified codebase
    • Uses about 42k LOC in the KVM API
    • TinyKVM's own codebase is about 9k LOC → much smaller than competing solutions
    • Example: Wasmtime 350k LOC, FireCracker 165k LOC
  • Static page table creation
    • Page tables cannot be modified at runtime → improved security
    • Performs page table integrity checks
  • Separate process context
    • KVM guests use a separate PCID/ASID → resistant to speculative execution attacks such as Spectre
  • Hardened kernel
    • SMEP and SMAP enabled
    • CPU exceptions can be handled in user mode

System call handling

  • API connection with the host
    • System calls are made via SYSCALL/SYSRET or the OUT instruction
    • A system call causes a VM exit → takes about 1μs
    • Recommended to design APIs with fewer small calls and larger I/O units

Benchmarks

  • VM call overhead
    • Tail latency measured with VM reset
    • Overhead is low for simple calls without reset
  • Memory performance
    • Memory performance is at normal levels
    • Example: 1500 AVIF encodes per second in an HTTP benchmark
  • JPEG → AVIF conversion performance
    • Converts about 1582 images per second
    • Lossless conversion possible using the YUV conversion path

Why the sandbox is fast

  • No I/O or drivers
    • No I/O, drivers, or virtual devices → avoids performance degradation
    • Uses only CPU resources → close to native speed
  • Hugepage optimization
    • Using hugepages reduces page walks → performance improvement
    • Achieves 99.7% of native performance on large LLM workloads
  • Fast VM calls
    • Minimal overhead for function calls into the guest
    • Data can be processed at native CPU speed

Limitations

  • Cannot reduce the number of vCPUs
    • The KVM API does not allow reducing the number of vCPUs
    • Multiprocessing can be handled by running multiple VMs in parallel
  • Reset performance degradation
    • Resetting VM state may degrade performance
    • However, this can be solved through state sharing and cloning

Future work

  • Add Intel TDX and AMD SEV support
  • AArch64 port
  • Add memory locking (KVM_MEM_READONLY) → improved security
  • Improve the API to be more user-friendly
  • Add dynamic link loading support → stronger integration with Varnish

Conclusion

  • TinyKVM is one of the smallest and fastest sandbox solutions
  • Achieves both stronger security and optimized performance
  • Small codebase makes maintenance easy
  • Available as an open-source library → check the code repository if interested

See also

Favorite site

author Zer0 sitemap