VRS
A file format for sensor data.
VRS Documentation

Overview

TL;DR: VRS is a file format optimized to record & playback streams of sensor data, such as images, audio samples, and any other discrete sensors (IMU, temperature, etc), stored in per-device streams of timestamped records.

If your data is not spread over time, with some kind of "packets" coming from one or multiple sources, with some level of format regularity, like camera frames, successive samples coming from sensors, or timestamps don't make sense for your use case, then VRS is probably not for you.

The VRS File Format in Short

  • VRS files contain multiple streams of time-sorted records generated by a set of "devices", typically one per stream.
  • File and streams contain a set of "tags", which are a set of string name-value pairs, to describe them.
  • Streams may contain Configuration, State and Data records, each with a timestamp in a common time domain for the whole file.
    Typically, streams contain one Configuration and one State record followed by one to millions of Data records.
  • Records are structured as a succession of typed content blocks.
    Typical content blocks are metadata, image, audio and custom content blocks.
  • Metadata content blocks contain raw sensor data described once per stream, making the file format very efficient. The marginal cost of adding 1 byte of data to each metadata content block of a stream is 1 byte per record (or less, when lossless compression happens).
  • Records can be losslessly compressed using lz4 or zstd, which can be fast enough to do realtime compression while recording.
  • Multiple threads can create records concurrently for the same file.
  • VRS supports huge file size (tested with multi terabytes use cases).
  • VRS supports chunked files: auto-chunking on creation, automated chunk detection for playback.
  • Playback is optimized for timestamp order (key for network streaming).
  • Random-access playback is fully supported (in memory file and stream indexes).
  • Customizable FileHandler support, to implement streaming from cloud storage (not provided yet).
This VRS API documentation was generated from the VRS source code.