Skip to main content

Subscribe to Aria Streaming Data

Overview​

This streaming_subscribe example shows how to subscribe to and unsubscribe from a streaming session as well as visualize the live stream, using the Project Aria Client SDK.

Stream and subscribe examples​

Example 1: Stream and subscribe over USB​

In this example, the CLI is used to initiate streaming and the Client SDK is used to subscribe to the stream. To find out how to start streaming using the SDK, go to Streaming Sensor Data.

  1. Plug your Aria glasses into your computer, using the provided cable
  2. From the samples directory in Terminal, run:
aria streaming start --interface usb --use-ephemeral-certs
  1. Wait for the stream to start then run:
python -m streaming_subscribe

You should then see:

Aria Live Stream Window
  1. There are several ways you can stop streaming:
  • Press q or ESC to quit the app
  • Use Ctrl-C to terminate in terminal
  • Press the Capture button on your glasses

Example 2: Using Wi-Fi​

To use Wi-Fi to initiate streaming or to stream data, alter the aria streaming start --interface usb --use-ephemeral-certs command.

  • To stream data over Wi-Fi, use --interface wifi
  • To initiate streaming over Wi-Fi, add --device-ip <glasses IP>

Code walkthrough​

1. Configure the subscription​

Use subscriber_data_type attribute to set the type of data the client subscribes to.

config = streaming_client.subscription_config
config.subscriber_data_type = (
    aria.StreamingDataType.Rgb | aria.StreamingDataType.Slam
)

2. Set message queue size​

The message queue size determines how many recent frames will be retained. A smaller queue size is utilized to process only the most recent data.

config.message_queue_size[aria.StreamingDataType.Rgb] = 1
config.message_queue_size[aria.StreamingDataType.Slam] = 1

3. Set streaming security options​

Security options are set to use ephemeral certificates through a StreamingSecurityOptions instance. Go to the Streaming Internals page for various aspects of streaming security and how certificates are used.

options = aria.StreamingSecurityOptions()
options.use_ephemeral_certs = True
config.security_options = options

4. Create an StreamingClient observer and attach it​

Find more description of observer in the streaming code sample

class StreamingClientObserver:
    def __init__(self):
        self.images = {}

    def on_image_received(self, image: np.array, record: ImageDataRecord):
        self.images[record.camera_id] = image

observer = StreamingClientObserver()
streaming_client.set_streaming_client_observer(observer)

5. Start subscribing and listen to the live stream​

The client begins listening for incoming streaming data from the subscribed data types.

streaming_client.subscribe()

6. Visualize the live stream​

  • RGB and SLAM images are visualized in separate windows using OpenCV. The images are processed and displayed the streaming stops or the application quit. We rotate the image and stack the SLAM images so that they are shown in a single window.
while not quit_keypress():
    # Render the RGB image
    if aria.CameraId.Rgb in observer.images:
        rgb_image = np.rot90(observer.images[aria.CameraId.Rgb], -1)
        rgb_image = cv2.cvtColor(rgb_image, cv2.COLOR_BGR2RGB)
        cv2.imshow(rgb_window, rgb_image)
        del observer.images[aria.CameraId.Rgb]

    # Stack and display the SLAM images
    if (
        aria.CameraId.Slam1 in observer.images
        and aria.CameraId.Slam2 in observer.images
    ):
        slam1_image = np.rot90(observer.images[aria.CameraId.Slam1], -1)
        slam2_image = np.rot90(observer.images[aria.CameraId.Slam2], -1)
        cv2.imshow(slam_window, np.hstack((slam1_image, slam2_image)))
        del observer.images[aria.CameraId.Slam1]
        del observer.images[aria.CameraId.Slam2]
note

Cameras on Aria glasses are installed sideways. The visualizer rotates the raw image data for a more natural view.

7. Unsubscribe from the stream and free resources​

  • Unsubscribing stops the client from listening to streaming data and cleans up resources.
streaming_client.unsubscribe()