spdl.io.Packets

class Packets[source]

Packets object. Returned from demux functions.

Packets objects represent the result of the demuxing. (Internally, it holds a series of FFmpeg’s AVPacket objects.)

Decode functions receive Packets objects and generate audio samples and visual frames. The Packets objects are exposed to public API to allow composing demux/decoding functions in ways that they are executed concurrently.

For example, when decoding multiple clips from a single audio and video file, by emitting Packets objects in between, decoding can be started while the demuxer is demuxing the subsequent windows.

The following code will kick-off the decoding job as soon as the streaming demux function yields a VideoPackets object.

Example

>>> src = "sample.mp4"
>>> windows = [
...     (3, 5),
...     (7, 9),
...     (13, 15),
... ]
>>>
>>> tasks = []
>>> async for packets in spdl.io.async_streaming_demux_video(src, windows):
>>>     # Kick off decoding job while demux function is demuxing the next window
>>>     task = asyncio.create_task(decode_packets(packets))
>>>     task.append(task)
>>>
>>> # Wait for all the decoding to be complete
>>> asyncio.wait(tasks)

Important

About the Lifetime of Packets Object

When packets objects are passed to a decode function, its ownership is also passed to the function. Therefore, accessing the packets object after it is passed to decode function will cause an error.

>>> # Demux an image
>>> packets = spdl.io.demux_image("foo.png").result()
>>> packets  # this works.
ImagePackets<src="foo.png", pixel_format="rgb24", bit_rate=0, bits_per_sample=0, codec="png", width=320, height=240>
>>>
>>> # Decode the packets
>>> frames = spdl.io.decode_packets(packets)
>>> frames
FFmpegImageFrames<pixel_format="rgb24", num_planes=1, width=320, height=240>
>>>
>>> # The packets object is no longer valid.
>>> packets
RuntimeWarning: nanobind: attempted to access an uninitialized instance of type 'spdl.lib._spdl_ffmpeg6.ImagePackets'!

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __repr__(): incompatible function arguments. The following argument types are supported:
    1. __repr__(self) -> str

Invoked with types: spdl.lib._spdl_ffmpeg6.ImagePackets
>>>

This design choice was made for two reasons;

  1. Decoding is performed in background thread, potentially long after since the job was created due to other decoding jobs. To ensure the existance of the packets, the decoding function should take the ownership of the packets, instead of a reference.

  2. An alternative approch to 1 is to share the ownership, however, in this approach, it is not certain when the Python variable holding the shared ownership of the Packets object is deleted. Python might keep the reference for a long time, or the garbage collection might kick-in when the execution is in critical code path. By passing the ownership to decoding function, the Packets object resouce is also released in background.

To decode packets multiple times, use the clone method.

>>> packets = spdl.io.demux_image("foo.png").result()
>>> # Decode the cloned packets
>>> packets2 = packets.clone()
>>> packets2
ImagePackets<src="foo.png", pixel_format="rgb24", bit_rate=0, bits_per_sample=0, codec="png", width=320, height=240>
>>> frames = spdl.io.decode_packets(packets)
>>>
>>> # The original packets object is still valid
>>> packets
ImagePackets<src="foo.png", pixel_format="rgb24", bit_rate=0, bits_per_sample=0, codec="png", width=320, height=240>

Note on clone method

The underlying FFmpeg implementation employs reference counting for AVPacket object.

Therefore, even though the method is called clone, this method does not copy the frame data.