Running Synchronous Functions¶
The asyncio
class has helper functions/class methods that allow to
execute a synchronous function in the async context.
asyncio.wrap_future()
- Wrap aconcurrent.futures.Future
object in aasyncio.Future
object.asyncio.loop.run_in_executor()
- Schedule the provided function to be executed in an executor, and returns anasyncio.Future
object.asyncio.to_thread()
- A wrapper aroundrun_in_executor
. Propagates the context variable.
This makes it possible to mix the execution of sync/async functions smoothly. The event loop can be regarded as a manager for dispatching synchronous execution to thread/process pool executor.
Using run_in_executor
to run synchronous function in Async I/O¶
SPDL uses asyncio.loop.run_in_executor()
method extensively.
See also
Pipeline Parallelism - Explains SPDL’s high-level API for switching executor.
The following example shows how to convert a synchronous function into asynchronous function.
executor = ThreadPoolExecutor() # or ProcessPoolExecutor
async def async_wrapper(input):
loop = asyncio.get_running_loop()
return await loop.run_in_executor(executor, sync_func, input)
If using multi-threading, you can also use the default ThreadPoolExecutor
attached to
the event loop †.
† The maximum concurrency of the default executor.
The maximum concurrency of the default executor is as follow
min(32, os.cpu_count() + 4)
(before Python 3.13)min(32, (os.process_cpu_count() or 1) + 4)
(since Python 3.13)
These values are intended for I/O tasks, and it is rather high for data loading, which might involve CPU tasks like media processing.
You can change this by explicitly setting the default executor with
asyncio.loop.set_default_executor()
function.
See also
Noisy Neighbour explains why it is important to always keep CPU utilization low.
The difference of multi-threading and multi-processing¶
There are difference between multi-threading and multi-processing. The following table summarizes them.
|
|
|
---|---|---|
Pros |
|
|
Cons |
|
|
To achieve high performance in multi-threading, we need to workaround the GIL.
SPDL provides spdl.io
module, which offers efficient media processing while releasing the GIL.
It complements the numerical computation like NumPy and PyTorch, so many AI applications are covered.