moolib.Rpc

class moolib.Rpc

A class to execute Remote Procedure Calls.

The class represents one peer in a cohort, and allows you to define functions to be called by other peers, or to call functions that other peers have defined.

Peers are referred to by name (as opposed to a network address) though an address is required for an initial connection, and there is an built-in discovery service to allow peers to find each other, so long as there is connection path between them.

Example

Using the Rpc class is quite simple:

import moolib

def foo(str):
    print(str)
    return 42

host = moolib.Rpc()
host.set_name("host")
host.define("bar", foo)
host.listen("127.0.0.1:1234")

client = moolib.Rpc()
client.connect("127.0.0.1:1234")

future = client.async_("host", "bar", "hello world")
print(future.get())

The output would then be:

hello world
42
__init__()

Methods

__init__

async_

Make an asynchronous call to function_name and return a Future.

async_callback

Make an asynchronous call to function_name and execute a callback with the result.

connect

Connect to a network address.

debug_info

Print debugging info.

define

Define a function to be available for peers to call.

define_deferred

Define a deferred function to be available for peers to call.

define_queue

Define a queue to be available for peers to populate, when they call the function_name.

get_name

Return the name of the peer.

listen

Listen on a network address.

set_name

Set the unique name of the peer.

set_timeout

Set the timeout in secs for each remote call.

set_transports

Set which transports are available for the rpc to communicate over.

sync

Make an synchronous call to function_name and return the result.
async_()

Make an asynchronous call to function_name and return a Future.

Note

If you send a Tensor as an argument, you may not modify the Tensor data until the result is returned.

Parameters

  • name (str) – the name of the peer.

  • function_name (str) – the name of the remote function to call on the peer.

  • args (Optional) – the args to call with the remote function.

  • kwargs (Optional) – the kwargs to call with the remote function.

Returns

a future result of the async call.

Return type

Future

async_callback()

Make an asynchronous call to function_name and execute a callback with the result.

Note

If you send a Tensor as an argument, you may not modify the Tensor data until the result is returned.

Parameters

  • name (str) – the name of the peer

  • function_name (str) – the name of the remote function to call on the peer.

  • callback (Callable) – a callback that accepts the result of the Future.

  • args (Optional) – the args to call with the remote function.

  • kwargs (Optional) – the kwargs to call with the remote function.

connect()

Connect to a network address.

This is a non-blocking call, and moolib will maintain this connection by regularly reconnecting if the connection is lost.

Parameters

address (str) – address to connect to (eg: "127.0.0.1:1234").

debug_info()

Print debugging info.

define()

Define a function to be available for peers to call.

This function will be run asynchronously, called from an internal thread. The maximum number of concurrent functions that can be run is limited by the maximum number of threads (see moolib.set_max_threads)

This function accepts two keyword arguments:
  • batch_size (int)

    any tensors in any arguments of calls to this function will be batched to this batch_size before remote execution, and split back on return.

  • device (str)

    any tensors in any arguments of calls to this function will be batched on this device before remote execution, and returns to the original device.

Note

It is best practice to call define methods before connecting to the network, to avoid race conditions.

Parameters

  • name (str) – a unique name for the function, for peers to use.

  • function (Callable[[], None]) – a Python function, for peers to call.

  • kwargs – specifications for how to treat Tensor arguments to the function.

define_deferred()

Define a deferred function to be available for peers to call.

The deferred function must accept a callback provided by moolib as its first argument. This defined function will not return until this callback is called, and the function returns the argument to the callback.

see define() for keyword arguments and Notes

Parameters

  • name (str) – a unique name for the function, for peers to use.

  • callback (Callable[[], None]) – a Python function, for peers to call.

  • kwargs – see define() for keyword arguments.

define_queue()

Define a queue to be available for peers to populate, when they call the function_name.

This queue will be populated on each “function call” by peers, and will be populated with a moolib provided callback and the arguments to the function call. The “function call” will be not return until the callback is called, and returns the arguments to the callback.

see define() for keyword arguments and Notes

Example

Usage:

# host.py
# ... setup host ...
queue = host.define_queue("foo")

while True:
    cb, arg1, arg2 = queue.result()
    print(arg1, arg2)
    cb(42) # returns 42 to client

# client.py
# ... setup client ...
print(host.sync("foo", arg1, arg2))  # eventually prints 42
Parameters

  • name (str) – a unique name for the function, for peers to use.

  • kwargs – see define() for keyword arguments.

get_name()

Return the name of the peer.

The name is a unique identifier of the peer within the current cohort of connected peers. In the default case this is randomly generated string which is guaranteed to be globally unique, or it can be actively set by calling set_name.

Returns

the name of the peer (as set by set_name or default).

Return type

str

listen()

Listen on a network address.

This is a non-blocking call.

Parameters

address (str) – address to listen to (eg: "127.0.0.1:1234").

set_name()

Set the unique name of the peer.

The name is a unique identifier of the peer within the current cohort of connected peers. The name chosen here must not exist among other peers in the cohort, and this function must be called before attempting to join the network (ie any listen or connect calls).

Parameters

name (str) – the name of the peer.

set_timeout()

Set the timeout in secs for each remote call.

Parameters

timeout (float) – the number of seconds before timeout.

set_transports()

Set which transports are available for the rpc to communicate over.

Moolib has an internal bandit that automatically selects the transport to use at any particular moment. This bandit tries to optimize for latency (and thus indirectly for throughput). Peers internally communicate their available transports and network addresses, and connections will be attempted with all available transports.

Parameters

transports (List[str]) – a list of transports to use ["tcp/ip", "shared memory", "infiniband"].

sync()

Make an synchronous call to function_name and return the result.

Parameters

  • peer_name (str) – a name.

  • function_name (Callable[[], None]) – the name of a callback.

  • args – the arguments to call with the callback.

  • kwargs – the arguments to call with the callback.