Parametrization API reference

Please note that parametrization is still a work in progress and changes are on their way (including for this documentation)! We are trying to update it to make it simpler and simpler to use (all feedbacks are welcome ;) ), with the side effect that there will be breaking changes.

The aim of parametrization is to specify what are the parameters that the optimization should be performed upon. The parametrization subpackage will help you do thanks to:

the parameter modules (accessed by the shortcut nevergrad.p) providing classes that should be used to specify each parameter.
the ops module (accessed through ng.ops) providing experimental objects for modifying a parameter behavior (eg: casting to int, adding complex constraints).
the FolderFunction which helps transform any code into a Python function in a few lines. This can be especially helpful to optimize parameters in non-Python 3.6+ code (C++, Octave, etc…) or parameters in scripts.

Parameters

Here are the current types of parameters currently provided:

`nevergrad.p.Array`(*[, init, shape, lower, ...])
`nevergrad.p.Scalar`([init, lower, upper, ...])	Parameter representing a scalar.
`nevergrad.p.Log`(*[, init, exponent, lower, ...])	Parameter representing a positive variable, mutated by Gaussian mutation in log-scale.
`nevergrad.p.Dict`(**parameters)	Dictionary-valued parameter.
`nevergrad.p.Tuple`(*parameters)	Tuple-valued parameter.
`nevergrad.p.Instrumentation`(args, *kwargs)	Container of parameters available through args and kwargs attributes.
`nevergrad.p.Choice`(choices[, repetitions, ...])	Unordered categorical parameter, randomly choosing one of the provided choice options as a value.
`nevergrad.p.TransitionChoice`(choices[, ...])	Categorical parameter, choosing one of the provided choice options as a value, with continuous transitions.

class nevergrad.p.Array(*, init: Optional[Union[Tuple[float, ...], List[float], ndarray]] = None, shape: Optional[Tuple[int, ...]] = None, lower: Optional[Union[float, int, int64, float64, Tuple[float, ...], List[float], ndarray]] = None, upper: Optional[Union[float, int, int64, float64, Tuple[float, ...], List[float], ndarray]] = None, mutable_sigma: bool = False)

Bases: Data

value: ValueProperty[Union[Tuple[float, ...], List[float], ndarray], ndarray]

Value of the Parameter, which should be sent to the function to optimize.

Example

>>> ng.p.Array(shape=(2,)).value
array([0., 0.])

class nevergrad.p.Choice(choices: Union[int, Iterable[Any]], repetitions: Optional[int] = None, deterministic: bool = False)

Bases: BaseChoice

Unordered categorical parameter, randomly choosing one of the provided choice options as a value. The choices can be Parameters, in which case there value will be returned instead. The chosen parameter is drawn randomly from the softmax of weights which are updated during the optimization.

Parameters

choices (list or int) – a list of possible values or Parameters for the variable (or an integer as a shortcut for range(num))
repetitions (None or int) – set to an integer n if you want n similar choices sampled independently (each with its own distribution) This is equivalent to Tuple(*[Choice(options) for _ in range(n)]) but can be 30x faster for large n.
deterministic (bool) – whether to always draw the most likely choice (hence avoiding the stochastic behavior, but loosing continuity)

Note

Since the chosen value is drawn randomly, the use of this variable makes deterministic functions become stochastic, hence “adding noise”
the “mutate” method only mutates the weights and the chosen Parameter (if it is not constant), leaving others untouched

Examples

>>> print(Choice(["a", "b", "c", "e"]).value)
"c"

>>> print(Choice(["a", "b", "c", "e"], repetitions=3).value)
("b", "b", "c")

class nevergrad.p.Dict(**parameters: Any)

Bases: Container

Dictionary-valued parameter. This Parameter can contain other Parameters, its value is a dict, with keys the ones provided as input, and corresponding values are either directly the provided values if they are not Parameter instances, or the value of those Parameters. It also implements a getter to access the Parameters directly if need be.

Parameters: **parameters (Any) – the objects or Parameter which will provide values for the dict

Note

This is the base structure for all container Parameters, and it is used to hold the internal/model parameters for all Parameter classes.

value: ValueProperty[Dict[str, Any], Dict[str, Any]]

Value of the Parameter, which should be sent to the function to optimize.

Example

>>> ng.p.Array(shape=(2,)).value
array([0., 0.])

class nevergrad.p.Instrumentation(*args: Any, **kwargs: Any)

Bases: Tuple

Container of parameters available through args and kwargs attributes. The parameter provided as input are used to provide values for an arg tuple and a kwargs dict. value attribue returns (args, kwargs), but each can be independantly accessed through the args and kwargs properties.

Parameters

*args – values or Parameters to be used to fill the tuple of args
*kwargs – values or Parameters to be used to fill the dict of kwargs

Note

When used in conjonction with the “minimize” method of an optimizer, functions call use func(*param.args, **param.kwargs) instead of func(param.value). This is for simplifying the parametrization of multiparameter functions.

value: ValueProperty[Tuple[Tuple[Any, ...], Dict[str, Any]], Tuple[Tuple[Any, ...], Dict[str, Any]]]

Value of the Parameter, which should be sent to the function to optimize.

Example

>>> ng.p.Array(shape=(2,)).value
array([0., 0.])

class nevergrad.p.Log(*, init: Optional[float] = None, exponent: Optional[float] = None, lower: Optional[float] = None, upper: Optional[float] = None, mutable_sigma: bool = False)

Bases: Scalar

Parameter representing a positive variable, mutated by Gaussian mutation in log-scale.

Parameters

init (float or None) – initial value of the variable. If not provided, it is set to the middle of lower and upper in log space
exponent (float or None) – exponent for the log mutation: an exponent of 2.0 will lead to mutations by factors between around 0.5 and 2.0 By default, it is set to either 2.0, or if the parameter is completely bounded to a factor so that bounds are at 3 sigma in the transformed space.
lower (float or None) – minimum value if any (> 0)
upper (float or None) – maximum value if any
mutable_sigma (bool) – whether the mutation standard deviation must mutate as well (for mutation based algorithms)

Note

This class is only a wrapper over Scalar.

class nevergrad.p.Scalar(init: Optional[float] = None, *, lower: Optional[float] = None, upper: Optional[float] = None, mutable_sigma: bool = True)

Bases: Data

Parameter representing a scalar.

Parameters

init (optional float) – initial value of the scalar (defaults to 0.0 if both bounds are not provided)
lower (optional float) – minimum value if any
upper (optional float) – maximum value if any
mutable_sigma (bool) – whether the mutation standard deviation must mutate as well (for mutation based algorithms)

Notes

by default, this is an unbounded scalar with Gaussian mutations.
if both lower and upper bounds are provided, sigma will be adapted so that the range spans 6 sigma. Also, if init is not provided, it will be set to the middle value.
More specific behaviors can be obtained throught the following methods: set_bounds, set_mutation, set_integer_casting

value: ValueProperty[float, float]

Value of the Parameter, which should be sent to the function to optimize.

Example

>>> ng.p.Array(shape=(2,)).value
array([0., 0.])

class nevergrad.p.TransitionChoice(choices: Union[int, Iterable[Any]], transitions: Union[Tuple[float, ...], List[float], ndarray, Array] = (1.0, 1.0), repetitions: Optional[int] = None, ordered: bool = True)

Bases: BaseChoice

Categorical parameter, choosing one of the provided choice options as a value, with continuous transitions. By default, this is ordered, and most algorithms except discrete OnePlusOne algorithms will consider it as ordered. The choices can be Parameters, in which case there value will be returned instead. The chosen parameter is drawn using transitions between current choice and the next/previous ones.

Parameters

choices (list or int) – a list of possible values or Parameters for the variable (or an integer as a shortcut for range(num))
transitions (np.ndarray or Array) – the transition weights. During transition, the direction (forward or backward will be drawn with equal probabilities), then the transitions weights are normalized through softmax, the 1st value gives the probability to remain in the same state, the second to move one step (backward or forward) and so on.
ordered (bool) – if False, changes the default behavior to be unordered and sampled uniformly when setting the data to a normalized and centered Gaussian (used in DiscreteOnePlusOne only)

Note

the “mutate” method only mutates the weights and the chosen Parameter (if it is not constant), leaving others untouched
in order to support export to standardized space, the index is encoded as a scalar. A normal distribution N(O,1) on this scalar yields a uniform choice of index. This may come to evolve for simplicity’s sake.
currently, transitions are computed through softmax, this may evolve since this is somehow impractical

class nevergrad.p.Tuple(*parameters: Any)

Bases: Container

Tuple-valued parameter. This Parameter can contain other Parameters, its value is tuple which values are either directly the provided values if they are not Parameter instances, or the value of those Parameters. It also implements a getter to access the Parameters directly if need be.

Parameters: **parameters (Any) – the objects or Parameter which will provide values for the tuple

Note

This is the base structure for all container Parameters, and it is used to hold the subparameters for all Parameter classes.

value: ValueProperty[Tuple[Any, ...], Tuple[Any, ...]]

Value of the Parameter, which should be sent to the function to optimize.

Example

>>> ng.p.Array(shape=(2,)).value
array([0., 0.])

Parameter API

class nevergrad.p.Parameter

Class providing the core functionality of a parameter, aka value, internal/model parameters, mutation, recombination and additional features such as shared random state, constraint check, hashes, generation and naming. The value field should sent to the function to optimize.

Example

>>> ng.p.Array(shape=(2,)).value
array([0., 0.])

add_layer(other: Layered) → L: Adds a layer which will modify the object behavior

property args: Tuple[Any, ...]: Value of the positional arguments. Used to input value in a function as func(*param.args, **param.kwargs) Use parameter.Instrumentation to set args and kwargs with full freedom.

copy() → P: Creates a full copy of the parameter (with new unique uid). Use spawn_child instead to make sure to add the parenthood information.

property dimension: int: Dimension of the standardized space for this parameter i.e size of the vector returned by get_standardized_data(reference=…)

freeze() → None: Prevents the parameter from changing value again (through value, mutate etc…)

property generation: int: Generation of the parameter (children are current generation + 1)

get_standardized_data(*, reference: P) → ndarray

Get the standardized data representing the value of the instance as an array in the optimization space. In this standardized space, a mutation is typically centered and reduced (sigma=1) Gaussian noise. The data only represent the value of this instance, not the parameters (eg.: mutable sigma), hence it does not fully represent the state of the instance. Also, in stochastic cases, the value can be non-deterministically deduced from the data (eg.: categorical variable, for which data includes sampling weights for each value)

Parameters: reference (Parameter) – the reference instance for representation in the standardized data space. This keyword parameter is mandatory to make the code clearer. If you use “self”, this method will always return a zero vector.
Returns: the representation of the value in the optimization space
Return type: np.ndarray

Note

Operations between different standardized data should only be performed if each array was produced by the same reference in the exact same state (no mutation)
to make the code more explicit, the “reference” parameter is enforced as a keyword-only parameter.

get_value_hash() → Hashable: Hashable object representing the current value of the instance

property kwargs: Dict[str, Any]: Value of the keyword arguments. Used to input value in a function as func(*param.args, **param.kwargs) Use parameter.Instrumentation to set args and kwargs with full freedom.

property losses: ndarray: Possibly multiobjective losses which were told to the optimizer along this parameter. In case of mono-objective loss, losses is the array containing this loss as sole element

Note

This API is highly experimental

mutate() → None: Mutate parameters of the instance, and then its value

property name: str: Name of the parameter This is used to keep track of how this Parameter is configured (included through internal/model parameters), mostly for reproducibility A default version is always provided, but can be overriden directly through the attribute, or through the set_name method (which allows chaining).

property random_state: RandomState: Random state the instrumentation and the optimizers pull from. It can be seeded/replaced.

recombine(*others: P) → None

Update value and parameters of this instance by combining it with other instances.

Parameters: *others (Parameter) – other instances of the same type than this instance.

register_cheap_constraint(func: Union[Callable[[Any], bool], Callable[[Any], float]], as_layer: bool = False) → None

Registers a new constraint on the parameter values.

Parameters: func (Callable) – function which, given the value of the instance, returns whether it satisfies the constraints (if output = bool), or a float which is >= 0 if the constraint is satisfied.

Note

this is only for checking after mutation/recombination/etc if the value still satisfy the constraints. The constraint is not used in those processes.
constraints should be fast to compute.
this function has an additional “as_layer” parameter which is experimental for now, and can have unexpected behavior

sample() → P: Sample a new instance of the parameter. This usually means spawning a child and mutating it. This function should be used in optimizers when creating an initial population, and parameter.heritage[“lineage”] is reset to parameter.uid instead of its parent’s

satisfies_constraints(ref: Optional[P] = None, no_tabu: bool = False) → bool

Whether the instance satisfies the constraints added through the register_cheap_constraint method

Parameters

ref – parameter of the optimization algorithm, if we want to check the tabu list.
no_tabu – if we want to ignore the Tabu system.

Returns

True iff the constraint is satisfied

Return type

bool

set_name(name: str) → L

Sets a name and return the current instrumentation (for chaining)

Parameters: name (str) – new name to use to represent the Parameter

set_standardized_data(data: Union[Tuple[float, ...], List[float], ndarray], *, reference: Optional[P] = None) → P

Updates the value of the provided reference (or self) using the standardized data.

Parameters

np.ndarray – the representation of the value in the optimization space
reference (Parameter) – the reference point for representing the data (“self”, if not provided)

Returns

self (modified)

Return type

Parameter

Note

To make the code more explicit, the “reference” is enforced as keyword-only parameters.

spawn_child(new_value: Optional[Any] = None) → P

Creates a new instance which shares the same random generator than its parent, is sampled from the same data, and mutates independently from the parentp. If a new value is provided, it will be set to the new instance

Parameters: new_value (anything (optional)) – if provided, it will update the new instance value (cannot be used at the same time as new_data).
Returns: a new instance of the same class, with same content/internal-model parameters/… Optionally, a new value will be set after creation
Return type: Parameter

value: ValueProperty[Any, Any]

Value of the Parameter, which should be sent to the function to optimize.

Example

>>> ng.p.Array(shape=(2,)).value
array([0., 0.])

Operators

These experimental operators are designed to be instantiated and called with a parameter as input, creating a new parameter with the required behavior.

class nevergrad.ops.Int(deterministic: bool = True)

Cast Data as integer (or integer array)

Parameters: deterministic (bool) – if True, the data is rounded to the closest integer, if False, both surrounded integers can be sampled inversely proportionally to how close the actual value is from the integers.

Example

0.2 is cast to 0 in deterministic mode, and either 0 (80% chance) or 1 (20% chance) in non-deterministic mode

Usage

param = ng.ops.Int(deterministic=True)(ng.p.Array(shape=(3,)))