Parametrization API reference

Please note that parametrization is still a work in progress and changes are on their way (including for this documentation)! We are trying to update it to make it simpler and simpler to use (all feedbacks are welcome ;) ), with the side effect that there will be breaking changes.

The aim of parametrization is to specify what are the parameters that the optimization should be performed upon. The parametrization subpackage will help you do thanks to:

  • the parameter modules (accessed by the shortcut nevergrad.p) providing classes that should be used to specify each parameter.

  • the FolderFunction which helps transform any code into a Python function in a few lines. This can be especially helpful to optimize parameters in non-Python 3.6+ code (C++, Octave, etc…) or parameters in scripts.

Parameters

Here are the current types of parameters currently provided:

nevergrad.p.Array(*, init, …], …)

Array parameter with customizable mutation and recombination.

nevergrad.p.Scalar(init, *, lower, upper, …)

Parameter representing a scalar.

nevergrad.p.Log(*, init, exponent, lower, …)

Parameter representing a positive variable, mutated by Gaussian mutation in log-scale.

nevergrad.p.Dict(**parameters)

Dictionary-valued parameter.

nevergrad.p.Tuple(*parameters)

Tuple-valued parameter.

nevergrad.p.Instrumentation(*args, **kwargs)

Container of parameters available through args and kwargs attributes.

nevergrad.p.Choice(choices, repetitions, …)

Unordered categorical parameter, randomly choosing one of the provided choice options as a value.

nevergrad.p.TransitionChoice(choices, …)

Ordered categorical parameter, choosing one of the provided choice options as a value, with continuous transitions.

class nevergrad.p.Array(*, init: Optional[Union[Tuple[float, ], List[float], numpy.ndarray]] = None, shape: Optional[Tuple[int, ]] = None, mutable_sigma: bool = False)

Bases: nevergrad.parametrization.core.Parameter

Array parameter with customizable mutation and recombination.

Parameters
  • init (np.ndarray, or None) – initial value of the array (defaults to 0, with a provided shape)

  • shape (tuple of ints, or None) – shape of the array, to be provided iff init is not provided

  • mutable_sigma (bool) – whether the mutation standard deviation must mutate as well (for mutation based algorithms)

Note

More specific behaviors can be obtained throught the following methods: set_bounds, set_mutation, set_integer_casting

set_bounds(lower: Optional[Union[float, int, numpy.ndarray]] = None, upper: Optional[Union[float, int, numpy.ndarray]] = None, method: str = 'bouncing', full_range_sampling: Optional[bool] = None, a_min: Optional[Union[float, int, numpy.ndarray]] = None, a_max: Optional[Union[float, int, numpy.ndarray]] = None) → A

Bounds all real values into [lower, upper] using a provided method

Parameters
  • lower (float or None) – minimum value

  • upper (float or None) – maximum value

  • method (str) –

    One of the following choices:

    • ”bouncing”: bounce on border (at most once). This is a variant of clipping,

      avoiding bounds over-samping (default).

    • ”clipping”: clips the values inside the bounds. This is efficient but leads to over-sampling on the bounds.

    • ”constraint”: adds a constraint (see register_cheap_constraint) which leads to rejecting mutations reaching beyond the bounds. This avoids oversampling the boundaries, but can be inefficient in large dimension.

    • ”arctan”: maps the space [lower, upper] to to all [-inf, inf] using arctan transform. This is efficient but it completely reshapes the space (a mutation in the center of the space will be larger than a mutation close to the bounds), and reaching the bounds is equivalent to reaching the infinity.

    • ”tanh”: same as “arctan”, but with a “tanh” transform. “tanh” saturating much faster than “arctan”, it can lead to unexpected behaviors.

  • full_range_sampling (Optional bool) – Changes the default behavior of the “sample” method (aka creating a child and mutating it from the current instance) or the sampling optimizers, to creating a child with a value sampled uniformly (or log-uniformly) within the while range of the bounds. The “sample” method is used by some algorithms to create an initial population. This is activated by default if both bounds are provided.

Notes

  • “tanh” reaches the boundaries really quickly, while “arctan” is much softer

  • only “clipping” accepts partial bounds (None values)

set_integer_casting() → A

Output will be cast to integer(s) through deterministic rounding.

Returns

Return type

self

Note

Using integer casting makes the parameter discrete which can make the optimization more difficult. It is especially ill-advised to use this with a range smaller than 10, or a sigma lower than 1. In those cases, you should rather use a TransitionChoice instead.

set_mutation(sigma: Optional[Union[float, nevergrad.parametrization.core.Parameter]] = None, exponent: Optional[float] = None, custom: Optional[Union[str, nevergrad.parametrization.core.Parameter]] = None) → A

Output will be cast to integer(s) through deterministic rounding.

Parameters
  • sigma (Array/Log or float) – The standard deviation of the mutation. If a Parameter is provided, it will replace the current value. If a float is provided, it will either replace a previous float value, or update the value of the Parameter.

  • exponent (float) – exponent for the logarithmic mode. With the default sigma=1, using exponent=2 will perform x2 or /2 “on average” on the value at each mutation.

  • custom (str or Parameter) – custom mutation which can be a string (“gaussian” or “cauchy”) or Mutation/Recombination like object or a Parameter which value is either of those

Returns

Return type

self

property sigma

Value for the standard deviation used to mutate the parameter

class nevergrad.p.Choice(choices: Iterable[Any], repetitions: Optional[int] = None, deterministic: bool = False)

Bases: nevergrad.parametrization.choice.BaseChoice

Unordered categorical parameter, randomly choosing one of the provided choice options as a value. The choices can be Parameters, in which case there value will be returned instead. The chosen parameter is drawn randomly from the softmax of weights which are updated during the optimization.

Parameters
  • choices (list) – a list of possible values or Parameters for the variable.

  • repetitions (None or int) – set to an integer n if you want n similar choices sampled independently (each with its own distribution) This is equivalent to Tuple(*[Choice(options) for _ in range(n)]) but can be 30x faster for large n.

  • deterministic (bool) – whether to always draw the most likely choice (hence avoiding the stochastic behavior, but loosing continuity)

Note

  • Since the chosen value is drawn randomly, the use of this variable makes deterministic functions become stochastic, hence “adding noise”

  • the “mutate” method only mutates the weights and the chosen Parameter (if it is not constant), leaving others untouched

Examples

>>> print(Choice(["a", "b", "c", "e"]).value)
"c"
>>> print(Choice(["a", "b", "c", "e"], repetitions=3).value)
("b", "b", "c")
property indices

Index of the chosen option

property probabilities

The probabilities used to draw the value

property weights

The weights used to draw the value

class nevergrad.p.Dict(**parameters: Any)

Bases: nevergrad.parametrization.core.Parameter

Dictionary-valued parameter. This Parameter can contain other Parameters, its value is a dict, with keys the ones provided as input, and corresponding values are either directly the provided values if they are not Parameter instances, or the value of those Parameters. It also implements a getter to access the Parameters directly if need be.

Parameters

**parameters (Any) – the objects or Parameter which will provide values for the dict

Note

This is the base structure for all container Parameters, and it is used to hold the internal/model parameters for all Parameter classes.

class nevergrad.p.Instrumentation(*args: Any, **kwargs: Any)

Bases: nevergrad.parametrization.container.Tuple

Container of parameters available through args and kwargs attributes. The parameter provided as input are used to provide values for an arg tuple and a kwargs dict. value attribue returns (args, kwargs), but each can be independantly accessed through the args and kwargs properties.

Parameters
  • *args – values or Parameters to be used to fill the tuple of args

  • *kwargs – values or Parameters to be used to fill the dict of kwargs

Note

When used in conjonction with the “minimize” method of an optimizer, functions call use func(*param.args, **param.kwargs) instead of func(param.value). This is for simplifying the parametrization of multiparameter functions.

class nevergrad.p.Log(*, init: Optional[float] = None, exponent: Optional[float] = None, lower: Optional[float] = None, upper: Optional[float] = None, mutable_sigma: bool = False, a_min: Optional[float] = None, a_max: Optional[float] = None)

Bases: nevergrad.parametrization.data.Scalar

Parameter representing a positive variable, mutated by Gaussian mutation in log-scale.

Parameters
  • init (float or None) – initial value of the variable. If not provided, it is set to the middle of lower and upper in log space

  • exponent (float or None) – exponent for the log mutation: an exponent of 2.0 will lead to mutations by factors between around 0.5 and 2.0 By default, it is set to either 2.0, or if the parameter is completely bounded to a factor so that bounds are at 3 sigma in the transformed space.

  • lower (float or None) – minimum value if any (> 0)

  • upper (float or None) – maximum value if any

  • mutable_sigma (bool) – whether the mutation standard deviation must mutate as well (for mutation based algorithms)

Note

This class is only a wrapper over Scalar.

class nevergrad.p.Scalar(init: Optional[float] = None, *, lower: Optional[float] = None, upper: Optional[float] = None, mutable_sigma: bool = True)

Bases: nevergrad.parametrization.data.Array

Parameter representing a scalar.

Parameters
  • init (optional float) – initial value of the scalar (defaults to 0.0 if both bounds are not provided)

  • lower (optional float) – minimum value if any

  • upper (optional float) – maximum value if any

  • mutable_sigma (bool) – whether the mutation standard deviation must mutate as well (for mutation based algorithms)

Notes

  • by default, this is an unbounded scalar with Gaussian mutations.

  • if both lower and upper bounds are provided, sigma will be adapted so that the range spans 6 sigma. Also, if init is not provided, it will be set to the middle value.

  • More specific behaviors can be obtained throught the following methods: set_bounds, set_mutation, set_integer_casting

class nevergrad.p.TransitionChoice(choices: Iterable[Any], transitions: Union[Tuple[float, ], List[float], numpy.ndarray, nevergrad.parametrization.data.Array] = (1.0, 1.0), repetitions: Optional[int] = None)

Bases: nevergrad.parametrization.choice.BaseChoice

Ordered categorical parameter, choosing one of the provided choice options as a value, with continuous transitions. The choices can be Parameters, in which case there value will be returned instead. The chosen parameter is drawn using transitions between current choice and the next/previous ones.

Parameters
  • choices (list) – a list of possible values or Parameters for the variable.

  • transitions (np.ndarray or Array) – the transition weights. During transition, the direction (forward or backward will be drawn with equal probabilities), then the transitions weights are normalized through softmax, the 1st value gives the probability to remain in the same state, the second to move one step (backward or forward) and so on.

Note

  • the “mutate” method only mutates the weights and the chosen Parameter (if it is not constant), leaving others untouched

  • in order to support export to standardized space, the index is encoded as a scalar. A normal distribution N(O,1) on this scalar yields a uniform choice of index. This may come to evolve for simplicity’s sake.

  • currently, transitions are computed through softmax, this may evolve since this is somehow impractical

property indices

Indices of the chosen options

property position

The continuous version of the index (used when working with standardized space)

property positions

The continuous version of the index (used when working with standardized space)

property transitions

The weights used to draw the step to the next value

class nevergrad.p.Tuple(*parameters: Any)

Bases: nevergrad.parametrization.core.Dict

Tuple-valued parameter. This Parameter can contain other Parameters, its value is tuple which values are either directly the provided values if they are not Parameter instances, or the value of those Parameters. It also implements a getter to access the Parameters directly if need be.

Parameters

**parameters (Any) – the objects or Parameter which will provide values for the tuple

Note

This is the base structure for all container Parameters, and it is used to hold the subparameters for all Parameter classes.

Parameter API

class nevergrad.p.Parameter(**parameters: Any)

Abstract class providing the core functionality of a parameter, aka value, internal/model parameters, mutation, recombination and additional features such as shared random state, constraint check, hashes, generation and naming.

property args

Value of the positional arguments. Used to input value in a function as func(*param.args, **param.kwargs) Use parameter.Instrumentation to set args and kwargs with full freedom.

copy() → P

Create a child, but remove the random state This is used to run multiple experiments

property dimension

Dimension of the standardized space for this parameter i.e size of the vector returned by get_standardized_data(reference=…)

freeze() → None

Prevents the parameter from changing value again (through value, mutate etc…)

property generation

Generation of the parameter (children are current generation + 1)

get_standardized_data(*, reference: P) → numpy.ndarray

Get the standardized data representing the value of the instance as an array in the optimization space. In this standardized space, a mutation is typically centered and reduced (sigma=1) Gaussian noise. The data only represent the value of this instance, not the parameters (eg.: mutable sigma), hence it does not fully represent the state of the instance. Also, in stochastic cases, the value can be non-deterministically deduced from the data (eg.: categorical variable, for which data includes sampling weights for each value)

Parameters

reference (Parameter) – the reference instance for representation in the standardized data space. This keyword parameter is mandatory to make the code clearer. If you use “self”, this method will always return a zero vector.

Returns

the representation of the value in the optimization space

Return type

np.ndarray

Note

  • Operations between different standardized data should only be performed if each array was produced by the same reference in the exact same state (no mutation)

  • to make the code more explicit, the “reference” parameter is enforced as a keyword-only parameter.

get_value_hash() → Hashable

Hashable object representing the current value of the instance

property kwargs

Value of the keyword arguments. Used to input value in a function as func(*param.args, **param.kwargs) Use parameter.Instrumentation to set args and kwargs with full freedom.

property losses

Possibly multiobjective losses which were told to the optimizer along this parameter. In case of mono-objective loss, losses is the array containing this loss as sole element

Note

This API is highly experimental

mutate() → None

Mutate parameters of the instance, and then its value

property name

Name of the parameter This is used to keep track of how this Parameter is configured (included through internal/model parameters), mostly for reproducibility A default version is always provided, but can be overriden directly through the attribute, or through the set_name method (which allows chaining).

property parameters

Internal/model parameters for this parameter

property random_state

Random state the instrumentation and the optimizers pull from. It can be seeded/replaced.

recombine(*others: P) → None

Update value and parameters of this instance by combining it with other instances.

Parameters

*others (Parameter) – other instances of the same type than this instance.

register_cheap_constraint(func: Callable[[Any], bool]) → None

Registers a new constraint on the parameter values.

Parameters

func (Callable) – function which, given the value of the instance, returns whether it satisfies the constraints.

Note

  • this is only for checking after mutation/recombination/etc if the value still satisfy the constraints. The constraint is not used in those processes.

  • constraints should be fast to compute.

sample() → P

Sample a new instance of the parameter. This usually means spawning a child and mutating it. This function should be used in optimizers when creating an initial population, and parameter.heritage[“lineage”] is reset to parameter.uid instead of its parent’s

satisfies_constraints() → bool

Whether the instance satisfies the constraints added through the register_cheap_constraint method

Returns

True iff the constraint is satisfied

Return type

bool

set_name(name: str) → P

Sets a name and return the current instrumentation (for chaining)

Parameters

name (str) – new name to use to represent the Parameter

set_standardized_data(data: Union[Tuple[float, ], List[float], numpy.ndarray], *, reference: Optional[P] = None, deterministic: bool = False) → P

Updates the value of the provided reference (or self) using the standardized data.

Parameters
  • np.ndarray – the representation of the value in the optimization space

  • reference (Parameter) – the reference point for representing the data (“self”, if not provided)

  • deterministic (bool) – whether the value should be deterministically drawn (max probability) in the case of stochastic parameters

Returns

self (modified)

Return type

Parameter

Note

To make the code more explicit, the “reference” and “deterministic” parameters are enforced as keyword-only parameters.

spawn_child(new_value: Optional[Any] = None) → P

Creates a new instance which shares the same random generator than its parent, is sampled from the same data, and mutates independently from the parentp. If a new value is provided, it will be set to the new instance

Parameters

new_value (anything (optional)) – if provided, it will update the new instance value (cannot be used at the same time as new_data).

Returns

a new instance of the same class, with same content/internal-model parameters/… Optionally, a new value will be set after creation

Return type

Parameter