Parametrization API reference¶
Please note that parametrization is still a work in progress and changes are on their way (including for this documentation)! We are trying to update it to make it simpler and simpler to use (all feedbacks are welcome ;) ), with the side effect that there will be breaking changes.
The aim of parametrization is to specify what are the parameters that the optimization should be performed upon. The parametrization subpackage will help you do thanks to:
the parameter modules (accessed by the shortcut nevergrad.p) providing classes that should be used to specify each parameter.
the FolderFunction which helps transform any code into a Python function in a few lines. This can be especially helpful to optimize parameters in nonPython 3.6+ code (C++, Octave, etc…) or parameters in scripts.
Parameters¶
Here are the current types of parameters currently provided:

Array parameter with customizable mutation and recombination. 

Parameter representing a scalar. 

Parameter representing a positive variable, mutated by Gaussian mutation in logscale. 

Dictionaryvalued parameter. 

Tuplevalued parameter. 

Container of parameters available through args and kwargs attributes. 

Unordered categorical parameter, randomly choosing one of the provided choice options as a value. 

Ordered categorical parameter, choosing one of the provided choice options as a value, with continuous transitions. 

class
nevergrad.p.
Array
(*, init: Optional[Union[Tuple[float, …], List[float], numpy.ndarray]] = None, shape: Optional[Tuple[int, …]] = None, mutable_sigma: bool = False)¶ Bases:
nevergrad.parametrization.core.Parameter
Array parameter with customizable mutation and recombination.
 Parameters
init (np.ndarray, or None) – initial value of the array (defaults to 0, with a provided shape)
shape (tuple of ints, or None) – shape of the array, to be provided iff init is not provided
mutable_sigma (bool) – whether the mutation standard deviation must mutate as well (for mutation based algorithms)
Note
More specific behaviors can be obtained throught the following methods: set_bounds, set_mutation, set_integer_casting

set_bounds
(lower: Optional[Union[float, int, numpy.ndarray]] = None, upper: Optional[Union[float, int, numpy.ndarray]] = None, method: str = 'bouncing', full_range_sampling: Optional[bool] = None, a_min: Optional[Union[float, int, numpy.ndarray]] = None, a_max: Optional[Union[float, int, numpy.ndarray]] = None) → A¶ Bounds all real values into [lower, upper] using a provided method
 Parameters
lower (float or None) – minimum value
upper (float or None) – maximum value
method (str) –
One of the following choices:
 ”bouncing”: bounce on border (at most once). This is a variant of clipping,
avoiding bounds oversamping (default).
”clipping”: clips the values inside the bounds. This is efficient but leads to oversampling on the bounds.
”constraint”: adds a constraint (see register_cheap_constraint) which leads to rejecting mutations reaching beyond the bounds. This avoids oversampling the boundaries, but can be inefficient in large dimension.
”arctan”: maps the space [lower, upper] to to all [inf, inf] using arctan transform. This is efficient but it completely reshapes the space (a mutation in the center of the space will be larger than a mutation close to the bounds), and reaching the bounds is equivalent to reaching the infinity.
”tanh”: same as “arctan”, but with a “tanh” transform. “tanh” saturating much faster than “arctan”, it can lead to unexpected behaviors.
full_range_sampling (Optional bool) – Changes the default behavior of the “sample” method (aka creating a child and mutating it from the current instance) or the sampling optimizers, to creating a child with a value sampled uniformly (or loguniformly) within the while range of the bounds. The “sample” method is used by some algorithms to create an initial population. This is activated by default if both bounds are provided.
Notes
“tanh” reaches the boundaries really quickly, while “arctan” is much softer
only “clipping” accepts partial bounds (None values)

set_integer_casting
() → A¶ Output will be cast to integer(s) through deterministic rounding.
 Returns
 Return type
self
Note
Using integer casting makes the parameter discrete which can make the optimization more difficult. It is especially illadvised to use this with a range smaller than 10, or a sigma lower than 1. In those cases, you should rather use a TransitionChoice instead.

set_mutation
(sigma: Optional[Union[float, nevergrad.parametrization.core.Parameter]] = None, exponent: Optional[float] = None, custom: Optional[Union[str, nevergrad.parametrization.core.Parameter]] = None) → A¶ Output will be cast to integer(s) through deterministic rounding.
 Parameters
sigma (Array/Log or float) – The standard deviation of the mutation. If a Parameter is provided, it will replace the current value. If a float is provided, it will either replace a previous float value, or update the value of the Parameter.
exponent (float) – exponent for the logarithmic mode. With the default sigma=1, using exponent=2 will perform x2 or /2 “on average” on the value at each mutation.
custom (str or Parameter) – custom mutation which can be a string (“gaussian” or “cauchy”) or Mutation/Recombination like object or a Parameter which value is either of those
 Returns
 Return type
self

property
sigma
¶ Value for the standard deviation used to mutate the parameter

class
nevergrad.p.
Choice
(choices: Iterable[Any], repetitions: Optional[int] = None, deterministic: bool = False)¶ Bases:
nevergrad.parametrization.choice.BaseChoice
Unordered categorical parameter, randomly choosing one of the provided choice options as a value. The choices can be Parameters, in which case there value will be returned instead. The chosen parameter is drawn randomly from the softmax of weights which are updated during the optimization.
 Parameters
choices (list) – a list of possible values or Parameters for the variable.
repetitions (None or int) – set to an integer
n
if you wantn
similar choices sampled independently (each with its own distribution) This is equivalent toTuple(*[Choice(options) for _ in range(n)])
but can be 30x faster for largen
.deterministic (bool) – whether to always draw the most likely choice (hence avoiding the stochastic behavior, but loosing continuity)
Note
Since the chosen value is drawn randomly, the use of this variable makes deterministic functions become stochastic, hence “adding noise”
the “mutate” method only mutates the weights and the chosen Parameter (if it is not constant), leaving others untouched
Examples
>>> print(Choice(["a", "b", "c", "e"]).value) "c"
>>> print(Choice(["a", "b", "c", "e"], repetitions=3).value) ("b", "b", "c")

property
indices
¶ Index of the chosen option

property
probabilities
¶ The probabilities used to draw the value

property
weights
¶ The weights used to draw the value

class
nevergrad.p.
Dict
(**parameters: Any)¶ Bases:
nevergrad.parametrization.core.Parameter
Dictionaryvalued parameter. This Parameter can contain other Parameters, its value is a dict, with keys the ones provided as input, and corresponding values are either directly the provided values if they are not Parameter instances, or the value of those Parameters. It also implements a getter to access the Parameters directly if need be.
 Parameters
**parameters (Any) – the objects or Parameter which will provide values for the dict
Note
This is the base structure for all container Parameters, and it is used to hold the internal/model parameters for all Parameter classes.

class
nevergrad.p.
Instrumentation
(*args: Any, **kwargs: Any)¶ Bases:
nevergrad.parametrization.container.Tuple
Container of parameters available through args and kwargs attributes. The parameter provided as input are used to provide values for an arg tuple and a kwargs dict. value attribue returns (args, kwargs), but each can be independantly accessed through the args and kwargs properties.
 Parameters
*args – values or Parameters to be used to fill the tuple of args
*kwargs – values or Parameters to be used to fill the dict of kwargs
Note
When used in conjonction with the “minimize” method of an optimizer, functions call use func(*param.args, **param.kwargs) instead of func(param.value). This is for simplifying the parametrization of multiparameter functions.

class
nevergrad.p.
Log
(*, init: Optional[float] = None, exponent: Optional[float] = None, lower: Optional[float] = None, upper: Optional[float] = None, mutable_sigma: bool = False, a_min: Optional[float] = None, a_max: Optional[float] = None)¶ Bases:
nevergrad.parametrization.data.Scalar
Parameter representing a positive variable, mutated by Gaussian mutation in logscale.
 Parameters
init (float or None) – initial value of the variable. If not provided, it is set to the middle of lower and upper in log space
exponent (float or None) – exponent for the log mutation: an exponent of 2.0 will lead to mutations by factors between around 0.5 and 2.0 By default, it is set to either 2.0, or if the parameter is completely bounded to a factor so that bounds are at 3 sigma in the transformed space.
lower (float or None) – minimum value if any (> 0)
upper (float or None) – maximum value if any
mutable_sigma (bool) – whether the mutation standard deviation must mutate as well (for mutation based algorithms)
Note
This class is only a wrapper over
Scalar
.

class
nevergrad.p.
Scalar
(init: Optional[float] = None, *, lower: Optional[float] = None, upper: Optional[float] = None, mutable_sigma: bool = True)¶ Bases:
nevergrad.parametrization.data.Array
Parameter representing a scalar.
 Parameters
init (optional float) – initial value of the scalar (defaults to 0.0 if both bounds are not provided)
lower (optional float) – minimum value if any
upper (optional float) – maximum value if any
mutable_sigma (bool) – whether the mutation standard deviation must mutate as well (for mutation based algorithms)
Notes
by default, this is an unbounded scalar with Gaussian mutations.
if both lower and upper bounds are provided, sigma will be adapted so that the range spans 6 sigma. Also, if init is not provided, it will be set to the middle value.
More specific behaviors can be obtained throught the following methods:
set_bounds
,set_mutation
,set_integer_casting

class
nevergrad.p.
TransitionChoice
(choices: Iterable[Any], transitions: Union[Tuple[float, …], List[float], numpy.ndarray, nevergrad.parametrization.data.Array] = (1.0, 1.0), repetitions: Optional[int] = None)¶ Bases:
nevergrad.parametrization.choice.BaseChoice
Ordered categorical parameter, choosing one of the provided choice options as a value, with continuous transitions. The choices can be Parameters, in which case there value will be returned instead. The chosen parameter is drawn using transitions between current choice and the next/previous ones.
 Parameters
choices (list) – a list of possible values or Parameters for the variable.
transitions (np.ndarray or Array) – the transition weights. During transition, the direction (forward or backward will be drawn with equal probabilities), then the transitions weights are normalized through softmax, the 1st value gives the probability to remain in the same state, the second to move one step (backward or forward) and so on.
Note
the “mutate” method only mutates the weights and the chosen Parameter (if it is not constant), leaving others untouched
in order to support export to standardized space, the index is encoded as a scalar. A normal distribution N(O,1) on this scalar yields a uniform choice of index. This may come to evolve for simplicity’s sake.
currently, transitions are computed through softmax, this may evolve since this is somehow impractical

property
indices
¶ Indices of the chosen options

property
position
¶ The continuous version of the index (used when working with standardized space)

property
positions
¶ The continuous version of the index (used when working with standardized space)

property
transitions
¶ The weights used to draw the step to the next value

class
nevergrad.p.
Tuple
(*parameters: Any)¶ Bases:
nevergrad.parametrization.core.Dict
Tuplevalued parameter. This Parameter can contain other Parameters, its value is tuple which values are either directly the provided values if they are not Parameter instances, or the value of those Parameters. It also implements a getter to access the Parameters directly if need be.
 Parameters
**parameters (Any) – the objects or Parameter which will provide values for the tuple
Note
This is the base structure for all container Parameters, and it is used to hold the subparameters for all Parameter classes.
Parameter API¶

class
nevergrad.p.
Parameter
(**parameters: Any)¶ Abstract class providing the core functionality of a parameter, aka value, internal/model parameters, mutation, recombination and additional features such as shared random state, constraint check, hashes, generation and naming.

property
args
¶ Value of the positional arguments. Used to input value in a function as func(*param.args, **param.kwargs) Use parameter.Instrumentation to set args and kwargs with full freedom.

copy
() → P¶ Create a child, but remove the random state This is used to run multiple experiments

property
dimension
¶ Dimension of the standardized space for this parameter i.e size of the vector returned by get_standardized_data(reference=…)

freeze
() → None¶ Prevents the parameter from changing value again (through value, mutate etc…)

property
generation
¶ Generation of the parameter (children are current generation + 1)

get_standardized_data
(*, reference: P) → numpy.ndarray¶ Get the standardized data representing the value of the instance as an array in the optimization space. In this standardized space, a mutation is typically centered and reduced (sigma=1) Gaussian noise. The data only represent the value of this instance, not the parameters (eg.: mutable sigma), hence it does not fully represent the state of the instance. Also, in stochastic cases, the value can be nondeterministically deduced from the data (eg.: categorical variable, for which data includes sampling weights for each value)
 Parameters
reference (Parameter) – the reference instance for representation in the standardized data space. This keyword parameter is mandatory to make the code clearer. If you use “self”, this method will always return a zero vector.
 Returns
the representation of the value in the optimization space
 Return type
np.ndarray
Note
Operations between different standardized data should only be performed if each array was produced by the same reference in the exact same state (no mutation)
to make the code more explicit, the “reference” parameter is enforced as a keywordonly parameter.

get_value_hash
() → Hashable¶ Hashable object representing the current value of the instance

property
kwargs
¶ Value of the keyword arguments. Used to input value in a function as func(*param.args, **param.kwargs) Use parameter.Instrumentation to set args and kwargs with full freedom.

property
losses
¶ Possibly multiobjective losses which were told to the optimizer along this parameter. In case of monoobjective loss, losses is the array containing this loss as sole element
Note
This API is highly experimental

mutate
() → None¶ Mutate parameters of the instance, and then its value

property
name
¶ Name of the parameter This is used to keep track of how this Parameter is configured (included through internal/model parameters), mostly for reproducibility A default version is always provided, but can be overriden directly through the attribute, or through the set_name method (which allows chaining).

property
parameters
¶ Internal/model parameters for this parameter

property
random_state
¶ Random state the instrumentation and the optimizers pull from. It can be seeded/replaced.

recombine
(*others: P) → None¶ Update value and parameters of this instance by combining it with other instances.
 Parameters
*others (Parameter) – other instances of the same type than this instance.

register_cheap_constraint
(func: Callable[[Any], bool]) → None¶ Registers a new constraint on the parameter values.
 Parameters
func (Callable) – function which, given the value of the instance, returns whether it satisfies the constraints.
Note
this is only for checking after mutation/recombination/etc if the value still satisfy the constraints. The constraint is not used in those processes.
constraints should be fast to compute.

sample
() → P¶ Sample a new instance of the parameter. This usually means spawning a child and mutating it. This function should be used in optimizers when creating an initial population, and parameter.heritage[“lineage”] is reset to parameter.uid instead of its parent’s

satisfies_constraints
() → bool¶ Whether the instance satisfies the constraints added through the register_cheap_constraint method
 Returns
True iff the constraint is satisfied
 Return type
bool

set_name
(name: str) → P¶ Sets a name and return the current instrumentation (for chaining)
 Parameters
name (str) – new name to use to represent the Parameter

set_standardized_data
(data: Union[Tuple[float, …], List[float], numpy.ndarray], *, reference: Optional[P] = None, deterministic: bool = False) → P¶ Updates the value of the provided reference (or self) using the standardized data.
 Parameters
np.ndarray – the representation of the value in the optimization space
reference (Parameter) – the reference point for representing the data (“self”, if not provided)
deterministic (bool) – whether the value should be deterministically drawn (max probability) in the case of stochastic parameters
 Returns
self (modified)
 Return type
Note
To make the code more explicit, the “reference” and “deterministic” parameters are enforced as keywordonly parameters.

spawn_child
(new_value: Optional[Any] = None) → P¶ Creates a new instance which shares the same random generator than its parent, is sampled from the same data, and mutates independently from the parentp. If a new value is provided, it will be set to the new instance
 Parameters
new_value (anything (optional)) – if provided, it will update the new instance value (cannot be used at the same time as new_data).
 Returns
a new instance of the same class, with same content/internalmodel parameters/… Optionally, a new value will be set after creation
 Return type

property