Parametrization API reference
Please note that parametrization is still a work in progress and changes are on their way (including for this documentation)! We are trying to update it to make it simpler and simpler to use (all feedbacks are welcome ;) ), with the side effect that there will be breaking changes.
The aim of parametrization is to specify what are the parameters that the optimization should be performed upon. The parametrization subpackage will help you do thanks to:
the parameter modules (accessed by the shortcut nevergrad.p) providing classes that should be used to specify each parameter.
the ops module (accessed through ng.ops) providing experimental objects for modifying a parameter behavior (eg: casting to int, adding complex constraints).
the FolderFunction which helps transform any code into a Python function in a few lines. This can be especially helpful to optimize parameters in nonPython 3.6+ code (C++, Octave, etc…) or parameters in scripts.
Parameters
Here are the current types of parameters currently provided:



Parameter representing a scalar. 

Parameter representing a positive variable, mutated by Gaussian mutation in logscale. 

Dictionaryvalued parameter. 

Tuplevalued parameter. 

Container of parameters available through args and kwargs attributes. 

Unordered categorical parameter, randomly choosing one of the provided choice options as a value. 

Categorical parameter, choosing one of the provided choice options as a value, with continuous transitions. 
 class nevergrad.p.Array(*, init: Optional[Union[Tuple[float, ...], List[float], ndarray]] = None, shape: Optional[Tuple[int, ...]] = None, lower: Optional[Union[float, int, int64, float64, Tuple[float, ...], List[float], ndarray]] = None, upper: Optional[Union[float, int, int64, float64, Tuple[float, ...], List[float], ndarray]] = None, mutable_sigma: bool = False)
Bases:
Data
 value: ValueProperty[Union[Tuple[float, ...], List[float], ndarray], ndarray]
Value of the Parameter, which should be sent to the function to optimize.
Example
>>> ng.p.Array(shape=(2,)).value array([0., 0.])
 class nevergrad.p.Choice(choices: Union[int, Iterable[Any]], repetitions: Optional[int] = None, deterministic: bool = False)
Bases:
BaseChoice
Unordered categorical parameter, randomly choosing one of the provided choice options as a value. The choices can be Parameters, in which case there value will be returned instead. The chosen parameter is drawn randomly from the softmax of weights which are updated during the optimization.
 Parameters
choices (list or int) – a list of possible values or Parameters for the variable (or an integer as a shortcut for range(num))
repetitions (None or int) – set to an integer
n
if you wantn
similar choices sampled independently (each with its own distribution) This is equivalent toTuple(*[Choice(options) for _ in range(n)])
but can be 30x faster for largen
.deterministic (bool) – whether to always draw the most likely choice (hence avoiding the stochastic behavior, but loosing continuity)
Note
Since the chosen value is drawn randomly, the use of this variable makes deterministic functions become stochastic, hence “adding noise”
the “mutate” method only mutates the weights and the chosen Parameter (if it is not constant), leaving others untouched
Examples
>>> print(Choice(["a", "b", "c", "e"]).value) "c"
>>> print(Choice(["a", "b", "c", "e"], repetitions=3).value) ("b", "b", "c")
 class nevergrad.p.Dict(**parameters: Any)
Bases:
Container
Dictionaryvalued parameter. This Parameter can contain other Parameters, its value is a dict, with keys the ones provided as input, and corresponding values are either directly the provided values if they are not Parameter instances, or the value of those Parameters. It also implements a getter to access the Parameters directly if need be.
 Parameters
**parameters (Any) – the objects or Parameter which will provide values for the dict
Note
This is the base structure for all container Parameters, and it is used to hold the internal/model parameters for all Parameter classes.
 value: ValueProperty[Dict[str, Any], Dict[str, Any]]
Value of the Parameter, which should be sent to the function to optimize.
Example
>>> ng.p.Array(shape=(2,)).value array([0., 0.])
 class nevergrad.p.Instrumentation(*args: Any, **kwargs: Any)
Bases:
Tuple
Container of parameters available through args and kwargs attributes. The parameter provided as input are used to provide values for an arg tuple and a kwargs dict. value attribue returns (args, kwargs), but each can be independantly accessed through the args and kwargs properties.
 Parameters
*args – values or Parameters to be used to fill the tuple of args
*kwargs – values or Parameters to be used to fill the dict of kwargs
Note
When used in conjonction with the “minimize” method of an optimizer, functions call use func(*param.args, **param.kwargs) instead of func(param.value). This is for simplifying the parametrization of multiparameter functions.
 value: ValueProperty[Tuple[Tuple[Any, ...], Dict[str, Any]], Tuple[Tuple[Any, ...], Dict[str, Any]]]
Value of the Parameter, which should be sent to the function to optimize.
Example
>>> ng.p.Array(shape=(2,)).value array([0., 0.])
 class nevergrad.p.Log(*, init: Optional[float] = None, exponent: Optional[float] = None, lower: Optional[float] = None, upper: Optional[float] = None, mutable_sigma: bool = False)
Bases:
Scalar
Parameter representing a positive variable, mutated by Gaussian mutation in logscale.
 Parameters
init (float or None) – initial value of the variable. If not provided, it is set to the middle of lower and upper in log space
exponent (float or None) – exponent for the log mutation: an exponent of 2.0 will lead to mutations by factors between around 0.5 and 2.0 By default, it is set to either 2.0, or if the parameter is completely bounded to a factor so that bounds are at 3 sigma in the transformed space.
lower (float or None) – minimum value if any (> 0)
upper (float or None) – maximum value if any
mutable_sigma (bool) – whether the mutation standard deviation must mutate as well (for mutation based algorithms)
Note
This class is only a wrapper over
Scalar
.
 class nevergrad.p.Scalar(init: Optional[float] = None, *, lower: Optional[float] = None, upper: Optional[float] = None, mutable_sigma: bool = True)
Bases:
Data
Parameter representing a scalar.
 Parameters
init (optional float) – initial value of the scalar (defaults to 0.0 if both bounds are not provided)
lower (optional float) – minimum value if any
upper (optional float) – maximum value if any
mutable_sigma (bool) – whether the mutation standard deviation must mutate as well (for mutation based algorithms)
Notes
by default, this is an unbounded scalar with Gaussian mutations.
if both lower and upper bounds are provided, sigma will be adapted so that the range spans 6 sigma. Also, if init is not provided, it will be set to the middle value.
More specific behaviors can be obtained throught the following methods:
set_bounds
,set_mutation
,set_integer_casting
 value: ValueProperty[float, float]
Value of the Parameter, which should be sent to the function to optimize.
Example
>>> ng.p.Array(shape=(2,)).value array([0., 0.])
 class nevergrad.p.TransitionChoice(choices: Union[int, Iterable[Any]], transitions: Union[Tuple[float, ...], List[float], ndarray, Array] = (1.0, 1.0), repetitions: Optional[int] = None, ordered: bool = True)
Bases:
BaseChoice
Categorical parameter, choosing one of the provided choice options as a value, with continuous transitions. By default, this is ordered, and most algorithms except discrete OnePlusOne algorithms will consider it as ordered. The choices can be Parameters, in which case there value will be returned instead. The chosen parameter is drawn using transitions between current choice and the next/previous ones.
 Parameters
choices (list or int) – a list of possible values or Parameters for the variable (or an integer as a shortcut for range(num))
transitions (np.ndarray or Array) – the transition weights. During transition, the direction (forward or backward will be drawn with equal probabilities), then the transitions weights are normalized through softmax, the 1st value gives the probability to remain in the same state, the second to move one step (backward or forward) and so on.
ordered (bool) – if False, changes the default behavior to be unordered and sampled uniformly when setting the data to a normalized and centered Gaussian (used in DiscreteOnePlusOne only)
Note
the “mutate” method only mutates the weights and the chosen Parameter (if it is not constant), leaving others untouched
in order to support export to standardized space, the index is encoded as a scalar. A normal distribution N(O,1) on this scalar yields a uniform choice of index. This may come to evolve for simplicity’s sake.
currently, transitions are computed through softmax, this may evolve since this is somehow impractical
 class nevergrad.p.Tuple(*parameters: Any)
Bases:
Container
Tuplevalued parameter. This Parameter can contain other Parameters, its value is tuple which values are either directly the provided values if they are not Parameter instances, or the value of those Parameters. It also implements a getter to access the Parameters directly if need be.
 Parameters
**parameters (Any) – the objects or Parameter which will provide values for the tuple
Note
This is the base structure for all container Parameters, and it is used to hold the subparameters for all Parameter classes.
 value: ValueProperty[Tuple[Any, ...], Tuple[Any, ...]]
Value of the Parameter, which should be sent to the function to optimize.
Example
>>> ng.p.Array(shape=(2,)).value array([0., 0.])
Parameter API
 class nevergrad.p.Parameter
Class providing the core functionality of a parameter, aka value, internal/model parameters, mutation, recombination and additional features such as shared random state, constraint check, hashes, generation and naming. The value field should sent to the function to optimize.
Example
>>> ng.p.Array(shape=(2,)).value array([0., 0.])
 add_layer(other: Layered) L
Adds a layer which will modify the object behavior
 property args: Tuple[Any, ...]
Value of the positional arguments. Used to input value in a function as func(*param.args, **param.kwargs) Use parameter.Instrumentation to set args and kwargs with full freedom.
 copy() P
Creates a full copy of the parameter (with new unique uid). Use spawn_child instead to make sure to add the parenthood information.
 property dimension: int
Dimension of the standardized space for this parameter i.e size of the vector returned by get_standardized_data(reference=…)
 freeze() None
Prevents the parameter from changing value again (through value, mutate etc…)
 property generation: int
Generation of the parameter (children are current generation + 1)
 get_standardized_data(*, reference: P) ndarray
Get the standardized data representing the value of the instance as an array in the optimization space. In this standardized space, a mutation is typically centered and reduced (sigma=1) Gaussian noise. The data only represent the value of this instance, not the parameters (eg.: mutable sigma), hence it does not fully represent the state of the instance. Also, in stochastic cases, the value can be nondeterministically deduced from the data (eg.: categorical variable, for which data includes sampling weights for each value)
 Parameters
reference (Parameter) – the reference instance for representation in the standardized data space. This keyword parameter is mandatory to make the code clearer. If you use “self”, this method will always return a zero vector.
 Returns
the representation of the value in the optimization space
 Return type
np.ndarray
Note
Operations between different standardized data should only be performed if each array was produced by the same reference in the exact same state (no mutation)
to make the code more explicit, the “reference” parameter is enforced as a keywordonly parameter.
 get_value_hash() Hashable
Hashable object representing the current value of the instance
 property kwargs: Dict[str, Any]
Value of the keyword arguments. Used to input value in a function as func(*param.args, **param.kwargs) Use parameter.Instrumentation to set args and kwargs with full freedom.
 property losses: ndarray
Possibly multiobjective losses which were told to the optimizer along this parameter. In case of monoobjective loss, losses is the array containing this loss as sole element
Note
This API is highly experimental
 mutate() None
Mutate parameters of the instance, and then its value
 property name: str
Name of the parameter This is used to keep track of how this Parameter is configured (included through internal/model parameters), mostly for reproducibility A default version is always provided, but can be overriden directly through the attribute, or through the set_name method (which allows chaining).
 property random_state: RandomState
Random state the instrumentation and the optimizers pull from. It can be seeded/replaced.
 recombine(*others: P) None
Update value and parameters of this instance by combining it with other instances.
 Parameters
*others (Parameter) – other instances of the same type than this instance.
 register_cheap_constraint(func: Union[Callable[[Any], bool], Callable[[Any], float]], as_layer: bool = False) None
Registers a new constraint on the parameter values.
 Parameters
func (Callable) – function which, given the value of the instance, returns whether it satisfies the constraints (if output = bool), or a float which is >= 0 if the constraint is satisfied.
Note
this is only for checking after mutation/recombination/etc if the value still satisfy the constraints. The constraint is not used in those processes.
constraints should be fast to compute.
this function has an additional “as_layer” parameter which is experimental for now, and can have unexpected behavior
 sample() P
Sample a new instance of the parameter. This usually means spawning a child and mutating it. This function should be used in optimizers when creating an initial population, and parameter.heritage[“lineage”] is reset to parameter.uid instead of its parent’s
 satisfies_constraints(ref: Optional[P] = None, no_tabu: bool = False) bool
Whether the instance satisfies the constraints added through the register_cheap_constraint method
 Parameters
ref – parameter of the optimization algorithm, if we want to check the tabu list.
no_tabu – if we want to ignore the Tabu system.
 Returns
True iff the constraint is satisfied
 Return type
bool
 set_name(name: str) L
Sets a name and return the current instrumentation (for chaining)
 Parameters
name (str) – new name to use to represent the Parameter
 set_standardized_data(data: Union[Tuple[float, ...], List[float], ndarray], *, reference: Optional[P] = None) P
Updates the value of the provided reference (or self) using the standardized data.
 Parameters
np.ndarray – the representation of the value in the optimization space
reference (Parameter) – the reference point for representing the data (“self”, if not provided)
 Returns
self (modified)
 Return type
Note
To make the code more explicit, the “reference” is enforced as keywordonly parameters.
 spawn_child(new_value: Optional[Any] = None) P
Creates a new instance which shares the same random generator than its parent, is sampled from the same data, and mutates independently from the parentp. If a new value is provided, it will be set to the new instance
 Parameters
new_value (anything (optional)) – if provided, it will update the new instance value (cannot be used at the same time as new_data).
 Returns
a new instance of the same class, with same content/internalmodel parameters/… Optionally, a new value will be set after creation
 Return type
 value: ValueProperty[Any, Any]
Value of the Parameter, which should be sent to the function to optimize.
Example
>>> ng.p.Array(shape=(2,)).value array([0., 0.])
Operators
These experimental operators are designed to be instantiated and called with a parameter as input, creating a new parameter with the required behavior.
 class nevergrad.ops.Int(deterministic: bool = True)
Cast Data as integer (or integer array)
 Parameters
deterministic (bool) – if True, the data is rounded to the closest integer, if False, both surrounded integers can be sampled inversely proportionally to how close the actual value is from the integers.
Example
0.2 is cast to 0 in deterministic mode, and either 0 (80% chance) or 1 (20% chance) in nondeterministic mode
Usage
param = ng.ops.Int(deterministic=True)(ng.p.Array(shape=(3,)))