Keyword reference¶
The keywords that are recognized by the Everest configuration system are described below. Each keyword indicates the expected data type, usually some basic type. It may also be a list, which indicates that the keyword can be provided as a (YAML) list. If a keyword is required or deprecated, this is also indicated.
EverestConfig
Type: objectNo Additional Properties
Controls
Type: arrayDefines a list of controls.
Each control defines a group of control variables. Each group of control variables is defined by a list of variable definitions, for example like this:
controls:
- name: point
type: generic_control
min: -1.0
max: 1.0
initial_guess: 0.1
perturbation_magnitude: 0.001
variables:
- name: x
initial_guess: 0.5
- name: y
min: 0.0
max: 1.0
- name: z
This defines a group of controls names point, with variables x,
y and z. The names of variables passed to the optimizer will be
composed of of the control name and the variable names: point.x,
point.y, and point.z.
Note that many variable properties may be defined both at the level
of the control group and at the variable level. Definitions at the
level of the control group set the properties for all variables in
the group, except when they are overridden by the same field at the
level of the variable. For instance, in the example above, all
variables have an initial guess of 0.1, except for the point.x
variable which has an initial value of 0.5.
Control names must be unique. Variables names must be unique within a control group, except when they are modified with an index, like this:
controls:
- name: point
max: 1.0
min: -1.0
initial_guess: 0.25
type: generic_control
variables:
- name: x
index: 1
- name: x
index: 2
- name: x
index: 3
This defines three variables point.x.1, point.x.2 and
point.x.3. When a variable has an index field, all variables in
the control group must have an index, which must be a unique
non-negative integer.
There is a shortcut for the case that the index fields form a
sequence starting at 1. In this case, the indexed variables can be
defined by providing a list of initial guesses. This is equivalent
to the example above:
controls:
- name: point
max: 1.0
min: -1.0
type: generic_control
variables:
- name: x
initial_guess: [0.25, 0.25, 0.25]
Must contain a minimum of 1 items
Each item of this array must be:
ControlConfig
Type: objectNo Additional Properties
Name
Type: stringThe control group name.
Controls name must be unique.
Type
Type: enum (of string)Control group type.
Only two allowed control types are accepted:
"well_control": Standard built-in Everest control type designed for field optimization"generic_control": Enables the user to define controls types to be employed for customized optimization jobs.
Must be one of:
- "well_control"
- "generic_control"
Variables
List of control variables.
No Additional Items
Each item of this array must be:
variable control
Type: objectNo Additional Properties
Name
Type: stringVariable name.
The name of the variable must be unique within a control group,
unless an index field is defined.
Control Type
Default: nullVariable data type.
If set, overrides the value of control_type of the control group.
Set to "integer" for discrete optimization, or "real" for
continuous optimization (default). This may be ignored if the
optimization algorithm that is used does not support this.
Must be one of:
- "real"
- "integer"
Enabled
Default: nullEnable/disable the variable.
If set to False the variable will be kept kept constant at the
initial guess value during optimization.
If set, overrides the value of enabled in the control group.
Scaled Range
Default: nullThe target range of the variable scaling.
Internally variables are scaled between their minimal and maximum
values (min/max settings) to the range given by target_range
(default = [0, 1]).
Overrides the value of scaled_range in the control group.
This option has no effect on discrete controls.
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Tuple Validation
Item at 1 must be:
Item at 2 must be:
Min
Default: nullThe minimum value of the variable kept during optimization.
Overrides the value of min in the control group.
The initial guess for this variable must respect this minimum value.
Max
Default: nullThe maximum value of the variable kept during optimization.
Overrides the value of max in the control group.
The initial guess for this variable must respect this maximum value.
Perturbation Type
Type: enum (of string) Default: "absolute"The perturbation type for the control group.
The perturbation_type keyword defines whether the perturbation
magnitude (perturbation_magnitude) should be treated as an
absolute value or as relative to the dynamic range of the controls.
Overrides the value of perturbation_type in the control group.
Must be one of:
- "absolute"
- "relative"
Perturbation Magnitude
Default: nullThe perturbation magnitude for this variable.
This controls the magnitude of perturbations (e.g. the standard deviation in case of a normal distribution) of controls, used to approximate the gradient.
The interpretation of this field depends on the value of the
perturbation_type field:
absolute: The given value is used as-is.relative: The perturbation magnitude is calculated by multiplying the given by value by the difference between themaxandminfields.
Overrides the value of perturbation_magnitude in the control group.
Value must be strictly greater than 0
Sampler configuration for this variable.
A sampler specifications is not required, if not provided, a normal distribution is used.
If at least one control group or variable has a sampler specification, only the groups or variables with a sampler specification are perturbed. Controls/variables that do not have a sampler section will not be perturbed at all. If that is not desired, make sure to specify a sampler for each control group and/or variable (or none at all to use a normal distribution for each control).
Within the sampler section, the shared keyword can be used to
direct the sampler to use the same perturbations for each
realization.
Overrides the value of sampler in the control group.
SamplerConfig
Type: objectNo Additional Properties
Backend
Default: null[Deprecated]
The correct backend will be inferred by the method. If several backends
have a method named A, pick a specific backend B by putting B/A in
the method field.
Method
Type: string Default: "norm"The sampling method or distribution used by the sampler backend.
The set of available methods depends on the sampler backend used. By
default a plugin based on scipy.stats is used, implementing the
following methods:
From Probability Distributions:
norm: Samples from a standard normal distribution (mean 0, standard deviation 1).truncnorm: Samples from a truncated normal distribution (mean 0, std. dev. 1), truncated to the range[-1, 1].uniform: Samples from a uniform distribution in the range[-1, 1].
From Quasi-Monte Carlo Sequences:
sobol: Uses Sobol' sequences.halton: Uses Halton sequences.lhs: Uses Latin Hypercube Sampling.
Note: QMC samples are generated in the unit hypercube
[0, 1]^dand then scaled to the hypercube[-1, 1]^d.
Options
Default: nullSpecifies a dict of optional parameters for the sampler backend.
This dict of values is passed unchanged to the selected method in the backend.
Additional Properties of any type are allowed.
Type: objectInitial Guess
Default: nullInitial guess for the variable.
Index
Default: nullA Non-negative integer value to distinguish variables with the same name within a control group.
index should be given either for all of the variables, or for none
of them.
Value must be greater or equal to 0
No Additional Items
Each item of this array must be:
ControlVariableGuessListConfig
Type: objectNo Additional Properties
Name
Type: stringVariable name.
The name of the variable must be unique within a control group,
unless an index field is defined.
Control Type
Default: nullVariable data type.
If set, overrides the value of control_type of the control group.
Set to "integer" for discrete optimization, or "real" for
continuous optimization (default). This may be ignored if the
optimization algorithm that is used does not support this.
Must be one of:
- "real"
- "integer"
Enabled
Default: nullEnable/disable the variable.
If set to False the variable will be kept kept constant at the
initial guess value during optimization.
If set, overrides the value of enabled in the control group.
Scaled Range
Default: nullThe target range of the variable scaling.
Internally variables are scaled between their minimal and maximum
values (min/max settings) to the range given by target_range
(default = [0, 1]).
Overrides the value of scaled_range in the control group.
This option has no effect on discrete controls.
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Tuple Validation
Item at 1 must be:
Item at 2 must be:
Min
Default: nullThe minimum value of the variable kept during optimization.
Overrides the value of min in the control group.
The initial guess for this variable must respect this minimum value.
Max
Default: nullThe maximum value of the variable kept during optimization.
Overrides the value of max in the control group.
The initial guess for this variable must respect this maximum value.
Perturbation Type
Type: enum (of string) Default: "absolute"The perturbation type for the control group.
The perturbation_type keyword defines whether the perturbation
magnitude (perturbation_magnitude) should be treated as an
absolute value or as relative to the dynamic range of the controls.
Overrides the value of perturbation_type in the control group.
Must be one of:
- "absolute"
- "relative"
Perturbation Magnitude
Default: nullThe perturbation magnitude for this variable.
This controls the magnitude of perturbations (e.g. the standard deviation in case of a normal distribution) of controls, used to approximate the gradient.
The interpretation of this field depends on the value of the
perturbation_type field:
absolute: The given value is used as-is.relative: The perturbation magnitude is calculated by multiplying the given by value by the difference between themaxandminfields.
Overrides the value of perturbation_magnitude in the control group.
Value must be strictly greater than 0
Sampler configuration for this variable.
A sampler specifications is not required, if not provided, a normal distribution is used.
If at least one control group or variable has a sampler specification, only the groups or variables with a sampler specification are perturbed. Controls/variables that do not have a sampler section will not be perturbed at all. If that is not desired, make sure to specify a sampler for each control group and/or variable (or none at all to use a normal distribution for each control).
Within the sampler section, the shared keyword can be used to
direct the sampler to use the same perturbations for each
realization.
Overrides the value of sampler in the control group.
SamplerConfig
Type: objectNo Additional Properties
Backend
Default: null[Deprecated]
The correct backend will be inferred by the method. If several backends
have a method named A, pick a specific backend B by putting B/A in
the method field.
Method
Type: string Default: "norm"The sampling method or distribution used by the sampler backend.
The set of available methods depends on the sampler backend used. By
default a plugin based on scipy.stats is used, implementing the
following methods:
From Probability Distributions:
norm: Samples from a standard normal distribution (mean 0, standard deviation 1).truncnorm: Samples from a truncated normal distribution (mean 0, std. dev. 1), truncated to the range[-1, 1].uniform: Samples from a uniform distribution in the range[-1, 1].
From Quasi-Monte Carlo Sequences:
sobol: Uses Sobol' sequences.halton: Uses Halton sequences.lhs: Uses Latin Hypercube Sampling.
Note: QMC samples are generated in the unit hypercube
[0, 1]^dand then scaled to the hypercube[-1, 1]^d.
Options
Default: nullSpecifies a dict of optional parameters for the sampler backend.
This dict of values is passed unchanged to the selected method in the backend.
Additional Properties of any type are allowed.
Type: objectInitial Guess
Type: array of number Default: nullA List of initial values for the variables.
If given, a series of variables with the same name are
generated, with their index field set by enumeration, staring
from 1. The initial value of these variables is taken from this
list, whereas all other fields keep the same given value.
Each item of this array must be:
Initial Guess
Default: nullInitial guess for the control group.
The initial guess value that is assigned to all control variables in the control group.
This default value can be overridden at the variable level.
Control Type
Type: enum (of string) Default: "real"Control data type for the control group.
The data type value that is assigned to all control variables in the control group.
Set to "integer" for discrete optimization, or "real" for
continuous optimization (default). This may be ignored if the
optimization algorithm that is used does not support this.
This default value can be overridden at the variable level.
Must be one of:
- "real"
- "integer"
Enabled
Type: boolean Default: trueEnable/disable control groups.
Whether all control variables in the control group are enabled or not. When not enabled, variables are kept constant at the initial guess value during optimization.
This default value can be overridden at the variable level.
Min
Default: nullThe minimum values of all control variables in the control group.
The initial_guess field of this control group and of all of its
variables must respect this minimum value.
This default value can be overridden at the variable level.
Max
Default: nullThe maximum values of all control variables in the control group.
The initial_guess field of this control group and of all of its
variables must respect this maximum value.
This default value can be overridden at the variable level.
Perturbation Type
Type: enum (of string) Default: "absolute"The perturbation type for the control group.
The perturbation_type keyword defines whether the perturbation
magnitude (perturbation_magnitude) should be treated as an
absolute value or as relative to the dynamic range of the controls.
Must be one of:
- "absolute"
- "relative"
Perturbation Magnitude
Type: numberThe magnitude of the perturbation of all control variables in the control group.
This controls the magnitude of perturbations (e.g. the standard deviation in case of a normal distribution) of controls, used to approximate the gradient.
The interpretation of this field depends on the value of the
perturbation_type field:
absolute: The given value is used as-is.relative: The perturbation magnitude is calculated by multiplying the given by value by the difference between themaxandminfields.
This default value can be overridden at the variable level.
Scaled Range
Type: array Default: [0.0, 1.0]The target range of the control variable scaling for the control group.
Internally control variables are scaled from their minimal and
maximum values (min/max settings) to the range given by
target_range (default = [0, 1]).
This default value can be overridden at the variable level.
This option has no effect on discrete controls.
Must contain a minimum of 2 items
Must contain a maximum of 2 items
Tuple Validation
Item at 1 must be:
Item at 2 must be:
Sampler configuration for a control group.
A sampler specification section applies to a group of controls, or to an individual control. Sampler specifications are not required, if no sampler sections are provided, a normal distribution is used.
This can be overridden at the variable level.
If at least one control group or variable has a sampler specification, only the groups or variables with a sampler specification are perturbed. Controls/variables that do not have a sampler section will not be perturbed at all. If that is not desired, make sure to specify a sampler for each control group and/or variable (or none at all to use a normal distribution for each control).
Within the sampler section, the shared keyword can be used to
direct the sampler to use the same perturbations for each
realization.
SamplerConfig
Type: objectNo Additional Properties
Backend
Default: null[Deprecated]
The correct backend will be inferred by the method. If several backends
have a method named A, pick a specific backend B by putting B/A in
the method field.
Method
Type: string Default: "norm"The sampling method or distribution used by the sampler backend.
The set of available methods depends on the sampler backend used. By
default a plugin based on scipy.stats is used, implementing the
following methods:
From Probability Distributions:
norm: Samples from a standard normal distribution (mean 0, standard deviation 1).truncnorm: Samples from a truncated normal distribution (mean 0, std. dev. 1), truncated to the range[-1, 1].uniform: Samples from a uniform distribution in the range[-1, 1].
From Quasi-Monte Carlo Sequences:
sobol: Uses Sobol' sequences.halton: Uses Halton sequences.lhs: Uses Latin Hypercube Sampling.
Note: QMC samples are generated in the unit hypercube
[0, 1]^dand then scaled to the hypercube[-1, 1]^d.
Options
Default: nullSpecifies a dict of optional parameters for the sampler backend.
This dict of values is passed unchanged to the selected method in the backend.
Additional Properties of any type are allowed.
Type: objectObjective Functions
Type: arrayList of objective function specifications.
Must contain a minimum of 1 items
Each item of this array must be:
ObjectiveFunctionConfig
Type: objectNo Additional Properties
Name
Type: stringThe name of the objective function.
Weight
Default: nullThe weight determines the importance of an objective function relative to the other objective functions.
Everest optimizes the weighted sum of all objectives. The weights of all objectives are normalized to a total value of one by dividing them by their sum. Weights may be zero or negative, but should add up to a positive value.
Scale
Default: nullOptional scaling of the objective function value.
Each objective function will be divided by this scale value. This can be used to change the overall range of the objective values. For instance, this may be needed to properly balance the relative scales of objectives and output constraints.
Note: This option should be used with care in case of multi-objective optimization, since scaling each objective differently will change their relative weights.
This option will be disabled if auto_scale is set in the
optimization section.
Value must be strictly greater than 0
Type
Type: enum (of string) Default: "mean"How to calculate the objective from realizations.
The objective function value is aggregated from their individual realization values, usually by averaging them. The calculation used for this operation can be modified by setting this field. Currently, the two values are supported:
"mean"(default): calculates the mean over the realizations."stddev": calculates the negative of the standard deviation over the realizations. The negative is used since in general the aim is to minimize the standard deviation (as opposed to the mean, which is preferred to be maximized).
Must be one of:
- "mean"
- "stddev"
OptimizationConfig
Type: objectOptimizer options.
Ensemble-based optimization in Everest is handled internally by the
ropt library for robust optimization, which in turn uses a plugin
mechanism to provide multiple low-level optimization backends. By
default Everest includes backends based on the Dakota optimization
package and on the SciPy package.
Many of the optimization options in this section are handled by
ropt in an uniform way, i.e. they will be handled identically by
the backend optimization algorithm. However, some options may be
handled differently, or not be supported, depending on the
optimization backend used. If this is the case, this will be
indicated in the description of the option.
Algorithm
Type: string Default: "optpp_q_newton"The optimization algorithm used by Everest. Defaults to
optpp_q_newton, a quasi-Newton algorithm in Dakota's OPT++
library.
Convergence Tolerance
Default: nullDefines the threshold value to test for convergence.
In most cases, this is used to set a relative convergence tolerance
for the objective function: if the change in the objective function
between one or more successive iterations divided by the previous
objective function is less than convergence_tolerance, the
optimization is terminated.
Note: this option is passed to the optimization backend unchanged, and therefore its actual interpretation depends on the algorithm used.
Backend
Default: null[Deprecated]
The correct backend will be inferred by the method. If several backends
have a method named A, pick a specific backend B by putting B/A in
the method field.
Backend Options
Default: null[Deprecated]
This field is deprecated. Please use options instead, it will
accept both objects and lists of strings.
Additional Properties of any type are allowed.
Type: objectOptions
Default: nullSpecify options that are passed unchanged to the optimization algorithm.
This field accepts dictionaries with key/values pair or lists of strings. The type required depends on the optimization backend that is used. The Dakota backend requires a list of a strings that are added to the Dakota configuration. The SciPy backend requires a dictionary with values, which will be passed as keyword arguments to the optimizer.
Consult the documentation of the optimization backend for valid options.
No Additional Items
Each item of this array must be:
Additional Properties of any type are allowed.
Type: objectConstraint Tolerance
Default: nullOutput constraint tolerance for Dakota.
Determines the maximum allowable value of infeasibility that any constraint in an optimization problem may possess and still be considered to be satisfied.
It is specified as a positive real value. If a constraint function is greater than this value then it is considered to be violated by the optimization algorithm. This specification gives some control over how tightly the constraints will be satisfied at convergence of the algorithm. However, if the value is set too small the algorithm may terminate with one or more constraints being violated.
This option is only used by the Dakota backend.
Directs the optimizer to use CVaR estimation.
When this section is present, Conditional Value at Risk (CVaR) will
be used to minimize risk. Effectively this means that at each
iteration the objective and constraint functions will be calculated
as the mean over the sub-set of the realizations that perform worst.
The size of this set is specified as an absolute number or as a
percentile value. These options are selected by setting either the
number_of_realizations option, or the percentile option, which
are mutually exclusive.
CVaRConfig
Type: objectNo Additional Properties
Number Of Realizations
Default: nullThe number of realizations used for CVaR estimation.
Sets the number of realizations that is used to calculate the total objective.
This option is mutually exclusive with the percentile option.
Percentile
Default: nullThe percentile used for CVaR estimation.
Sets the percentile of distribution of the objective over the realizations that is used to calculate the total objective.
This option is mutually exclusive with the number_of_realizations
option.
Value must be greater or equal to 0.0 and lesser or equal to 1.0
Max Batch Num
Default: nullLimit the number of batches of simulations.
The optimization will be terminated if the given number of batches has been reached.
Value must be strictly greater than 0
Max Function Evaluations
Default: nullLimits the maximum number of function evaluations.
If the given number of function evaluations is reached, the
optimization will be terminated. This differs from the
max_batch_num and max_iterations options, because these may
require multiple (or no) function evaluations.
Note: Evaluations of perturbed values are not included in this limit.
Value must be strictly greater than 0
Max Iterations
Default: nullLimits the maximum number of iterations.
This limits the number of iterations that the optimization algorithm will perform.
This differs from max_batch_num and max_functions options, since
an iteration may require multiple batches and/or function
evaluations.
This option is passed through unchanged, and its exact interpretation depends on the backend optimization algorithm used.
Value must be strictly greater than 0
Min Pert Success
Default: nullThe number of perturbations that must succeed.
To calculate gradients, the optimizer requests a number of forward model evaluations for perturbed variables. There are two possible ways to calculate the gradient of the objective function:
- For each realization,
perturbation_numobjective functions are evaluated with perturbed controls, which will be used to calculate a gradient for that realization. The gradient calculation for a single realization is considered successful if at leastmin_pert_successforward model runs for that realization are successful. The overall gradient can be calculated from the set of realization gradients if a sufficient number succeeded (see themin_realizations_successoption). If not, the optimization terminates with an error. - If the
perturbation_numfield is equal to 1, the overall gradient is directly calculated from a single perturbation for each realization. In this casemin_pert_successis internally set to 1, and the number of successful realizations is equal to the number of successful perturbed runs. The gradient calculation succeeds if this is at least equal tomin_realizations_success.
Note: The value of min_pert_success is internally adjusted to be
capped by perturbation_num.
Value must be strictly greater than 0
Min Realizations Success
Default: nullMinimum number of successful realizations.
The overall objective functions and gradients are calculated from a set of forward model runs for a number of realizations of an ensemble. This is done by aggregating the functions and gradients, usually by averaging. If one or more of the forward model evaluations fail, this may lead to a failure of either the function or the gradient evaluation. However, the number of realizations that are successfully evaluated may still be sufficient to calculate robust aggregated functions and gradients.
To calculate the aggregated objective function and gradient, the
number of successful forward models that calculate the objective
function for each realization must be at least equal to
min_realizations_success. The number of realizations that are
successful in terms of the gradient calculation is additionally
determined by the min_pert_success field, see its documentation
for more details.
If these requirements are not met, the optimizer will stop and report an error.
Note: The value of min_realizations_success is internally adjusted
to be capped by the number of realizations. It is possible to set
the minimum number of successful realizations equal to zero. Some
optimization algorithms are able to handle this and will proceed
even if all realizations failed. Most algorithms are not capable of
this and will internally adjust the value to be equal to one.
Value must be greater or equal to 0
Perturbation Num
Default: nullThe number of perturbed control vectors per realization.
The number of simulation runs used for estimating the gradient is
equal to the the product of perturbation_num and
model.realizations.
Value must be strictly greater than 0
Speculative
Type: boolean Default: falseCalculate gradients, even if not strictly needed.
When running forward models in parallel, e.g. on a computing cluster, it may be advantageous to calculate a gradient along with each function evaluation. The reason is that the optimizer may be able to use the gradient in a next iteration. Although this wastes resources if the gradient is not used, it may save clock time, since the gradient does not need to be calculated again.
Notes:
- This is mostly useful if a full set of functions and perturbations (for the gradient) can run in parallel. Additional batches may be needed anyway if this is not the case.
- Running an optimization with or without this disabled will yield slightly different results, since the perturbed variables will differ because different (longer) sequences of random numbers will be generated.
- This does potentially waste a lot of computational resources!
Parallel
Type: boolean Default: trueEnable parallel function evaluation.
If set to False, a single function evaluation is performed in each
batch. In case of gradient-free optimizers this can be highly
inefficient, since these tend to need many independent function
evaluations at each iteration. By setting parallel to True,
multiple functions may be evaluated in parallel, if supported by the
optimization algorithm.
The default is to use parallel evaluation if supported.
Auto Scale
Type: boolean Default: falseEnable auto-scaling of objectives and input/output constraints.
If set to True, objectives, input constraints and output
constraints are automatically scaled.
In the case of objectives and output constraints this is done by scaling by the values obtained in the first batch. Input constraints are scaled by dividing by the maximum values of their coefficients or right-hand-sides.
Note: If enabled, the individual scale options for objectives and constraints are not allowed.
ModelConfig
Type: objectConfiguration of the ensemble model that is optimized.
No Additional PropertiesRealizations
Type: array of integerRealizations to use from the ensemble.
This should be a list of realization ID's, indicating which realizations of the ensemble should be used.
The list can be specified directly as a (YAML) list, or as a string consisting of comma separated values and ranges.
For example 1, 2, 5-7, 10 is equivalent to [1, 2, 5, 6, 7, 10].
Must contain a minimum of 1 items
Each item of this array must be:
Value must be greater or equal to 0
Realizations Weights
Type: array of numberA list of realization weights.
If provided, the list must consist of one weight for each
realization (as determined by the realizations field). If not
provided, the weight of each realization is equal to one divided by
the number or realizations.
Each item of this array must be:
EnvironmentConfig
Type: objectThe environment of the optimization run.
This specifies which folders are used for simulation and output, as well as the level of detail in the logs
No Additional PropertiesSimulation Folder
Type: string Default: "simulation_folder"Folder where the forward model outputs are stored.
It the provided path is not absolute, it is assumed to be relative
with respect to the path given by output_folder.
Output Folder
Type: string Default: "everest_output"Folder for Everest output.
Log Level
Type: enum (of string) Default: "info"Defines the verbosity of logs output by Everest.
The default log level is info. The supported log levels are:
debug: Detailed information, typically of interest only when diagnosing problems.info: Confirmation that things are working as expected.warning: An indication that something unexpected happened, or indicative of some problem in the near future (e.g.disk space low). The software is still working as expected.error: Due to a more serious problem, the software has not been able to perform some function.critical: A serious error, indicating that the program will not be able to continue running.
Must be one of:
- "debug"
- "info"
- "warning"
- "error"
- "critical"
Random Seed
Type: integer Default: nullRandom seed.
A positive integer used to seed the random number generator that is used to generate perturbed controls for gradient estimation.
Value must be strictly greater than 0
Wells
Default: nullAn optional list of well configurations.
Each well configuration consists of a name field, and an optional
drill_time field. All variables in control groups with
control.type == "well_control" must also be listed in this
section.
If not present, a minimal well configuration containing only the
names of wells will be derived from controls that have
control.type == "well_control".
No Additional Items
Each item of this array must be:
WellConfig
Type: objectNo Additional Properties
Name
Type: stringThe unique name of the well.
Drill Time
Type: integer Default: 0Specifies the time it takes to drill the well.
Value must be strictly greater than 0
Definitions
Type: objectSection for specifying variables.
Used to specify variables that will be replaced in the file when
encountered as r{{ NAME }}.
Additional Properties of any type are allowed.
Type: objectInput Constraints
Type: arrayA list of input constraint definitions.
Input constraints (linear constraints) are defined by a set of linear equations that must be equal to (or less- or greater-than) a given right-hand-side value. The optimizer will be directed to limit the control variables, during optimization, to the set of of values that obey all of these equations (i.e. find feasible control values).
The values of the linear constraints are calculated directly from the control vector values and do not require the results of the forward model that is being optimized.
How linear constraints are handled depends on the backend optimization algorithm: some may keep the control variables feasible during the entire optimization, some may allow constraint violations at intermediate trial points.
No Additional ItemsEach item of this array must be:
InputConstraintConfig
Type: objectNo Additional Properties
Weights
Type: objectThe coefficients of the linear equation that defines the input constraint.
Note: Although the term "weights" suggests that these coefficients must be normalized, or are internally normalized, this is not the case: they can take any value.
upper_bound: 0.2
weights:
- point_3D.x.0: 1
- point_3D.y.1: 1
- point_3D.z.2: 1
Each additional property must conform to the following schema
Type: numberTarget
Default: nullSets the right-hand-side value for a linear equality constraint.
This will direct the optimizer to limit the control values during
optimization to the set of values where the sum of the controls
multiplied by their coefficients (weights) is equal to target.
Lower Bound
Type: number Default: -InfinitySets the right-hand-side value for a linear equality constraint.
This will direct the optimizer to limit the control values during
optimization to the set of values where the sum of the controls
multiplied by their coefficients (weights) is equal to, or greater
than lower_bound.
Upper Bound
Type: number Default: InfinitySets the right-hand-side value for a linear equality constraint.
This will direct the optimizer to limit the control values during
optimization to the set of values where the sum of the controls
multiplied by their coefficients (weights) is equal to, or less
than upper_bound.
Scale
Default: nullScaling of input constraints.
scale is a normalization factor which can be used to scale the
input constraint. The bounds or target, and the weights will be
scaled with this number.
This option will be disabled if auto_scale is set in the
optimization section.
Value must be strictly greater than 0
Output Constraints
Type: arrayA list of output constraint definitions.
Output constraints (non-linear constraints) are defined by a set of arbitrary functions that must be equal to (or less- or greater-than) a given right-hand-side value. The optimizer will attempt to find a solution that maximizes the objective function and does not violate these constraints.
The values of the non-linear constraints must be provided by the forward model that is being optimized.
How non-linear constraints are handled depends on the backend optimization algorithm: in most cases some intermediate trial points will violate the constraint.
No Additional ItemsEach item of this array must be:
OutputConstraintConfig
Type: objectNo Additional Properties
Name
Type: stringThe unique name of the output constraint.
Target
Default: nullDefines the equality constraint
f(x) = b,
where f is a function of the control vector x and b is the target.
Lower Bound
Type: number Default: -InfinityDefines the equality constraint
f(x) >= b,
where f is a function of the control vector x and b is the lower bound.
Upper Bound
Type: number Default: InfinityDefines the equality constraint
f(x) <= b,
where f is a function of the control vector x and b is the upper bound.
Scale
Default: nullScaling of constraints.
scale is a normalization factor which can be used to scale the
constraint to control its importance relative to the objective.
Both the bounds or target and the function evaluation value will be scaled with this number. For example, if the upper bound is 0.5 and the scaling is 10, then the function evaluation value will be divided by 10 and bounded from above by 0.05.
This option will be disabled if auto_scale is set in the
optimization section.
Value must be strictly greater than 0
Install Jobs
Type: arrayA list of jobs to install.
This defines the set of executables that can be referenced in the
forward_model section.
Each item of this array must be:
InstallJobConfig
Type: objectNo Additional Properties
Name
Type: stringThe name of the installed job.
This name is used to identify the job in the list of jobs to be
executed, as defined by the forward_model field.
Source
Default: nullThe source file of the ert job.
source is deprecated, please use executable instead.
Executable
Default: nullThe executable linked to the job name.
Install Workflow Jobs
Type: arrayA list of workflow jobs to install.
This defines the set of executables that can be referenced in the
workflows section.
Each item of this array must be:
InstallJobConfig
Type: objectNo Additional Properties
Name
Type: stringThe name of the installed job.
This name is used to identify the job in the list of jobs to be
executed, as defined by the forward_model field.
Source
Default: nullThe source file of the ert job.
source is deprecated, please use executable instead.
Executable
Default: nullThe executable linked to the job name.
Install Data
Type: arrayA list of directories and/or files to copy or link to the directory where the evaluation jobs will run.
No Additional ItemsEach item of this array must be:
InstallDataConfig
Type: objectNo Additional Properties
Source
Default: nullPath to file or directory that needs to be copied or linked into the directory where the forward model executes.
Target
Type: stringRelative destination path to copy or link the given source to.
Link
Type: boolean Default: falseIf set to true will create a link to the given source at the given target, if not set the source will be copied at the given target.
Data
Default: nullData to be exported to the target file.
If provided, the data is exported to a JSON file. If target has no
.json extension, it is added.
The data and source fields are mutually exclusive.
If data is provided, link will be ignored.
Additional Properties of any type are allowed.
Type: objectInstall Templates
Type: arrayGenerate files using a jinja2 template.
A list of template files to be rendered to an output file. Extra JSON or YAML files can be provided with data that will be exposed as variables to the jinja2 renderer.
Example:
Given two files my_input.json and tmpl.jinja, stored in a
files directory:
files/my_input.json:
{
"my_variable": "my_value"
}
and files/tmpl.jinja:
This is written in my file: {{my_input.my_variable}}
Add this to the config file:
install_data:
- source: files
target: files
install_templates:
- template: files/tmpl.jinja
output_file: my_output.txt
extra_data: files/my_input.json
This will produce an output file with the content:
This is written in my file: my_value
Each item of this array must be:
InstallTemplateConfig
Type: objectNo Additional Properties
Template
Type: stringThe jinja2 template file.
Output File
Type: stringThe name of the output file.
Extra Data
Default: nullExtra input files.
The content of each extra JSON or YAML file is exposed to the jinja2 renderer, using the name of the file as a variable name.
ServerConfig
Type: objectDefines Everest server settings, i.e., which queue system, queue name and queue options are used for the everest server. This makes it possible to reduce the resource requirements for the server that tend to be low compared with the forward models. This may prevent situations where everest times out because it can not add the server to the queue.
Queue system and queue name defaults to the same as simulator, and
the server should not need to be configured by most users. This is
also true for the --include-host and --exclude-host options that are
used by the SLURM driver.
Note that changing values in this section has no impact on the resource requirements of the forward models.
No Additional PropertiesQueue System
Default: nullDefines which queue system the everest server is submitted to.
For example, to run the server locally even if the forward model may run on another system:
server:
queue_system:
name: local
LocalQueueOptions
Type: objectNo Additional Properties
Name
Type: const Default: "local"Specific value:
"local"
Max Running
Type: integer Default: 0Value must be greater or equal to 0
Submit Sleep
Type: number Default: 0.0Value must be greater or equal to 0
Num Cpu
Type: integer Default: 1Value must be greater or equal to 0
Realization Memory
Type: integer Default: 0When set, a job is only allowed to use realization_memory of memory.
realization_memory may be an integer value, indicating the number of bytes, or a
string consisting of a number followed by a unit. The unit indicates the
multiplier that is applied, and must start with one of these characters:
* b, B: bytes
* k, K: kilobytes (1024 bytes)
* m, M: megabytes (1024**2 bytes)
* g, G: gigabytes (1024**3 bytes)
* t, T: terabytes (1024**4 bytes)
* p, P: petabytes (1024**5 bytes)
Spaces between the number and the unit are ignored, and so are any
characters after the first. For example: 2g, 2G, and 2 GB all resolve
to the same value: 2 gigabytes, equaling 2 * 1024**3 bytes.
If not set, or set to zero, the allowed amount of memory is unlimited.
Value must be greater or equal to 0
Job Script
Type: string Default: "/home/docs/checkouts/readthedocs.org/user_builds/everest/envs/16.0.7/bin/fm_dispatch.py"Project Code
Default: nullActivate Script
Default: nullLsfQueueOptions
Type: objectNo Additional Properties
Name
Type: const Default: "lsf"Specific value:
"lsf"
Max Running
Type: integer Default: 0Value must be greater or equal to 0
Submit Sleep
Type: number Default: 0.0Value must be greater or equal to 0
Num Cpu
Type: integer Default: 1Value must be greater or equal to 0
Realization Memory
Type: integer Default: 0When set, a job is only allowed to use realization_memory of memory.
realization_memory may be an integer value, indicating the number of bytes, or a
string consisting of a number followed by a unit. The unit indicates the
multiplier that is applied, and must start with one of these characters:
* b, B: bytes
* k, K: kilobytes (1024 bytes)
* m, M: megabytes (1024**2 bytes)
* g, G: gigabytes (1024**3 bytes)
* t, T: terabytes (1024**4 bytes)
* p, P: petabytes (1024**5 bytes)
Spaces between the number and the unit are ignored, and so are any
characters after the first. For example: 2g, 2G, and 2 GB all resolve
to the same value: 2 gigabytes, equaling 2 * 1024**3 bytes.
If not set, or set to zero, the allowed amount of memory is unlimited.
Value must be greater or equal to 0
Job Script
Type: string Default: "/home/docs/checkouts/readthedocs.org/user_builds/everest/envs/16.0.7/bin/fm_dispatch.py"Project Code
Default: nullActivate Script
Default: nullBhist Cmd
Default: nullMust be at least 1 characters long
Bjobs Cmd
Default: nullMust be at least 1 characters long
Bkill Cmd
Default: nullMust be at least 1 characters long
Bsub Cmd
Default: nullMust be at least 1 characters long
Exclude Host
Default: nullLsf Queue
Default: nullMust be at least 1 characters long
Lsf Resource
Default: nullSlurmQueueOptions
Type: objectNo Additional Properties
Name
Type: const Default: "slurm"Specific value:
"slurm"
Max Running
Type: integer Default: 0Value must be greater or equal to 0
Submit Sleep
Type: number Default: 0.0Value must be greater or equal to 0
Num Cpu
Type: integer Default: 1Value must be greater or equal to 0
Realization Memory
Type: integer Default: 0When set, a job is only allowed to use realization_memory of memory.
realization_memory may be an integer value, indicating the number of bytes, or a
string consisting of a number followed by a unit. The unit indicates the
multiplier that is applied, and must start with one of these characters:
* b, B: bytes
* k, K: kilobytes (1024 bytes)
* m, M: megabytes (1024**2 bytes)
* g, G: gigabytes (1024**3 bytes)
* t, T: terabytes (1024**4 bytes)
* p, P: petabytes (1024**5 bytes)
Spaces between the number and the unit are ignored, and so are any
characters after the first. For example: 2g, 2G, and 2 GB all resolve
to the same value: 2 gigabytes, equaling 2 * 1024**3 bytes.
If not set, or set to zero, the allowed amount of memory is unlimited.
Value must be greater or equal to 0
Job Script
Type: string Default: "/home/docs/checkouts/readthedocs.org/user_builds/everest/envs/16.0.7/bin/fm_dispatch.py"Project Code
Default: nullActivate Script
Default: nullSbatch
Type: string Default: "sbatch"Must be at least 1 characters long
Scancel
Type: string Default: "scancel"Must be at least 1 characters long
Scontrol
Type: string Default: "scontrol"Must be at least 1 characters long
Sacct
Type: string Default: "sacct"Must be at least 1 characters long
Squeue
Type: string Default: "squeue"Must be at least 1 characters long
Exclude Host
Type: string Default: ""Include Host
Type: string Default: ""Partition
Default: nullMust be at least 1 characters long
Squeue Timeout
Type: number Default: 2Value must be strictly greater than 0
Max Runtime
Default: nullValue must be greater or equal to 0
TorqueQueueOptions
Type: objectNo Additional Properties
Name
Type: const Default: "torque"Specific value:
"torque"
Max Running
Type: integer Default: 0Value must be greater or equal to 0
Submit Sleep
Type: number Default: 0.0Value must be greater or equal to 0
Num Cpu
Type: integer Default: 1Value must be greater or equal to 0
Realization Memory
Type: integer Default: 0When set, a job is only allowed to use realization_memory of memory.
realization_memory may be an integer value, indicating the number of bytes, or a
string consisting of a number followed by a unit. The unit indicates the
multiplier that is applied, and must start with one of these characters:
* b, B: bytes
* k, K: kilobytes (1024 bytes)
* m, M: megabytes (1024**2 bytes)
* g, G: gigabytes (1024**3 bytes)
* t, T: terabytes (1024**4 bytes)
* p, P: petabytes (1024**5 bytes)
Spaces between the number and the unit are ignored, and so are any
characters after the first. For example: 2g, 2G, and 2 GB all resolve
to the same value: 2 gigabytes, equaling 2 * 1024**3 bytes.
If not set, or set to zero, the allowed amount of memory is unlimited.
Value must be greater or equal to 0
Job Script
Type: string Default: "/home/docs/checkouts/readthedocs.org/user_builds/everest/envs/16.0.7/bin/fm_dispatch.py"Project Code
Default: nullActivate Script
Default: nullQsub Cmd
Default: nullMust be at least 1 characters long
Qstat Cmd
Default: nullMust be at least 1 characters long
Qdel Cmd
Default: nullMust be at least 1 characters long
Queue
Default: nullMust be at least 1 characters long
Cluster Label
Default: nullMust be at least 1 characters long
Job Prefix
Default: nullMust be at least 1 characters long
Keep Qsub Output
Type: boolean Default: falseSimulatorConfig
Type: objectDefines simulator settings, i.e., which queue system, queue name and queue options are used for running the forward models.
No Additional Propertiesqueue_system:
max_running: 3
name: local
Cores Per Node
Default: nullDefines the number of CPUs when running the forward models. This can for example be used in conjunction with the Eclipse parallel keyword for multiple CPU simulation runs.
This keyword has no effect when running with the local queue.
Value must be strictly greater than 0
Delete Run Path
Type: boolean Default: falseIf True, delete the output folder of a successful forward model run.
Max Runtime
Default: nullMaximum allowed running time for each individual forward model job specified in seconds.
A value of 0 means unlimited runtime.
Value must be greater or equal to 0
Max Memory
Default: nullMaximum allowed memory usage of a forward model.
When set, a job is only allowed to use max_memory of memory.
max_memory may be an integer value, indicating the number of
bytes, or a string consisting of a number followed by a unit. The
unit indicates the multiplier that is applied, and must start with
one of these characters:
- b, B: bytes
- k, K: kilobytes (1024 bytes)
- m, M: megabytes (1024**2 bytes)
- g, G: gigabytes (1024**3 bytes)
- t, T: terabytes (1024**4 bytes)
- p, P: petabytes (1024**5 bytes)
Spaces between the number and the unit are ignored, and so are any characters after the first. For example: 2g, 2G, and 2 GB all resolve to the same value: 2 gigabytes, equaling 2 * 1024**3 bytes.
If not set, or a set to zero, the allowed amount of memory is unlimited.
Queue System
Default: nullDefines which queue system the everest submits jobs to.
LocalQueueOptions
Type: objectNo Additional Properties
Name
Type: const Default: "local"Specific value:
"local"
Max Running
Type: integer Default: 0Value must be greater or equal to 0
Submit Sleep
Type: number Default: 0.0Value must be greater or equal to 0
Num Cpu
Type: integer Default: 1Value must be greater or equal to 0
Realization Memory
Type: integer Default: 0When set, a job is only allowed to use realization_memory of memory.
realization_memory may be an integer value, indicating the number of bytes, or a
string consisting of a number followed by a unit. The unit indicates the
multiplier that is applied, and must start with one of these characters:
* b, B: bytes
* k, K: kilobytes (1024 bytes)
* m, M: megabytes (1024**2 bytes)
* g, G: gigabytes (1024**3 bytes)
* t, T: terabytes (1024**4 bytes)
* p, P: petabytes (1024**5 bytes)
Spaces between the number and the unit are ignored, and so are any
characters after the first. For example: 2g, 2G, and 2 GB all resolve
to the same value: 2 gigabytes, equaling 2 * 1024**3 bytes.
If not set, or set to zero, the allowed amount of memory is unlimited.
Value must be greater or equal to 0
Job Script
Type: string Default: "/home/docs/checkouts/readthedocs.org/user_builds/everest/envs/16.0.7/bin/fm_dispatch.py"Project Code
Default: nullActivate Script
Default: nullLsfQueueOptions
Type: objectNo Additional Properties
Name
Type: const Default: "lsf"Specific value:
"lsf"
Max Running
Type: integer Default: 0Value must be greater or equal to 0
Submit Sleep
Type: number Default: 0.0Value must be greater or equal to 0
Num Cpu
Type: integer Default: 1Value must be greater or equal to 0
Realization Memory
Type: integer Default: 0When set, a job is only allowed to use realization_memory of memory.
realization_memory may be an integer value, indicating the number of bytes, or a
string consisting of a number followed by a unit. The unit indicates the
multiplier that is applied, and must start with one of these characters:
* b, B: bytes
* k, K: kilobytes (1024 bytes)
* m, M: megabytes (1024**2 bytes)
* g, G: gigabytes (1024**3 bytes)
* t, T: terabytes (1024**4 bytes)
* p, P: petabytes (1024**5 bytes)
Spaces between the number and the unit are ignored, and so are any
characters after the first. For example: 2g, 2G, and 2 GB all resolve
to the same value: 2 gigabytes, equaling 2 * 1024**3 bytes.
If not set, or set to zero, the allowed amount of memory is unlimited.
Value must be greater or equal to 0
Job Script
Type: string Default: "/home/docs/checkouts/readthedocs.org/user_builds/everest/envs/16.0.7/bin/fm_dispatch.py"Project Code
Default: nullActivate Script
Default: nullBhist Cmd
Default: nullMust be at least 1 characters long
Bjobs Cmd
Default: nullMust be at least 1 characters long
Bkill Cmd
Default: nullMust be at least 1 characters long
Bsub Cmd
Default: nullMust be at least 1 characters long
Exclude Host
Default: nullLsf Queue
Default: nullMust be at least 1 characters long
Lsf Resource
Default: nullSlurmQueueOptions
Type: objectNo Additional Properties
Name
Type: const Default: "slurm"Specific value:
"slurm"
Max Running
Type: integer Default: 0Value must be greater or equal to 0
Submit Sleep
Type: number Default: 0.0Value must be greater or equal to 0
Num Cpu
Type: integer Default: 1Value must be greater or equal to 0
Realization Memory
Type: integer Default: 0When set, a job is only allowed to use realization_memory of memory.
realization_memory may be an integer value, indicating the number of bytes, or a
string consisting of a number followed by a unit. The unit indicates the
multiplier that is applied, and must start with one of these characters:
* b, B: bytes
* k, K: kilobytes (1024 bytes)
* m, M: megabytes (1024**2 bytes)
* g, G: gigabytes (1024**3 bytes)
* t, T: terabytes (1024**4 bytes)
* p, P: petabytes (1024**5 bytes)
Spaces between the number and the unit are ignored, and so are any
characters after the first. For example: 2g, 2G, and 2 GB all resolve
to the same value: 2 gigabytes, equaling 2 * 1024**3 bytes.
If not set, or set to zero, the allowed amount of memory is unlimited.
Value must be greater or equal to 0
Job Script
Type: string Default: "/home/docs/checkouts/readthedocs.org/user_builds/everest/envs/16.0.7/bin/fm_dispatch.py"Project Code
Default: nullActivate Script
Default: nullSbatch
Type: string Default: "sbatch"Must be at least 1 characters long
Scancel
Type: string Default: "scancel"Must be at least 1 characters long
Scontrol
Type: string Default: "scontrol"Must be at least 1 characters long
Sacct
Type: string Default: "sacct"Must be at least 1 characters long
Squeue
Type: string Default: "squeue"Must be at least 1 characters long
Exclude Host
Type: string Default: ""Include Host
Type: string Default: ""Partition
Default: nullMust be at least 1 characters long
Squeue Timeout
Type: number Default: 2Value must be strictly greater than 0
Max Runtime
Default: nullValue must be greater or equal to 0
TorqueQueueOptions
Type: objectNo Additional Properties
Name
Type: const Default: "torque"Specific value:
"torque"
Max Running
Type: integer Default: 0Value must be greater or equal to 0
Submit Sleep
Type: number Default: 0.0Value must be greater or equal to 0
Num Cpu
Type: integer Default: 1Value must be greater or equal to 0
Realization Memory
Type: integer Default: 0When set, a job is only allowed to use realization_memory of memory.
realization_memory may be an integer value, indicating the number of bytes, or a
string consisting of a number followed by a unit. The unit indicates the
multiplier that is applied, and must start with one of these characters:
* b, B: bytes
* k, K: kilobytes (1024 bytes)
* m, M: megabytes (1024**2 bytes)
* g, G: gigabytes (1024**3 bytes)
* t, T: terabytes (1024**4 bytes)
* p, P: petabytes (1024**5 bytes)
Spaces between the number and the unit are ignored, and so are any
characters after the first. For example: 2g, 2G, and 2 GB all resolve
to the same value: 2 gigabytes, equaling 2 * 1024**3 bytes.
If not set, or set to zero, the allowed amount of memory is unlimited.
Value must be greater or equal to 0
Job Script
Type: string Default: "/home/docs/checkouts/readthedocs.org/user_builds/everest/envs/16.0.7/bin/fm_dispatch.py"Project Code
Default: nullActivate Script
Default: nullQsub Cmd
Default: nullMust be at least 1 characters long
Qstat Cmd
Default: nullMust be at least 1 characters long
Qdel Cmd
Default: nullMust be at least 1 characters long
Queue
Default: nullMust be at least 1 characters long
Cluster Label
Default: nullMust be at least 1 characters long
Job Prefix
Default: nullMust be at least 1 characters long
Keep Qsub Output
Type: boolean Default: falseResubmit Limit
Type: integer Default: 1Specifies how many times a forward model may be submitted to the queue system.
A forward model may fail for external reasons, examples include node in
the cluster crashing, network issues, etc. Therefore, it might make
sense to resubmit a forward model in case it fails. resubmit_limit
defines the number of times we will resubmit a failing forward model. If
not specified, a default value of 1 will be used.
Value must be greater or equal to 0
Forward Model
Type: arrayThe list of jobs to run.
No Additional ItemsEach item of this array must be:
ForwardModelStepConfig
Type: objectJob
Type: stringThe forward model step to execute.
A string that consists of the name of the job, followed by zero or more job options.
Results
Default: nullSummaryResults
Type: objectFile Name
Type: stringThe output file produced by the forward model.
Type
Type: constSpecific value:
"summary"
Keys
Default: "*"The list of keys to include in the result.
Defaults to '*' indicating all keys.
Specific value:
"*"
No Additional Items
Each item of this array must be:
GenDataResults
Type: objectFile Name
Type: stringThe output file produced by the forward model.
Type
Type: constSpecific value:
"gen_data"
WorkflowConfig
Type: objectOptional workflow jobs to run during optimization.
No Additional PropertiesPre Simulation
Type: array of stringThe list of workflow jobs triggered pre-simulation.
Each entry consists of the name of the job, followed by zero or more job options.
No Additional ItemsEach item of this array must be:
Post Simulation
Type: array of stringThe list of workflow jobs triggered post-simulation.
Each entry consists of the name of the job, followed by zero or more job options.
No Additional ItemsEach item of this array must be:
ExportConfig
Type: objectSettings to control the exports of an optimization run by everest.
No Additional PropertiesKeywords
Type: array of stringA list of Eclipse keywords to export.
No Additional Items