Project Configuration

A Provengo project might have many settings and parameters. The configuration system allows setting these parameters using a key-value interface. A value of a given parameter can be specified in the following ways:

Using a commandline parameter (e.g. provengo --output-dir=/var/test1 …)
Using an environment variable (e.g. PROVENGO_viewer_html)
Using provengo.yml, a configuration file. Setting a value can be done either at the configuration’s top level, or in a profile.
Using a default value

The full list of configuration keys can be found here.

When looking up a parameter’s value, Provengo’s configuration system will search the above locations using the above specified order, stopping once the sought value was found. For example, if a parameter is specified both as an environment variable and in the configuration file, the value specified by the environment variable will be used.

Figure 1. Lookup Order

Use this lookup order to easily test various settings using the commandline. When the correct value is found, to store it in the configuration file.

Effective Configuration: The project’s Effective Configuration for a given run is the set of configuration key-value pairs that Provengo Tool uses in that run, after it completed the configuration keys lookup process.

Environment Variables

Provengo pulls values from environment variables that begin with PROVENGO_ into its effective configuration. In order to play nice with shells and scripts, the . character is replaced with _. For example, the PROVENGO_viewer_html environment variable will set the viewer.html configuration key.

Example: Setting the ensemble iteration count via an environment variable.

$ PROVENGO_ensemble_iterations=10000 provengo ensemble MyProj

Configuration Files

Each Provengo project has a configuration file called provengo.yml in its conf folder. This is a YAML file, listing configuration keys and their values. The full list of keys and their meaning can be found here.

Keys are composed of names concatenated with dots, e.g. area.detail.parameter: value. For convenience, brevity, and readability, it is possible to nest YAML maps to compose the keys, like so:

Same configuration, different key notations

# Explicit (1)
ensemble.iterations: 2000
ensemble.suite-size: 20
ensemble.genetic.generation-attempts: 3000

# Nested  (2)
ensemble:
    iterations: 2000
    suite-size: 20
    genetic.generation-attempts: 3000 (3)

1	In explicit notations, each row has all the key parts, concatenated by dots.
2	In nested notations, the keys from the map’s path are concatenated to form the name. This saves typing and groups related parameters together.
3	You can combine nested and explicit notations.

Below is a sample configuration file defining various execution parameters, such as output file location, Selenium modes, and ensemble parameters.

Sample Configuration File

version: 2
selenium:
    server: none
    headless: yes
output.file: /var/sys1/ (1)
source:
    highlight: products/run-source/garbanzo-19.json(2)
ensemble:
    iterations: 5000
    genetic.generation-attempts: 3000

1	An absolute path
2	A relative path, resolved from the project directory.

Paths that begin with / are absolute. Otherwise, paths are relative to the project folder. For example, if a project is located at /home/jane/p1, these two paths are resolve to the same location:

/home/jane/p1/out
out

Profiles

Profiles allow users to tweak a project settings in order to fit a specific scenario. For example, consider a project has to test system for two different interest:

General errors throughout the app ("smoke tests"), and
Specific errors in a funds management area.

These different targets imply different optimization goal for generated test suites. So, the project can be configured with two profiles, each one specifying different sample and ensemble parameters.

Profiles override settings specified in the main file, or in their parent profile.

The settings at the top level to a provengo.yml file can be viewed as the project’s default profile.

Below is a sample configuration file for the project mentioned above.

Sample provengo.yml with profiles

version: 2
ensemble.ranking-function: maxSpread
ensemble.suite-size: 2000
ensemble.iterations: 4000
profiles:
    funds:
        ensemble.ranking-function: fundsArea
    quick_funds:
        parent: funds
        ensemble:
            iterations: 50
            suite-size: 100

The above provengo.yml, contains two profiles. The default behavior performs a rigorous smoke test, as it scores ensembles using the maxSpread functions (which, as its name implies, prefers ensembles that test aspects spread across the application).

The funds profile performs a rigorous testing of the funds management parts of the application. It only overrides the scoring function setting, but leaves the ensemble creations parameters unchanged.

Lastly, the quickFunds profile performs a quick test of the funds area. Since is declares funds as its parent profile, it only needs to alter the ensemble creation parameters to achieve this.