`ensemble`

Create an optimized test suite (a.k.a "ensemble") from a scenarios file.

Synopsys

provengo ensemble [--size=200] <path-to-project>

Description

Ensembles are collections of scenarios, optimized to cover specific intentions. The optimization is done using a user-provided function, which takes a collection of scenarios and returns a numeric score for said collection.

For example, the following function prefers test ensemble that have many SELL_COMPLETED events:

const SELL_COMPLETED = ...; (1)

function rankingFunction( ensemble ) { (2)
  let hits = 0;
  for ( let test of ensemble ) { (3)
    for ( let event of test ) {
      if ( SELL_COMPLETED.contains(event) ) {
        hits = hits + 1; (4)
      }
    }
  }
  return hits; (5)
}

1	An event / set of events we want to focus on
2	Defining the ranking function used to score ensembles
3	A double-nested loop, going over all events in all scenarios in the ensemble candidate
4	If an event of interest is detected, increase the score
5	Return the score to Provengo’s algorithm.

The presented function is a very simple one, intended for explanation purposes. The logic can be as complex (or as simple!) as needed.

Parameters

-a/--algorithm: Optimizer type options: brute-force, genetic, goals. Defaults to genetic.
--size: Number of required tests in the generated test suite.
-s/--run-source: Name of the file containing scenarios.
-o/--output-file: Name of file to store the generated ensemble in.
--iterations: Number of algorithmic iterations for generating the test suite.
--ranking-function: Ranking function name. Defaults to rankingFunction.
--crossover-prob: Probability of executing a crossover combinator (genetic optimizer only).
--mutation-prob: Probability of executing a mutation mutator (genetic optimizer only).
--num-of-generations: Number of generations to execute (genetic optimizer only).
--max-phenotype-age: Maximal phenotype age (genetic optimizer only).