Scenario Objects

To a very large extent, Provengo is a tool for creating, analyzing, and reasoning about scenarios that a specification allows. Because scenarios are so central, Provengo provides built-in objects and methods to easily work with them. This section describes these objects and how to work with them.

The main class is Scenario. Object of this class contains a sequence of events (basically, the run through the Provengo model), and some metadata, such as title and a set of tags. Setting the metadata of a scenario is done by the function specified by the scenario.generator.metadata configuration key. This function accepts a Scenario object, accesses its event sequence, and then assigns it a title and assigns tags accordingly.

By default, adding metadata to scenarios is done the by the addMetadata() function, in the meta-spec/scenario-metadata.js file. This function calls default implementations for adding tags and titles. If the default methods work for you, there’s no need to change anything.

Objects and Methods

addMetadata(aScenario)

Receives a Scenario object, sets its title sand assigns tags to it. This function is called by the Provengo tool for each executed scenario. To change the name metadata function, use the scenario.generator.metadata configuration key.

aScenario

Scenario object to be tagged and titled.

Returns: nothing.

Scenario Object

A Scenario object represents a single run of the model. In most cases, a single scenario describes a single test case. A scenario contains a series of events, starting at the model’s start node and ending in a node with no outbound arrows. A Scenario object also contains metadata — a title and a set of tags.

scn.addTag(key, value)

Adds a tag to the scenario. Tags are composed of a key and value. The key is an array of strings (which may be empty), and the value is a string. This allows users to create structured tag hierarchies, which can be later used for more structured reporting and machine learning.

key

Optional string or string array. The tag’s "name". If omitted, an empty array is used and the tag in interpreted as a "label" - a "boolean" value that is either present or absent.

value

string. The value of the tag.

Returns: nothing.

Example

Consider a test model that involves costly API calls, for example to an external service that bills for every call. We’d like to know which tests are going to be costly to run, and which will be cheaper. For this, we count the number of API calls, and tag scenarios accordingly. The snippet below shows three examples for creating such a tag.

For this function to be called as a metadata generator, its name should be added to the project config under the scenario.generator.metadata config key. 
function customAddMetadata( aScenario ) {
    let apiCalls = countApiCalls(aScenario.events); (1)
    if ( apiCalls > 10 ) {                 (2)
        aScenario.addTag("api-heavy");
    } else {
        aScenario.addTag("api-light");
    }

    aScenario.addTag("apiCalls", String(apiCalls));           (3)
    aScenario.addTag(["api", "callCount"], String(apiCalls)); (4)
}

  1. Counting costly API calls. Implementation not shown, but it basically iterates over the scenario’s events field, looks for API call events, and counts them.

  2. Option 1: Labeling scenarios as api-heavy or api-light.

  3. Option 2: Labeling each scenario with the exact call count, under the key apiCalls.

  4. Option 3: Labeling each scenario with the exact call count, under the hierarchical key ["api","callCount"].

scn.events

An array of Event objects forming the scenario.

Example

Printing the scenario’s events to the log.

for ( let idx=0; idx<scenario.events.length; idx++ ) {
    bp.log.info( scenario.events[idx] );
}

scn.length

number. The number of events in the scenario.

Example

Printing the scenario’s length to the log.

function logScenarioLength( aScenario ) {
    bp.log.info( `This scenario has ${aScenario.length} events`);
}

scn.title

string. The title of the scenario.

Example

Printing the scenario title to the log, in a crocodile’s speech bubble. This is useful in many cases (we’re just not sure which cases. But it’s useful).

function crocTitle( aScenario ) {
    let bubbleEdge = " ".padEnd(aScenario.title.length+3,"-");
    bp.log.info(bubbleEdge);
    bp.log.info(`( ${aScenario.title} )`);
    bp.log.info(bubbleEdge);
    bp.log.info("\\ ");
    bp.log.info("  \\ ");
    bp.log.info("      _____--\\__A_A_A_A_A_A_A_A_A_A_A ");
    bp.log.info("     (____ oo                        \\A_A_A_A_A_A_A_A_ ");
    bp.log.info("     (____     /           /                          \\ ");
    bp.log.info("          \\__ /   /_______/   /________________________\\ ");
    bp.log.info("             /,,,/ \\__\\  /,,,/ \\__\\ ");
}

Sample result:

[RUN>random] INFO B-program started
[RUN>random] INFO Selected: [Greeting: Howdy {JS_Obj type:"selection", name:"Greeting", value:"Howdy", lib:"bp-base"}]
[RUN>random] INFO Selected: [Choice: Venus {JS_Obj type:"choice", value:"Venus", lib:"bp-base"}]
[RUN>random] INFO Selected: [Howdy, Venus]
[RUN>random] INFO Super-step done
[RUN>random] INFO [BP][Info]  -----------------------
[RUN>random] INFO [BP][Info] ( Greeting Howdy, Venus )
[RUN>random] INFO [BP][Info]  -----------------------
[RUN>random] INFO [BP][Info] \
[RUN>random] INFO [BP][Info]   \
[RUN>random] INFO [BP][Info]       _____--\__A_A_A_A_A_A_A_A_A_A_A
[RUN>random] INFO [BP][Info]      (____ oo                        \A_A_A_A_A_A_A_A_
[RUN>random] INFO [BP][Info]      (____     /           /                          \
[RUN>random] INFO [BP][Info]           \__ /   /_______/   /________________________\
[RUN>random] INFO [BP][Info]              /,,,/ \__\  /,,,/ \__\
[RUN  ] INFO Test Result: SUCCESS

ScenarioUtils Object

A utility object storing methods for working with scenarios.

ScenarioUtils.addMetadata(aScenario)

A default implementation for adding metadata to scenarios. Assigns both tags and title, based on standard Provengo events such as choose() and select().from(), and on standard libraries such as State Machines and Combi.

aScenario

Scenario object. The scenario to be tagged and titled.

Returns: true if either tags or title was added, false otherwise.

Example

Using the ScenarioUtils.addMetadata function as a fallback when custom logic does not yield metadata.

For this function to be called as a metadata generator, its name should be added to the project config under the scenario.generator.metadata config key. 
function customAddMetadata( aScenario ) {
    if ( ! generateSpecificMetadata(aScenario) ) {  (1)
        ScenarioUtils.addMetadata(aScenario);
    }
}

  1. generateSpecificMetadata is defined somewhere else in the project.

ScenarioUtils.autoTag(aScenario)

Adds tags to a scenario, based on events from standard libraries and bp-base.

aScenario

Scenario object. The scenario to be tagged.

Returns: true if one or more tags were added, false otherwise.

Example

Using the ScenarioUtils.autoTag function to add an initial set of tags.

For this function to be called as a metadata generator, its name should be added to the project config under the scenario.generator.metadata config key. 
function customAddMetadata( aScenario ) {
    ScenarioUtils.autoTag(aScenario);
    if ( aScenario.length > 20 ) {
        aScenario.addTag("Lengthy");
    }
}

ScenarioUtils.autoTitle(aScenario)

Adds a title to the passed scenario, based on events from standard libraries and bp-base.

aScenario

Scenario object. The scenario to be tagged.

Returns: true if the title was set, false otherwise.

Example

Using the ScenarioUtils.autoTitle function to add a title to a scenario, then adding the event count to the scenario’s title.

For this function to be called as a metadata generator, its name should be added to the project config under the scenario.generator.metadata config key. 
function customAddMetadata( aScenario ) {
    if ( ! ScenarioUtils.autoTitle(aScenario) ) {
        aScenario.title="Test scenario"; // fallback
    }
    aScenario.title = aScenario.title + ` (${aScenario.length})`';
}