`run`

Execute tests specified by the project, and store the results in the project’s run log database.

Synopsis

provengo run [--PARAMS] [--run-source PATH] <path-to-project>

Description

Execute test scenarios directly on the tested system, using scenarios defined by the Provengo project. Test scenarios come from a run source, which is specified by the optional Run Source parameter (--run-source/-s). When no run source is specified, provengo derives a random scenario from the current model (effectively defaulting to --run-source :random). Execution results are stored in the project’s test execution database.

In addition to executing scenarios, the Provengo tool can run various Lifecycle Scripts for set-up, tear-down, etc.

Run Source

Several run sources are available:

:random: The default mode. Randomly selects a single run from the model. This source is useful while developing the model, or when trying out some code changes. Often coupled with --dry to prevent actuations from actually happening.
:log: Re-runs a previous run from the run log database. The run id must be specified, using the --run-id parameter.
filename.json: A run file. These are commonly generated by the sample and ensemble run modes, or are downloaded from an execution report. It it possible to run only a specific run from the file, using --run-id and the run’s number (1-based).

Examples

Run a random test

These type of runs are often useful when you start working on a model, and want to quickly check how it behaves.

The following lines are all equivalent

provengo run --run-source :random path/to/project (1)
provengo run -s :random path/to/project (2)
provengo run path/to/project (3)

1	Specifying the run source to be a random run.
2	Specifying the run source to be a random run using abbreviated syntax.
3	Not specifying the run source, so it defaults to `:random`.

Same as above, but skip actuations. Useful when you want to see what would happen if you ran the tests, but not actually have that happen.

provengo run --dry path/to/project

Run Tests from a File

Execute all tests stored in customer-area-checks.json.

provengo run -s customer-area-checks.json path/to/project

Execute test number 42 from file customer-area-checks.json. The number of the first run is 1.

provengo run -s customer-area-checks.json --run-id 42 path/to/project

Re-Run a Test from the Log

Each run in the log has an id. This id is shown in the Test Log Report (see report run mode). The following snippet re-runs test number 42.

provengo run -s :log --run-id 42 path/to/project

Using the :log run source requires specifying a --run-id.

Develop Selenium Automations with `--dev-mode`

Use --dev-mode while developing or repairing Selenium automations. While in this mode, provengo makes browser sessions visible, and pauses on the first automation failure without closing those sessions. This enables automation engineers to inspect the page and correct problematic locators or wrong test expectations quickly, without having to re-create the UI state manually.

provengo run --dev-mode path/to/project

--dev-mode is intended for interactive local development. Avoid it in unattended CI runs because any test failure would block the build waiting for user interaction.

Parameters

-s/--run-source RUN_SOURCE: Where to bring the runs from. Either :random, :log, or a run filename. See Run Source.
--random-seed RANDOM_SEED: Seed number for random number generation.
--dry: Do not actuate external systems.
--run-id RUN_ID: Specify a specific run in the current run source. For :log it’s the run id, for files it the 1-based index of the run.
-o/--output-file OUTPUT_FILE: Run result location.
-m/--max-depth MAX-DEPTH: Maximum number of events in a single test. Applies only when using the :random run source.
--dev-mode: Developer-friendly mode, especially for when testing automations. In this mode provengo will show web sessions (similar to --show-sessions) and will pause on the first failure without terminating browser sessions. So if a locator is missing, you’ll have a chance to fix it.
--junit-report: Create a JUnit report while the test is running.
--max-length: Maximal scenario length, measured in events.

Lifecycle Scripts

See Scripts Page for more details.

--setup SETUP: A command to run before the tests start. Typically, used for bringing the test system up.
--before-test BEFORE_TEST: A command to run before each test.
--after-success AFTER_SUCCESS: A command to run after a successful test.
--after-failure AFTER_FAILURE: A command to run after a test failed test. Typically, cleans up and resets the test system.
--teardown TEARDOWN: A command to run after all tests are executed. Typically, cleans up and frees resources.

Selenium Related Parameters

--selenium-server SELENIUM_SERVER: URL of a selenium server for driving web UIs.
--action-delay ACTION_DELAY: Delay between Selenium actions, in milliseconds.
--selenium-browser: Browser to use for Selenium tests. One of chrome (default), firefox, edge, safari.
--show-sessions: Show Selenium sessions. If this flag is not set we run in 'headless' mode.
--max-windows: Maximize browser windows when opened.

Safari browsers do not support headless mode - they must run with a visible window. Thus, when using a Safari browser, the controlled windows are shown, regardless of the --show-sessions switch.

TN5250 Related Parameters

--tn5250 TN5250_SERVER: URL of a tn5250 Provengo’s terminal for driving AS400/IBM System I.

run