run
Execute tests specified by the project, and store the results in the project’s run log database.
Description
Execute test scenarios directly on the tested system, using scenarios defined by the Provengo project. Test scenarios come from a run source, which is specified by the optional Run Source parameter (--run-source/-s). Execution results are stored in the project’s test execution database.
In addition to executing scenarios, the Provengo tool can run various Lifecycle Scripts for set-up, tear-down, etc.
Run Source
Several run sources are available:
:random-
The default mode. Randomly selects a single run from the model. This source is useful while developing the model, or when trying out some code changes. Often coupled with
--dryto prevent actuations from actually happening. :log-
Re-runs a previous run from the run log database. The run id must be specified, using the
--run-idparameter. filename.json-
A run file. These are commonly generated by the sample and ensemble run modes, or are downloaded from an execution report. It it possible to run only a specific run from the file, using
--run-idand the run’s number (1-based).
Examples
Run a random test
These type of runs are often useful when you start working on a model, and want to quickly check how it behaves.
The following lines are all equivalent
provengo run --run-source :random path/to/project (1)
provengo run -s :random path/to/project (2)
provengo run path/to/project (3)| 1 | Specifying the run source to be a random run. |
| 2 | Specifying the run source to be a random run using abbreviated syntax. |
| 3 | Not specifying the run source, so it defaults to :random. |
Same as above, but skip actuations. Useful when you want to see what would happen if you ran the tests, but not actually have that happen.
provengo run --dry path/to/projectRun Tests from a File
Execute all tests stored in customer-area-checks.json.
provengo run -s customer-area-checks.json path/to/projectExecute test number 42 from file customer-area-checks.json. The number of the first run is 1.
provengo run -s customer-area-checks.json --run-id 42 path/to/projectRe-Run a Test from the Log
Each run in the log has an id. This id is shown in the Test Log Report (see report run mode). The following snippet re-runs test number 42.
provengo run -s :log --run-id 42 path/to/project
Using the :log run source requires specifying a --run-id.
|
Parameters
-s/--run-source RUN_SOURCE-
Where to bring the runs from. Either
:random,:log, or a run filename. See Run Source. --random-seed RANDOM_SEED-
Seed number for random number generation.
--dry-
Do not actuate external systems.
--run-id RUN_ID-
Specify a specific run in the current run source. For
:logit’s the run id, for files it the 1-based index of the run. -o/--output-file OUTPUT_FILE-
Run result location.
Lifecycle Scripts
See Scripts Page for more details.
--setup SETUP-
A command to run before the tests start. Typically, used for bringing the test system up.
--before-test BEFORE_TEST-
A command to run before each test.
--after-success AFTER_SUCCESS-
A command to run after a successful test.
--after-failure AFTER_FAILURE-
A command to run after a test failed test. Typically, cleans up and resets the test system.
--teardown TEARDOWN-
A command to run after all tests are executed. Typically, cleans up and frees resources.
Selenium Related Parameters
--selenium-server SELENIUM_SERVER-
URL of a selenium server for driving web UIs.
--action-delay ACTION_DELAY-
Delay between Selenium actions, in milliseconds.
--selenium-browser-
Browser to use for Selenium tests. One of
chrome(default),firefox,edge,safari. --show-sessions-
Show Selenium sessions. If this flag is not set we run in 'headless' mode.
--max-windows-
Maximize browser windows when opened.
Safari browsers do not support headless mode - they must run with a visible window. Thus, when using a Safari browser, the controlled windows are shown, regardless of the --show-sessions switch.
|