QuerySurge Technical Whitepaper No. 2
When QuerySurge is installed you are given the option to install the command line Execution API component (called the "Command Line API" option in the installer), which allows you to interact with QuerySurge via the command line to execute QuerySurge tests. This includes the options to create and run new QuerySurge Scenarios from Test Suites, as well as check on the execution status of Scenarios executed via the command line.
List of Available Commands
The following commands are available:
runTestSuiteByName– Runs an available Test Suite in QuerySurge, specified by name.
runTestSuiteById– Runs an available Test Suite in QuerySurge, specified by Test Suite ID.
getScenarioOutcome– Returns the outcome of a Scenario, specified by Scenario ID (which is
help– Prints the arguments for the specified command
- <blank> – Prints the list of available commands
modifyReusableQueryTextById- modify a Reusable Query Snippet value, specified by the Snippet ID
modifyReusableQueryTextByName- modify a Reusable Query Snippet value, specified by the Snippet Name
Simple Method for Starting Test Runs
The simplest case that involves using the Command Line Execution API is simply to run a Test Suite directly from the Windows Command Prompt or a Linux shell.1 Here, the appropriate command (either
runTestSuiteById) is given with the desired arguments. When a Test Suite is run using the
runTestSuiteById command, a new QuerySurge Scenario is created and is scheduled to run immediately. The Id of this Scenario is then returned to the command line call, which can be used to check on the status of the run.
Figure 1 shows the command to run a Test Suite specified by name, which returns the Scenario Id (which in this example is "44ca0f07-7516-42d6-b5e1-e39346246b5a"). Using the Command Line API "manually" in this manner is useful in many situations, such as when running in a headless mode or via SSH.
Note: While all of the examples in this whitepaper are Windows-based, the Command Line can be installed and run from supported Linux systems as well.
Basic Scripts for Starting Test Runs
Another common Scenario that can be easily implemented using the Command Line API is to start a test run from an external process or script. This allows you to easily run multiple commands, so that you can begin the execution of multiple Test Suites. One particularly useful instance in which a script can be used is when you’d like to schedule a Test Suite or multiple Test Suites to run on a repeating schedule. In Windows, this is possible using the Windows Scheduler to run a batch script of QuerySurge commands according to the schedule you choose. For example, you can set up regression tests to run every night after business hours. This can also be accomplished in Linux using cron jobs with shell scripts. Figure 2 is an example of a simple batch script that runs two different Test Suites, one specified by name, and the other by Id. The script first outputs the message that it has been started, calls into the API to run the desired Test Suites, and then outputs the message that the run has completed.
Advanced Event Driven Scripts
Creating scripts to run Test Suites is useful, but these scripts can be made even more powerful with additional logic.
Figure 3 shows an example of a batch script that incorporates logic to determine when each Test Suite is ready to be run, based on whether the previous Test Suite has completed. It also incorporates kicking off multiple legs of an ETL process (two legs, in this illustration).
First, the script runs the initial ETL process. Once that is complete, it runs the Test Suite in QuerySurge that has been set up for that ETL leg. It then captures the Scenario Id that is returned (in a variable called %ScenarioId%) and uses that Id with the
getScenarioOutcome command to check on the status of that run in QuerySurge. The script checks whether the Scenario has started, and once it has, it continues to check on the run status until the Scenario has completed. Once the first Scenario is completed, the second ETL process is run. Finally, the second Test Suite execution in QuerySurge is executed, which runs the tests prepared for the second ETL leg.
This example shows how the different commands can be used together to build a comprehensive run script that effectively is incorporated into an ETL process.
Parameterizing QueryPairs with Reusable Query Snippets and the Execution API
You can modify Reusable Snippet values from the Execution API using the methods
modifyReusableQueryTextByName. This can be used to provide data-driven parameterized QueryPairs. This technique is fully described in the Knowledge Base article "Parameterizing your QueryPairs using Reusable Query Snippets and the Execution API", which you can find here.
The Command Line API gives QuerySurge significant flexibility and allows QuerySurge integration into a wide range of workflows, from "functional" and regression testing workflows owned by Data Warehouse project teams to daily monitoring workflows owned by Operations teams. QuerySurge of course keeps results organized with full access to past runs, data drilldown and export, and results reporting. Scenarios that are run from the Command Line API appear in the QuerySurge user interface the same as user-initiated runs appear, so all of the data access features are available for these executions.
If the QuerySurge Email Notification feature is used, QuerySurge will notify users about Command Line API run outcomes via email. This means that a fully automated testing information flow can be managed by QuerySurge – using either a scheduling tool or an ETL tool to kick off QuerySurge executions automatically, QuerySurge will execute the designated tests, analyze the results, and notify users by email of results.