QuerySurge has long emphasized team communication and information-sharing in its design. As part of that emphasis, webhook support was introduced in QuerySurge 10.0. With QuerySurge 10.3, we've added built-in support for TestRail Result creation via webhook.
This article describes the webhook setup for TestRail Add Result option.
Prerequisite: Enable the TestRail API
The TestRail API must be enabled in order to send in requests. You can enable the API but navigating to the Administration section of your project.
Then click on the Site Settings section:
Under the API section, be sure the Enable API option is checked:
Configuring the TestRail Webhook in QuerySurge
TestRail provides a REST API, and in the following example, we set up a QuerySurge webhook that adds a test run result to your TestRail project.
1. This is the URL template for the TestRail REST API calls. Note that you'll need your specific TestRail project and test case ID to complete the URL.
https://<yourTestRailProject>.testrail.io/index.php?/api/v2/add_result/<yourTestCaseID>
Note: The test case ID should only contain the numerical value of the ID. In the example below, the test case ID used in the URL should be 4090.
2. TestRail uses basic authentication to authenticate API requests. A user with access to the TestRail project specified will be able to fire webhooks against that project:
username=<yourUserName>
password=<yourPassword>
3. The JSON payload in this example is simple to moderate in complexity. The JSON must follow TestRail API syntax rules (consult the TestRail API documentation). There are a few items of note here. You can insert QuerySurge values into the message, denoted by the ${...} syntax. If you have required custom fields, you'll need to find out how they are named in the JSON (again, consult the documentation) and configure them with QuerySurge values or hardcoded values as needed.
The "status_id" field can set the status for your test run result in TestRail based on the outcome of a QuerySurge Scenario. Since QuerySurge has its own Outcome statuses which may differ slightly from TestRail, a mapping (shown below) was implemented to translate the QuerySurge outcomes values to the corresponding TestRail result statuses.
|
status_id Mapping |
|
|
QuerySurge Scenario Outcome |
TestRail Result Status |
| Pass | Passed |
| Fail | Failed |
| Killed | Retest |
| Error | Blocked |
| Warning | Retest |
{
"status_id": "${status_id}",
"comment": "Scenario ${scenarioName} has an outcome of: ${outcome} \nPercent passing: ${passRate} %\nNumber of QueryPairs: ${numberOfQueryPairs}\n${queryPairsPassed} / ${numberOfQueryPairs} passed\n${queryPairsFailed} / ${numberOfQueryPairs} failed\n${queryPairsKilled}, ${queryPairWarnings}, ${queryPairErrors} killed, warning, errors\n\nRun Owner: ${owner}\nRun Start: ${startTime}\nRun End: ${endTime}",
"elapsed": "${duration}"
}
4. You have the option to add additional filters on an event. For example, you could limit the firing of the webhook to a specific owner (i.e. user), or in the "Scenario Ended" event, to the result status of the execution. The webhook Event Filters are identical to Email Notification Event Filters. For more information, see this.
Your TestRail run result should look like a standard result (unless you have customized it):