The Scheme:
QuerySurge Test Management REST API allows tools such as Azure DevOps Build Agents to execute suites and retrieve results from QuerySurge. Following diagram illustrates the relationships:
Prerequisites:
- QuerySurge installation
- QuerySurge Test Management Samples
- QuerySurge Test Management REST API License
- Azure DevOps Account
- Visual Studio Client
Azure DevOps Project Setup
1. After logging in to Azure DevOps, click the "Create Project" button (You will need the appropriate privileges to create a new project)
2. Complete the new project form and Click the Create button. Make sure that you select the version control option that you plan to use to store your Visual Studio unit test project files.
2. You should see confirmation that your project has been created.
Connect Visual Studio to the new Azure DevOps project
1. Launch Visual Studio and and from the main menu, select Team -> Manage Connections
2. When the Connect to Project window opens, click on the Add TFS Server link
3. Enter the server URL to your Azure Devops server and click Add
4. Expand the tree to the newly created project and click the Connect button.
5. If the connection is successful, you should see project listed in the Team Explorer window
Visual Studio Unit Test Project Setup
1. From the main Visual Studio menu, select File->New->Project...
2. Select Unit Test Project to create a new Azure DevOps Unit Test Project.
Note: Your Unit Test Project name needs to match the Test Files pattern (see Step 8 in the section "Create Visual Studio Team Services Build Verification Test" below). You can either modify your Unit Test Project name to match the default pattern (see Step 8 in the section "Create Visual Studio Team Build Verification Test" for the default pattern) or you can modify the pattern to match your Unit Test Project name.
3. On the machine where you have installed the Test Management Samples locate the samples installation directory. In this directory, you'll find a samples/ subdirectory containing sample .cs files. In the default installation, the path is: C:\Program Files\QuerySurge\rest\samples\testmanagement-samples. The sample file for this setup is: TFS_ScenarioBySuiteId.cs. You will need to modify the following string variables in this sample .cs file for your usage: userName, password, projectId, scenarioName , suiteIdListand outputPath. (Note: Make sure the appropriate users and the Service Account has access to the outputPath directory.
4. Open the TFS_QuerySurgeConnection_Rest.cs file in your Visual Studio Project and update the hostName, and port string variables appropriately.
5. If not already configured during the project creation, add the solution to Source Control by right clicking on the solution and selecting Add Solution to Source Control.... Add the Solution to the project and clicking the OK button.
5. "Check In" the updated .cs files to your Team Project in Azure DevOps by right clicking on the solution and selecting Check in.... Fill out the form in the Team Explorer window and click Check In button.
Configure the Visual Studio Build Agent Machine
1. Navigate to your Azure DevOps Project. Click the Project Settings option on the bottom of the left side menu.
2. Under the Pipelines menu, Select the Agent pools option.
3. Select the appropriate agent pool, (This example uses Default) and click the Download Agent button in the upper right corner of the screen.
4. Follow the instructions displayed in the browser on the designated agent machine.
Note: When you install the agent, it is important that you install the agent as a service.
Note: Make sure to choose the proper architecture (x84 or x64)
Note: Part of the agent setup is done with PowerShell. Your PowerShell Version must be 4.0 or greater to complete this process.
Note: If using a personal access token for authentication, follow the steps to create the token listed here.
5. Once agent set up is completed, go to Start -> Services and make sure the agent is started.
6. In the Services console, right click on Visual Studio Build Agent and select Properties. Click on the Log On tab and make sure Local System account is selected.
7. Navigate back to your Azure DevOps Project, and return to the Agent Queues page. You should now see your newly created agent:
Create an Azure DevOps Pipeline Build
1. Navigate to the Pipelines -> Build from the main menu and click the New ->New Build Pipeline option.
2. Select the appropriate source option and and Workspace mappings and click Continue.
Note: The Server path should include the project that contains the Visual Studio Unit Test project that was created earlier.
3. Select the .NET Desktop template and click the Apply button.
4. On right pane, provide an Name for the build and select the Agent pool that contains the VS Build agent that was configured.
8. Select the Visual Studio Test task and apply the appropriate Test Files and Test Filter Criteria information for your project.
Note: The unit tests that are used in this example have the test method names of QuerySurgeConnectionBySuiteIdTest.
Note: See Step 2 in the section "Visual Studio Unit Test Project Setup" above. Your Assembly name, which is typically the name of your "Unit Test Project" needs to contain the sub-string 'test' to be recognized by the default Test Files pattern in this step (**\*test*.dll;-:**\obj\**). Alternately, you can change the Test Files pattern here to match your Assembly naming convention.
9. Click the Save & queue button to save the Build Definition and queue the build for execution.
10. Confirm that the appropriate agent pool is selected from the dropdown menu and click the Save and queue button.
10. A message will be displayed indicated that the build has been queued. To the view the details of the build, click on the Build# link to view the status of the build
11. Once the build has been completed, click the Tests tab to see the results of the execution.
Optional: Run Tests from the Test Hub
1. In order to run tests from the Test Hub, follow the setup instructions from this link:
https://www.visualstudio.com/en-us/docs/test/continuous-testing/run-automated-tests-from-test-hub
Troubleshooting
Note: The steps in this section can be done with any PC on you corporate network that has Microsoft Visual Studio installed. Some steps in this section refer to the install directories for the QuerySurge Host, so please make sure that you have access to a QuerySurge Host installation.
The generated QuerySurge HTML results attachment will be made available on Build Verification Test failure to provide more details on the current test run.