Although it has always been possible to integrate AscentialTest with Azure manually, the new AscentialTest Azure pipeline task extension makes that integration much easier to accomplish.
The extension configures and runs the AscentialTest command lines to execute tests and generate reports. It also outputs a set of results files to summarize ‘passed’, ‘failed’, ‘blocked’ and ‘not run’ tests to make it easy to integrate with other extensions for generating custom reports.
The AscentialTest Azure extension can be installed from Microsoft Azure Marketplace.
The Task panel indicates required fields with a symbol and provides ‘help’, using standard the Azure tool tip.
If the user clicks on the icon, a help panel will display to describe the associated parameter.
When the user completes the Task Panel and clicks on the ‘Add’ button, the configured task is displayed in the pipeline as pictured in the image below:
When the pipeline is triggered, ‘atrun’ and ‘atreports’ will be executed accordingly.
Once the run is complete, the reports selected in the new plugin panel will be available in the ‘Report Path’ folder as set in the task panel.
In addition to running the command lines to execute tests and generate AscentialTest Test Set reports, the extension also generates a set of result data files that can be graphed using third-party plotting tools:
Each of these files is updated for every run, recording the number of tests with the respective outcomes:
To facilitate integration of AscentialTest results with Azure Dev Ops dashboards, the following project components are available from Zeenyx Support.
To successfully run the AscentialTest Azure extension, you will need to target a self-hosted agent in interactive mode. Running a self-hosted agent as a service, although more convenient, will most likely result in permissions problems when attempting to run tests. We found in our trials that self-hosted agents run as a service are not allowed to interact with the desktop even if the related windows service setting is configured. This appears to be a limitation of Azure self-hosted agents.
If you have any questions about the AscentialTest Azure Pipeline Task Extension or would like a copy of the Junit Results generation, please contact Novalys Support at email@example.com. We are here to support you and answer your questions.