Test Hub Reporting and Trace Analysis with Modern Requirements4DevOps

Modern Requirements4DevOps includes the ability to generate test hub information reporting and trace analysis. Users can generate Smart Reports and Horizontal Traceability Matrix on Test Plans, Test Suite, Test Cases, Test Runs, Test Case Run, and Test Case Step Run details. Since much of the information in Test Hub is not based on work items and links, Modern Requirements introduced the concept of virtual work items and links to abstract the data into work items and links on which reports can be generated. This is a quickly evolving set of functions with new data being exposed from Azure DevOps as virtual Work Items and Links so that users can generate Smart Reports & Traceability Reports. This reduces the testing effort, improves resource utilization, speed, and quality of the tests.

Introducción

In the simplest form, the test case management software consolidates several aspects of testing into one place. This
reduces the testing effort, improves resource utilization, speed, and quality of the tests.

Most often, the tools differ in their scope, ability, effectiveness, flexibility, and more. While some organizations settle on one comprehensive solution, others opt to use a combination of different tools.

The normal testing hierarchy supported by Azure DevOps is as following:

Whereas organizations normally have a more comprehensive Test Case Management as illustrated below:

The Requirement work item will be available when Requirement suite is created from the Test Hub.

Note: The Requirement work item is supported in Trace Analysis

What does Modern Requirements4DevOps have to offer?

In order to fill the gap between what test strategy companies usually follow and what Azure DevOps has to offer,
Modern Requirements4DevOps now offers Virtual work items.

In Azure DevOps, the data for these virtual work items can be fetched from TFS using the following tabs:

Prueba de funcionamiento

Test Result

Test Step Run

These Virtual work items are linked with existing work items using Virtual Link.

Virtual Work Items Properties

Support for Attachment and Comments Field

Attachments and Comments are now supported in Trace Analysis and Smart Report module (Trace Analysis, Test Plan Hub, Queries tab and Smart Doc)

Comment added on run test case is mapped to “Detail Comments”
Attachment added on run test case is mapped to “Result Attachment”

Comment and attachment added from test run are mapped in test results virtual work item.

Smart Report Configuration

Output

Comments added from update analysis will be mapped to “Comment” field in test

Comment added from update analysis is added to the “Comment” field in Test Results.

Comments added on fail steps on run test case are mapped to the “Comment” field in Test Step Run.

Comment for fail step added in “Comment” field of Test Step Run in Trace Analysis and Smart Report.

Configuration is mapped to “Config Name” field in Test Results

Configuration can be seen mapped below in test results.

Captured screenshot from Test Run are mapped as an attachment in Test Step Run virtual work

User can take screenshot from the Capture screenshot option as shown in the image below.

The captured screenshot can be seen in ADO mapped as an attachment. Users can download the captured screenshot by clicking on it.

The captured screenshot can be seen as an attachment in the Smart Report. Users can download the attachment by clicking on it.

The options Capture user actions, Record Screen, and Record screen with audio are supported in the Test Result virtual work item. Once a user adds these attachments from ADO, they will be visible in the Smart Report/Trace Analysis section of the Test Result work item under the “Result attachment” property.

For multiple iterations, Comments and Attachments added in each iteration is shown. On run test case, if there are multiple iterations, user can add comments in each iteration which can be seen by adding “Details” field from column options in Trace analysis and through “Report Designer” in Smart Report.

ADO

Comments and attachments added against each iteration can been seen in the “Summary” section.

Análisis de trazas

Comments and Attachments added by the user against each of the iterations will be visible in the Traceability Matrix.

Informe inteligente

By adding “Details” field from the column options, comments and attachments for each iteration can be seen in “Report Designer” in Smart Report.

Test Parameter

Parameter added in Test case steps can be seen by adding “Test Parameter” field from column options in Trace analysis and through “Report Designer” in Smart Report.

ADO

Análisis de trazas

Comments and Attachments added by the user against each of the iterations will be visible in the Traceability Matrix.

Informe inteligente

By selecting the “Test Parameter” field from column options, user can see “Parameters” column in Smart Report.

Linking of Virtual Work Items

In Smart Report, the virtual linking is automatically applied. However, in Trace Analysis you need to manually select the Virtual Link option to link the Virtual work items as shown in the image below:

Hierarchy

The hierarchy below can be created and shown in Smart Report as well as while working in Traceability module:

Test Plan vs Test Suite vs Test Case/Test Point vs Test Run vs Test Result vs Test Step Run vs Bugs

Same hierarchy can be created while showing the work in Trace Analysis.

Paths to Generates Output

Following are the different paths you can define to generate the output:

Test plan vs Test suite
Test suite vs Test case
Test plan vs Test suite vs Test case
Test run
Test run vs Test result
Test run vs Test result vs Test step run
Test run vs Test result v––s Bugs
Test run vs Test result vs Test step run vs Bugs
Test run vs Test result vs Bugs vs Test step run
Test suite vs Test case vs Test run vs Test result vs Test step run
Test plan vs Test suite vs Test run vs Test result vs Bugs
Test suite vs Test case vs Test run vs Test result vs Test step run vs Bugs
Test plan vs Test suite vs Test run vs Test result vs Test step run vs Bugs
Test suite vs Test case vs Test run vs Test result vs Bugs vs Test step run
Test plan vs Test suite vs Test run vs Test result vs Bugs vs Test step run
Test plan vs Test suite vs Test case vs Test run vs Test result vs Test step run vs Bugs
Test Plan vs Test Suite vs Test Point vs Test Run vs Test Result vs Test Step Run vs Bugs
Test Plan vs Test Suite vs Test Case vs Test Point vs Test Run vs Test Result vs Test Step Run vs Bugs
Test Plan vs Test Suite vs Test Point vs Test Case vs Test Run vs Test Result vs Test Step Run vs Bugs
Test Plan vs Test Suite vs Test Point
Test Plan vs Requirement vs Test Case vs Test Run vs Test Result vs Test Step Run Vs Bugs

Usability of the Virtual Work Items

These work items can be used in the following modules/sections:

Informe inteligente

Virtual work items can be used in Smart Report available under the Queries tab, Test Hub and Smart Docs.

Análisis de trazas

In the Traceability module, these virtual items can be used to create a horizontal matrix using:

Work item type
Query type

Using Virtual Work Items to Create a Horizontal Matrix Using Query Type

Using Virtual Work Items to Create a Horizontal Matrix Using Work Item Type

Creating Test Plan / Test Suite from Test Hub in Azure DevOps

Limitations

Smart Report | Queries Tab

The linking of virtual work items is supported only when the report is generated from the

In Smart Report (Queries Tab), all the sections should be hierarchical except for the ones that do not have any subordinates and, for these, the hierarchy would be

If you have work items other than Test Plan, Test Suite and Test Case, you cannot generate a test case If other work items are also a part of the query then the smart report will be generated on its normal functionality.

Smart Report | Smart Docs Module

While using Smart Report in Smart Docs, the default section should have work item type “Any” or root work item

The Report will only be generated on adding Test Plan / Test Siute / Test Case work items from Insert Work Item These work items are to be created from Test Plan Hub.

Empiece a utilizar Modern Requirements hoy mismo.

✅ Defina, gestione y realice un seguimiento de los requisitos en Azure DevOps
✅ Colabore sin problemas entre equipos regulados
✅ Empiece GRATIS, sin necesidad de tarjeta de crédito

Comience la prueba gratuita