Skip to content

Test Hub Reporting and Trace Analysis with Modern Requirements4DevOps

Modern Requirements’ latest capability includes the ability to generate test hub information reporting and trace analysis. Users can now generate Smart Reports and Horizontal Traceability Matrix on Test Plans, Test Suite, Test Cases, Test Runs, Test Case Run, and Test Case Step Run details. Since much of the information in Test Hub are not based on work items and links, Modern Requirements introduced the concept of virtual work items and links to abstract the data into work items and links on which reports can be generated. This is a quickly evolving set of functions with new data being exposed from Azure DevOps as virtual Work Items and Links so that users can generate Smart Reports & Traceability Reports. This reduces the testing effort, improves resource utilization, speed, and quality of the tests. 

Table of Contents

Introduction  

In general, test case management tools aim to consolidate several aspects of testing into one place. Most often, tools differ in their scope, ability, effectiveness, flexibility, and more. While some organizations settle on using a combination of different tools, others prefer to use one comprehensive solution.

The basic testing hierarchy supported by Azure DevOps is as follows:

Whereas, organizations normally have a more comprehensive Test Case Management as illustrated below: 

What does Modern Requirements4DevOps have to offer?

In order to fill the gap between what test strategy companies usually follow and what Azure DevOps has
to offer, Modern Requirements4DevOps now offers Virtual work items. 

In Azure DevOps, the data for these virtual work items can be fetched from TFS using the following tabs:

Test Run
Test Result
Test Step Run

These Virtual work items are linked with existing work items using Virtual Link. 

Ready to streamline your requirements management today?

Start free trial

VIRTUAL WORK ITEMS PROPERTIES

LINKING OF VIRTUAL WORK ITEMS
In Smart Report, the virtual linking is automatically applied. However, in Trace Analysis you need to manually select the Virtual Link option to link the Virtual work items as shown in the image below:
HIERARCHY
The hierarchy below can be created and shown in Smart Report as well as while working in Traceability
module:

Test Plan vs Test Suite vs Test Case vs Test Run vs Test Result vs Test Step Run vs Bugs.
Same hierarchy can be created while showing the work in Trace Analysis.
 
PATHS TO GENERATE OUTPUT

Following are the different paths you can define to generate the output. 

  • Test plan vs Test suite 
  • Test suite vs Test case 
  • Test plan vs Test suite vs Test case
  • Test run
  • Test run vs Test result 
  • Test run vs Test result vs Test step run 
  • Test run vs Test result vs Bugs 
  • Test run vs Test result vs Test step run vs Bugs
  • Test run vs Test result vs Bugs vs Test step run
  • Test suite vs Test case vs Test run vs Test result vs Test step run 
  • Test plan vs Test suite vs Test run vs Test result vs Bugs 
  • Test suite vs Test case vs Test run vs Test result vs Test step run vs Bugs 
  • Test plan vs Test suite vs Test run vs Test result vs Test step run vs Bugs 
  • Test suite vs Test case vs Test run vs Test result vs Bugs vs Test step run 
  • Test plan vs Test suite vs Test run vs Test result vs Bugs vs Test step run 
  • Test plan vs Test suite vs Test case vs Test run vs Test result vs Test step run vs Bugs

Usability of the Virtual Work Items

These work items can be used in the following modules/sections:
 
SMART REPORT
Virtual work items can be used in Smart Report available under the Queries tab, Test Hub and Smart
Docs.
TRACE ANALYSIS
In the Traceability module, these virtual items can be used to create a horizontal matrix using: 
  • Work item type 
  • Query type
USING VIRTUAL WORK ITEMS TO CREATE A HORIZONTAL MATRIX USING QUERY TYPE

USING VIRTUAL WORK ITEMS TO CREATE A HORIZONTAL MATRIX USING WORK ITEM TYPE

Support for attachment and comments field

Comments added on run test case is mapped to “Detail Comments” field. 

Attachment added on run test case is mapped to “Result Attachment” field.

Comment and attachment added from test run are mapped in test results. 

COMMENTS ADDED FROM UPDATE ANALYSIS MAPPED TO “COMMENT” FIELD 

Comments added from update analysis will be mapped to “Comment” field in test results. 

Comment added from update analysis is added to the “Comment” field in Test Results. 

COMMENTS ADDED ON FAIL STEPS ON RUN TEST CASE MAPPED TO “COMMENT” FIELD 

Comments added on fail steps on run test case are mapped to the “Comment” field in Test Step Run. 

Comment for fail step added in “Comment” field of Test Step Run.
CONFIGURATION IS MAPPED TO “CONFIG NAME” FIELD IN TEST RESULTS 

Configuration can be seen mapped below in test results. 

Creating test plan / test suite from test hub in Azure DevOps

Limitations

SMART REPORT | QUERIES TAB

    • The linking of virtual work items is supported only when the report is generated from the toolbar. 
    • In Smart Report (Queries Tab), all the sections should be hierarchical except for the ones that do not have any subordinates, and, for these, the hierarchy would be false.
  • If you have other work items other than Test Plan, Test Suite and Test Case, you cannot generate a test case API. If other work items are also a part of the query then the smart report will be generated on its normal functionality.
 

SMART REPORT | SMART DOCS MODULE

  • While using Smart Report in Smart Docs, the default section should have work item type “Any” or root work item type. 
  • The Report will only be generated on adding Test Plan / Test Suite / Test Case work items from Insert Work Item option. These work items are to be created from Test Plan Hub. 
Ready to streamline your requirements management? Start your free trial of Modern Requirements4DevOps today!