VSTest retries all examples of a scenario if one fails

[kellylarkin]

We migrated from SpecFlow to ReqNRoll this week. Our project uses MSTest as the test framework and the VSTest runner in Azure DevOps to run the tests/retries. We also upgraded from dotnet6 to dotnet9 as a part of this migration.

With SpecFlow, if one example of a scenario failed-- only that one example would be retried. After the migration to ReqNRoll, if one example fails, all examples of the test are retried which is bloating our pipeline duration.

I noticed when running SpecFlow that each example of a scenario seemed to be discovered as its own test variant, whereas the test discovery with ReqNRoll seems to group all examples under the one test (the scenario).

Is there something I'm missing to configure this correctly? Or should I move away from using VSTest for retries?

SpecFlow:

ReqnRoll:

VSTest config:

[ajeckmans]

You can revert to the old behavior by setting allowRowTests
See the documentation https://docs.reqnroll.net/latest/installation/configuration.html

[kellylarkin]

This worked! Thanks so much.