Document MSTestAnalysisMode property #44632
Conversation
dotnet-policy-service
bot
added
dotnet-fundamentals/svc
community-contribution
| > [!NOTE] | ||
| > The following rules are completely opt-in and are not enabled in `Default`, `Recommended`, or `All` modes: | ||
| > | ||
| > - [MSTEST0015: Test method should not be ignored](mstest0015.md) | ||
| > - [MSTEST0019: Prefer TestInitialize methods over constructors](mstest0019.md) | ||
| > - [MSTEST0020: Prefer constructors over TestInitialize methods](mstest0020.md) | ||
| > - [MSTEST0021: Prefer Dispose over TestCleanup methods](mstest0021.md) | ||
| > - [MSTEST0022: Prefer TestCleanup over Dispose methods](mstest0022.md) |
There was a problem hiding this comment.
Not sure where to best document this. Options:
- In this page only as I did in this PR
- In doc page of the specific analyzer, with only a note here mentioning "some rules" may have this behavior
- Both as I did here plus in the specific analyzer page.
There was a problem hiding this comment.
Probably 3 but the specific analyzer page update can be done on a follow-up PR IMO.
|
|
||
| ### `Recommended` | ||
|
|
||
| This mode is more aggressive than `Default`. Rules that are enabled by default with Info (`suggestion`) severity are escalated to be warnings. Moreover, certain rules may decide that they are escalated to be errors in both `Recommended` and `All` modes. For example, [MSTEST0003: Test methods should have valid layout](mstest0003.md) is escalated to error in `Recommended` and `All` modes. |
There was a problem hiding this comment.
The first sentence could feel a little scary to users. I would instead say that in Default we are ensuring to not have rules that would cause warning or error and that don't result in runtime error. Recommended is what we think most dev should use but this means that a minor update of the package could cause new warnings.
There was a problem hiding this comment.
Recommended is what we think most dev should use but this means that a minor update of the package could cause new warnings.
But a minor update can cause new warnings for "Default" as well?
There was a problem hiding this comment.
Only for cases where there is an execution error. Otherwise we mark the new rules as info
| - Rules that are enabled by default will use their default severity. | ||
| - Rules that are disabled by default will use `none` severity. | ||
|
|
||
| ### `Recommended` |
There was a problem hiding this comment.
It might be a bit too boring to maintain the doc but it would probably be awesome if we could list here the rules enabled by this mode.
There was a problem hiding this comment.
It's easy that it gets outdated, esp when changes happen between releases. I'm not sure how to best handle this. Maybe the generated globalconfigs we produce should be part of version control with CI check that ensures the file is up-to-date? In that case we can instruct users to go to the relevant release branch and look into the globalconfig exactly for the specific release. Alternatively, we can tell them that they can look into the NuGet package contents with NuGet Package Explorer and find the globalconfig there (it's less convenient though 😕)
I'm not really sure what's best.
There was a problem hiding this comment.
Maybe we could generate a markdown per version and we would include it here.
@IEvangelist is this something that is common?
| > [!NOTE] | ||
| > The following rules are completely opt-in and are not enabled in `Default`, `Recommended`, or `All` modes: | ||
| > | ||
| > - [MSTEST0015: Test method should not be ignored](mstest0015.md) | ||
| > - [MSTEST0019: Prefer TestInitialize methods over constructors](mstest0019.md) | ||
| > - [MSTEST0020: Prefer constructors over TestInitialize methods](mstest0020.md) | ||
| > - [MSTEST0021: Prefer Dispose over TestCleanup methods](mstest0021.md) | ||
| > - [MSTEST0022: Prefer TestCleanup over Dispose methods](mstest0022.md) |
There was a problem hiding this comment.
Probably 3 but the specific analyzer page update can be done on a follow-up PR IMO.
Fixes microsoft/testfx#4749
cc @Evangelink
Internal previews