Compare the coverage

We have tree ways of checking running the doctests on SciPy:

1. The old way: refguide-check
2. The doctest way: via the doctesting layer
3. The pytest-plugin way: via the pytest plugin layer.

We want to make sure that the "new" ways have at least the same coverage as the refguide-check way. Here's how.

---

The pytest plugin way

• Patch the scpdt plugin to include functions with no examples in the docstrings: comment out the if test.examples: # skip empty doctests clause at https://github.com/ev-br/scpdt/blob/main/scpdt/plugin.py#L184

• Use the scipy conftest mods from https://github.com/scipy/scipy/compare/main...ev-br:scipy:doctest_plugin (the git branch is at ev-br/scipy/doctest_plugin) to activate the plugin

• Run the plugin and stop at the collection stage:

$ python dev.py shell
$ cd path/to-scipy/folder
$ pytest build-install/lib/python3.10/site-packages/scipy/ --doctest-modules --ignore=build-install/lib/python3.10/site-packages/scipy/interpolate/_interpnd_info.py --ignore=build-install/lib/python3.10/site-packages/scipy/_lib --collect-only > plugin.log

Then run the plugin.log through a small utility to extract the names of functions and classes:

"""
convert the result of 
$ pytest ... --doctest-modules --collect-only
to simplify comparing to the doctest_.log from doctest-based runs
"""

if __name__ == "__main__":
    import sys
    if len(sys.argv) != 2:
        print("Usage: modify_log.py LOGFILE")

    fname = sys.argv[1]
    with open(fname, 'r') as inf:
        in_lines = inf.readlines()

    out_lines = []
    for line in in_lines:
        line_ = line.strip()
         
        if line_.startswith("<DoctestItem"):
            out_lines.append(line_[13:-1])

    print("\n".join(sorted(out_lines)))

---

The refguide-check way

• Use https://github.com/scipy/scipy/pull/16391 which patches the refguide-check utility so that running

$ python dev.py refguide-check

drops a file refguide_check.log in the SciPy root folder.

• Run the log through

"""
strip module names from the refguide-check.log
to simplify comparing to the doctest_.log from doctest-based runs
"""

PACKAGE = "scipy."

if __name__ == "__main__":
    import sys
    if len(sys.argv) != 2:
        print("Usage: modify_log.py LOGFILE")

    fname = sys.argv[1]
    with open(fname, 'r') as inf:
        in_lines = inf.readlines()

    out_lines = []
    for j in range(len(in_lines)-1):
        line = in_lines[j]
        next_line = in_lines[j+1]

        if line.startswith("="):
            continue

        if next_line.startswith("="):
            # this is a package name, add and reset
            out_lines.append(line)
            PACKAGE=line.strip()
            continue

        out_lines.append(f"{PACKAGE}.{line}")

    print("".join(sorted(out_lines)))