Allow reuse of test code in other projects #5068
Conversation
|
Any comments? Or should I add a changelog entry (as mentioned by one of the CI checks) first? |
|
I'm going to review this in full over the next few hours but I want to be clear that beets does not promise that these, or any, testing objects will remain static. These can and will be changed, so relying on them for your own project will be something you'll have to take on at your own risk and keep an eye on PRs here that might change those parts of our testing framework. |
|
If you could make sure that the tests pass with your changes, I will review after that. |
sumpfralle
force-pushed
the
reusable-test-code
branch
from
3341fd2 to
7d89021
Compare
Done. Thanks! |
Yes, I understand that the exposure of these components will make them part of the public interface of beets. From my point of view, these pieces are only useful in testing environments. Thus, breaking these interfaces would not cause damage (only consuming time of developers). And users ("developers") probably understand, that such pieces of code are not bound to the same level of stability as the rest of beets' code. But these are just my subjective assessments. I could perfectly understand, if you reach a different conclusion. |
|
Our CI testing is still failing. Are they working on your local machine? |
sumpfralle
force-pushed
the
reusable-test-code
branch
from
7d89021 to
8bd33c5
Compare
Thanks for your reminder. Indeed the windows-CI failure was caused by my changes. This is fixed now. The docs-related failure seems to be a recent problem of sphinx (fixed to an older version) combined with the latest |
|
See #5080 for the "docs" failure. |
|
Could you please rebase this in light of the Sphinx changes being merged? That'll get rid of the failing CI and I can review in depth. |
The import path mangling is not relevant (anymore?) for the two ways of running tests: * `python3 test/testall.py` (see CONTRIBUTING.rst): The `testall.py` script already adds the project path to `sys.path`. * `tox -e py-cov`: this command is supposed to be run from the project path. Thus, the current directory is already the first of location in `sys.path`. The previous mangling of the import path while loading a module could lead to unwanted side-effects hidden in an unexpected location. Instead, import path mangling should take place in the script being called by the user (here: `testall.py`).
…lper` `ImportHelper` and `AutotagStub` are used in many tests. Thus, they should reside in a module which is obviously used by multiple tests.
…` to `test.helper` This class is imported by some other test modules. Thus, it should reside in a module, which is obviously used by other tests.
External Python packages interfacing beets may want to use an in-memory beets library instance for testing beets-related code. The `TestHelper` class is very helpful for this purpose. Previously `TestHelper` was located in the `test/` directory. Now it is part of `beets` itself (`beets.test.helper.TestHelper`) and can be easily imported.
sumpfralle
force-pushed
the
reusable-test-code
branch
from
8bd33c5 to
508d28f
Compare
Done. |
|
Alright, this seems good to me. I just want to reiterate that beets developers may change these in future PRs; we can't really promise that the testing framework is stable or even tied to the version number, so be aware of that. |
Description
Currently the test-related code of
beetsresides in thetestdirectory, which is not part of thebeetspackage.Thus, external projects (in my case: mopidy-beets) cannot use any parts of beets' test infrastructure.
Specifically the
TestHelperclass (and indirectly some parts oftest._common) may be useful for external projects (e.g. for working with an in-memory instance of a beets library).This proposal consists of four commits:
test.test_importertotest.helperTerminalImportSessionSetupfromtests.test_ui_importertotest.helpertest._commonandtest.helper) into thebeetspackageThese changes do not interfere with the current structure or behavior of the beets tests.
These changes would allow me to improve the tests of the mopidy-beets project.