-
Notifications
You must be signed in to change notification settings - Fork 31
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
## Description This PR brings a major refactoring for the overall Python packaging as well as the CI employed in the project. ### Packaging We are (finally) moving away from `setuptools` and towards modern `scikit-build-core` (https://scikit-build-core.readthedocs.io/) This brings many advantages such as - truly editable installs that also include the C++ part; this implies incredibly fast rebuilds that can be further improved by installing ccache. - completely eliminates the `setup.py` (and the `MANIFEST.in`) file and all custom build code that was required previously. As part of this change, we adopt the established best practice of a src-layout for the Python package. This implies that the Python package no longer lives in the main package directory, but is moved to `src/mqt/...`. This avoids several pitfalls when working with Python packages and makes the overall handling way easier. This also means there is no direct build-time dependency on cmake and ninja anymore as this is automatically handled by scikit-build-core on demand. As part of this rewrite, several `nox` sessions are also updated ### CI MQT Core has introduced reusable workflows in - cda-tum/mqt-core#396 These have been further refined in - cda-tum/mqt-core#407 Overall, this allows to - remove almost all workflow files of this repository - eliminate a lot of maintenance required in this repository to keep up with the latest best practices - rely on dependabot for workflow updates once MQT Core has settled enough so that it follows proper versioning and we can use tags for the workflows here instead of `@main` while at the same time: - having automatic change detection and only triggering jobs that are really necessary; with a single pass check that can be used for branch protection - setting up ccache for all supported workflow jobs which greatly improves compilation times - setting up mold as a better linker under linux - splitting jobs into as many subjobs as possible and feasible to distribute work - automatically creating wheels for all supported platforms (including exotic platforms that need emulation) and Python versions using PyPI trusted publishing - enabling automatic retries for flaky jobs such as MSVC builds or coverage uploads - enabling automatic build parallelization via using Ninja wherever suitable - running an automatic workflow using the minimal versiosn for all Python dependencies to guarantee compatibility As a nice side effect, the new CI has unveiled several issues in the HSF simulator and it's task logic that have subsequently been resolved. Upon writing regression tests, an endianness bug in the Amplitude Mode HSF simulation surfaced. With this PR, the expected results are returned. ## Checklist: <!--- This checklist serves as a reminder of a couple of things that ensure your pull request will be merged swiftly. --> - [x] The pull request only contains commits that are related to it. - [x] I have added appropriate tests and documentation. - [x] I have made sure that all CI jobs on GitHub pass. - [x] The pull request introduces no new warnings and follows the project's style guidelines.
- Loading branch information
Showing
65 changed files
with
1,270 additions
and
1,014 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| .clone_script: &clone | | ||
| if [ -z "$CIRRUS_PR" ]; then | ||
| git clone --recursive --branch=$CIRRUS_BRANCH https://x-access-token:${CIRRUS_REPO_CLONE_TOKEN}@github.com/${CIRRUS_REPO_FULL_NAME}.git $CIRRUS_WORKING_DIR | ||
| git reset --hard $CIRRUS_CHANGE_IN_REPO | ||
| else | ||
| git clone https://x-access-token:${CIRRUS_REPO_CLONE_TOKEN}@github.com/${CIRRUS_REPO_FULL_NAME}.git $CIRRUS_WORKING_DIR | ||
| git fetch origin pull/$CIRRUS_PR/head:pull/$CIRRUS_PR | ||
| git checkout $CIRRUS_BASE_BRANCH | ||
| git -c user.email="you@example.com" merge --no-commit pull/$CIRRUS_PR | ||
| git submodule update --init --recursive | ||
| fi | ||
|
|
||
| .statistics_script: &statistics | | ||
| ccache -s -v | ||
| echo $(python -m pip cache dir) | ||
| echo $(python -m pip cache list) | ||
|
|
||
| macos_arm64_test_task: | ||
| name: 🐍 Test / 🍎 arm64 | ||
| alias: macos_arm64_test | ||
| clone_script: *clone | ||
| macos_instance: | ||
| image: ghcr.io/cirruslabs/macos-monterey-xcode:latest | ||
| ccache_cache: | ||
| folder: .ccache | ||
| populate_script: | ||
| - mkdir -p .ccache | ||
| fingerprint_key: ccache-macosx_arm64 | ||
| pip_cache: | ||
| folder: /Users/admin/Library/Caches/pip | ||
| prepare_env_script: | | ||
| brew install python@3.10 ccache pipx ninja nox | ||
| ln -s $(which python3.10) python | ||
| export PATH=/opt/homebrew/opt/python@3.10/libexec/bin:$PATH | ||
| export PATH=/opt/homebrew/opt/ccache/libexec:$PATH | ||
| export PATH=/opt/homebrew/opt/pipx/libexec:$PATH | ||
| export PATH=/opt/homebrew/opt/ninja/libexec:$PATH | ||
| export PATH=/opt/homebrew/opt/nox/libexec:$PATH | ||
| echo "PATH=$PATH" >> $CIRRUS_ENV | ||
| echo "CCACHE_DIR=$PWD/.ccache" >> $CIRRUS_ENV | ||
| run_nox_script: | ||
| - nox -s tests-3.10 --verbose | ||
| statistics_script: *statistics | ||
|
|
||
| build_and_store_wheels: &BUILD_AND_STORE_WHEELS | ||
| install_cibuildwheel_script: | ||
| - python -m pip install cibuildwheel~=2.15.0 | ||
| run_cibuildwheel_script: | ||
| - cibuildwheel | ||
| wheels_artifacts: | ||
| path: "wheelhouse/*" | ||
|
|
||
| linux_aarch64_wheels_task: | ||
| only_if: "$CIRRUS_RELEASE != ''" | ||
| name: 🐍 Packaging / 🎡 🐧 arm64 | ||
| alias: linux_aarch64_wheels | ||
| clone_script: *clone | ||
| compute_engine_instance: | ||
| image_project: cirrus-images | ||
| image: family/docker-builder-arm64 | ||
| architecture: arm64 | ||
| platform: linux | ||
| cpu: 8 | ||
| setup_pyhton_script: | ||
| - apt-get install -y python3-venv python-is-python3 | ||
| <<: *BUILD_AND_STORE_WHEELS | ||
|
|
||
| macos_arm64_wheels_task: | ||
| only_if: "$CIRRUS_RELEASE != ''" | ||
| name: 🐍 Packaging / 🎡 🍎 arm64 | ||
| alias: macos_arm64_wheels | ||
| clone_script: *clone | ||
| macos_instance: | ||
| image: ghcr.io/cirruslabs/macos-monterey-xcode:latest | ||
| ccache_cache: | ||
| folder: .ccache | ||
| populate_script: | ||
| - mkdir -p .ccache | ||
| fingerprint_key: ccache-macosx_arm64-wheels | ||
| pip_cache: | ||
| folder: /Users/admin/Library/Caches/pip | ||
| prepare_env_script: | | ||
| brew install python@3.10 ccache pipx ninja nox | ||
| ln -s $(which python3.10) python | ||
| export PATH=/opt/homebrew/opt/python@3.10/libexec/bin:$PATH | ||
| export PATH=/opt/homebrew/opt/ccache/libexec:$PATH | ||
| export PATH=/opt/homebrew/opt/pipx/libexec:$PATH | ||
| export PATH=/opt/homebrew/opt/ninja/libexec:$PATH | ||
| export PATH=/opt/homebrew/opt/nox/libexec:$PATH | ||
| echo "PATH=$PATH" >> $CIRRUS_ENV | ||
| echo "CCACHE_DIR=$PWD/.ccache" >> $CIRRUS_ENV | ||
| <<: *BUILD_AND_STORE_WHEELS | ||
| statistics_script: *statistics | ||
|
|
||
| publish_task: | ||
| name: 🚀 Deploy to PyPI | ||
| container: { image: "python:3.10-bullseye" } | ||
| depends_on: | ||
| - linux_aarch64_wheels | ||
| - macos_arm64_wheels | ||
| only_if: "$CIRRUS_RELEASE != ''" | ||
| env: | ||
| TWINE_REPOSITORY: pypi | ||
| TWINE_USERNAME: __token__ | ||
| TWINE_PASSWORD: "ENCRYPTED\ | ||
| [87601b93dead1db95c3ccb4b2ea2e0da333a0da3e52d5eb1\ | ||
| e5eb83dc24ae3c704779789acb68a9d53719a6d823c0692e]" | ||
| install_script: pip install twine | ||
| publish_script: | ||
| - curl -L https://api.cirrus-ci.com/v1/artifact/build/$CIRRUS_BUILD_ID/wheels.zip -o wheels.zip | ||
| - unzip wheels.zip | ||
| - python -m twine check wheelhouse/* | ||
| - python -m twine upload wheelhouse/* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,22 +1,54 @@ | ||
| ignore: | ||
| - "extern/**/*" | ||
| - "apps/*" | ||
| - "extern/**/*" | ||
| - "include/python/**/*" | ||
| - "src/python/*.cpp" | ||
| - "test/*.cpp" | ||
| - "mqt/**/*" | ||
|
|
||
| coverage: | ||
| range: 60..95 | ||
| range: 60..90 | ||
| precision: 1 | ||
| status: | ||
| project: | ||
| default: | ||
| project: off | ||
| patch: off | ||
|
|
||
| flag_management: | ||
| default_rules: | ||
| carryforward: true | ||
| statuses: | ||
| - type: project | ||
| target: auto | ||
| threshold: 0.5% | ||
| patch: | ||
| default: | ||
| removed_code_behavior: adjust_base | ||
| - type: patch | ||
| target: 90% | ||
| threshold: 1% | ||
| individual_flags: | ||
| - name: cpp | ||
| paths: | ||
| - "include" | ||
| - "src" | ||
| - "!src/python/*.cpp" | ||
| after_n_builds: 1 | ||
| - name: python | ||
| paths: | ||
| - "src/mqt/**/*.py" | ||
| after_n_builds: 3 | ||
| statuses: | ||
| - type: project | ||
| threshold: 0.5% | ||
| removed_code_behavior: adjust_base | ||
| - type: patch | ||
| target: 95% | ||
| threshold: 1% | ||
|
|
||
| parsers: | ||
| gcov: | ||
| branch_detection: | ||
| conditional: no | ||
| loop: no | ||
|
|
||
| comment: | ||
| layout: "reach, diff, flags, files" | ||
| require_changes: true | ||
| show_carryforward_flags: true |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,31 +1,36 @@ | ||
| version: 2 | ||
| updates: | ||
| # Enable version updates for git submodules | ||
| - package-ecosystem: "gitsubmodule" | ||
| # Look for `.gitmodules` in the `root` directory | ||
| directory: "/" | ||
| # Check the submodules for updates every week | ||
| groups: | ||
| submodules: | ||
| patterns: | ||
| - "*" | ||
| schedule: | ||
| interval: "monthly" | ||
| day: "friday" | ||
| time: "05:00" | ||
| timezone: "Europe/Vienna" | ||
|
|
||
| # Enable version updates for GitHub Actions | ||
| - package-ecosystem: "github-actions" | ||
| # Look for `*.yml` files in the `.github/workflows` directory | ||
| directory: "/" | ||
| # Check for updates to GitHub Actions every week | ||
| groups: | ||
| github-actions: | ||
| patterns: | ||
| - "*" | ||
| schedule: | ||
| interval: "monthly" | ||
| interval: "weekly" | ||
| day: "friday" | ||
| time: "05:00" | ||
| timezone: "Europe/Vienna" | ||
|
|
||
| - package-ecosystem: "pip" | ||
| directory: "/" | ||
| groups: | ||
| python-dependencies: | ||
| patterns: | ||
| - "*" | ||
| schedule: | ||
| interval: "monthly" | ||
| interval: "weekly" | ||
| day: "friday" | ||
| time: "05:00" | ||
| timezone: "Europe/Vienna" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| { | ||
| "problemMatcher": [ | ||
| { | ||
| "severity": "warning", | ||
| "pattern": [ | ||
| { | ||
| "regexp": "^([^:]+):(\\d+):(\\d+): ([A-DF-Z]\\d+): \\033\\[[\\d;]+m([^\\033]+).*$", | ||
| "file": 1, | ||
| "line": 2, | ||
| "column": 3, | ||
| "code": 4, | ||
| "message": 5 | ||
| } | ||
| ], | ||
| "owner": "pylint-warning" | ||
| }, | ||
| { | ||
| "severity": "error", | ||
| "pattern": [ | ||
| { | ||
| "regexp": "^([^:]+):(\\d+):(\\d+): (E\\d+): \\033\\[[\\d;]+m([^\\033]+).*$", | ||
| "file": 1, | ||
| "line": 2, | ||
| "column": 3, | ||
| "code": 4, | ||
| "message": 5 | ||
| } | ||
| ], | ||
| "owner": "pylint-error" | ||
| } | ||
| ] | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| ## Description | ||
|
|
||
| Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. | ||
|
|
||
| Fixes #(issue) <!--- Replace (issue) with the issue number that is fixed by this pull request. --> | ||
|
|
||
| ## Checklist: | ||
|
|
||
| <!--- | ||
| This checklist serves as a reminder of a couple of things that ensure your pull request will be merged swiftly. | ||
| --> | ||
|
|
||
| - [ ] The pull request only contains commits that are related to it. | ||
| - [ ] I have added appropriate tests and documentation. | ||
| - [ ] I have made sure that all CI jobs on GitHub pass. | ||
| - [ ] The pull request introduces no new warnings and follows the project's style guidelines. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.