Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Entrypoint Consolidation: All Functionality accessible via mlflow-export-import #30

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

juftin
Copy link

@juftin juftin commented Jul 26, 2022

This PR addresses Issue #29

Changes

1) All Click Commands have been added to a new master Click Group

2) All entrypoints have been replaced with the new Click Group:

mlflow-export-import --help
Usage: mlflow-export-import [OPTIONS] COMMAND [ARGS]...

  MLflow Export / Import CLI: Command Line Interface

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  export-all          Export the entire tracking server.
  export-experiment   Exports an experiment to a directory.
  export-experiments  Exports experiments to a directory.
  export-model        Export a registered model.
  export-models       Exports models and their versions' backing.
  export-run          Exports a run to a directory.
  find-artifacts      Find artifacts that match a filename.
  http-client         Interact with the MLflow / Databricks HTTP Client.
  import-all          Imports models and their experiments and runs.
  import-experiment   Imports an experiment from a directory.
  import-experiments  Import a list of experiments from a directory.
  import-model        Import a registered model.
  import-run          Imports a run from a directory.
  list-models         Lists all registered models.

3) All packaging configuration moved to setup.cfg - setup.py inherits from this automatically

  • This also includes a pyproject.toml file which instructs Python to use setuptools as the build tool. Again nothing actually changes here - we're just using the setup.cfg file to be a little more declarative and having no arbitrary code execution at install time via setup.py

4) Version management moved to the _version.py file - this allows us to do things like from mlflow_export_import import __version__

5) Standard Python .gitignore added (with mlruns/ exclusion)

Notes

  • Missing Entrypoints
    • import-models - this was a duplicate of import-all
    • export-model-list - this code is missing

Signed-off-by: Justin Flannery <juftin@juftin.com>
@juftin juftin force-pushed the feature/click_group_consolidation branch from 877ae18 to d6d05ba Compare July 26, 2022 23:14
Signed-off-by: Justin Flannery <juftin@juftin.com>
@juftin juftin marked this pull request as ready for review July 26, 2022 23:17
Signed-off-by: Justin Flannery <juftin@juftin.com>
Comment on lines +26 to +54
@click.group()
@click.version_option(version=__version__, prog_name=__application__)
@click.pass_context
def cli(ctx: click.core.Context) -> None:
"""
MLflow Export / Import CLI: Command Line Interface
"""
ctx.ensure_object(dict)


CLI_COMMANDS: List[click.Command] = [
export_all,
export_models,
import_all,
export_run,
import_run,
export_experiment,
import_experiment,
export_experiments,
import_experiments,
export_model,
import_model,
list_models,
http_client,
find_artifacts,
]

for command in CLI_COMMANDS:
cli.add_command(command)
Copy link
Author

@juftin juftin Jul 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's where all the magic happens to consolidate each click command. You can distribute a click CLI over different files in a few ways, but this way results in the least refactor.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's how the official MLflow CLI is organized using a similar method: https://github.com/mlflow/mlflow/blob/fd1f3aa246ca55a6dc0121ecf582c48b1c0287d0/mlflow/cli.py#L578-L583

Comment on lines +11 to +16
@click.command("list-models")
@click.option("--output-dir", help="Output directory.", default=".", type=str)
def main(output_dir): # pragma: no cover
"""
Lists all registered models.
"""
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's where I tell the main function that it's click command is "list-models". Click will handle this for us automatically if the function being decorated with click.command() was named list_models.

I also place the docstring on the function - click will automatically pass this to the CLI

@juftin juftin changed the title Entrypoint Consolidation Entrypoint Consolidation: All Functionality accessible via mlflow-export-import Aug 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant