Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Centralize argument parsing #948

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Centralize argument parsing #948

wants to merge 2 commits into from

Conversation

matthiasdiener
Copy link
Member

@matthiasdiener matthiasdiener commented Aug 7, 2023

Questions for the review:

  • Is the scope and purpose of the PR clear?
    • The PR should have a description.
    • The PR should have a guide if needed (e.g., an ordering).
  • Is every top-level method and class documented? Are things that should be documented actually so?
  • Is the interface understandable? (I.e. can someone figure out what stuff does?) Is it well-defined?
  • Does the implementation do what the docstring claims?
  • Is everything that is implemented covered by tests?
  • Do you see any immediate risks or performance disadvantages with the design? Example: what do interface normals attach to?

@matthiasdiener matthiasdiener requested review from majosm and MTCam August 7, 2023 20:09
@matthiasdiener matthiasdiener self-assigned this Aug 7, 2023
@matthiasdiener matthiasdiener force-pushed the argparse-new branch 2 times, most recently from 362031b to 03e03b4 Compare August 7, 2023 20:11
Base automatically changed from numpyactx-support to main August 9, 2023 21:11
@matthiasdiener matthiasdiener marked this pull request as ready for review August 9, 2023 21:43
@@ -733,25 +733,14 @@ def my_post_step(step, t, dt, state):


if __name__ == "__main__":
from mirgecom.simutil import add_general_args
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add_predefined_cmd_line_options?

dest="restart_file", nargs="?", action="store",
help="simulation restart file")
parser.add_argument("--casename", help="casename to use for i/o")
add_general_args(parser, overintegration=False)
Copy link
Collaborator

@majosm majosm Aug 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd prefer if this was a little more customizable, and if the options were opt-in rather than opt-out. Otherwise a driver might end up accepting options that it doesn't actually support. Maybe something like:

add_predefined_cmd_line_options(parser, [
    "lazy",
    "profiling",
    "log",
    "leap",
    "numpy",
    "restart_file",
    "casename"])

?

Copy link
Member Author

@matthiasdiener matthiasdiener Aug 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see your point, but wouldn't that lead to similar issues we have right now? E.g., adding a new option, like --numpy or --esdg, requires changes in every driver?

A driver can also choose to ignore certain options (by not accessing the corresponding args.foo option, or throwing an error).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but the driver would have to be modified in order to actually use the new option anyway. Doing the above would at least reduce the amount of boilerplate.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A driver can also choose to ignore certain options (by not accessing the corresponding args.foo option, or throwing an error).

The ignoring thing scares me a little... if a particular driver doesn't support an option, IMO it shouldn't accept it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As someone who is constantly writing and maintaining drivers, I would like it very much if the options for actx creation, the MPI entry point, and any other non-physics options (even logpyle init) could be parsed by the "front-matter" code, and the rest of the "application-specific" or "simulation-specific" args could be handled by the specific physics-oriented driver.

casename, restart file specification, etc, all belong to the physics driver domain, and not the "front-matter" stuff. We can easily write generalized components to handle common sets of physics or simulation control args.

There is a built-in argparse mechanism for doing this parse_known_args that lets us parse command line args in stages, by different components.

Copy link
Member

@MTCam MTCam Aug 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A driver can also choose to ignore certain options (by not accessing the corresponding args.foo option, or throwing an error).

The ignoring thing scares me a little... if a particular driver doesn't support an option, IMO it shouldn't accept it.

This should be up to the driver developer imo. parse_known_args will continue to accumulate the un-parsed args, and at the end if there are any unknown args, let the driver toss an error if the driver developer deems that necessary, or give a warning or ignore.

(of course we can require that our examples toss an error, but don't force everyone to do it)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be up to the driver developer imo. parse_known_args will continue to accumulate the un-parsed args, and at the end if there are any unknown args, let the driver toss an error if the driver developer deems that necessary, or give a warning or ignore.

(of course we can require that our examples toss an error, but don't force everyone to do it)

Makes sense. Could have the "front matter" call parse_known_args and then the examples can call parse_args on whatever's left I guess?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's along the lines of what I was thinking, then we can make like 2 or 3 more drivers (to rule them all) which drive fluid-only or coupled problems and parse all the common args there, and so on. This would probably be sufficient for all the examples and reduce our maintenance task to just those few drivers.

Each component can use the standard built-in argparse facility for defining the args and the help strings, etc, and we can just gather up all the common ones into convenient blocks.

@matthiasdiener matthiasdiener mentioned this pull request Aug 17, 2023
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants