Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Add a project launcher #67 #75

Open
wants to merge 21 commits into
base: main
Choose a base branch
from

Conversation

nikhiljangra264
Copy link
Contributor

@nikhiljangra264 nikhiljangra264 commented Nov 19, 2024

closes #67

Added launcher.py

  1. launch web application
python launcher.py --mode=web
  1. install dependencies
python launcher.py --install
  1. rebuilding backend MediaProcessor
python launcher.py --rebuild
  1. help
python launcher.py --help

Copy link
Owner

@omeryusufyagci omeryusufyagci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @nikhiljangra264, thanks a lot for this great start! I've also fixed 2 bugs while testing this version and have already pushed them here. For the rest, I've taken a quick look and left some comments.

launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
@omeryusufyagci omeryusufyagci added feature New feature or request platform iteration Requires further iteration for approval labels Nov 20, 2024
MediaProcessor/tests/TestUtils.cpp Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
Copy link
Owner

@omeryusufyagci omeryusufyagci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes @nikhiljangra264! I've also tested it locally on an Ubuntu system that already had all dependencies installed. Looks good already; please see my comments for details.

MediaProcessor/cmake/src.cmake Outdated Show resolved Hide resolved
MediaProcessor/cmake/test.cmake Outdated Show resolved Hide resolved
MediaProcessor/tests/TestUtils.cpp Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
config.json Outdated
Comment on lines 2 to 5
"deep_filter_path": "MediaProcessor/res/deep-filter-0.5.6-x86_64-unknown-linux-musl",
"deep_filter_tarball_path": "MediaProcessor/res/DeepFilterNet3_ll_onnx.tar.gz",
"deep_filter_encoder_path": "MediaProcessor/res/DeepFilterNet3_ll_onnx/tmp/export/enc.onnx",
"deep_filter_decoder_path": "MediaProcessor/res/DeepFilterNet3_ll_onnx/tmp/export/df_dec.onnx",
"deep_filter_path": "MediaProcessor\\res\\deep-filter-0.5.6-x86_64-unknown-linux-musl",
"deep_filter_tarball_path": "MediaProcessor\\res\\DeepFilterNet3_ll_onnx.tar.gz",
"deep_filter_encoder_path": "MediaProcessor\\res\\DeepFilterNet3_ll_onnx\\tmp\\export\\enc.onnx",
"deep_filter_decoder_path": "MediaProcessor\\res\\DeepFilterNet3_ll_onnx\\tmp\\export\\df_dec.onnx",
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may fix it for windows, but breaks it for Unix. While testing this PR on macOS I got some panics from the library which was caused by a file not found due to this. This happened after running the launcher.

Please dynamically update the paths for Windows and let's keep the default paths with the Unix slashes. There seems to be an issue with the dynamic update of this. Could you please check ? Thanks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is launcher.py successfully updates the path but config.json modified can be accidently pushed with other changes (In this case).

we can change them back after running but what if user runs via command python app.py. That will not work because of paths. So for windows development we have to be careful not to push the config.json.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, the launcher will be the official method to launch any application or even for setting up the repository. Of course it's open source, so everyone is free to hack-away at the code, but we don't necessarily have to support use cases that fall outside of the officially supported methods -that's why we provide a launcher.

I agree with you on your other point though. I'd suggest exploring a runtime override (potentially even a second runtime_config.json) instead of directly overwriting the file. The config.json would serve as a base.

On a second note, we should add a check on app.py to warn the user if it's called from outside the launcher.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should add a runtime json that we dont have to warn user about launching outside the launcher.py
we should standardize to use launcher.py

launcher.py Outdated Show resolved Hide resolved
1. Capitalize virutal env constants `VENV_NAME` and `VENV_DIR`
2. Use pathlib lib for paths
Add python dependencies and MSYS2 as dependency
Now `config.json` is converted to `runtime_config.json` file for cross paltform compatibility.
1. Fix launcher.py runtime errors
2. utf_16 size error on Windows
1. Move config file handling and logging config to `Config`
2. Move Virtual env handling, running command, check internet connectivity to `Utils` class.
3. Move running web application to `WebApplication` class.
Copy link
Owner

@omeryusufyagci omeryusufyagci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Please see my comments for details

launcher.py Outdated Show resolved Hide resolved
launcher.py Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
launcher.py Outdated Show resolved Hide resolved
Copy link
Owner

@omeryusufyagci omeryusufyagci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nikhiljangra264, looks all good for the updates since the last review!

Have you tested this on Win10, Darwin, and Linux? I don't have access to my HW to do platform test for a few more days.

Otherwise, this is quite OK for an initial implementation. I'll test it over the weekend, and if all goes well, this should get merged before next week!

@omeryusufyagci omeryusufyagci removed the iteration Requires further iteration for approval label Jan 1, 2025
@nikhiljangra264
Copy link
Contributor Author

Thanks @nikhiljangra264, looks all good for the updates since the last review!

Have you tested this on Win10, Darwin, and Linux? I don't have access to my HW to do platform test for a few more days.

Otherwise, this is quite OK for an initial implementation. I'll test it over the weekend, and if all goes well, this should get merged before next week!

Tested on linux, Windows 11, deleted everthing and download again working fine.

This fixes an issue where the .dylib was pointing at a hardcoded
local path instead of @rpath.
@omeryusufyagci omeryusufyagci self-requested a review January 8, 2025 21:38
Copy link
Owner

@omeryusufyagci omeryusufyagci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested on Darwin macOS.

The lack of any logging threw me off initially. We should tell the user at least the steps the launcher is taking. Not debug level details, but enough to make them understand things are progressing. I thought the launcher was hanging until I started with debug mode.

Then, I ran into an issue with the .dylib pointing to a local hardcoded path instead of @ rpath.
I had seen this the last time around and fixed it, but apparently that was gone with the rebase. Now it's fixed again and I've updated the PR as you'll see.

I'll still take a look at it on Windows, but in the meantime please double check to see we haven't missed anything and take another look at the logging we're providing.

Thanks!

@nikhiljangra264
Copy link
Contributor Author

Checked the code.
Please take a look. contributed after Long time. took some time. :)

@omeryusufyagci
Copy link
Owner

Checked the code. Please take a look. contributed after Long time. took some time. :)

Thanks, and no worries!

Trying to understand why I wasn't seeing any logs, I discovered that the logging setup is delegated to the WebApplication class.
In general, control flow seems to depend on WebApplication and for generic things in the launcher we should have common methods.
I have added FIXMEs in-place please check my last commit here to see more details.

I would recommend adding dedicated methods to setup/initialize and cleanup (if needed) the launcher itself. That would include setting up the logging, resolving common dependencies, etc. So, we shouldn't treat it as if WebApp will be the only user of the launcher.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request platform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature: Add a project launcher
2 participants