Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mlflow.exceptions.RestException: INVALID_PARAMETER_VALUE: Tag name mlflow.source.type contains one or more of the reserved characters: List(., ,, -, =, /, :) make: *** [Makefile:58: steps/register/outputs/registered_model_version.json] Error 1 #58

Open
ahmadjami opened this issue Jan 29, 2025 · 0 comments
Open

mlflow.exceptions.RestException: INVALID_PARAMETER_VALUE: Tag name mlflow.source.type contains one or more of the reserved characters: List(., ,, -, =, /, :) make: *** [Makefile:58: steps/register/outputs/registered_model_version.json] Error 1 #58

ahmadjami opened this issue Jan 29, 2025 · 0 comments

Comments

@ahmadjami
Copy link

ERROR while registering model in Databricks

2025/01/29 08:10:54 INFO mlflow.recipes.utils.execution: ingest, split, transform, train, evaluate: No changes. Skipping.
Run MLflow Recipe step: register
2025/01/29 08:10:59 INFO mlflow.recipes.step: Running step register...
2025/01/29 08:11:00 WARNING mlflow.recipes.step: Encountered unexpected failure while initializing Spark connection. Spark operations may not succeed. Exception: [CONTEXT_UNAVAILABLE_FOR_REMOTE_CLIENT] Remote client cannot create a SparkContext. Create SparkSession instead.
Wed Jan 29 08:11:00 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:00 2025 Initialized gateway on port 41459
Wed Jan 29 08:11:01 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:01 2025 Initialized gateway on port 44379
Wed Jan 29 08:11:01 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:01 2025 Initialized gateway on port 39197
Successfully registered model 'sd_common_dev_databricks_workspace01.default.insurance_recipe_model_exp1'.
Wed Jan 29 08:11:01 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:01 2025 Initialized gateway on port 37569
Wed Jan 29 08:11:01 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:01 2025 Initialized gateway on port 35021
Wed Jan 29 08:11:01 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:01 2025 Initialized gateway on port 36887
Wed Jan 29 08:11:01 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:01 2025 Initialized gateway on port 41477
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 40515
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 37327
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 43345
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 34531
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 44275
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 41891
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 35741
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 35761
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 37473
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 46617
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 45719
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 46367
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 34771
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 37591
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 45583
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 34313
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 36041
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 41245
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 43329
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 41351
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 40225
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 33605
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 38055
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 39995
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 43849
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 33983
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 35787
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 34907
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 43039
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 39677
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 42015
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 36413
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 38485
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 46197
Wed Jan 29 08:11:02 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:02 2025 Initialized gateway on port 34131
Wed Jan 29 08:11:03 2025 Connection to spark from PID 43576
Wed Jan 29 08:11:03 2025 Initialized gateway on port 39367
Traceback (most recent call last):
File "", line 1, in
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/recipes/step.py", line 132, in run
self.step_card = self._run(output_directory=output_directory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/recipes/steps/register.py", line 79, in _run
self.model_details = mlflow.register_model(
^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/tracking/_model_registry/fluent.py", line 77, in register_model
return _register_model(
^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/tracking/_model_registry/fluent.py", line 111, in _register_model
create_version_response = client._create_model_version(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/tracking/client.py", line 3669, in _create_model_version
return self._get_registry_client().create_model_version(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 221, in create_model_version
mv = self.store.create_model_version(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/store/_unity_catalog/registry/rest_store.py", line 803, in create_model_version
model_version = self._call_endpoint(
^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/store/model_registry/base_rest_store.py", line 44, in _call_endpoint
return call_endpoint(
^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/utils/rest_utils.py", line 379, in call_endpoint
response = verify_rest_response(response, endpoint)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/utils/rest_utils.py", line 249, in verify_rest_response
raise RestException(json.loads(response.text))
mlflow.exceptions.RestException: INVALID_PARAMETER_VALUE: Tag name mlflow.source.type contains one or more of the reserved characters: List(., ,, -, =, /, :)
make: *** [Makefile:58: steps/register/outputs/registered_model_version.json] Error 1

=========

MlflowException: Failed to run step 'register' of recipe 'recipes_regression_exp1':
The following error occurred while running step 'Step:register':
Traceback (most recent call last):
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/recipes/step.py", line 132, in run
self.step_card = self._run(output_directory=output_directory)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/recipes/steps/register.py", line 79, in _run
self.model_details = mlflow.register_model(
^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/tracking/_model_registry/fluent.py", line 77, in register_model
return _register_model(
^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/tracking/_model_registry/fluent.py", line 111, in _register_model
create_version_response = client._create_model_version(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/tracking/client.py", line 3669, in _create_model_version
return self._get_registry_client().create_model_version(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/tracking/_model_registry/client.py", line 221, in create_model_version
mv = self.store.create_model_version(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/store/_unity_catalog/registry/rest_store.py", line 803, in create_model_version
model_version = self._call_endpoint(
^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/store/model_registry/base_rest_store.py", line 44, in _call_endpoint
return call_endpoint(
^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/utils/rest_utils.py", line 379, in call_endpoint
response = verify_rest_response(response, endpoint)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/utils/rest_utils.py", line 249, in verify_rest_response
raise RestException(json.loads(response.text))
mlflow.exceptions.RestException: INVALID_PARAMETER_VALUE: Tag name mlflow.source.type contains one or more of the reserved characters: List(., ,, -, =, /, :)

Last step status: 'StepStatus.FAILED'
File , line 1
----> 1 r.run("register")
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.11/site-packages/mlflow/recipes/recipe.py:106, in BaseRecipe.run(self, step)
100 last_step_error_mesg = (
101 f"The following error occurred while running step '{last_executed_step}':\n"
102 f"{last_executed_step_state.stack_trace}\n"
103 f"Last step status: '{last_executed_step_state.status}'\n"
104 )
105 if step is not None:
--> 106 raise MlflowException(
107 f"Failed to run step '{step}' of recipe '{self.name}':\n{last_step_error_mesg}",
108 error_code=BAD_REQUEST,
109 )
110 else:
111 raise MlflowException(
112 f"Failed to run recipe '{self.name}':\n{last_step_error_mesg}",
113 error_code=BAD_REQUEST,
114 )
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ahmadjami