Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add/port metrics to the new Execution interface #45511

Open
kaxil opened this issue Jan 9, 2025 · 0 comments
Open

Add/port metrics to the new Execution interface #45511

kaxil opened this issue Jan 9, 2025 · 0 comments
Labels
area:metrics area:task-execution-interface-aip72 AIP-72: Task Execution Interface (TEI) aka Task SDK area:task-sdk
Milestone

Comments

@kaxil
Copy link
Member

kaxil commented Jan 9, 2025

Find a way to emit metrics from the new supervisor process.

Some examples of what needs porting over:

Stats.incr("local_task_job_prolonged_heartbeat_failure", 1, 1)

def _log_return_code_metric(self, return_code: int):
Stats.incr(
"local_task_job.task_exit."
f"{self.job.id}.{self.task_instance.dag_id}.{self.task_instance.task_id}.{return_code}"
)
# Same metric with tagging
Stats.incr(
"local_task_job.task_exit",
tags={
"job_id": self.job.id,
"dag_id": self.task_instance.dag_id,
"task_id": self.task_instance.task_id,
"return_code": return_code,
},
)

@kaxil kaxil converted this from a draft issue Jan 9, 2025
@kaxil kaxil added area:task-execution-interface-aip72 AIP-72: Task Execution Interface (TEI) aka Task SDK area:task-sdk labels Jan 9, 2025
@dosubot dosubot bot added the area:metrics label Jan 9, 2025
@kaxil kaxil added this to the Airflow 3.0.0 milestone Jan 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:metrics area:task-execution-interface-aip72 AIP-72: Task Execution Interface (TEI) aka Task SDK area:task-sdk
Development

No branches or pull requests

1 participant