Skip to content

Add missing ti.start and ti.finish metrics in Airflow 3#62019

Merged
amoghrajesh merged 2 commits into
apache:mainfrom
astronomer:missing-ti-metrics
Feb 18, 2026
Merged

Add missing ti.start and ti.finish metrics in Airflow 3#62019
amoghrajesh merged 2 commits into
apache:mainfrom
astronomer:missing-ti-metrics

Conversation

@amoghrajesh

@amoghrajesh amoghrajesh commented Feb 16, 2026

Copy link
Copy Markdown
Contributor
Was generative AI tooling used to co-author this PR?
  • No

closes: #62015

Some metrics like ti.start and ti.finish were not ported over to Airflow 3.

This PR intends to add those to Airflow 3 and emit it from task execution.

Testing

Deployed Airflow with breeze with --integration statsd

Earlier:

image (67) image (68)

After:

Prometheus
image image
Statsd

Finish:

➜  ~ curl -s http://localhost:29102/metrics | grep -i "finish"

# HELP airflow_ti_finish Metric autogenerated by statsd_exporter.
# TYPE airflow_ti_finish counter
airflow_ti_finish 2
# HELP airflow_ti_finish_sleeping_sleep_success Metric autogenerated by statsd_exporter.
# TYPE airflow_ti_finish_sleeping_sleep_success counter
airflow_ti_finish_sleeping_sleep_success 2

Start:

➜  ~ curl -s http://localhost:29102/metrics | grep -i "start"
# HELP airflow_job_start Metric autogenerated by statsd_exporter.
# TYPE airflow_job_start counter
airflow_job_start 2
# HELP airflow_ti_start Metric autogenerated by statsd_exporter.
# TYPE airflow_ti_start counter
airflow_ti_start 4
# HELP airflow_ti_start_sleeping_sleep Metric autogenerated by statsd_exporter.
# TYPE airflow_ti_start_sleeping_sleep counter
airflow_ti_start_sleeping_sleep 4
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.77124692275e+09
  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@amoghrajesh amoghrajesh added this to the Airflow 3.1.8 milestone Feb 16, 2026
@amoghrajesh amoghrajesh self-assigned this Feb 16, 2026
@amoghrajesh

amoghrajesh commented Feb 16, 2026

Copy link
Copy Markdown
Contributor Author

cc @kacpermuda @xBis7 apologies if wrongly tagged, but I think you are involved with metrics and could help with reviews

kacpermuda
kacpermuda approved these changes Feb 16, 2026
View reviewed changes

@kacpermuda kacpermuda left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@kacpermuda

kacpermuda commented Feb 16, 2026

Copy link
Copy Markdown
Collaborator

I think you are involved with metrics and could help with reviews

I'm not, but the code looks good, left an approve, thanks !

xBis7
xBis7 reviewed Feb 16, 2026
View reviewed changes

@xBis7 xBis7 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amoghrajesh Thanks for pinging me! I left two comments.

Comment thread task-sdk/src/airflow/sdk/execution_time/task_runner.py
Comment thread task-sdk/src/airflow/sdk/execution_time/task_runner.py
xBis7
xBis7 approved these changes Feb 17, 2026
View reviewed changes

@xBis7 xBis7 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amoghrajesh Based on the provided context and that this fix is meant for 3.1.8, ignore my comments for using the DualStatsManager. LGTM!

@amoghrajesh amoghrajesh merged commit 8d8048c into apache:main Feb 18, 2026
101 checks passed
@amoghrajesh amoghrajesh deleted the missing-ti-metrics branch February 18, 2026 07:44
github-actions Bot pushed a commit that referenced this pull request Feb 18, 2026
@amoghrajesh
[v3-1-test] Add missing ti.start and ti.finish metrics in Airflow 3 (#…
aaf3bf6 
…62019)

(cherry picked from commit 8d8048c)

Co-authored-by: Amogh Desai <[email protected]>
@github-actions

github-actions Bot commented Feb 18, 2026

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-1-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
✅ v3-1-test PR Link

amoghrajesh added a commit that referenced this pull request Feb 18, 2026
@github-actions @amoghrajesh
[v3-1-test] Add missing ti.start and ti.finish metrics in Airflow 3 (#…
a173777 
…62019) (#62110)

(cherry picked from commit 8d8048c)

Co-authored-by: Amogh Desai <[email protected]>
sjyangkevin pushed a commit to sjyangkevin/airflow that referenced this pull request Feb 18, 2026
@amoghrajesh @sjyangkevin
Add missing ti.start and ti.finish metrics in Airflow 3 (apache#62019)
7e25b15
choo121600 pushed a commit to choo121600/airflow that referenced this pull request Feb 22, 2026
@amoghrajesh @choo121600
Add missing ti.start and ti.finish metrics in Airflow 3 (apache#62019)
03fa3d0
@xBis7 xBis7 mentioned this pull request Mar 2, 2026
Closed
2 tasks
Subham-KRLX pushed a commit to Subham-KRLX/airflow that referenced this pull request Mar 4, 2026
@amoghrajesh @Subham-KRLX
Add missing ti.start and ti.finish metrics in Airflow 3 (apache#62019)
fe1caf7
vatsrahul1001 pushed a commit that referenced this pull request Mar 4, 2026
@github-actions @amoghrajesh @vatsrahul1001
[v3-1-test] Add missing ti.start and ti.finish metrics in Airflow 3 (#…
8f9e513 
…62019) (#62110)

(cherry picked from commit 8d8048c)

Co-authored-by: Amogh Desai <[email protected]>
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
@amoghrajesh @dominikhei
Add missing ti.start and ti.finish metrics in Airflow 3 (apache#62019)
9d10bf8
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
@amoghrajesh @Ankurdeewan
Add missing ti.start and ti.finish metrics in Airflow 3 (apache#62019)
7ff792e
@myps6415 myps6415 mentioned this pull request May 27, 2026
Open
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vatsrahul1001 vatsrahul1001 vatsrahul1001 approved these changes
@ashb ashb Awaiting requested review from ashb ashb is a code owner
@kaxil kaxil Awaiting requested review from kaxil kaxil is a code owner
@potiuk potiuk Awaiting requested review from potiuk
@mobuchowski mobuchowski Awaiting requested review from mobuchowski
+2 more reviewers
@xBis7 xBis7 xBis7 approved these changes
@kacpermuda kacpermuda kacpermuda approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@amoghrajesh amoghrajesh

Labels

area:task-sdk

Projects

None yet

Milestone

Airflow 3.1.8

Development

Successfully merging this pull request may close these issues.

Missing ti.start and ti.finish metrics in Airflow 3

5 participants

@amoghrajesh @kacpermuda @ashb @vatsrahul1001 @xBis7