Senior Data Engineer · Data Platform · Workflow Orchestration · GitOps
Senior Data Engineer with 12 years of experience. From batch and pipeline orchestration (Apache Airflow) to query engines (Trino), transformation (dbt), and the Kubernetes + GitOps platform that runs it all, I design and reliably operate every stage that data flows through. I currently run an Airflow 3 platform with 3,600 DAGs on Kubernetes.
- 🛠️ Data platform ownership — operating the Airflow / Trino / dbt stack across multiple clusters (dev · stag · prd)
- 🔀 Orchestration at scale — Kubernetes-based Airflow 3.x, Git-bundle DAGs, custom image pipelines
- 🚀 GitOps first — declarative delivery with ArgoCD; every change goes Git → review → sync
- 🔎 Root-cause driven — digging down to the transaction and session level to find the real cause, and contributing upstream when needed
- 🔐 Governed access — data access control built on OPA / Ranger / OIDC
🎯 Running a 3,600-DAG production platform, I trace the bugs and race conditions I hit down to their root cause and contribute the fixes directly upstream (Apache Airflow, Trino).
Apache Airflow
| Contribution | Status | |
|---|---|---|
| apache/airflow#69845 | Fix DAG import errors being dropped during parsing | 🟢 under review |
| apache/airflow#70002 | Fix dag_tag UniqueViolation race with HA dag processors |
🟢 under review |
| apache/airflow#74722 | Don't fail requests with a bare 500 on transient JWT refresh errors (re-submitted from #70057) | 🟢 under review |
| apache/airflow#70056 | Report: scheduler silently skips queued DagRuns with unresolvable pinned versions | ✅ fixed upstream in 3.3.2 (#71696) |
| apache/airflow#70058 | Add refresh_on_initialize option to GitDagBundle |
⚪ closed after review (kept as an internal subclass) |
| apache/airflow#70003 | Add pin_symlink_on_refresh to LocalDagBundle for git-sync checkouts |
⚪ closed (stale) |
Trino
| Contribution | Status | |
|---|---|---|
| trinodb/trino#28630 | "SHOW COLUMNS only returns the first row" — traced to an OPA batched filterColumns policy corner case (not a Trino bug); the reporter confirmed the fix and closed the issue |
✅ root-caused & closed |
| trinodb/trino#28602 | OPA access control: on partial-permission denial for SELECT *, the error message lists every column name — shared code pointers and a local repro; maintainers are fixing it per access-control plugin (Ranger: #30435), OPA still open |
💬 in discussion |
| Area | Details |
|---|---|
| Pipeline orchestration | Apache Airflow 3.x on Kubernetes — Git-bundle DAGs, custom images (dbt/ClickHouse/Impala venvs), multi-env (dev · stag · prd) |
| Query & analytics | Trino coordinators/workers, OPA-based access policies, group sync automation |
| Transformation | dbt (core + ClickHouse / Impala adapters) |
| Platform / GitOps | ArgoCD-managed EKS & on-prem clusters, declarative manifests, selective sync & rollout |
| Reliability | multi-replica DAG-processor race conditions, transaction-safety, TLS & auth-proxy hardening |
Languages
Pipeline & Streaming
Data Platform
Infra & DevOps
Observability & Security
Backend & DB
