@@ -225,9 +225,10 @@ and the requirement to record a performance audit. Hold the PR to both.
 
 ### Generated, review the source instead
 
-The compatibility pages under `docs/source/user-guide/latest/compatibility/expressions/` and
-`docs/source/user-guide/latest/configs.md` are produced by `GenerateDocs` and regenerated by CI on
-merge. Never ask the contributor to hand-edit them. Check what feeds them:
+`GenerateDocs` fills in the expression compatibility tables and the config tables when the docs are
+built, and CI rebuilds and publishes the docs when a merge changes their inputs. The **Generated Content** section of
+`docs/source/contributor-guide/style_guide.md` says exactly which text is generated. Never ask the
+contributor to hand-edit that text. Check what feeds it:
 
 - Expression marked `Incompatible(Some("..."))` but `getIncompatibleReasons()` is empty, so the
   compat page shows it as supported with no caveats.
 
@@ -100,19 +100,27 @@ configuration, from the inside out:
 
 - [ ] A new decorator forwards **every** `MemoryPool` method to its inner pool. A partial
       implementation makes `reserved()` disagree between levels.
-- [ ] **`fair_unified` checks the requesting consumer against its share, and the total against
-      the pool.** It divides `pool_size` by the number of registered consumers and rejects if what
-      the consumer holds plus the request exceeds that quotient, or if the pool's total reserved
-      plus the request exceeds `pool_size`. With two consumers and an 8 GiB pool, once one holds
-      3 GiB the other can still reserve up to 4 GiB. The pool keeps a running total for each
-      consumer id, so the sibling reservations that `new_empty()`, `split()` and `take()` create
-      count against one share. A PR that checks `reservation.size()` instead lets an operator with
-      several reservations, such as a sort's streaming merge, take other consumers' shares. It
-      also depends on when DataFusion updates the size, which is after it calls `try_grow` but
-      before it calls `shrink`. If the PR changes either comparison it changes when every query
-      spills, so it needs benchmark evidence, not reasoning.
-- [ ] **`num_consumers` counts every consumer in the task**, across every native plan, because the
-      pool is task-shared.
+- [ ] **`fair_unified` holds the consumers that can spill to a share of the task's share.** The
+      task's share is `spark.memory.offHeap.size` divided by the tasks on the executor running
+      native plans, which `RunningTasks` counts while each task holds its task-shared pool, or
+      `pool_size` if that is smaller. A consumer that can spill is
+      refused if what it holds plus the request exceeds what the consumers that cannot spill leave
+      of the task's share, divided by the registered consumers that can spill, or if the task's
+      total plus the request exceeds the task's share. A consumer that cannot spill is refused only
+      if the task's total plus the request exceeds `pool_size`, which binds before Spark's own
+      limit only while the deprecated fraction is below `1 / N`. With eight running tasks, a 16 GiB
+      pool and two consumers that can spill, each may hold 1 GiB, or 768 MiB while a merge holds
+      512 MiB. The pool keeps a running total for each consumer id, so the sibling reservations
+      that `new_empty()`, `split()` and `take()` create count against one share. A PR that checks
+      `reservation.size()` instead lets an operator with several reservations, such as a sort's
+      streaming merge, take other consumers' shares. It also depends on when DataFusion updates the
+      size, which is after it calls `try_grow` but before it calls `shrink`. A PR that limits a
+      consumer that cannot spill to a share, or sizes the shares from the whole pool, brings back
+      #7088: one operator takes the task's share from Spark and a merge or hash join build fails
+      the task. If the PR changes any of these comparisons it changes when every query spills, so
+      it needs benchmark evidence, not reasoning.
+- [ ] **The consumers that divide the task's share are every consumer that can spill in the
+      task**, across every native plan, because the pool is task-shared.
 - [ ] **Task-shared pool lifetime.** `acquire_task_shared_pool` keeps a process-wide
       `HashMap<task_attempt_id, Weak<TaskSharedMemoryPool>>`. The returned `Arc` is the only
       lifetime handle, so there is no explicit release to forget and a `createPlan` that fails
@@ -143,14 +151,16 @@ reclamation, that is a significant change and needs to say what it does to in-fl
 ## 5. Budget and Configuration
 
 `CometExecIterator.getMemoryConfig` computes the budget once per executor and passes it across JNI
-as `memory_limit`:
+as `memory_limit`, along with `spark.memory.offHeap.size` itself as `off_heap_size`, which
+`fair_unified` divides among the tasks running native plans:
 
 ```text
 memory_limit = spark.memory.offHeap.size * spark.comet.exec.memoryPool.fraction
+off_heap_size = spark.memory.offHeap.size
 ```
 
 - [ ] A new memory config goes through `getMemoryConfig` rather than being read separately in
-      native code.
+      native code, and a new value crossing JNI is added to `createPlan` on both sides.
 - [ ] Changing `memoryPool.fraction` semantics affects every deployment that tuned it as a haircut
       for the accounting gap.
 - [ ] On Kubernetes, `spark.memory.offHeap.size` is **part of** the pod limit, not headroom on top
@@ -204,8 +214,8 @@ will not catch. Check:
 - The **pool stack** diagram, if a decorator is added, removed, or reordered
 - The **pool type** table, if a pool type is added, removed, or changes how it is sized
 - The `memory_limit` formula, if the budget calculation changes
-- The **fair pool** description, which spells out the share and pool-total checks and their
-  consequences
+- The **fair pool** description, which spells out the task share, the share and task-total
+  checks, and their consequences
 - The **Crossing the FFI boundary** section, which names the operators that reserve for imported
   batches
 - The **Open problems** list. If the PR closes one of these gaps, the entry must be removed or
 
@@ -1,6 +1,6 @@
 ---
 name: review-comet-pr
-description: Use when reviewing a DataFusion Comet pull request. Covers the workflow that applies to every PR and routes to the area-specific review skills for expressions, FFI, memory management, shuffle, and Iceberg writes. Provides guidance to a human reviewer rather than posting comments.
+description: Use when reviewing a DataFusion Comet pull request. Covers the workflow that applies to every PR and routes to the area-specific review skills for expressions, FFI, memory management, shuffle, Iceberg writes, and documentation. Provides guidance to a human reviewer rather than posting comments.
 argument-hint: <pr-number>
 ---
 
@@ -43,6 +43,7 @@ change.
 | `native/core/src/execution/memory_pools/`, `CometTaskMemoryManager`, `CometShuffleMemoryAllocator`, `MemoryConsumer` / `try_grow` / `reserve` call sites, pool configs                     | `review-comet-memory-pr`        |
 | `native/shuffle/`, `spark/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/`, `spark/src/main/java/org/apache/spark/shuffle/`, `CometShuffleExchangeExec`                       | `review-comet-shuffle-pr`       |
 | `IcebergWriteStrategy`, `IcebergWriteExec`, `IcebergCommitExec`, `CometIcebergWriteExec`, `CometIcebergNativeWrite`, `iceberg_write.rs`, `iceberg_partition_path.rs`, the iceberg-rust pin | `review-comet-iceberg-write-pr` |
+| `docs/source/`, `README.md`, `AGENTS.md`, `.ai/skills/**/SKILL.md`, `.doc(...)` strings in config classes, doc comments                                                                    | `review-comet-docs`             |
 
 For a new operator, read `docs/source/contributor-guide/adding_a_new_operator.md` alongside this
 skill. There is no dedicated operator review skill yet.
@@ -52,8 +53,8 @@ For a PR that deals with timestamps or the session timezone, read
 cuts across the serde, the native kernels, the scans and the JVM/native boundary, so no area skill
 owns it. "Timestamps and timezones" in step 5 says what to check.
 
-If the PR falls outside all of these, for example build, CI, docs only, or release tooling, this
-skill on its own is the review.
+If the PR falls outside all of these, for example build, CI, or release tooling, this skill on its
+own is the review.
 
 ## 2. Gather PR Metadata
 
@@ -95,7 +96,7 @@ Anything that reports `getSupportLevel()` must report it honestly:
 
 - `Compatible()`, matches Spark exactly
 - `Incompatible(Some("reason"))`, differs in documented ways and requires
-  `spark.comet.expr.allowIncompatible=true`
+  `spark.comet.expression.<Name>.allowIncompatible=true`
 - `Unsupported(Some("reason"))`, cannot be implemented
 
 A PR that quietly marks something `Compatible` while the diff shows a known divergence is the single
@@ -173,7 +174,7 @@ Report the comparison in the review even when nothing changed, so the reviewer k
 
 Comet supports several Spark versions. The check on the Spark version belongs in Scala, either in a
 shim under `spark/src/main/spark-*/` or in one named predicate such as `isSpark42Plus` or
-`RegrSparkVersions.slopeFiltersVarByPairNulls`. It does not belong in an ad hoc comparison on the
+`SparkFixes.slopeFiltersVarByPairNulls`. It does not belong in an ad hoc comparison on the
 version string in shared code, or in native Rust. If the PR adds a shim for one 4.x version, check
 that the sibling 4.x source sets got it too.
 
@@ -264,7 +265,8 @@ For a change on a default path, look for tests with the inputs that broke earlie
 
 ### CI
 
-`gh pr checks <pr> --repo apache/datafusion-comet`, and again with `--failed` for detail.
+`gh pr checks <pr> --repo apache/datafusion-comet`. For only the failed checks, add
+`--json name,state,bucket,link --jq '.[] | select(.bucket == "fail")'`.
 
 Summarize failures in the review. Do not compare against failures on `main`.
 
@@ -299,10 +301,13 @@ same PR, because a follow-up that is not filed as an issue does not happen.
 Each area skill names the doc for its subsystem and the specific claims in it that go stale. When you
 load an area skill, read its doc and hold the diff against it.
 
-**Do not ask contributors to hand-edit generated docs.** The compatibility guide pages under
-`docs/source/user-guide/latest/compatibility/expressions/` and
-`docs/source/user-guide/latest/configs.md` are produced by `GenerateDocs` and regenerated by CI on
-every merge to `main`. Review the source that feeds them instead.
+**Do not ask contributors to hand-edit generated docs.** `GenerateDocs` fills in the config tables
+and the expression compatibility tables when the docs are built, and CI rebuilds and publishes the
+docs when a merge to `main` changes their inputs. The
+**Generated Content** section of `docs/source/contributor-guide/style_guide.md` says exactly which
+text is generated. Review the source that feeds it instead. The prose around the generated tables,
+the `_category_template/` files, and the `index.md` pages are written by hand and are reviewed
+normally.
 
 ## Review Bar
 
 
@@ -116,6 +116,9 @@ github:
     branch-1.1:
       required_pull_request_reviews:
         required_approving_review_count: 1
+    branch-1.2:
+      required_pull_request_reviews:
+        required_approving_review_count: 1
   # GitHub's merge queue. Classic `protected_branches` above has no way to
   # express this, but .asf.yaml rulesets accept a raw Rulesets API payload, so
   # no INFRA ticket is needed. Same approach as apache/iceberg-python and
@@ -179,6 +182,13 @@ github:
     # Lets a committer queue a PR before its checks have finished, which is
     # most of the point of having a queue.
     allow_auto_merge: true
+    # Limit open pull requests per user without write access.
+    #
+    # With coding agents, the number of open PRs now far exceeds our review capacity.
+    # Setting a limit helps reviewers focus on fewer PRs at a time.
+    creation_cap:
+      enabled: true
+      max_open_pull_requests: 3
 # publishes the content of the `asf-site` branch to
 # https://datafusion.apache.org/comet/
 publish:
 
@@ -23,12 +23,28 @@ runs:
   steps:
     # Note: cargo fmt check is now handled by the lint job that gates this workflow
 
+    - name: Set up embedded Python for native UDF tests
+      shell: bash
+      run: |
+        apt-get update
+        apt-get install -y --no-install-recommends python3 python3-dev python3-venv
+        python3 -m venv /tmp/comet-rust-python
+        /tmp/comet-rust-python/bin/pip install "pyarrow>=14" cloudpickle
+        echo "PYO3_PYTHON=/tmp/comet-rust-python/bin/python" >> "$GITHUB_ENV"
+        echo "PYTHONPATH=$(/tmp/comet-rust-python/bin/python -c 'import sysconfig; print(sysconfig.get_path("purelib"))')" >> "$GITHUB_ENV"
+
     - name: Check Cargo clippy
       shell: bash
       run: |
         cd native
         cargo clippy --color=never --all-targets --workspace -- -D warnings
 
+    - name: Check native Python UDF feature
+      shell: bash
+      run: |
+        cd native
+        cargo clippy --color=never -p datafusion-comet --all-targets --features python-udf -- -D warnings
+
     - name: Check compilation
       shell: bash
       run: |
@@ -83,6 +99,13 @@ runs:
         export LD_LIBRARY_PATH=${JAVA_HOME}/lib/server:${LD_LIBRARY_PATH}
         RUST_BACKTRACE=1 cargo nextest run
 
+    - name: Run native Python UDF tests
+      shell: bash
+      run: |
+        cd native
+        export LD_LIBRARY_PATH=${JAVA_HOME}/lib/server:${LD_LIBRARY_PATH}
+        RUST_BACKTRACE=1 cargo nextest run -p datafusion-comet --lib --features python-udf python_udf
+
     # The steps above lint and test the accounting allocator over the system allocator. Lint and
     # test it over jemalloc too, since that is a different allocator backend and nothing else in CI
     # builds it.
 
@@ -53,6 +53,42 @@ updates:
         update-types:
           - "minor"
           - "patch"
+  - package-ecosystem: cargo
+    directory: "/native/native-executors"
+    schedule:
+      interval: weekly
+    target-branch: main
+    labels: [dependencies]
+    ignore:
+      # major version bumps of datafusion*, arrow*, parquet are handled manually
+      - dependency-name: "arrow*"
+        update-types: ["version-update:semver-major"]
+      - dependency-name: "parquet"
+        update-types: ["version-update:semver-major"]
+      - dependency-name: "datafusion*"
+        update-types: ["version-update:semver-major"]
+    groups:
+      proto:
+        applies-to: version-updates
+        patterns:
+          - "prost*"
+          - "pbjson*"
+      # Catch-all: group only minor/patch into a single PR,
+      # excluding deps we want always separate (and excluding arrow/parquet which have their own group)
+      all-other-cargo-deps:
+        applies-to: version-updates
+        patterns:
+          - "*"
+        exclude-patterns:
+          - "arrow*"
+          - "parquet"
+          - "object_store"
+          - "sqlparser"
+          - "prost*"
+          - "pbjson*"
+        update-types:
+          - "minor"
+          - "patch"
   - package-ecosystem: "github-actions"
     directory: "/"
     schedule:
 
@@ -95,12 +95,9 @@ area:ffi:
   - changed-files:
       - any-glob-to-any-file:
           - 'native/jni-bridge/**'
-          - 'native/core/src/execution/columnar_to_row.rs'
           - 'spark/src/main/**/vector/**'
           - 'common/src/main/**/vector/**'
-          - 'spark/src/main/scala/org/apache/comet/NativeColumnarToRowConverter.scala'
           - 'spark/src/main/scala/org/apache/spark/sql/comet/CometColumnarToRowExec.scala'
-          - 'spark/src/main/scala/org/apache/spark/sql/comet/CometNativeColumnarToRowExec.scala'
           - 'spark/src/main/scala/org/apache/spark/sql/comet/CometSparkToColumnarExec.scala'
 
 area:memory: