Update README occording to the new examples (#18529) by cj-zhukov · Pull Request #19257 · apache/datafusion

cj-zhukov · 2025-12-10T12:35:50Z

Which issue does this PR close?

Closes #Update datafusion-examples/README.md occording to the new examples format #18529.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

cj-zhukov · 2025-12-10T12:42:25Z

High Level Overview

This PR restructures and improves the organization of the README for datafusion-examples directory to make it significantly easier for users to discover, understand, and run examples. The main goals are consistency, clarity, and improved developer experience. The tables provide a quick and complete overview of the available examples.

Note: The ffi_xyz crate is not included, as it's not a runnable example. It also currently does not appear in the README. If needed, a follow-up PR can clarify its purpose and documentation.

alamb · 2025-12-10T14:25:21Z

For anyone else who is curious, you can see it rendered at this page: https://github.com/cj-zhukov/datafusion/tree/cj-zhukov/update-readme-occording-to-the-new-examples/datafusion-examples

alamb

Thanks @cj-zhukov -- this look like a nice improvement to me

I wonder if there is any way to ensure this document stays in sync with the code. For example, if someone adds a new example, how do we ensure that an appropriate entry is added here 🤔

Maybe we could add a ci check similar to the config doc check

datafusion/.github/workflows/rust.yml

Lines 683 to 710 in 2a08013

    
             config-docs-check: 
        
               name: check configs.md and ***_functions.md is up-to-date 
        
               needs: linux-build-lib 
        
               runs-on: ubuntu-latest 
        
               container: 
        
                 image: amd64/rust 
        
               steps: 
        
                 - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8  # v6.0.1 
        
                   with: 
        
                     submodules: true 
        
                     fetch-depth: 1 
        
                 - name: Setup Rust toolchain 
        
                   uses: ./.github/actions/setup-builder 
        
                   with: 
        
                     rust-version: stable 
        
                 - uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f  # v6.1.0 
        
                   with: 
        
                     node-version: "20" 
        
                 - name: Check if configs.md has been modified 
        
                   run: | 
        
                     # If you encounter an error, run './dev/update_config_docs.sh' and commit 
        
                     ./dev/update_config_docs.sh 
        
                     git diff --exit-code 
        
                 - name: Check if any of the ***_functions.md has been modified 
        
                   run: | 
        
                     # If you encounter an error, run './dev/update_function_docs.sh' and commit 
        
                     ./dev/update_function_docs.sh 
        
                     git diff --exit-code

🤔

alamb · 2025-12-10T14:26:09Z

datafusion-examples/README.md

+
+| Group             | Subcommand       | Category       | File Path                                        | Description                                                |
+| ----------------- | ---------------- | -------------- | ------------------------------------------------ | ---------------------------------------------------------- |
+| builtin_functions | date_time        | Single Process | `examples/builtin_functions/date_time.rs`        | Examples of date-time related functions and queries        |


Could we make the file path a link so readers can easily navigate to the code with a single click?

That's a good point! Let's do it.

cj-zhukov · 2025-12-10T14:49:22Z

@Jefffrey since you originally suggested using the tabular format for organizing the examples, you might be interested in taking a look at this PR.

Jefffrey · 2025-12-11T00:46:47Z

datafusion-examples/README.md

- [`examples/flight/client.rs`](examples/flight/client.rs) and [`examples/flight/server.rs`](examples/flight/server.rs): Run DataFusion as a standalone process and execute SQL queries from a client using the Arrow Flight protocol.
+## Builtin Functions Examples
+
+| Group             | Subcommand       | Category       | File Path                                        | Description                                                |


We can probably remove the Group and Category columns as they seem constant per header; we can mention them upfront above the table & beneath the header, to reduce the table size

That's a good point! I think reducing the table size is a good idea.

Jefffrey

Should be good to go once the CI checks pass

…me-occording-to-the-new-examples

alamb · 2025-12-11T21:43:26Z

Thanks again @cj-zhukov and @Jefffrey

cj-zhukov · 2025-12-12T05:50:44Z

Thanks @cj-zhukov -- this look like a nice improvement to me

I wonder if there is any way to ensure this document stays in sync with the code. For example, if someone adds a new example, how do we ensure that an appropriate entry is added here 🤔

Maybe we could add a ci check similar to the config doc check

datafusion/.github/workflows/rust.yml

Lines 683 to 710 in 2a08013

config-docs-check:

name: check configs.md and ***_functions.md is up-to-date

needs: linux-build-lib

runs-on: ubuntu-latest

container:

image: amd64/rust

steps:

- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1

with:

submodules: true

fetch-depth: 1

- name: Setup Rust toolchain

uses: ./.github/actions/setup-builder

with:

rust-version: stable

- uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6.1.0

with:

node-version: "20"

- name: Check if configs.md has been modified

run: |

# If you encounter an error, run './dev/update_config_docs.sh' and commit

./dev/update_config_docs.sh

git diff --exit-code

- name: Check if any of the ***_functions.md has been modified

run: |

# If you encounter an error, run './dev/update_function_docs.sh' and commit

./dev/update_function_docs.sh

git diff --exit-code

🤔

This would be great if automated! I've opened a follow-up PR for that #19294

Update README occording to the new examples (apache#18529)

3ee1ed9

alamb approved these changes Dec 10, 2025

View reviewed changes

Jefffrey reviewed Dec 11, 2025

View reviewed changes

remove Group & Category, file path as a link

439c9d6

Jefffrey approved these changes Dec 11, 2025

View reviewed changes

alamb added 2 commits December 11, 2025 15:19

prettier

d03e3c4

Merge remote-tracking branch 'apache/main' into cj-zhukov/update-read…

a1ac4b8

…me-occording-to-the-new-examples

alamb added the documentation Improvements or additions to documentation label Dec 11, 2025

alamb added this pull request to the merge queue Dec 11, 2025

Merged via the queue into apache:main with commit c8add5c Dec 11, 2025
5 checks passed

cj-zhukov mentioned this pull request Dec 12, 2025

Automatically generate examples documentation and add CI sync check #19294

Closed

cj-zhukov deleted the cj-zhukov/update-readme-occording-to-the-new-examples branch December 12, 2025 05:51

cj-zhukov mentioned this pull request Dec 12, 2025

Update datafusion-examples/README.md occording to the new examples format #18529

Closed

kumarUjjawal mentioned this pull request Dec 15, 2025

refactor: refactor spark like function to use datafusion like #19324

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Update README occording to the new examples (#18529)#19257

Update README occording to the new examples (#18529)#19257
alamb merged 4 commits intoapache:mainfrom
cj-zhukov:cj-zhukov/update-readme-occording-to-the-new-examples

cj-zhukov commented Dec 10, 2025

Uh oh!

cj-zhukov commented Dec 10, 2025

Uh oh!

alamb commented Dec 10, 2025

Uh oh!

alamb left a comment

Uh oh!

alamb Dec 10, 2025

Uh oh!

cj-zhukov Dec 11, 2025

Uh oh!

cj-zhukov commented Dec 10, 2025

Uh oh!

Jefffrey Dec 11, 2025

Uh oh!

cj-zhukov Dec 11, 2025

Uh oh!

Jefffrey left a comment

Uh oh!

alamb commented Dec 11, 2025

Uh oh!

Uh oh!

cj-zhukov commented Dec 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	config-docs-check:
	name: check configs.md and ***_functions.md is up-to-date
	needs: linux-build-lib
	runs-on: ubuntu-latest
	container:
	image: amd64/rust
	steps:
	- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
	with:
	submodules: true
	fetch-depth: 1
	- name: Setup Rust toolchain
	uses: ./.github/actions/setup-builder
	with:
	rust-version: stable
	- uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6.1.0
	with:
	node-version: "20"
	- name: Check if configs.md has been modified
	run: \|
	# If you encounter an error, run './dev/update_config_docs.sh' and commit
	./dev/update_config_docs.sh
	git diff --exit-code
	- name: Check if any of the ***_functions.md has been modified
	run: \|
	# If you encounter an error, run './dev/update_function_docs.sh' and commit
	./dev/update_function_docs.sh
	git diff --exit-code

Comments

Conversation

cj-zhukov commented Dec 10, 2025

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

cj-zhukov commented Dec 10, 2025

High Level Overview

Uh oh!

alamb commented Dec 10, 2025

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

alamb Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

cj-zhukov Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

cj-zhukov commented Dec 10, 2025

Uh oh!

Jefffrey Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

cj-zhukov Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Jefffrey left a comment

Choose a reason for hiding this comment

Uh oh!

alamb commented Dec 11, 2025

Uh oh!

Uh oh!

cj-zhukov commented Dec 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants