Skip to content

feat: Support Show runtime settings#18564

Merged
Weijun-H merged 6 commits intoapache:mainfrom
Weijun-H:18452-show-runtime
Nov 22, 2025
Merged

feat: Support Show runtime settings#18564
Weijun-H merged 6 commits intoapache:mainfrom
Weijun-H:18452-show-runtime

Conversation

@Weijun-H
Copy link
Member

@Weijun-H Weijun-H commented Nov 9, 2025

Which issue does this PR close?

Rationale for this change

Previously, the SHOW command could not display runtime configuration settings such as datafusion.runtime.memory_limit, datafusion.runtime.max_temp_directory_size, etc. When users attempted to view these settings, they would receive an error:

Error during planning: 'datafusion.runtime.memory_limit' is not a variable which can be viewed with 'SHOW'

This PR enables users to query runtime settings using the SHOW command, making them accessible in the same way as other DataFusion configuration variables.

What changes are included in this PR?

  1. Added config_entries() method to RuntimeEnv - Returns runtime configuration as ConfigEntry objects with human-readable values (e.g., "100M", "unlimited")

  2. Exposed getter methods in DiskManager - Added max_temp_directory_size() and temp_dir_paths() to access disk manager configuration

  3. Updated InformationSchema - Modified make_df_settings() to include runtime configuration entries from RuntimeEnv

  4. Modified SQL statement validation - Added check for datafusion.runtime.* variables to allow SHOW command

Are these changes tested?

Yes, comprehensive tests have been added:

  • set_variable.slt - Tests SHOW and SET for all 4 runtime variables (memory_limit, max_temp_directory_size, metadata_cache_limit, temp_directory)
  • information_schema.slt - Verifies runtime variables appear in information_schema.df_settings
  • Tests cover default values, setting custom values, and querying via information_schema

Are there any user-facing changes?

Yes, users can now:

View runtime settings with SHOW:

SHOW datafusion.runtime.memory_limit;
-- Returns: datafusion.runtime.memory_limit | unlimited

@github-actions github-actions bot added sql SQL Planner sqllogictest SQL Logic Tests (.slt) catalog Related to the catalog crate execution Related to the execution crate labels Nov 9, 2025
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Weijun-H -- I tried it out locally and it works very nicely 👌

@Omega359
Copy link
Contributor

LGTM, thanks @Weijun-H !

@Weijun-H Weijun-H added this pull request to the merge queue Nov 22, 2025
Merged via the queue into apache:main with commit c8d26ba Nov 22, 2025
28 checks passed
@Weijun-H
Copy link
Member Author

Thanks @Jefffrey , @alamb and @Omega359 for reviewing

logan-keede pushed a commit to logan-keede/datafusion that referenced this pull request Nov 23, 2025
## Which issue does this PR close?

- Closes apache#18452

## Rationale for this change

Previously, the `SHOW` command could not display runtime configuration
settings such as `datafusion.runtime.memory_limit`,
`datafusion.runtime.max_temp_directory_size`, etc. When users attempted
to view these settings, they would receive an error:

Error during planning: 'datafusion.runtime.memory_limit' is not a
variable which can be viewed with 'SHOW'

This PR enables users to query runtime settings using the `SHOW`
command, making them accessible in the same way as other DataFusion
configuration variables.

## What changes are included in this PR?

1. **Added `config_entries()` method to `RuntimeEnv`** - Returns runtime
configuration as `ConfigEntry` objects with human-readable values (e.g.,
"100M", "unlimited")

2. **Exposed getter methods in `DiskManager`** - Added
`max_temp_directory_size()` and `temp_dir_paths()` to access disk
manager configuration

3. **Updated `InformationSchema`** - Modified `make_df_settings()` to
include runtime configuration entries from `RuntimeEnv`

4. **Modified SQL statement validation** - Added check for
`datafusion.runtime.*` variables to allow SHOW command


## Are these changes tested?

Yes, comprehensive tests have been added:

- **set_variable.slt** - Tests SHOW and SET for all 4 runtime variables
(memory_limit, max_temp_directory_size, metadata_cache_limit,
temp_directory)
- **information_schema.slt** - Verifies runtime variables appear in
`information_schema.df_settings`
- Tests cover default values, setting custom values, and querying via
information_schema

## Are there any user-facing changes?

Yes, users can now:

**View runtime settings with SHOW**:
   ```sql
   SHOW datafusion.runtime.memory_limit;
   -- Returns: datafusion.runtime.memory_limit | unlimited
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

catalog Related to the catalog crate execution Related to the execution crate sql SQL Planner sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Show runtime config

4 participants