Skip to content

Conversation

@sfc-gh-nbellante
Copy link
Contributor

@sfc-gh-nbellante sfc-gh-nbellante commented Oct 20, 2025

Describe your changes

This PR refactors the AudioInput component and adds audio recording capabilities to the ChatInput component:

  1. In AudioInput:

    • Improved code organization by moving the transcodeAndUploadFile function before controller initialization
    • Removed the unnecessary ref for transcodeAndUploadFile and passed the function directly to onApprove
    • Added cleanup for blob URLs to prevent memory leaks
    • Added explanatory comments for error handling cases
  2. In ChatInput:

    • Added audio recording functionality with mic button when acceptAudio is enabled
    • Implemented recording UI with waveform visualization and approve/cancel buttons
    • Added audio file upload and submission logic
    • Ensured proper cleanup of audio resources
    • Fixed setValue handling to prevent duplicate processing

Testing Plan

  • The AudioInput refactoring maintains the same functionality with improved code organization
  • Manually tested audio recording in ChatInput with various browsers
  • Verified that audio files are properly uploaded and submitted with chat messages
  • Tested error handling for recording and upload failures
  • Verified that UI state transitions correctly between text input and recording modes
  • Confirmed that audio resources are properly cleaned up when component unmounts

Contribution License Agreement

By submitting this pull request you agree that all contributions to this project are made under the Apache 2.0 license.

@snyk-io
Copy link
Contributor

snyk-io bot commented Oct 20, 2025

Snyk checks have passed. No issues have been found so far.

Status Scanner Critical High Medium Low Total (0)
Licenses 0 0 0 0 0 issues
Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Copy link
Contributor Author

sfc-gh-nbellante commented Oct 20, 2025

@github-actions
Copy link
Contributor

github-actions bot commented Oct 20, 2025

✅ PR preview is ready!

Name Link
📦 Wheel file https://core-previews.s3-us-west-2.amazonaws.com/pr-12835/streamlit-1.51.0-py3-none-any.whl
📦 @streamlit/component-v2-lib Download from artifacts
🕹️ Preview app pr-12835.streamlit.app (☁️ Deploy here if not accessible)

@github-actions
Copy link
Contributor

github-actions bot commented Oct 20, 2025

📉 Python coverage change detected

The Python unit test coverage has decreased by 0.0000%

  • Current PR: 92.7643% (20081 statements, 1453 missed)
  • Latest develop: 92.7643% (20081 statements, 1453 missed)

✅ Coverage change is within normal range.

Coverage by files
Name Stmts Miss Cover
streamlit/__init__.py 141 0 100%
streamlit/__main__.py 3 3 0%
streamlit/auth_util.py 100 25 75%
streamlit/cli_util.py 39 6 85%
streamlit/column_config.py 3 0 100%
streamlit/commands/__init__.py 0 0 100%
streamlit/commands/echo.py 54 11 80%
streamlit/commands/execution_control.py 59 10 83%
streamlit/commands/experimental_query_params.py 40 2 95%
streamlit/commands/logo.py 41 6 85%
streamlit/commands/navigation.py 106 2 98%
streamlit/commands/page_config.py 101 4 96%
streamlit/components/__init__.py 0 0 100%
streamlit/components/lib/__init__.py 0 0 100%
streamlit/components/lib/local_component_registry.py 35 2 94%
streamlit/components/types/__init__.py 0 0 100%
streamlit/components/types/base_component_registry.py 14 0 100%
streamlit/components/types/base_custom_component.py 49 6 88%
streamlit/components/v1/__init__.py 5 0 100%
streamlit/components/v1/component_arrow.py 33 8 76%
streamlit/components/v1/component_registry.py 41 3 93%
streamlit/components/v1/components.py 4 4 0%
streamlit/components/v1/custom_component.py 85 7 92%
streamlit/components/v2/__init__.py 24 0 100%
streamlit/components/v2/bidi_component/__init__.py 4 0 100%
streamlit/components/v2/bidi_component/constants.py 5 0 100%
streamlit/components/v2/bidi_component/main.py 109 12 89%
streamlit/components/v2/bidi_component/serialization.py 81 5 94%
streamlit/components/v2/bidi_component/state.py 13 0 100%
streamlit/components/v2/component_definition_resolver.py 30 0 100%
streamlit/components/v2/component_file_watcher.py 117 9 92%
streamlit/components/v2/component_manager.py 96 16 83%
streamlit/components/v2/component_manifest_handler.py 24 0 100%
streamlit/components/v2/component_path_utils.py 68 5 93%
streamlit/components/v2/component_registry.py 118 8 93%
streamlit/components/v2/get_bidi_component_manager.py 8 1 88%
streamlit/components/v2/manifest_scanner.py 224 25 89%
streamlit/components/v2/presentation.py 84 19 77%
streamlit/components/v2/types.py 8 8 0%
streamlit/config.py 412 12 97%
streamlit/config_option.py 79 3 96%
streamlit/config_util.py 288 7 98%
streamlit/connections/__init__.py 6 0 100%
streamlit/connections/base_connection.py 45 0 100%
streamlit/connections/snowflake_connection.py 60 13 78%
streamlit/connections/snowpark_connection.py 44 3 93%
streamlit/connections/sql_connection.py 56 6 89%
streamlit/connections/util.py 33 0 100%
streamlit/cursor.py 82 2 98%
streamlit/dataframe_util.py 501 45 91%
streamlit/delta_generator.py 205 6 97%
streamlit/delta_generator_singletons.py 76 8 89%
streamlit/deprecation_util.py 59 4 93%
streamlit/development.py 1 0 100%
streamlit/elements/__init__.py 0 0 100%
streamlit/elements/alert.py 60 0 100%
streamlit/elements/arrow.py 198 15 92%
streamlit/elements/balloons.py 10 0 100%
streamlit/elements/bokeh_chart.py 27 0 100%
streamlit/elements/code.py 20 1 95%
streamlit/elements/deck_gl_json_chart.py 104 10 90%
streamlit/elements/dialog_decorator.py 38 0 100%
streamlit/elements/doc_string.py 227 9 96%
streamlit/elements/empty.py 16 4 75%
streamlit/elements/exception.py 101 10 90%
streamlit/elements/form.py 54 2 96%
streamlit/elements/graphviz_chart.py 35 1 97%
streamlit/elements/heading.py 56 0 100%
streamlit/elements/html.py 48 0 100%
streamlit/elements/iframe.py 29 0 100%
streamlit/elements/image.py 32 0 100%
streamlit/elements/json.py 39 2 95%
streamlit/elements/layouts.py 140 3 98%
streamlit/elements/lib/__init__.py 0 0 100%
streamlit/elements/lib/built_in_chart_utils.py 387 26 93%
streamlit/elements/lib/color_util.py 100 4 96%
streamlit/elements/lib/column_config_utils.py 168 1 99%
streamlit/elements/lib/column_types.py 189 4 98%
streamlit/elements/lib/dialog.py 67 1 99%
streamlit/elements/lib/dicttools.py 39 2 95%
streamlit/elements/lib/file_uploader_utils.py 30 0 100%
streamlit/elements/lib/form_utils.py 26 0 100%
streamlit/elements/lib/image_utils.py 176 21 88%
streamlit/elements/lib/js_number.py 28 3 89%
streamlit/elements/lib/layout_utils.py 109 1 99%
streamlit/elements/lib/mutable_status_container.py 73 4 95%
streamlit/elements/lib/options_selector_utils.py 90 0 100%
streamlit/elements/lib/pandas_styler_utils.py 80 2 98%
streamlit/elements/lib/policies.py 56 1 98%
streamlit/elements/lib/streamlit_plotly_theme.py 49 0 100%
streamlit/elements/lib/subtitle_utils.py 76 13 83%
streamlit/elements/lib/utils.py 76 5 93%
streamlit/elements/map.py 110 1 99%
streamlit/elements/markdown.py 63 2 97%
streamlit/elements/media.py 181 8 96%
streamlit/elements/metric.py 91 0 100%
streamlit/elements/pdf.py 49 2 96%
streamlit/elements/plotly_chart.py 114 4 96%
streamlit/elements/progress.py 36 0 100%
streamlit/elements/pyplot.py 39 2 95%
streamlit/elements/snow.py 10 0 100%
streamlit/elements/space.py 12 0 100%
streamlit/elements/spinner.py 34 0 100%
streamlit/elements/text.py 16 0 100%
streamlit/elements/toast.py 26 0 100%
streamlit/elements/vega_charts.py 226 3 99%
streamlit/elements/widgets/__init__.py 0 0 100%
streamlit/elements/widgets/audio_input.py 68 10 85%
streamlit/elements/widgets/button.py 214 3 99%
streamlit/elements/widgets/button_group.py 161 1 99%
streamlit/elements/widgets/camera_input.py 62 10 84%
streamlit/elements/widgets/chat.py 193 54 72%
streamlit/elements/widgets/checkbox.py 52 0 100%
streamlit/elements/widgets/color_picker.py 56 2 96%
streamlit/elements/widgets/data_editor.py 239 14 94%
streamlit/elements/widgets/file_uploader.py 103 18 83%
streamlit/elements/widgets/multiselect.py 105 4 96%
streamlit/elements/widgets/number_input.py 143 5 97%
streamlit/elements/widgets/radio.py 83 5 94%
streamlit/elements/widgets/select_slider.py 97 0 100%
streamlit/elements/widgets/selectbox.py 91 2 98%
streamlit/elements/widgets/slider.py 241 8 97%
streamlit/elements/widgets/text_widgets.py 130 6 95%
streamlit/elements/widgets/time_widgets.py 248 15 94%
streamlit/elements/write.py 166 30 82%
streamlit/emojis.py 4 0 100%
streamlit/env_util.py 21 3 86%
streamlit/error_util.py 33 2 94%
streamlit/errors.py 188 25 87%
streamlit/external/__init__.py 0 0 100%
streamlit/external/langchain/__init__.py 2 0 100%
streamlit/external/langchain/streamlit_callback_handler.py 141 82 42%
streamlit/file_util.py 84 8 90%
streamlit/git_util.py 100 5 95%
streamlit/logger.py 54 0 100%
streamlit/material_icon_names.py 1 0 100%
streamlit/navigation/__init__.py 0 0 100%
streamlit/navigation/page.py 78 2 97%
streamlit/net_util.py 55 3 95%
streamlit/platform.py 10 1 90%
streamlit/runtime/__init__.py 8 0 100%
streamlit/runtime/app_session.py 447 94 79%
streamlit/runtime/caching/__init__.py 19 0 100%
streamlit/runtime/caching/cache_data_api.py 164 3 98%
streamlit/runtime/caching/cache_errors.py 45 4 91%
streamlit/runtime/caching/cache_resource_api.py 122 0 100%
streamlit/runtime/caching/cache_type.py 11 1 91%
streamlit/runtime/caching/cache_utils.py 165 9 95%
streamlit/runtime/caching/cached_message_replay.py 108 1 99%
streamlit/runtime/caching/hashing.py 311 25 92%
streamlit/runtime/caching/legacy_cache_api.py 14 0 100%
streamlit/runtime/caching/storage/__init__.py 2 0 100%
streamlit/runtime/caching/storage/cache_storage_protocol.py 31 2 94%
streamlit/runtime/caching/storage/dummy_cache_storage.py 21 0 100%
streamlit/runtime/caching/storage/in_memory_cache_storage_wrapper.py 60 0 100%
streamlit/runtime/caching/storage/local_disk_cache_storage.py 86 4 95%
streamlit/runtime/connection_factory.py 85 9 89%
streamlit/runtime/context.py 140 0 100%
streamlit/runtime/context_util.py 18 0 100%
streamlit/runtime/credentials.py 139 4 97%
streamlit/runtime/forward_msg_cache.py 23 2 91%
streamlit/runtime/forward_msg_queue.py 63 4 94%
streamlit/runtime/fragment.py 112 2 98%
streamlit/runtime/media_file_manager.py 69 7 90%
streamlit/runtime/media_file_storage.py 15 0 100%
streamlit/runtime/memory_media_file_storage.py 68 0 100%
streamlit/runtime/memory_session_storage.py 15 0 100%
streamlit/runtime/memory_uploaded_file_manager.py 41 1 98%
streamlit/runtime/metrics_util.py 193 12 94%
streamlit/runtime/pages_manager.py 59 2 97%
streamlit/runtime/runtime.py 248 18 93%
streamlit/runtime/runtime_util.py 30 1 97%
streamlit/runtime/script_data.py 16 0 100%
streamlit/runtime/scriptrunner/__init__.py 5 0 100%
streamlit/runtime/scriptrunner/exec_code.py 49 5 90%
streamlit/runtime/scriptrunner/magic.py 83 1 99%
streamlit/runtime/scriptrunner/magic_funcs.py 10 1 90%
streamlit/runtime/scriptrunner/script_cache.py 27 0 100%
streamlit/runtime/scriptrunner/script_runner.py 230 27 88%
streamlit/runtime/scriptrunner_utils/__init__.py 0 0 100%
streamlit/runtime/scriptrunner_utils/exceptions.py 11 1 91%
streamlit/runtime/scriptrunner_utils/script_requests.py 106 5 95%
streamlit/runtime/scriptrunner_utils/script_run_context.py 135 2 99%
streamlit/runtime/secrets.py 242 25 90%
streamlit/runtime/session_manager.py 60 1 98%
streamlit/runtime/state/__init__.py 7 0 100%
streamlit/runtime/state/common.py 52 2 96%
streamlit/runtime/state/presentation.py 19 4 79%
streamlit/runtime/state/query_params.py 110 3 97%
streamlit/runtime/state/query_params_proxy.py 71 0 100%
streamlit/runtime/state/safe_session_state.py 77 11 86%
streamlit/runtime/state/session_state.py 433 29 93%
streamlit/runtime/state/session_state_proxy.py 62 8 87%
streamlit/runtime/state/widgets.py 15 1 93%
streamlit/runtime/stats.py 42 0 100%
streamlit/runtime/theme_util.py 46 1 98%
streamlit/runtime/uploaded_file_manager.py 39 3 92%
streamlit/runtime/websocket_session_manager.py 66 0 100%
streamlit/source_util.py 36 1 97%
streamlit/string_util.py 88 7 92%
streamlit/temporary_directory.py 18 1 94%
streamlit/testing/__init__.py 0 0 100%
streamlit/testing/v1/__init__.py 2 0 100%
streamlit/testing/v1/app_test.py 239 6 97%
streamlit/testing/v1/element_tree.py 1327 87 93%
streamlit/testing/v1/local_script_runner.py 71 2 97%
streamlit/testing/v1/util.py 17 0 100%
streamlit/time_util.py 28 1 96%
streamlit/type_util.py 138 12 91%
streamlit/url_util.py 39 5 87%
streamlit/user_info.py 87 8 91%
streamlit/util.py 38 1 97%
streamlit/version.py 3 0 100%
streamlit/watcher/__init__.py 3 0 100%
streamlit/watcher/event_based_path_watcher.py 181 24 87%
streamlit/watcher/folder_black_list.py 14 1 93%
streamlit/watcher/local_sources_watcher.py 127 9 93%
streamlit/watcher/path_watcher.py 43 3 93%
streamlit/watcher/polling_path_watcher.py 55 2 96%
streamlit/watcher/util.py 49 1 98%
streamlit/web/__init__.py 0 0 100%
streamlit/web/bootstrap.py 138 18 87%
streamlit/web/cache_storage_manager_config.py 5 0 100%
streamlit/web/cli.py 186 17 91%
streamlit/web/server/__init__.py 5 0 100%
streamlit/web/server/app_static_file_handler.py 29 3 90%
streamlit/web/server/authlib_tornado_integration.py 18 1 94%
streamlit/web/server/bidi_component_request_handler.py 65 8 88%
streamlit/web/server/browser_websocket_handler.py 115 31 73%
streamlit/web/server/component_file_utils.py 24 0 100%
streamlit/web/server/component_request_handler.py 55 4 93%
streamlit/web/server/media_file_handler.py 65 9 86%
streamlit/web/server/oauth_authlib_routes.py 118 18 85%
streamlit/web/server/oidc_mixin.py 44 0 100%
streamlit/web/server/routes.py 90 7 92%
streamlit/web/server/server.py 188 11 94%
streamlit/web/server/server_util.py 67 5 93%
streamlit/web/server/stats_request_handler.py 53 4 92%
streamlit/web/server/upload_file_request_handler.py 53 9 83%
streamlit/web/server/websocket_headers.py 19 1 95%
TOTAL 20081 1453 93%

📊 View detailed coverage comparison

@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr8-chatinput-integration branch from d8dc6fd to d8de826 Compare October 21, 2025 14:07
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr7-chatinput-components branch from deb8f05 to 53062ef Compare October 21, 2025 14:07
@sfc-gh-nbellante sfc-gh-nbellante added security-assessment-completed Security assessment has been completed for PR impact:internal PR changes only affect internal code change:feature PR contains new feature or enhancement implementation labels Oct 21, 2025
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr8-chatinput-integration branch from d8de826 to bcfdda6 Compare October 21, 2025 14:33
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr7-chatinput-components branch 2 times, most recently from 662c525 to fa19cbe Compare October 21, 2025 14:54
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr8-chatinput-integration branch 2 times, most recently from 2109c8a to 7f52651 Compare October 21, 2025 15:43
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr7-chatinput-components branch from fa19cbe to 1d2cf42 Compare October 21, 2025 15:43
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr8-chatinput-integration branch from 7f52651 to d8c8b84 Compare October 21, 2025 15:51
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr7-chatinput-components branch from 1d2cf42 to 9c1fd45 Compare October 21, 2025 15:51
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr8-chatinput-integration branch from d8c8b84 to b68c2ad Compare October 21, 2025 16:09
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr8-chatinput-integration branch from b68c2ad to bb13287 Compare October 21, 2025 16:15
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr7-chatinput-components branch from 9c1fd45 to 5dbf8af Compare October 21, 2025 16:15
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr8-chatinput-integration branch from bb13287 to f71021d Compare October 21, 2025 21:01
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr7-chatinput-components branch 2 times, most recently from 53b4659 to 9ba9e8d Compare October 23, 2025 14:18
bottom: "0px",
// Calculate the right padding to account for button(s) on the right
// Each button is: iconSizes.xl + 2 * spacing.sm
// When acceptAudio is true, there are 2 buttons (mic + send) with extra margin
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: My previous comment about re-structuring this so we aren't unnecessarily coupling this still stands: #12835

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea where this is being used actually. I was trying to tweak this earlier and get it to not overlap the placeholder text, but it wasn't changing anything and I found that the actual padding was coming from a paddingRight elsewhere. Let me see if this can be removed entirely.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, gem from claude.

The wrapper exists solely to push the character count left to avoid the absolutely positioned buttons.

Comment on lines +673 to +676
onClick={() => {
handleRecordingApprove().catch(_error => {
// Error handling is done via controller events
})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (non-blocking): This should be wrapped in a useCallback.

Comment on lines +694 to +698
onClick={(e: React.MouseEvent) => {
handleMicClick(e).catch(_error => {
// Error handling is done via controller events
})
}}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (non-blocking): This should be wrapped in a useCallback.

@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr8-chatinput-integration branch from 317e4cf to 8ab4bd4 Compare October 31, 2025 21:19
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr8-chatinput-integration branch from 8ab4bd4 to 804f153 Compare October 31, 2025 22:01
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr6-audio-infra branch 2 times, most recently from 244738a to 41b2ef3 Compare November 1, 2025 07:03
@sfc-gh-nbellante sfc-gh-nbellante requested a review from a team as a November 1, 2025 07:03
@sfc-gh-nbellante sfc-gh-nbellante changed the base branch from nico/pr6-audio-infra to graphite-base/12835 November 1, 2025 07:30
Add audio recording capability to the ChatInput widget with waveform visualization and approve/cancel controls. Includes shared logger for ChatInput components.

- Add WaveformController integration for live audio recording
- Add recording state management and UI controls
- Add shared logger.ts for ChatInput component logging
- Add comprehensive test coverage for audio recording flow
Update AudioInput and useWaveformController to support being controlled externally by ChatInput. Changes include:

- Add controllerRef pattern to allow parent components to access controller methods
- Fix circular dependency by moving upload logic out of controller events
- Add error handling comments for silent catch blocks
- Update AudioInput test to match new controller pattern
Move StyledChatAudioWave from removed component library to widget's styled-components file and update import path.
- Consolidate styled-components imports into single block
- Add empty line between import groups
- Move StyledChatAudioWave to correct import location
When copying StyledChatAudioWave from PR7, we missed the width: 100% that was added in PR8's inline styles refactor. Without explicit width, the waveform collapses to 0 width and becomes invisible despite recording working.
When recording, use static positioning instead of absolute so buttons participate in flex layout alongside waveform. This fixes overlapping layout issues.
Add acceptAudio prop to StyledInputInstructionsContainer to account for both mic and send buttons when audio is enabled. Doubles the right padding calculation from 1 button to 2 buttons to prevent overlap.
Add extra spacing.sm to account for the gap between mic and send buttons when calculating right padding for InputInstructions.
Double the final margin from spacing.sm to 2*spacing.sm when acceptAudio is true to prevent overlap with mic button.
Apply the same conditional padding logic to the textarea input that we use for InputInstructions - account for 2 buttons when acceptAudio is true.
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr8-chatinput-integration branch from 804f153 to ddd2d4a Compare November 1, 2025 07:31
@graphite-app graphite-app bot changed the base branch from graphite-base/12835 to develop November 1, 2025 07:32
Remove extra spacing.sm margin - 80px (2 × 40px buttons) is sufficient padding.
@sfc-gh-nbellante sfc-gh-nbellante force-pushed the nico/pr8-chatinput-integration branch from ddd2d4a to ee72aaf Compare November 1, 2025 07:32
@sfc-gh-nbellante sfc-gh-nbellante merged commit 6e52e24 into develop Nov 1, 2025
37 checks passed
Copy link
Contributor Author

Merge activity

@sfc-gh-nbellante sfc-gh-nbellante deleted the nico/pr8-chatinput-integration branch November 1, 2025 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

change:feature PR contains new feature or enhancement implementation impact:internal PR changes only affect internal code security-assessment-completed Security assessment has been completed for PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants