Skip to content

Conversation

@ianthomas23
Copy link
Collaborator

This fixes a bug in the caching of received message headers at various places in the code which are later used for reply or iopub messages. The existing code on main branch assumes that the shell processes a single request at a time so that the cached header is valid until that request is replied to, but now of course there can be multiple subshells in different threads executing code concurrently.

The solution is to cache a different header per subshell, or to be more precise a different header per thread. This is accomplished using a ContextVar which is essentially thread-local storage that also supports asynchronous execution. We are already using a ContextVar at

self._parent_header: contextvars.ContextVar[dict[str, Any]] = contextvars.ContextVar(

so it is already a dependency of ipykernel.

I found this whilst debugging use of ipympl in jupyterlab, trying to create multiple matplotlib plots using the option of one subshell per comm-target (the default option). The best way to view this bug is to use kernelspy as it shows that in this scenerio some messages (e.g. some idle status messages) have the wrong parent header and are therefore displayed in the wrong section of the kernelspy output.

All new parent_header attributes are private (leading underscore). Where they were previously public I have added a public parent_header getter to extract the appropriate value from the ContextVar so that downstream libraries that have been reading the parent_header in the main shelll should have the same functionality as before.

@ianthomas23 ianthomas23 force-pushed the parent-header-contextvar-2 branch from 6ddffcf to f7e52eb Compare August 4, 2025 16:20
@ianthomas23 ianthomas23 force-pushed the parent-header-contextvar-2 branch from f7e52eb to 5b8ad2e Compare August 7, 2025 13:28
@ianthomas23
Copy link
Collaborator Author

I'm merging this as is, with the downstream jupyter_client and ipyparallel workflows disabled as they are waiting on PRs in those repositories. I'll create an issue to track the downstream changes.

@ianthomas23 ianthomas23 merged commit efd53d7 into ipython:main Aug 8, 2025
44 of 45 checks passed
@ianthomas23 ianthomas23 deleted the parent-header-contextvar-2 branch August 8, 2025 09:56
@ianthomas23
Copy link
Collaborator Author

Note Kernel._parent_ident reinstated in PR #1427 for backward compatibility.

@minrk
Copy link
Member

minrk commented Oct 14, 2025

Looks like there's more to it - we need to resolve parent_header to the main shell thread from any thread that could be spawned (#1450).

I think this actually relates to #1289 because we do a lot wrong with output from threads right now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants