Skip to content

Allow null execution_count in execute_result. [API change] - #4

Closed
JayOfTheKeyboard wants to merge 1 commit into
jgm:masterfrom
JayOfTheKeyboard:execute-result-null-count
Closed

JayOfTheKeyboard wants to merge 1 commit into
jgm:masterfrom
JayOfTheKeyboard:execute-result-null-count

Conversation

@JayOfTheKeyboard

Copy link
Copy Markdown

See jgm/pandoc#12008.

The nbformat v4 schema allows null for an execute_result's execution_count, as it does for a code cell's ("type": ["integer", "null"] in every schema from nbformat.v4.0 to v4.5). executeCount is an Int, so a notebook with one of these fails to decode as v4, and pandoc then reports the error from its v3 attempt:

% pandoc -f ipynb -t markdown null-ec.ipynb
ipynb decoding error: Error in $: expected nbformat <= 3
% echo $?
93

with this input:

{"nbformat": 4, "nbformat_minor": 5, "metadata": {},
 "cells": [{"cell_type": "code", "id": "a1", "execution_count": null, "metadata": {}, "source": ["1+1"],
   "outputs": [{"output_type": "execute_result", "execution_count": null, "metadata": {}, "data": {"text/plain": ["2"]}}]}]}

That was pandoc 3.13. The file passes nbformat.validate (nbformat 5.11.1). Four of the Ray project's Lightning and Tune example notebooks have such outputs (for example doc/source/tune/examples/tune-pytorch-lightning.ipynb), and all four validate.

executeCount is now Maybe Int, like codeExecutionCount. In v4, null reads as Nothing and is written back as null, so the key the schema requires stays in place. In v3, a pyout's prompt_number now works the way a code cell's does: it is read with .:? and left out when there is no count. Writing null there would break the v3 schema, which wants an integer, and the reader can still read what the writer produces.

Because this changes an exported type, pandoc needs two edits. The writer stops compiling, which makes that one easy to spot. The reader still compiles but would quietly produce execution_count="Just 3":

  • Readers/Ipynb.hs, outputToBlock: replace [("execution_count",tshow ec)] with maybe [] (\n -> [("execution_count", tshow n)]) ec, as the code-cell case already does.
  • Writers/Ipynb.hs, blockToOutput: drop the fromMaybe 0, so an execute_result div with no count is written as null rather than 0. None of the files in test/ipynb has an execute_result without a count.

I didn't touch the version or Changelog.md, since you have always updated those yourself at release.

Test: test/rt-files/test4execute_result_null.ipynb, a one-cell v4.5 notebook, goes through the existing round trip. The round trip compares decoded values, so it can't see whether null survives in the JSON. I added two small cases to test/roundtrip.hs for that: one checks that encoding the new notebook gives "execution_count": null on the output, and one checks that a v3 pyout with "prompt_number": null is written without prompt_number and reads back the same. Without the change all three fail with parsing Int failed, expected Number, but encountered Null. With it, all 8 tests pass (GHC 9.6.7, aeson 2.3.2.0), and -Wall reports only the existing unused x at line 241. I also broke the change four ways (v4 writes 0, v4 leaves out the key, v3 writes null, v3 reads with .:), and each one makes a test fail. The four Ray notebooks decode and round-trip with the change and fail without it.

Prepared with Claude Code.

The nbformat v4 schema gives an execute_result's execution_count the
type ["integer", "null"], the same as a code cell's. Such notebooks
failed to decode as v4. executeCount is now Maybe Int, like
codeExecutionCount, and null is written back as null.

For v3, a pyout's prompt_number is now handled like a code cell's:
it is read with .:? and left out when there is no count.
@JayOfTheKeyboard

Copy link
Copy Markdown
Author

Heads up on the red CI here: it isn't from this change. The Linux jobs fail at cabal v2-update with root.json does not have enough signatures signed with the appropriate keys — that's Hackage's TUF root-key rotation, and the old cabal versions the matrix pins can't verify the new root. It happens before anything in this PR compiles (macOS passes because it uses stack).

I've opened #5 to fix the workflow (a current cabal via haskell-actions/setup@v2, GHC and aeson matrix unchanged). Once that lands, this PR should re-run green.

@jgm

jgm commented Nov 14, 2026

Copy link
Copy Markdown
Owner

If you rebase against master, the CI issues are now fixed.

@jgm

jgm commented Nov 14, 2026

Copy link
Copy Markdown
Owner

I'm closing this because I'm fixing this together with a number of other things in an upcoming push.

@jgm jgm closed this Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants