Skip to content

Work around various issues with Python 3.12#222

Closed
johnkerl wants to merge 3 commits intomainfrom
kerl/python-3.12
Closed

Work around various issues with Python 3.12#222
johnkerl wants to merge 3 commits intomainfrom
kerl/python-3.12

Conversation

@johnkerl
Copy link
Copy Markdown
Contributor

@johnkerl johnkerl commented Sep 16, 2024 •
edited
Loading

TL;DR we are trying to support Python 3.12 in TileDB-SOMA

Solved challenge:

As tracked from parent issue single-cell-data/TileDB-SOMA#1849

Current unsolved challenges:

  • Latest Python 3.12 is 3.12.5
    • As debug I tried Python 3.12.3 which worked as debug, but isn't a good option for shipping
  • TileDB-SOMA's current typeguard version is 4.2.0
  • Python 3.12.4 added a new required argument to evaluate_forward_ref within Python's built-in typing module which broke many packages including typeguard
  • As a result, our TileDB-SOMA CI for PR 3001 fails like this
    • TypeError: ForwardRef._evaluate() missing 1 required keyword-only argument: 'recursive_guard'
  • typeguard 4.3.0 contains a fix for this problem
  • But this surfaces another problem which is:
E       typeguard.TypeCheckError: item 0 of argument "coords" (tuple) did not match any element in the union:
E         NoneType: is not an instance of NoneType
E         int: is not an instance of int
E         somacore.types.Slice[int]: is not compatible with the Slice protocol because it has no attribute named '__orig_bases__'

/home/ubuntu/.pyenv/versions/3.12.5/lib/python3.12/site-packages/typeguard/_checkers.py:433: TypeCheckError

The current PR is an attempt to deal with this.

Once this works, we'll need to:

  • Merge this PR
  • Tag a release of this repo (somacore)
  • Depend on 1.0.next of somacore within the TileDB-SOMA repo

I've currently got the following setup on an EC2 instance:

@johnkerl johnkerl changed the title WIP Work around various issues with Python 3.12 Sep 16, 2024
@johnkerl johnkerl changed the title Work around various issues with Python 3.12 Work around various issues with Python 3.12 [WIP] Sep 16, 2024
@johnkerl johnkerl mentioned this pull request Sep 16, 2024
Closed
11 tasks
@johnkerl johnkerl mentioned this pull request Sep 16, 2024
Merged
@johnkerl johnkerl mentioned this pull request Sep 17, 2024
Closed
@johnkerl johnkerl changed the title Work around various issues with Python 3.12 [WIP] Work around various issues with Python 3.12 Sep 17, 2024
@johnkerl johnkerl marked this pull request as ready for review September 17, 2024 14:21
thetorpedodog
thetorpedodog reviewed Sep 17, 2024
View reviewed changes
Comment thread python-spec/src/somacore/options.py

SparseNDCoord = Union[
None,
Tuple[int, ...],
Copy link
Copy Markdown
Contributor

@thetorpedodog thetorpedodog Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is necessary that’s fine, I guess, but this should already be covered under ValSliceOrSequence[int], since that includes a Sequence[int], which is a supertype of Tuple[int, ...].

Comment thread python-spec/src/somacore/types.py
Comment on lines +62 to +68
@property
def __orig_bases__(self) -> Any:
# Only exists in Python 3.12 and above
from types import get_original_bases

return get_original_bases(int)
# return get_original_bases(_T_co)
Copy link
Copy Markdown
Contributor

@thetorpedodog thetorpedodog Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a typeguard bug—it’s checking too many built-in attributes of the protocol that don’t actually matter. (See, for example, a case where it rejects objects with a completely empty protocol that I found.

That said, if implementing this makes typeguard stop complaining for now, I would do two things:

  1. Add a comment to that effect, and that it should be removed at some point
  2. Make this function instead raise AssertionError, since it should never be called.

@ryan-williams ryan-williams self-requested a review October 21, 2024 18:49
@ryan-williams
Copy link
Copy Markdown
Contributor

ryan-williams commented Oct 21, 2024

I believe this PR is no longer necessary; tests+lint have passed on 3.12 in this repo since #186, and single-cell-data/TileDB-SOMA#3001 has 3.12 passing without requiring any somacore changes.

@johnkerl
Copy link
Copy Markdown
Contributor Author

johnkerl commented Oct 21, 2024

Awesome, thanks @ryan-williams !

@johnkerl johnkerl closed this Oct 21, 2024
@ryan-williams ryan-williams mentioned this pull request Oct 28, 2024
Merged
@johnkerl johnkerl deleted the kerl/python-3.12 branch May 15, 2025 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ryan-williams ryan-williams Awaiting requested review from ryan-williams

1 more reviewer

@thetorpedodog thetorpedodog thetorpedodog left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@johnkerl @ryan-williams @thetorpedodog