Skip to content

Comments

Initial example of Python TypeSystem#58

Merged
JacobHayes merged 2 commits intogoldenfrom
trivial_artifact
Jul 21, 2021
Merged

Initial example of Python TypeSystem#58
JacobHayes merged 2 commits intogoldenfrom
trivial_artifact

Conversation

@mikss
Copy link
Contributor

@mikss mikss commented Jul 7, 2021

This is a first attempt at what we talked about yesterday w.r.t. types. I do think there needs to be some additional thought here, since I don't like the fact that we need to frequently compare type classes and instances. E.g.,

def test_python_TypeSystem() -> None:
    from arti.types.python import python

    assert type(python.to_core(int())) == Int64
    assert python.from_core(Int64()) == int

@mikss mikss requested review from JacobHayes and joycex99 July 7, 2021 14:58
@codecov
Copy link

codecov bot commented Jul 7, 2021

Codecov Report

Merging #58 (43d6199) into golden (191e54a) will not change coverage.
The diff coverage is 100.00%.

❗ Current head 43d6199 differs from pull request most recent head 809878b. Consider uploading reports for the commit 809878b to get more accurate results
Impacted file tree graph

@@            Coverage Diff            @@
##            golden       #58   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           16        17    +1     
  Lines          583       619   +36     
  Branches        72        74    +2     
=========================================
+ Hits           583       619   +36     
Impacted Files Coverage Δ
src/arti/formats/core.py 100.00% <ø> (ø)
src/arti/internal/models.py 100.00% <100.00%> (ø)
src/arti/storage/core.py 100.00% <100.00%> (ø)
src/arti/types/core.py 100.00% <100.00%> (ø)
src/arti/types/python.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e91c46c...809878b. Read the comment docs.

Copy link
Member

@JacobHayes JacobHayes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this up so quick!

Do you mind adding src/arti/types/python.py, think that was missed. 😅

I don't feel so bad about checking types/instances specifically (the purpose of this code is type checking 😁), but I agree w/ the unwieldiness matching to the Adapter's external type (and that interface overall). Still processing a few more thoughts/paths, but might wait for python.py

Comment on lines 46 to 47
assert type(python.to_core(int())) == Int64
assert python.from_core(Int64()) == int
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, int() is 0 so shouldn't int() be int in the first line (akin to what from_core returns)?

Also, think we can tidy these a little bit (but still fundamentally type checking):

Suggested change
assert type(python.to_core(int())) == Int64
assert python.from_core(Int64()) == int
assert isinstance(python.to_core(int), Int64)
assert python.from_core(Int64()) is int

(I think the is may be slightly better than == since it's a singleton, but 🤷)

--

I think there's still a fundamental design oddity here as you note.

Copy link
Contributor Author

@mikss mikss Jul 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the tidying is fine, but we have in https://github.com/replicahq/artigraph/blob/cb0283674a148a3b9315ae7bc5df071898449fef/src/arti/types/core.py#L131

self.adapter_by_external_priority[type(type_)].to_internal(type_)

i.e., checking the type of type_, because that's how other (non Python) type systems work (e.g., pyarrow), so you want to pass int() to type(...) not int.

@mikss mikss changed the title draft of TypeSystem and TypeAdapter RAD-2740: initial example of Python TypeSystem Jul 19, 2021
@mikss mikss marked this pull request as ready for review July 19, 2021 15:23
@mikss
Copy link
Contributor Author

mikss commented Jul 19, 2021

@JacobHayes I think this is ready now; after thinking over some of our proposed solutions, I think just allowing a Python-weirdness-specific release valve in the form of "check if metaclass" (see this commit b1f6ea3) is lightest weight.

@mikss mikss requested a review from JacobHayes July 20, 2021 18:47
Copy link
Member

@JacobHayes JacobHayes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking much better overall, but I don't think the strictly type-based matching (even with system_metaclass) will work for everything: BigQuery's SchemaField is a single type with an instance level field_type string attribute. Should we call this good for now or try to iron out?

Copy link
Member

@JacobHayes JacobHayes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for iterating though this, it's in a much better state now! Gonna tidy commits up a bit and merge. 🚀

@JacobHayes JacobHayes merged commit ce84794 into golden Jul 21, 2021
@JacobHayes JacobHayes deleted the trivial_artifact branch July 21, 2021 14:58
@JacobHayes JacobHayes changed the title RAD-2740: initial example of Python TypeSystem Initial example of Python TypeSystem Mar 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants