Skip to content

Adding initial MOA parser#74

Merged
saulshanabrook merged 2 commits intoQuansight-Labs:masterfrom
costrouc:costrouc/add-initial-parser
Dec 8, 2018
Merged

Adding initial MOA parser#74
saulshanabrook merged 2 commits intoQuansight-Labs:masterfrom
costrouc:costrouc/add-initial-parser

Conversation

@costrouc
Copy link
Copy Markdown
Contributor

@costrouc costrouc commented Dec 6, 2018 •
edited
Loading

@saulshanabrook what are your thoughts?

  • tests are included with parser

It can handle pretty much any moa expression. There are still a few symbols that need to be added but the fundamentals are covered. Additionally it will complain when it does not understand an expression.

Future vision of expressions.

  • We need an ability for composability. For instance <1 2> psi A can be given an name like my_index and can be used in a later expression like my_index * B.
  • Supplying variables can be done with a dictionary like so moa.eval('<1 2> psi A', {'A': np.random.random((2, 3, 4)))

Currently pretty much any moa expression will evaluation but it will complain that uarray does not have support for the operation such as ('/' and '-').

Examples. Available tokens can be found in lexer.py.

<1 2> + <3 4>

A + B + C + D + E

A + (B + C) + D psi E

A ^ <1 2> + B psi C

@saulshanabrook
Copy link
Copy Markdown
Contributor

saulshanabrook commented Dec 6, 2018

I think you should be able to use vector from core.py:

uarray/uarray/core.py

Lines 238 to 243 in c1542e4

def vector(*values: int) -> CArray:
# vc: CCallableUnary[CContent, CContent] = VectorCallable(*map(Int, values))
# getitem = Compose(scalar_fn, vc)
return vector_of(*(Scalar(Int(v)) for v in values))

and with_shape from core as well. I think you could do something like:

with_shape(unbound("A"), [Int(1), Int(2)])

To get an an array named "A" with shape <1 2>.

I would continue to based off of core. My PR is in a lot of flux and we should be able to move this over easily if we do end up merging it eventually.

saulshanabrook
saulshanabrook approved these changes Dec 6, 2018
View reviewed changes
Copy link
Copy Markdown
Contributor

@saulshanabrook saulshanabrook left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good! Once you substitute in the real arrays and shapes, I am curious when you call replace on resulting ASTs, how the results look.

Also, it might be nice to have a little notebook just demoing what this is doing.

@costrouc
Copy link
Copy Markdown
Contributor Author

costrouc commented Dec 6, 2018

I will add a notebook to demonstrate the features! Perfect you gave me exactly what I needed to know.

@costrouc costrouc force-pushed the costrouc/add-initial-parser branch from b849a2f to 8ab2602 Compare December 6, 2018 21:52
@costrouc
Copy link
Copy Markdown
Contributor Author

costrouc commented Dec 6, 2018 •
edited
Loading

Tests fail due to the following not being equal.

 A = core.with_shape(core.unbound('j'), [moa.Int(1), moa.Int(2)])
 B = core.with_shape(core.unbound('j'), [moa.Int(1), moa.Int(2)])
 print(A)
 print(B)
 assert A == B

Sequence(1, (i6 -> Sequence(2, (i7 -> Scalar(Content(CallUnary(GetItem(CallUnary(GetItem(j), i6)), i7)))))))
Sequence(1, (i8 -> Sequence(2, (i9 -> Scalar(Content(CallUnary(GetItem(CallUnary(GetItem(j), i8)), i9)))))))

@costrouc costrouc force-pushed the costrouc/add-initial-parser branch 2 times, most recently from 3a301f0 to c9dc585 Compare December 6, 2018 22:42
@saulshanabrook
Copy link
Copy Markdown
Contributor

saulshanabrook commented Dec 7, 2018

@costrouc I would do something like:

 A = core.with_shape(core.unbound('j'), [moa.Int(1), moa.Int(2)])
 B = core.with_shape(core.unbound('j'), [moa.Int(1), moa.Int(2)])

 assert replace(Shape(A)) == replace(Shape(B))
idxs = vector_of(unbound("index_1"), unbound("index_2"))
assert replace(Index(idxs, A)) == replace(Index(idxs, B))

@saulshanabrook
Copy link
Copy Markdown
Contributor

saulshanabrook commented Dec 7, 2018

To make sure that shape is equal and indexed forms are equal

@saulshanabrook
Copy link
Copy Markdown
Contributor

saulshanabrook commented Dec 7, 2018

Also, you might not need assertions for now. Even a notebook just showing off this functionality with no tests is useful!

@costrouc
Copy link
Copy Markdown
Contributor Author

costrouc commented Dec 7, 2018 •
edited
Loading

@saulshanabrook this is what I was thinking. For now I have the specific tests that test A ^ <1 2> logic disabled. Everything else works with tests enabled. I am developing notebooks now.

@costrouc
Adding initial MOA parser
e700ac2 
 - tests are included with parser
 - adding notebook to demontrate parser
@costrouc costrouc force-pushed the costrouc/add-initial-parser branch from c9dc585 to e700ac2 Compare December 7, 2018 22:08
@costrouc
Copy link
Copy Markdown
Contributor Author

costrouc commented Dec 7, 2018

Added notebook and ready for merge. I think that in combination with the visualization writing expressions should be much easier

@saulshanabrook
Copy link
Copy Markdown
Contributor

saulshanabrook commented Dec 8, 2018

Wow GIthub has a merge conflict UI now! Pretty sweet.

@saulshanabrook saulshanabrook merged commit 7b8b8ae into Quansight-Labs:master Dec 8, 2018
@costrouc
Copy link
Copy Markdown
Contributor Author

costrouc commented Dec 8, 2018

Yeah it does! But be careful. At least from my experience I've had it duplicate code when I though I was editing a single line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@saulshanabrook saulshanabrook saulshanabrook approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@costrouc @saulshanabrook