Introduce Numba-based FSM utilities#272
Merged
brandonwillard merged 3 commits intodottxt-ai:mainfrom Sep 29, 2023
Merged
Conversation
23fe929 to
80a2fb1
Compare
27feb9d to
7ae64b3
Compare
Closed
9f416a5 to
d79967e
Compare
9a186fe to
c7b3cc8
Compare
c7b3cc8 to
6739b30
Compare
4 tasks
6739b30 to
bd032c4
Compare
bd032c4 to
61cf813
Compare
ab84dc5 to
b4d4b2b
Compare
87c97cd to
fb37a1c
Compare
fb37a1c to
9004440
Compare
Contributor
|
will it help to speed up the "self.regex_fsm = regex_pattern.to_fsm().reduce()" in outlines 0.0.8,i found when the set the constrain field long like maxLength=1000, it takes very long in regex_fsm construction |
Member
|
Did you try with |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces Numba JITed FSM utilities with 20x speed-ups over the current pure Python implementations.
It also introduces a more memory efficient "end-to-end" means of producing FSM indices. I avoided implementing it this way originally because it involves multiple iterations through a vocabulary, but, in order to address some memory-related shortcomings of the CFG indexing approaches tested in #178, this might be the better approach for now. It's a clear trade-off between processing and memory—now leaning toward processing—but, with the JIT speed-ups, it's reasonable.
Closes #226 (for now), closes #239, and should help with #192.
Regeximplementation.This was always how it was supposed to work, but our previous prototype didn't implement it. Since we're updating/replacing that prototype, it might be best to add it now.
We need to make sure that caching works exactly as expected (i.e. only once for all the index-building code).
uints for the states, instead ofint64.