Centillion 2.0 #3
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add Google Drive doctypes classes
Add Github doctypes classes
Add errors classes and utilities
Add doctypes submodule
Add a static Config class for consistent config access
* add stemming analyzer to gdrive docx doctype schema * add makefile, basic requirements.txt, bin checks * improve config constructor * apply black to doctypes submodule * lint doctypes submodule * add flake8 lint prefs file * [tests part 1] Add basic test code (#18, #20, #21) * add flake8 lint prefs file * remove tests * add test __init__ with utils and decorators * [tests part 2] Add Config class tests (#20) * add Config class test * fix tests * update makefile lint rule * one more fix * fix install process for doctypes submodule * fix Config class typos * add mypy to requirements * fix staticmethod/classmethod refs * update doctypes to fix mypy errors * fix import in doctype * add GithubBaseDoctype back into github.py * improve config tests * add centillion config exception class * use centillion config exception class * [tests part 3] Add doctypes tests (#21) * split tests/__init__.py into separate importable components * fix decorator import in config tests * add doctypes registry test * fixup config tests too * lint * fix registry bug * add github doctype tests (wip) * add github doctype integration tests * update github integration tests, use common integration test config * fix integration test decorator check * split up doctype tests * use a context manager to handle temp config, use standalone/integration to set test config files * add centillion 2.0 example config class for doctype/config tests * update source files with test fixes * update config.py and error.py to get gdrive doctype tests passing * update gdrive exception/type handling, get abs path to token file * add gdrive doctype tests * add gdrive creds to example config * add test mixins * update tests to get doctypes github/gdrive tests working * s/application/access/g
chmreid
force-pushed
the
chmreid-centillion-2
branch
from
49c3ac3 to
5cb8be9
Compare
…lasses (#24) * add github jinja template and render search result methods, restructure utils * lint * refactor and move some utility methods out into utils * update dotdict class * add tests for github jinja templates functionality * add render_search_results and get_jinja_template methods to gdrive doctypes * fixup issue/pr doctype template title * add readme with some doctype notes * add note in test for later * refactor/reorganize github doctype tests * update tests * fix return statements * fix return type * clean up config dir * update tests readme
* update gitignore * add util to check if path is abs * update how we look for gdrive credentials (check 3 paths) * improve config credentials finding and error messages * misc gdrive fixes - mimetype, err handling, pagination * multiple improvements and fixes to github doctype classes * fix import in utils * restore old config file when using TempCentillionConfig context manager * add invalid google creds to test fixtures * fix test mixin name * add placeholder tests, fix integration tests * fix invalid credentials test in github doctype, plus other fixes from integration tests * fix gdrive doctype tests for integration tests
* add make test/all_test/integration_test/standalone_test to makefile * fix make test commands in makefile * fix up travis yml (no integration stage) * use secret archive on travis * enable integration stage in travis * update decorators * move check for token_path existence to method that constructs gdrive service * fix parent class methods * fix test decorators * update decorators and context to fix build errors * fix attribute error * run lint before tests * update requirements.txt * make temp dir * update requirements.txt.in
* remove payloads from search submodule * fix up makefile comments * add config method to get index directory * one comment * implement schema methods, create/open index methods, sync methods * add imports of Search to search/__init__.py * update manifest.in, no search payloads * add schema labels to doctype module * add search submodule to setup.py * add search submodule to makefile * fix doctype imports in search module * add initial tests for search submodule * implement add/delete doc capabilities, handle index creation * fix label name * add tests for search submodule (work in progress) * Config class: ensure creation of directories; add reset() method * improve logic of config tests * use Config.reset() method in temp config context manager * add search index exception class * import temp centillion config in all tests * fix config file handling in github doctype tests * add tests for context managers in test dir * split utilities into separate files, import from __init__ * use modularized config/doc utilities * fix method signatures, comment out unused imports * Add tests for search submodule (no search() method) (#37) * lint * add dir/module for test doctypes * fix up config tests * fix up/start crud search test with plain doc * use "plain" doctype in doctype tests * add plain doctype to tests.doctypes module * fix how doctypes are imported (in test __init__, not test search) * update how integration and standalone decorators set configuration * add registry to plain doctypes class * change iterate doctypes mixins to take a function as an input arg * add plain doc registration step to test search * update config utils * update get_remote_list -> get_remote_map in doctypes submodule * fix typing signature in gh doctypes * make plain doctype register_document a classmethod * improve/fix action() formulation in test mixins * fix remote_list -> remote_map * fix remote_list -> remote_map again * finish crud test * update tests readme/secrets md files * no microseconds * add get_by_id to search class * modify plain doctype to add get_remote_map and get_by_id, add placeholder methods for other virtual methods * update method name in gdrive doctypes test * fix docstrings and comments in github doctypes test * no microseconds in searchdocs utilities * add some more add/update tests for Search class * add tests for plain doctype (used in tests only) * remove unused import * fix whoosh update_document() call * fix test search update_docs() test * remove sleep command * make plain doctype a static class again * test plain doctype register_document function * fix search submodule test, use updated plaindoctype class and update_docs method call * use PlainDoctype instance, not class * fix add/update/delete/sync docs methods in search submodule * update typing hints for doctype classes * lint * fix typing hint * add requirements-dev files, add pytest codecov plugin * any branch i guess * add pytest-cov to CI/CD, add it to make test command * update requirements file to use in .travis.yml * add test for Search.delete_docs method * add test for Search.delete_doc method * finish crud tests, break apart add tests * update tests of Search methods add_doc and add_docs * make a temp dir variable for integration tests * add code to create/delete temporary directory for integration tests * run tests together for "make test" rule * fix stages in .travis.yml * add loggers * remove get_schema and get_common_schema from Doctype class * remove references to get_schema and get_common_schema from github/gdrive doctypes * logging.warn -> logging.warning * remove references to get_schema and get_common_schema from search * add logging import * remove references to get_schema and get_common_schema from doctypes tests * clean up/organize Search test, add schema tests * add tests of indexer init * lint * test opening search index in test_search * functionalize method to update remote documents in PlainDoctype * add test of search index update functionality
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update/reimplement/reorganize centillion.
Make the search submodule and the doctype submodule separate, improve the logic around adding new doctypes, split out functionality into reusable parts.
New functionality will be implemented in PRs based on this PR.
This pull request removes the old search submodule and adds stub methods for a new Search class.