Skip to content

Clickhouse driver [WIP]#8722

Closed
ei-grad wants to merge 12 commits intometabase:masterfrom
ei-grad:clickhouse_driver
Closed

Clickhouse driver [WIP]#8722
ei-grad wants to merge 12 commits intometabase:masterfrom
ei-grad:clickhouse_driver

Conversation

@ei-grad
Copy link
Copy Markdown
Contributor

@ei-grad ei-grad commented Oct 18, 2018

@enqueue
Copy link
Copy Markdown

enqueue commented Nov 6, 2018

Regarding the NPE: This is probably due to the JDBC driver returning a null result set for queries starting with comments, e. g. -- MetaBase\n SELECT 42 I created PR ClickHouse/clickhouse-java#256 to counter this.

Regarding the test data, I would like to ask @camsaul for advice: The test-data/checkins table has a date column defined with type base-type :type/Date. Then data is loaded via a timestamp, e.g. #inst "2014-09-15T07 resulting in 2014-09-15 07:00:00 on the wire. This is rejected by the ClickHouse server because it cannot parse this input as a Date ("Date must be in YYYY-MM-DD format"). What is expected of the driver in this case? What would be the right place in the code to adjust the format of the timestamp depending on the target column type when loading data?

@enqueue
Copy link
Copy Markdown

enqueue commented Nov 24, 2018

@ei-grad is there a way for me to create a PR against this branch that is already rebased on master? See changes in enqueue/clickhouse_update

We are far away from completion (ca. 40 tests still failing), but if you compile JDBC driver including your changes ClickHouse/clickhouse-java#247 you will be able to play around with the database and perhaps improve the driver further ;-)

@ei-grad
Copy link
Copy Markdown
Contributor Author

ei-grad commented Nov 26, 2018

@enqueue rebased, moved the project.clj change to separate commit to easier rebase in future.

@ei-grad
Copy link
Copy Markdown
Contributor Author

ei-grad commented Nov 26, 2018

Merged, removed the TODO_clickhouse.txt file, not sure if it should go in the source tree, it is better to put it here, imo (added to the PR first comment).

@ei-grad
Copy link
Copy Markdown
Contributor Author

ei-grad commented Nov 26, 2018

When I run the tests with ENGINES=h2,clickhouse lein test the result is:

Ran 3446 tests containing 3342 assertions in 91516 msecs
1 failures, 8 errors.

Did something changed with running the tests?..

@ei-grad
Copy link
Copy Markdown
Contributor Author

ei-grad commented Nov 26, 2018

@ei-grad
Copy link
Copy Markdown
Contributor Author

ei-grad commented Nov 26, 2018

@enqueue I didn't have a look to the latest driver structure refactoring yet, do you have any ideas why this extra fields appeared? ^


(driver/register! :clickhouse, :parent :sql-jdbc)

(def ^:private default-base-types
Copy link
Copy Markdown

@kzaitsev kzaitsev Nov 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Added the data type to the test.

@enqueue
Copy link
Copy Markdown

enqueue commented Nov 27, 2018

@ei-grad thanks for your help! Let's try to get this PR ready for christmas...

  • Will remove TODO_clickhouse.txt when we are done with this branch (unless we forget ;-) )
  • You have to run the tests using DRIVERS=h2,ClickHouse instead of ENGINE
  • I do not know about the buggy extra fields. Are these specific to ClickHouse driver?

@enqueue
Copy link
Copy Markdown

enqueue commented Dec 17, 2018

The previous comment is obsolete because the structure has changed. Unfortunately this PR seems to have stalled, so I have added the current state of my branch as PR #9469

@notarseniy notarseniy mentioned this pull request Jan 9, 2019
@CheatEx
Copy link
Copy Markdown

CheatEx commented Feb 6, 2019

Hi. Does anyone working on finishing this effort? Specifically on porting to new plugin system?

@enqueue
Copy link
Copy Markdown

enqueue commented Feb 6, 2019

Hello @CheatEx I have been working on this driver, and refactored it for the new module structure. Unfortunately I cannot update this PR myself (see ei-grad#6). If you want to check out a more recent version, see clickhouse_module_2 branch. What should I do?

@zeronewb
Copy link
Copy Markdown

imho it should be rebased to new PR :/
@ei-grad, @enqueue WDYT?

@ei-grad ei-grad closed this Mar 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants