Skip to content

Add PostgreSQL table function, dictionary source, database engine#18554

Merged
alexey-milovidov merged 52 commits intoClickHouse:masterfrom
kssenii:pg2ch
Jan 16, 2021
Merged

Add PostgreSQL table function, dictionary source, database engine#18554
alexey-milovidov merged 52 commits intoClickHouse:masterfrom
kssenii:pg2ch

Conversation

@kssenii
Copy link
Copy Markdown
Member

@kssenii kssenii commented Dec 27, 2020

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Added PostgreSQL table engine (both select/insert, with support for multidimensional arrays), also as table function. Added PostgreSQL dictionary source. Added PostgreSQL database engine.

@robot-clickhouse robot-clickhouse added doc-alert pr-feature Pull request with new product feature labels Dec 27, 2020
@robot-clickhouse robot-clickhouse added the submodule changed At least one submodule changed in this PR. label Dec 27, 2020
@alexey-milovidov alexey-milovidov added the 🎅 🎁 gift🎄 To make people wonder label Dec 27, 2020
auto parsed_host_port = parseAddress(args[0]->as<ASTLiteral &>().value.safeGet<String>(), 5432);
remote_table_name = args[2]->as<ASTLiteral &>().value.safeGet<String>();

connection_str = fmt::format("dbname={} host={} port={} user={} password={}",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be unsafe, need to check it...

}


/// Cannot just use serializeAsText for array data type even though it converts perfectly
Copy link
Copy Markdown
Member

@alexey-milovidov alexey-milovidov Jan 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it fails with copy command:

E helpers.client.QueryRuntimeException: Client failed! Return code: 233, stderr: Received exception from server (version 21.1.1):
E Code: 1001. DB::Exception: Received from 172.21.0.3:9000. DB::Exception: pqxx::data_exception: ERROR: malformed array literal: "array['2000-05-12','2000-05-12']"
E DETAIL: Array value must start with "{" or dimension information.
E CONTEXT: COPY test_array_dimensions, line 1, column a: "array['2000-05-12','2000-05-12']"

E helpers.client.QueryRuntimeException: Client failed! Return code: 233, stderr: Received exception from server (version 21.1.1):
E Code: 1001. DB::Exception: Received from 172.21.0.3:9000. DB::Exception: pqxx::data_exception: ERROR: malformed array literal: "array[[[1,1],[1,1]],[[3,3],[3,3]],[[4,4],[5,5]]]"
E DETAIL: Array value must start with "{" or dimension information.
E CONTEXT: COPY array_columns, line 1, column b: "array[[[1,1],[1,1]],[[3,3],[3,3]],[[4,4],[5,5]]]"


/// Conversion is done via column casting because with writeText(Array..) got incorrect conversion
/// of Date and DateTime data types and it added extra quotes for values inside array.
static std::string clickhouseToPostgresArray(const Array & array_field, const DataTypePtr & data_type)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FieldVisitorToString may also suffice.

@alexey-milovidov alexey-milovidov merged commit 7de745c into ClickHouse:master Jan 16, 2021
@tisonkun
Copy link
Copy Markdown
Contributor

After this commit darwin build failed. See also #19277

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🎅 🎁 gift🎄 To make people wonder pr-feature Pull request with new product feature submodule changed At least one submodule changed in this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants