-
Notifications
You must be signed in to change notification settings - Fork 547
Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
when setting up a source of MariaDB to MySQL, (using the mysql connector), i get an char() error.
ec2 AMI: ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-20230516
Error:
Error: failed to sync v1 source mysql: rpc error: code = Unknown desc = got unknown MySQL type "char(32)" for column "USER" of table "users" while trying to convert it to CloudQuery internal schema type. Supported MySQL types are ["bigint" "bigint unsigned" "bigint(20)" "bigint(20) unsigned" "binary(16)" "blob" "datetime" "double" "float" "int" "int unsigned" "int(10) unsigned" "int(11)" "json" "smallint" "smallint unsigned" "smallint(5)" "smallint(5) unsigned" "smallint(6)" "text" "tinyint" "tinyint unsigned" "tinyint(1)" "tinyint(3) unsigned" "tinyint(4)"]
Expected Behavior
no errors.
CloudQuery (redacted) config
<cloudquery-config/mysql_source.yaml>
kind: source
spec:
name: "mysql"
path: "cloudquery/mysql"
version: "v1.2.0"
tables: ["*"]
destinations: ["mysql"]
spec:
connection_string: "user:password@tcp(dns:port)/dbname"
<cloudquery-config/mysql_destination.yaml>
kind: destination
spec:
name: "mysql"
registry: "github"
path: "cloudquery/mysql"
version: "v2.1.1"
spec:
connection_string: "user:password@tcp(dns:port)/dbname"
Steps To Reproduce
- In ubuntu-jammy-22.04, sudo nano the following cloudquery config files
- run ./cloudquery sync cloudquery-config/mysql_source.yaml cloudquery-config/mysql_destination.yaml
CloudQuery (redacted) logs
023-06-07T07:43:43Z INF Loading spec(s) args=["./cloudquery-config/mysql_source.yaml","cloudquery-config/mysql_destination.yaml"] module=cli
2023-06-07T07:43:44Z INF Source plugin server listening address=/tmp/cq-kOBmPLmBrRLPvAec.sock module=cli
2023-06-07T07:43:47Z INF Destination plugin server listening address=/tmp/cq-DsZoSMKgqDgBBPdd.sock module=cli
2023-06-07T07:43:48Z INF started call grpc.code=OK grpc.component=server grpc.method=GetVersions grpc.method_type=unary grpc.service=cloudquery.discovery.v0.Discovery grpc.start_time=2023-06-07T07:43>
2023-06-07T07:43:48Z INF finished call grpc.code=OK grpc.component=server grpc.method=GetVersions grpc.method_type=unary grpc.service=cloudquery.discovery.v0.Discovery grpc.start_time=2023-06-07T07:4>
2023-06-07T07:43:48Z INF Start sync destinations=["mysql"] module=cli source="mysql (v1.2.0)" sync_time=2023-06-07T07:43:48Z
2023-06-07T07:43:48Z INF started call grpc.code=OK grpc.component=server grpc.method=Init grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:43:48Z grpc.tim>
2023-06-07T07:43:48Z INF finished call grpc.code=OK grpc.component=server grpc.method=Init grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:43:48Z grpc.ti>
2023-06-07T07:43:48Z INF started call grpc.code=OK grpc.component=server grpc.method=GetDynamicTables grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:43:>
2023-06-07T07:43:48Z INF finished call grpc.code=OK grpc.component=server grpc.method=GetDynamicTables grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:43>
2023-06-07T07:43:48Z INF started call grpc.code=OK grpc.component=server grpc.method=Configure grpc.method_type=unary grpc.service=proto.Destination grpc.start_time=2023-06-07T07:43:48Z grpc.time_ms=>
2023-06-07T07:43:48Z INF finished call grpc.code=OK grpc.component=server grpc.method=Configure grpc.method_type=unary grpc.service=proto.Destination grpc.start_time=2023-06-07T07:43:48Z grpc.time_ms>
2023-06-07T07:43:48Z INF started call grpc.code=OK grpc.component=server grpc.method=Migrate grpc.method_type=unary grpc.service=proto.Destination grpc.start_time=2023-06-07T07:43:48Z grpc.time_ms=0.>
2023-06-07T07:43:48Z INF End sync destinations=["mysql"] module=cli source="mysql (v1.2.0)" sync_time=2023-06-07T07:43:48Z
2023-06-07T07:43:48Z INF Sending sync summary to analyticsv1.cloudquery.io:443 module=cli
2023-06-07T07:43:48Z ERR finished call grpc.code=Unknown grpc.component=server grpc.error="Error 1045 (28000): Access denied for user 'pandai'@'172.31.18.25' (using password: YES)" grpc.method=Migrat>
2023-06-07T07:43:48Z INF waiting for destination plugin to terminate module=cli
2023-06-07T07:43:48Z INF Got stop signal. Destination plugin server shutting down address=/tmp/cq-DsZoSMKgqDgBBPdd.sock module=cli signal=interrupt
2023-06-07T07:43:48Z INF waiting for source plugin to terminate module=cli
2023-06-07T07:43:48Z INF Got stop signal. Source plugin server shutting down address=/tmp/cq-kOBmPLmBrRLPvAec.sock module=cli signal=interrupt
2023-06-07T07:43:48Z ERR exiting with error error="failed to sync v1 source mysql: rpc error: code = Unknown desc = Error 1045 (28000): Access denied for user 'pandai'@'172.31.18.25' (using password:>
2023-06-07T07:44:32Z INF Loading spec(s) args=["./cloudquery-config/mysql_source.yaml","cloudquery-config/mysql_destination.yaml"] module=cli
2023-06-07T07:44:32Z INF Source plugin server listening address=/tmp/cq-tBYhtZFoLHLCECUn.sock module=cli
2023-06-07T07:44:33Z INF Destination plugin server listening address=/tmp/cq-WHUkEeZrPawTomVX.sock module=cli
2023-06-07T07:44:34Z INF started call grpc.code=OK grpc.component=server grpc.method=GetVersions grpc.method_type=unary grpc.service=cloudquery.discovery.v0.Discovery grpc.start_time=2023-06-07T07:44>
2023-06-07T07:44:34Z INF finished call grpc.code=OK grpc.component=server grpc.method=GetVersions grpc.method_type=unary grpc.service=cloudquery.discovery.v0.Discovery grpc.start_time=2023-06-07T07:4>
2023-06-07T07:44:34Z INF Start sync destinations=["mysql"] module=cli source="mysql (v1.2.0)" sync_time=2023-06-07T07:44:34Z
2023-06-07T07:44:34Z INF started call grpc.code=OK grpc.component=server grpc.method=Init grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:44:34Z grpc.tim>
2023-06-07T07:44:34Z INF finished call grpc.code=OK grpc.component=server grpc.method=Init grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:44:34Z grpc.ti>
2023-06-07T07:44:34Z INF started call grpc.code=OK grpc.component=server grpc.method=GetDynamicTables grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:44:>
2023-06-07T07:44:34Z INF finished call grpc.code=OK grpc.component=server grpc.method=GetDynamicTables grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:44>
2023-06-07T07:44:34Z INF started call grpc.code=OK grpc.component=server grpc.method=Configure grpc.method_type=unary grpc.service=proto.Destination grpc.start_time=2023-06-07T07:44:34Z grpc.time_ms=>
2023-06-07T07:44:34Z INF finished call grpc.code=OK grpc.component=server grpc.method=Configure grpc.method_type=unary grpc.service=proto.Destination grpc.start_time=2023-06-07T07:44:34Z grpc.time_ms>
2023-06-07T07:44:34Z INF started call grpc.code=OK grpc.component=server grpc.method=Migrate grpc.method_type=unary grpc.service=proto.Destination grpc.start_time=2023-06-07T07:44:34Z grpc.time_ms=0.>
2023-06-07T07:44:34Z ERR finished call grpc.code=Unknown grpc.component=server grpc.error="got unknown MySQL type "char(32)" for column "USER" of table "users" while trying to convert it to Clo>
2023-06-07T07:44:34Z INF End sync destinations=["mysql"] module=cli source="mysql (v1.2.0)" sync_time=2023-06-07T07:44:34Z
2023-06-07T07:44:34Z INF Sending sync summary to analyticsv1.cloudquery.io:443 module=cli
2023-06-07T07:44:35Z INF waiting for destination plugin to terminate module=cli
2023-06-07T07:44:35Z INF Got stop signal. Source plugin server shutting down address=/tmp/cq-tBYhtZFoLHLCECUn.sock module=cli signal=interrupt
2023-06-07T07:44:35Z ERR exiting with error error="failed to sync v1 source mysql: rpc error: code = Unknown desc = got unknown MySQL type "char(32)" for column "USER" of table "users" while tr>
2023-06-07T07:48:27Z INF Loading spec(s) args=["./cloudquery-config/mysql_source.yaml","cloudquery-config/mysql_destination.yaml"] module=cli
2023-06-07T07:48:27Z INF Source plugin server listening address=/tmp/cq-eNBnZaYPOjpFcLmN.sock module=cli
2023-06-07T07:48:28Z INF Destination plugin server listening address=/tmp/cq-FEpEaiyxbmMVSsrj.sock module=cli
2023-06-07T07:48:29Z INF started call grpc.code=OK grpc.component=server grpc.method=GetVersions grpc.method_type=unary grpc.service=cloudquery.discovery.v0.Discovery grpc.start_time=2023-06-07T07:48>
2023-06-07T07:48:29Z INF finished call grpc.code=OK grpc.component=server grpc.method=GetVersions grpc.method_type=unary grpc.service=cloudquery.discovery.v0.Discovery grpc.start_time=2023-06-07T07:4>
2023-06-07T07:48:29Z INF Start sync destinations=["mysql"] module=cli source="mysql (v1.2.0)" sync_time=2023-06-07T07:48:29Z
2023-06-07T07:48:29Z INF started call grpc.code=OK grpc.component=server grpc.method=Init grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:48:29Z grpc.tim>
2023-06-07T07:48:29Z INF finished call grpc.code=OK grpc.component=server grpc.method=Init grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:48:29Z grpc.ti>
2023-06-07T07:48:29Z INF started call grpc.code=OK grpc.component=server grpc.method=GetDynamicTables grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:48:>
2023-06-07T07:48:29Z INF finished call grpc.code=OK grpc.component=server grpc.method=GetDynamicTables grpc.method_type=unary grpc.service=cloudquery.source.v1.Source grpc.start_time=2023-06-07T07:48>
2023-06-07T07:48:29Z INF started call grpc.code=OK grpc.component=server grpc.method=Configure grpc.method_type=unary grpc.service=proto.Destination grpc.start_time=2023-06-07T07:48:29Z grpc.time_ms=>
2023-06-07T07:48:29Z INF finished call grpc.code=OK grpc.component=server grpc.method=Configure grpc.method_type=unary grpc.service=proto.Destination grpc.start_time=2023-06-07T07:48:29Z grpc.time_ms>
2023-06-07T07:48:29Z INF started call grpc.code=OK grpc.component=server grpc.method=Migrate grpc.method_type=unary grpc.service=proto.Destination grpc.start_time=2023-06-07T07:48:29Z grpc.time_ms=0.>
2023-06-07T07:48:30Z ERR finished call grpc.code=Unknown grpc.component=server grpc.error="got unknown MySQL type "varchar(255)" for column "log_name" of table "activity_log" while trying to co>
2023-06-07T07:48:30Z INF End sync destinations=["mysql"] module=cli source="mysql (v1.2.0)" sync_time=2023-06-07T07:48:29Z
2023-06-07T07:48:30Z INF Sending sync summary to analyticsv1.cloudquery.io:443 module=cli
2023-06-07T07:48:30Z INF waiting for destination plugin to terminate module=cli
2023-06-07T07:48:30Z INF Got stop signal. Destination plugin server shutting down address=/tmp/cq-FEpEaiyxbmMVSsrj.sock module=cli signal=interrupt
2023-06-07T07:48:30Z INF waiting for source plugin to terminate module=cli
2023-06-07T07:48:30Z INF Got stop signal. Source plugin server shutting down address=/tmp/cq-eNBnZaYPOjpFcLmN.sock module=cli signal=interrupt
2023-06-07T07:48:30Z ERR exiting with error error="failed to sync v1 source mysql: rpc error: code = Unknown desc = got unknown MySQL type "varchar(255)" for column "log_name" of table "activity>
CloudQuery version
cloudquery command not found?
Additional Context
No response
Pull request (optional)
- I can submit a pull request