old one is 1.2.20 release, new one is commit ac8802a30

Created Diff never expires
50 removals
61 lines
42 additions
57 lines
./sling run --src-conn CLICKHOUSE --src-stream 'v2' --tgt-conn CLICKHOUSE --tgt-object newv --mode full-refresh
./sling run --src-conn CLICKHOUSE --src-stream 'v2' --tgt-conn CLICKHOUSE --tgt-object newv --mode full-refresh
2024-10-09 00:15:19 DBG Sling version: 1.2.20 (linux amd64)
2024-10-09 00:16:05 DBG Sling version: dev (linux amd64)
2024-10-09 00:15:19 DBG type is db-db
2024-10-09 00:16:05 DBG type is db-db
2024-10-09 00:15:19 DBG using: {"columns":null,"mode":"full-refresh","transforms":null}
2024-10-09 00:16:05 DBG using: {"columns":null,"mode":"full-refresh","transforms":null}
2024-10-09 00:15:19 DBG using source options: {"empty_as_null":false,"null_if":"NULL","datetime_format":"AUTO","max_decimals":11}
2024-10-09 00:16:05 DBG using source options: {"empty_as_null":false,"null_if":"NULL","datetime_format":"AUTO","max_decimals":11}
2024-10-09 00:15:19 DBG using target options: {"batch_limit":100000,"datetime_format":"auto","file_max_rows":0,"max_decimals":11,"use_bulk":true,"add_new_columns":true,"adjust_column_type":false,"column_casing":"source"}
2024-10-09 00:16:05 DBG using target options: {"batch_limit":100000,"datetime_format":"auto","file_max_rows":0,"max_decimals":11,"use_bulk":true,"add_new_columns":true,"adjust_column_type":false,"column_casing":"source"}
2024-10-09 00:15:19 DBG opened "clickhouse" connection (conn-clickhouse-MjV)
2024-10-09 00:16:05 DBG opened "clickhouse" connection (conn-clickhouse-1nO)
2024-10-09 00:15:19 DBG opened "clickhouse" connection (conn-clickhouse-gRT)
2024-10-09 00:16:05 DBG opened "clickhouse" connection (conn-clickhouse-bZ0)
2024-10-09 00:15:19 INF connecting to source database (clickhouse)
2024-10-09 00:16:05 INF connecting to source database (clickhouse)
2024-10-09 00:15:19 INF connecting to target database (clickhouse)
2024-10-09 00:16:05 INF connecting to target database (clickhouse)
2024-10-09 00:15:19 INF reading from source database
2024-10-09 00:16:05 INF reading from source database
2024-10-09 00:15:19 DBG select * from `default`.`v2`
2024-10-09 00:16:05 DBG select * from `default`.`v2`
2024-10-09 00:15:19 INF writing to target database [mode: full-refresh]
2024-10-09 00:16:05 INF writing to target database [mode: full-refresh]
2024-10-09 00:15:19 DBG drop table if exists `default`.`newv_tmp`
2024-10-09 00:16:05 DBG drop table if exists `default`.`newv_tmp`
2024-10-09 00:15:19 DBG table `default`.`newv_tmp` dropped
2024-10-09 00:16:05 DBG table `default`.`newv_tmp` dropped
2024-10-09 00:15:19 DBG create table `default`.`newv_tmp` (`id` Nullable(Int64),
2024-10-09 00:16:05 DBG create table `default`.`newv_tmp` (`id` Nullable(Int64),
`t` Nullable(DateTime64(6))) engine=MergeTree ORDER BY tuple()
`t` Nullable(DateTime64(6))) engine=MergeTree ORDER BY tuple()
2024-10-09 00:15:19 INF streaming data
2024-10-09 00:16:05 INF created table `default`.`newv_tmp`
2024-10-09 00:15:19 DBG use `default`
2024-10-09 00:16:05 INF streaming data
2s 480,887 264999 r/s 19 MB | 18% MEM | 11% CPU ^C
2024-10-09 00:16:05 DBG use `default`
2s 515,313 261631 r/s 20 MB | 18% MEM | 11% CPU ^C
interrupting...
interrupting...
2024-10-09 00:15:22 DBG opened "clickhouse" connection (conn-clickhouse-ROI)
2024-10-09 00:16:08 DBG opened "clickhouse" connection (conn-clickhouse-ABX)
2024-10-09 00:15:22 DBG drop table if exists `default`.`newv_tmp`
2024-10-09 00:16:08 DBG drop table if exists `default`.`newv_tmp`
2s 751,323 544233 r/s 30 MB | 18% MEM | 10% CPU
2024-10-09 00:16:08 DBG table `default`.`newv_tmp` dropped
2024-10-09 00:15:22 DBG table `default`.`newv_tmp` dropped
2024-10-09 00:16:08 DBG closed "clickhouse" connection (conn-clickhouse-ABX)
2024-10-09 00:15:22 DBG closed "clickhouse" connection (conn-clickhouse-ROI)
2024-10-09 00:16:08 DBG closed "clickhouse" connection (conn-clickhouse-1nO)
2024-10-09 00:15:22 DBG closed "clickhouse" connection (conn-clickhouse-MjV)
2024-10-09 00:16:08 DBG closed "clickhouse" connection (conn-clickhouse-bZ0)
2024-10-09 00:15:22 DBG closed "clickhouse" connection (conn-clickhouse-gRT)
3s 779,585 488702 r/s 31 MB | 18% MEM | 12% CPU
2024-10-09 00:15:22 INF execution failed
2024-10-09 00:16:08 INF execution failed
fatal:
fatal:
--- proc.go:271 main ---
--- proc.go:271 main ---
--- sling_cli.go:458 main ---
--- sling_cli.go:458 main ---
--- sling_cli.go:494 cliInit ---
--- sling_cli.go:494 cliInit ---
--- cli.go:286 CliProcess ---
--- cli.go:286 CliProcess ---
--- sling_run.go:225 processRun ---
--- sling_run.go:225 processRun ---
~ failure running task (see docs @ https://docs.slingdata.io/sling-cli)
~ failure running task (see docs @ https://docs.slingdata.io/sling-cli)
--- sling_run.go:396 runTask ---
--- sling_run.go:396 runTask ---
--- task_run.go:138 Execute ---
--- task_run.go:155 Execute ---




--- task_run.go:99 func1 ---
--- task_run.go:116 func2 ---
--- task_run.go:543 runDbToDb ---
--- task_run.go:559 runDbToDb ---
--- task_run_write.go:331 WriteToDb ---
--- task_run_write.go:228 WriteToDb ---
--- database.go:2313 BulkImportFlow ---
--- database.go:2313 BulkImportFlow ---
~ could not bulk import
~ could not bulk import
--- database.go:2300 func1 ---
--- database.go:2300 func1 ---
~ could not copy data
~ could not copy data
--- database_clickhouse.go:261 BulkImportStream ---
--- database_clickhouse.go:261 BulkImportStream ---
~ could not commit transaction
~ could not commit transaction
--- database_clickhouse.go:257 func2 ---
--- database_clickhouse.go:257 func2 ---


--- task_run.go:99 func1 ---
--- task_run.go:116 func2 ---
--- task_run.go:559 runDbToDb ---
~ Could not WriteToDb
~ Could not WriteToDb
--- task_run.go:543 runDbToDb ---
--- task_run_write.go:234 WriteToDb ---
~ could not insert into `default`.`newv_tmp`
could not insert into `default`.`newv_tmp`
--- task_run_write.go:337 WriteToDb ---

--- datastream.go:944 func9 ---
~ error during iteration
--- datastream.go:854 2 ---