How to detect errors during processing in binary formats over HTTP? #46426
Description
ClickHouse can send errors during requests if the processing fails.
For instance,
$ curl -v -sS 'http://localhost:8123/?max_execution_time=0.01' -d 'SELECT toUInt8(65 + number % 5) FROM system.numbers LIMIT 9000000 FORMAT RowBinary' > data
$ tail -c 300 data | hexdump -CResult:
00000000 42 43 44 45 41 42 43 44 45 41 42 43 44 45 41 42 |BCDEABCDEABCDEAB|
00000010 43 44 45 41 42 43 44 45 41 42 43 44 45 41 42 43 |CDEABCDEABCDEABC|
00000020 44 45 41 42 43 44 45 41 42 43 44 45 41 42 43 44 |DEABCDEABCDEABCD|
00000030 45 41 42 43 44 45 41 42 43 44 45 41 42 43 44 45 |EABCDEABCDEABCDE|
00000040 41 42 43 44 45 41 42 43 44 45 41 42 43 44 45 41 |ABCDEABCDEABCDEA|
00000050 42 43 44 45 41 42 43 44 45 41 42 43 44 45 41 42 |BCDEABCDEABCDEAB|
00000060 43 44 45 41 42 43 44 45 41 42 43 44 45 41 42 43 |CDEABCDEABCDEABC|
00000070 44 45 41 42 43 44 45 41 42 43 44 45 41 42 43 44 |DEABCDEABCDEABCD|
00000080 45 41 42 43 44 45 41 42 43 44 45 41 42 43 44 45 |EABCDEABCDEABCDE|
00000090 41 42 43 44 45 41 42 43 44 45 41 43 6f 64 65 3a |ABCDEABCDEACode:|
000000a0 20 31 35 39 2e 20 44 42 3a 3a 45 78 63 65 70 74 | 159. DB::Except|
000000b0 69 6f 6e 3a 20 54 69 6d 65 6f 75 74 20 65 78 63 |ion: Timeout exc|
000000c0 65 65 64 65 64 3a 20 65 6c 61 70 73 65 64 20 30 |eeded: elapsed 0|
000000d0 2e 30 31 33 31 37 30 35 37 33 20 73 65 63 6f 6e |.013170573 secon|
000000e0 64 73 2c 20 6d 61 78 69 6d 75 6d 3a 20 30 2e 30 |ds, maximum: 0.0|
000000f0 31 2e 20 28 54 49 4d 45 4f 55 54 5f 45 58 43 45 |1. (TIMEOUT_EXCE|
00000100 45 44 45 44 29 20 28 76 65 72 73 69 6f 6e 20 32 |EDED) (version 2|
00000110 32 2e 31 30 2e 32 2e 31 31 20 28 6f 66 66 69 63 |2.10.2.11 (offic|
00000120 69 61 6c 20 62 75 69 6c 64 29 29 0a |ial build)).|
0000012c
wait_end_of_query=1is not always applicable, so I cannot rely on it in my generic client.- AFAIK, there is no support for trailers: Use HTTP trailers. #21136.
- An error can be included in the same HTTP chunk as data, so I cannot check only the initial bytes of chunks.
- Trailing bytes of a chunk can be checked, but another question here: can this message be split into several chunks?
- Checks should rely on
(official build))orCode:, but there are no guarantees by CH for these messages, innit? - Binary formats can include bytes, such checks can reject valid data.
So, how to detect an error during parsing reliably?