input_format_allow_errors_num not working #10668
Description
Describe the bug
input_format_allow_errors_num and input_format_allow_errors_ratio not working as expected.
How to reproduce
- Version
20.3.8.53 - Interface
clickhouse-client - Table
CREATE TABLE nums2 (
num1 Int64,
num2 Int64
) ENGINE = MergeTree() ORDER BY num1
- Data
nums.csv
1,0
2,0
3,0
4,0
5,0
6,0
7,0
8,0
9,0
10,0
11,0
12,AAAAAAA
13,0
14,0
15,0
16,0
17,0
18,0
19,0
20,0
Command
WORKS(using -n 10 to only get first 10 lines):
$ head -n 10 /nums/nums.csv | tr '\r\n' '\n' | clickhouse-client -h clickhouse -q "INSERT INTO nums2(num1,num2) FORMAT CSV" --format_csv_delimiter="," --compression 0 --format_csv_allow_double_quotes=0 --format_csv_allow_single_quotes=0 --input_format_allow_errors_num=10 --input_format_allow_errors_ratio=0.1
DOESN'T WORK(using -n 15 to include 12th line, 12,AAAAAAA):
$ head -n 15 /nums/nums.csv | tr '\r\n' '\n' | clickhouse-client -h clickhouse -q "INSERT INTO nums2(num1,num2) FORMAT CSV" --format_csv_delimiter="," --compression 0 --format_csv_allow_double_quotes=0 --format_csv_allow_single_quotes=0 --input_format_allow_errors_num=10 --input_format_allow_errors_ratio=0.1
Code: 117, e.displayText() = DB::Exception: Expected end of line, Stack trace (when copying this message, always include the lines below):
Gets this error:
0. Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x10542450 in /usr/bin/clickhouse
1. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x8f4272d in /usr/bin/clickhouse
2. ? @ 0xdc5f2ad in /usr/bin/clickhouse
3. ? @ 0xdc57971 in /usr/bin/clickhouse
4. DB::CSVRowInputFormat::readRow(std::__1::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, DB::RowReadExtension&) @ 0xdc5da39 in /usr/bin/clickhouse
5. DB::IRowInputFormat::generate() @ 0xdc2b8d1 in /usr/bin/clickhouse
6. DB::ISource::work() @ 0xdba8ceb in /usr/bin/clickhouse
7. DB::InputStreamFromInputFormat::readImpl() @ 0xdb5986d in /usr/bin/clickhouse
8. DB::IBlockInputStream::read() @ 0xce5701f in /usr/bin/clickhouse
9. DB::ParallelParsingBlockInputStream::parserThreadFunction(unsigned long) @ 0xdb5e462 in /usr/bin/clickhouse
10. ThreadPoolImpl<ThreadFromGlobalPool>::worker(std::__1::__list_iterator<ThreadFromGlobalPool, void*>) @ 0x8f6792b in /usr/bin/clickhouse
11. ThreadFromGlobalPool::ThreadFromGlobalPool<void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(void&&, void ThreadPoolImpl<ThreadFromGlobalPool>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()&&...)::'lambda'()::operator()() const @ 0x8f68608 in /usr/bin/clickhouse
12. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x8f667eb in /usr/bin/clickhouse
13. ? @ 0x8f64c33 in /usr/bin/clickhouse
14. start_thread @ 0x76db in /lib/x86_64-linux-gnu/libpthread-2.27.so
15. clone @ 0x12188f in /lib/x86_64-linux-gnu/libc-2.27.so
(version 20.3.8.53 (official build))
Code: 117. DB::Exception: Expected end of line