-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Dictionary loading 0 rows in v20.5 (worked properly in v20.4) #12269
Copy link
Copy link
Closed
Labels
bugConfirmed user-visible misbehaviour in official releaseConfirmed user-visible misbehaviour in official releasecomp-dictionaryDictionaries (in-memory key-value, periodically refreshed from sources).Dictionaries (in-memory key-value, periodically refreshed from sources).
Description
Describe the bug
Same dictionary that works on version 20.4.2.9 (official build) doesn't work on version 20.5.2.7 (official build)
The dictionary is loaded with 0 rows
How to reproduce
Dictionary file: http://51.15.227.24/GeoLite2-Country-Blocks-IPv4.csv
Dictionary config:
<?xml version="1.0"?>
<dictionaries>
<dictionary>
<name>geoip_country_blocks_ipv4</name>
<source>
<file>
<path>/etc/clickhouse-server/GeoLite2-Country-CSV_20200414/GeoLite2-Country-Blocks-IPv4.csv</path>
<format>CSVWithNames</format>
</file>
</source>
<lifetime>300</lifetime>
<layout>
<ip_trie/>
</layout>
<structure>
<key>
<attribute>
<name>prefix</name>
<type>String</type>
</attribute>
</key>
<attribute>
<name>geoname_id</name>
<type>UInt32</type>
<null_value>0</null_value>
</attribute>
<attribute>
<name>registered_country_geoname_id</name>
<type>UInt32</type>
<null_value>0</null_value>
</attribute>
<attribute>
<name>represented_country_geoname_id</name>
<type>UInt32</type>
<null_value>0</null_value>
</attribute>
<attribute>
<name>is_anonymous_proxy</name>
<type>UInt8</type>
<null_value>0</null_value>
</attribute>
<attribute>
<name>is_satellite_provider</name>
<type>UInt8</type>
<null_value>0</null_value>
</attribute>
</structure>
</dictionary>
Expected behavior
Expected (as v20.4)
element_count: 297733
SELECT *
FROM system.dictionaries
WHERE name = 'geoip_country_blocks_ipv4'
FORMAT Vertical
Row 1:
──────
database:
name: geoip_country_blocks_ipv4
status: LOADED
origin: /etc/clickhouse-server/geoip_country_blocks_ipv4_dictionary.xml
type: Trie
key: (String)
attribute.names: ['geoname_id','registered_country_geoname_id','represented_country_geoname_id','is_anonymous_proxy','is_satellite_provider']
attribute.types: ['UInt32','UInt32','UInt32','UInt8','UInt8']
bytes_allocated: 26505896
query_count: 132306077
hit_rate: 1
element_count: 297733
load_factor: 1
source: File: /etc/clickhouse-server/GeoLite2-Country-CSV_20200414/GeoLite2-Country-Blocks-IPv4.csv CSVWithNames
lifetime_min: 300
lifetime_max: 300
loading_start_time: 2020-06-30 18:28:16
last_successful_update_time: 2020-06-30 18:28:19
loading_duration: 2.637
last_exception:
Current behavior
element_count: 0
SELECT *
FROM system.dictionaries
WHERE name = 'geoip_country_blocks_ipv4'
FORMAT Vertical
Row 1:
──────
database:
name: geoip_country_blocks_ipv4
status: LOADED
origin: /etc/clickhouse-server/geoip_country_blocks_ipv4_dictionary.xml
type: Trie
key: (String)
attribute.names: ['geoname_id','registered_country_geoname_id','represented_country_geoname_id','is_anonymous_proxy','is_satellite_provider']
attribute.types: ['UInt32','UInt32','UInt32','UInt8','UInt8']
bytes_allocated: 0
query_count: 162597
hit_rate: 1
element_count: 0
load_factor: nan
source: File: /etc/clickhouse-server/GeoLite2-Country-CSV_20200414/GeoLite2-Country-Blocks-IPv4.csv CSVWithNames
lifetime_min: 300
lifetime_max: 300
loading_start_time: 2020-07-07 17:34:11
last_successful_update_time: 2020-07-07 17:34:11
loading_duration: 0.018
last_exception:
Error message and/or stacktrace
No error messages in error log file
Dictionary reload in std log seems good except Peak memory usage (for query): 0.00 B.
2020.07.07 18:20:11.027898 [ 8366 ] {44f96fac-dc55-4f23-90e2-b6b4f131e22a} <Debug> executeQuery: (from XXXX) SYSTEM RELOAD DICTIONARY geoip_country_blocks_ipv4
2020.07.07 18:20:11.028047 [ 8366 ] {44f96fac-dc55-4f23-90e2-b6b4f131e22a} <Trace> ContextAccess (default): Access granted: SYSTEM RELOAD DICTIONARY ON *.*
2020.07.07 18:20:11.029249 [ 8366 ] {44f96fac-dc55-4f23-90e2-b6b4f131e22a} <Trace> ExternalDictionariesLoader: Will load the object 'geoip_country_blocks_ipv4' in background, force = true, loading_id = 15
2020.07.07 18:20:11.029423 [ 8321 ] {} <Trace> ExternalDictionariesLoader: Start loading object 'geoip_country_blocks_ipv4'
2020.07.07 18:20:11.030134 [ 8321 ] {} <Trace> DictionaryFactory: Created dictionary source 'File: /etc/clickhouse-server/GeoLite2-Country-CSV_20200414/GeoLite2-Country-Blocks-IPv4.csv CSVWithNames' for dictionary 'geoip_country_blocks_ipv4'
2020.07.07 18:20:11.049378 [ 8321 ] {} <Trace> FileDictionary: loadAll File: /etc/clickhouse-server/GeoLite2-Country-CSV_20200414/GeoLite2-Country-Blocks-IPv4.csv CSVWithNames
2020.07.07 18:20:11.050711 [ 8321 ] {} <Trace> ExternalDictionariesLoader: Supposed update time for 'geoip_country_blocks_ipv4' is 2020-07-07 18:25:11 (loaded, lifetime [300, 300], no errors)
2020.07.07 18:20:11.050736 [ 8321 ] {} <Trace> ExternalDictionariesLoader: Next update time for 'geoip_country_blocks_ipv4' was set to 2020-07-07 18:25:11
2020.07.07 18:20:11.051050 [ 8366 ] {44f96fac-dc55-4f23-90e2-b6b4f131e22a} <Debug> MemoryTracker: Peak memory usage (for query): 0.00 B.
2020.07.07 18:20:11.051176 [ 8366 ] {} <Information> TCPHandler: Processed in 0.023664076 sec.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugConfirmed user-visible misbehaviour in official releaseConfirmed user-visible misbehaviour in official releasecomp-dictionaryDictionaries (in-memory key-value, periodically refreshed from sources).Dictionaries (in-memory key-value, periodically refreshed from sources).