Skip to content

Dictionary loading 0 rows in v20.5 (worked properly in v20.4) #12269

@inakivb

Description

@inakivb

Describe the bug
Same dictionary that works on version 20.4.2.9 (official build) doesn't work on version 20.5.2.7 (official build)
The dictionary is loaded with 0 rows

How to reproduce
Dictionary file: http://51.15.227.24/GeoLite2-Country-Blocks-IPv4.csv

Dictionary config:

<?xml version="1.0"?>
<dictionaries>
    <dictionary>
        <name>geoip_country_blocks_ipv4</name>
        <source>
            <file>
                <path>/etc/clickhouse-server/GeoLite2-Country-CSV_20200414/GeoLite2-Country-Blocks-IPv4.csv</path>
                <format>CSVWithNames</format>
            </file>
        </source>
        <lifetime>300</lifetime>
        <layout>
            <ip_trie/>
        </layout>
        <structure>
            <key>
                <attribute>
                    <name>prefix</name>
                    <type>String</type>
                </attribute>
            </key>
            <attribute>
                <name>geoname_id</name>
                <type>UInt32</type>
                <null_value>0</null_value>
            </attribute>
            <attribute>
                <name>registered_country_geoname_id</name>
                <type>UInt32</type>
                <null_value>0</null_value>
            </attribute>
            <attribute>
                <name>represented_country_geoname_id</name>
                <type>UInt32</type>
                <null_value>0</null_value>
            </attribute>
            <attribute>
                <name>is_anonymous_proxy</name>
                <type>UInt8</type>
                <null_value>0</null_value>
            </attribute>
            <attribute>
                <name>is_satellite_provider</name>
                <type>UInt8</type>
                <null_value>0</null_value>
            </attribute>
        </structure>
    </dictionary>

Expected behavior
Expected (as v20.4)

element_count: 297733

SELECT *
FROM system.dictionaries
WHERE name = 'geoip_country_blocks_ipv4'
FORMAT Vertical

Row 1:
──────
database:                    
name:                        geoip_country_blocks_ipv4
status:                      LOADED
origin:                      /etc/clickhouse-server/geoip_country_blocks_ipv4_dictionary.xml
type:                        Trie
key:                         (String)
attribute.names:             ['geoname_id','registered_country_geoname_id','represented_country_geoname_id','is_anonymous_proxy','is_satellite_provider']
attribute.types:             ['UInt32','UInt32','UInt32','UInt8','UInt8']
bytes_allocated:             26505896
query_count:                 132306077
hit_rate:                    1
element_count:               297733
load_factor:                 1
source:                      File: /etc/clickhouse-server/GeoLite2-Country-CSV_20200414/GeoLite2-Country-Blocks-IPv4.csv CSVWithNames
lifetime_min:                300
lifetime_max:                300
loading_start_time:          2020-06-30 18:28:16
last_successful_update_time: 2020-06-30 18:28:19
loading_duration:            2.637
last_exception:              

Current behavior
element_count: 0

SELECT *
FROM system.dictionaries
WHERE name = 'geoip_country_blocks_ipv4'
FORMAT Vertical

Row 1:
──────
database:                    
name:                        geoip_country_blocks_ipv4
status:                      LOADED
origin:                      /etc/clickhouse-server/geoip_country_blocks_ipv4_dictionary.xml
type:                        Trie
key:                         (String)
attribute.names:             ['geoname_id','registered_country_geoname_id','represented_country_geoname_id','is_anonymous_proxy','is_satellite_provider']
attribute.types:             ['UInt32','UInt32','UInt32','UInt8','UInt8']
bytes_allocated:             0
query_count:                 162597
hit_rate:                    1
element_count:               0
load_factor:                 nan
source:                      File: /etc/clickhouse-server/GeoLite2-Country-CSV_20200414/GeoLite2-Country-Blocks-IPv4.csv CSVWithNames
lifetime_min:                300
lifetime_max:                300
loading_start_time:          2020-07-07 17:34:11
last_successful_update_time: 2020-07-07 17:34:11
loading_duration:            0.018
last_exception:              

Error message and/or stacktrace
No error messages in error log file

Dictionary reload in std log seems good except Peak memory usage (for query): 0.00 B.

2020.07.07 18:20:11.027898 [ 8366 ] {44f96fac-dc55-4f23-90e2-b6b4f131e22a} <Debug> executeQuery: (from XXXX) SYSTEM RELOAD DICTIONARY geoip_country_blocks_ipv4
2020.07.07 18:20:11.028047 [ 8366 ] {44f96fac-dc55-4f23-90e2-b6b4f131e22a} <Trace> ContextAccess (default): Access granted: SYSTEM RELOAD DICTIONARY ON *.*
2020.07.07 18:20:11.029249 [ 8366 ] {44f96fac-dc55-4f23-90e2-b6b4f131e22a} <Trace> ExternalDictionariesLoader: Will load the object 'geoip_country_blocks_ipv4' in background, force = true, loading_id = 15
2020.07.07 18:20:11.029423 [ 8321 ] {} <Trace> ExternalDictionariesLoader: Start loading object 'geoip_country_blocks_ipv4'
2020.07.07 18:20:11.030134 [ 8321 ] {} <Trace> DictionaryFactory: Created dictionary source 'File: /etc/clickhouse-server/GeoLite2-Country-CSV_20200414/GeoLite2-Country-Blocks-IPv4.csv CSVWithNames' for dictionary 'geoip_country_blocks_ipv4'
2020.07.07 18:20:11.049378 [ 8321 ] {} <Trace> FileDictionary: loadAll File: /etc/clickhouse-server/GeoLite2-Country-CSV_20200414/GeoLite2-Country-Blocks-IPv4.csv CSVWithNames
2020.07.07 18:20:11.050711 [ 8321 ] {} <Trace> ExternalDictionariesLoader: Supposed update time for 'geoip_country_blocks_ipv4' is 2020-07-07 18:25:11 (loaded, lifetime [300, 300], no errors)
2020.07.07 18:20:11.050736 [ 8321 ] {} <Trace> ExternalDictionariesLoader: Next update time for 'geoip_country_blocks_ipv4' was set to 2020-07-07 18:25:11
2020.07.07 18:20:11.051050 [ 8366 ] {44f96fac-dc55-4f23-90e2-b6b4f131e22a} <Debug> MemoryTracker: Peak memory usage (for query): 0.00 B.
2020.07.07 18:20:11.051176 [ 8366 ] {} <Information> TCPHandler: Processed in 0.023664076 sec.

Metadata

Metadata

Assignees

Labels

bugConfirmed user-visible misbehaviour in official releasecomp-dictionaryDictionaries (in-memory key-value, periodically refreshed from sources).

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions