Skip to content

RabbitMQ ENGINE randomly re publishing messages #21112

@khlystov

Description

@khlystov

Describe the bug
I have a table based on RabbitMQ ENGINE, I do inserts in that table and it works as expected and I am able to see incoming messages in rabbitmq. BUT from time to time I am getting this log message in the error log and exactly after that one of my previously posted messages becomes republished.

2021.02.23 17:43:16.443563 [ 21028 ] {} StorageRabbitMQ (test): Producer's channel 20 error: connection lost
2021.02.23 17:43:16.443671 [ 21028 ] {} StorageRabbitMQ (test): Library error report: connection lost

Does it reproduce on recent release?
ClickHouse server version 21.2.4 revision 54447

How to reproduce

  • Which ClickHouse server version to use
    ClickHouse server version 21.2.4 revision 54447
  • Queries to run that lead to unexpected result
    `drop table if exists clickhouse.test;
    drop table if exists clickhouse.input;
    drop table if exists clickhouse.clickhouse_mv;
    drop table if exists clickhouse.clickhouse_mv;

CREATE TABLE clickhouse.test (
key UInt64,
value UInt64,
date DateTime
) ENGINE = RabbitMQ SETTINGS rabbitmq_host_port = ' ',
rabbitmq_exchange_name = 'test',
rabbitmq_format = 'JSONEachRow',
rabbitmq_persistent = 1,
rabbitmq_num_consumers = 1;

CREATE TABLE clickhouse.input (
key UInt64,
value UInt64,
date DateTime
) Engine = MergeTree ORDER BY (date);

CREATE MATERIALIZED VIEW clickhouse.clickhouse_mv TO clickhouse.test AS
SELECT * FROM clickhouse.input;

INSERT INTO clickhouse.input (key, value) VALUES(52, 5221);
`

Expected behavior
I think once a message is published it should not be published again.

Error message and/or stacktrace
image

2021.02.23 17:43:16.443563 [ 21028 ] {} StorageRabbitMQ (test): Producer's channel 20 error: connection lost
2021.02.23 17:43:16.443671 [ 21028 ] {} StorageRabbitMQ (test): Library error report: connection lost

Metadata

Metadata

Assignees

Labels

bugConfirmed user-visible misbehaviour in official releasecomp-message-queuesMessage queue integrations (Kafka, RabbitMQ, NATS table engines for stream ingestion/egress).

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions