Skip to content

tests/integration: suppress exceptions during logging (due to pytest)#44618

Merged
alexey-milovidov merged 1 commit intoClickHouse:masterfrom
azat:tests/integration/pytest-logging-fix
Dec 27, 2022
Merged

tests/integration: suppress exceptions during logging (due to pytest)#44618
alexey-milovidov merged 1 commit intoClickHouse:masterfrom
azat:tests/integration/pytest-logging-fix

Conversation

@azat
Copy link
Copy Markdown
Member

@azat azat commented Dec 26, 2022

Due to pytest play games with logging 1 it is better to ignore it.

This is due to incorrect kazoo client usage (not all clients calls stop(), this is in progress)

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)

Cc: @alexey-milovidov
Cc: @quickhouse (#41344 (comment))

Due to pytest play games with logging [1] it is better to ignore it.

  [1]: pytest-dev/pytest#5502

This is due to incorrect kazoo client usage (not all clients calls
stop())

Signed-off-by: Azat Khuzhin <[email protected]>
@robot-ch-test-poll2 robot-ch-test-poll2 added the pr-not-for-changelog This PR should not be mentioned in the changelog label Dec 26, 2022
@azat azat changed the title tests/integration: disable logging.raiseExceptions tests/integration: fix internal "Logging error" due to kazoo client Dec 26, 2022
@azat azat changed the title tests/integration: fix internal "Logging error" due to kazoo client tests/integration: suppress exceptions during logging (due to pytest) Dec 27, 2022
@azat azat marked this pull request as ready for review December 27, 2022 09:40
@alexey-milovidov
Copy link
Copy Markdown
Member

#44569

@alexey-milovidov
Copy link
Copy Markdown
Member

#44573

@alexey-milovidov alexey-milovidov merged commit 7e44759 into ClickHouse:master Dec 27, 2022
azat added a commit to azat/ClickHouse that referenced this pull request Dec 28, 2022
pytest play games with logging output [1].

  [1]: pytest-dev/pytest#5502

But this does not actually affect ClickHouse tests, the only reason is
that the kazoo client is not stopped correctly without calling
kazoo.client.KazooClient.stop(), and that's why you can see the
following messages:

<details>

    test_storage_rabbitmq/test.py::test_rabbitmq_big_message --- Logging error ---
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 179, in _socket_error_handling
        yield
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 659, in _connect
        self._socket = self.handler.create_connection(
      File "/usr/local/lib/python3.8/dist-packages/kazoo/handlers/threading.py", line 178, in create_connection
        return utils.create_tcp_connection(socket, *args, **kwargs)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/handlers/utils.py", line 265, in create_tcp_connection
        sock = module.create_connection(address, timeout_at)
      File "/usr/lib/python3.8/socket.py", line 808, in create_connection
        raise err
      File "/usr/lib/python3.8/socket.py", line 796, in create_connection
        sock.connect(sa)
    socket.timeout: timed out

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 579, in _connect_attempt
        read_timeout, connect_timeout = self._connect(host, hostip, port)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 659, in _connect
        self._socket = self.handler.create_connection(
      File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
        self.gen.throw(type, value, traceback)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 182, in _socket_error_handling
        raise ConnectionDropped("socket connection error: %s" % (err,))
    kazoo.exceptions.ConnectionDropped: socket connection error: None

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/lib/python3.8/logging/__init__.py", line 1088, in emit
        stream.write(msg + self.terminator)
    ValueError: I/O operation on closed file.
    Call stack:
      File "/usr/lib/python3.8/threading.py", line 890, in _bootstrap
        self._bootstrap_inner()
      File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
        self.run()
      File "/usr/lib/python3.8/threading.py", line 870, in run
        self._target(*self._args, **self._kwargs)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 512, in zk_loop
        if retry(self._connect_loop, retry) is STOP_CONNECTING:
      File "/usr/local/lib/python3.8/dist-packages/kazoo/retry.py", line 126, in __call__
        return func(*args, **kwargs)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 552, in _connect_loop
        status = self._connect_attempt(host, hostip, port, retry)
      File "/usr/local/lib/python3.8/dist-packages/kazoo/protocol/connection.py", line 622, in _connect_attempt
        self.logger.warning('Connection dropped: %s', e)
    Message: 'Connection dropped: %s'
    Arguments: (ConnectionDropped('socket connection error: None'),)

</details>

Which eventually leads to incorrectly parsed report.

This patch is an addition to the `logging.raiseExceptions=False` - ClickHouse#44618

Signed-off-by: Azat Khuzhin <[email protected]>
@azat azat deleted the tests/integration/pytest-logging-fix branch December 28, 2022 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-not-for-changelog This PR should not be mentioned in the changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants