QueueIterator: speedup via __anext_impl w/wo timeout in __init__#627
QueueIterator: speedup via __anext_impl w/wo timeout in __init__#627gglluukk wants to merge 2 commits intomosquito:masterfrom
Conversation
| self._queue = asyncio.Queue() | ||
| self._consume_kwargs = kwargs | ||
|
|
||
| if kwargs.get("timeout"): |
There was a problem hiding this comment.
Hm... that's should be handled by wait_for implementation: https://github.com/python/cpython/blob/main/Lib/asyncio/tasks.py#L494-L507
There was a problem hiding this comment.
yep... and asyncio.wait_for calls:
https://github.com/python/cpython/blob/main/Lib/asyncio/timeouts.py#L145-L162 and so on which gives ~5% overhead even if we don't use timeout= argument.
my idea is to eliminate this calls' chain if we don't need it
|
How does #615 do in this regard? |
|
@Darsstar my custom task was to speed up the particular code (cancellation/connection check there is done by its own logic). during that i've researched the speed degradation and proposed patch. decision will it be accepted or not is out of my responsibility. |
|
9.5.0 now contains a refactored version of the branch you benchmarked. I have incorperated the same optimizations you did, just differently. And an extra one as well. ( |
4 participants
Hi!
During investigation of optimization RabbitMQ's reader I've faced up performance gap between aio-pika 6.6.1 and 6.7.0 (benchmarks for 10,000 reads/seconds):
More: https://github.com/gglluukk/rmq2psql
The performance gap arose due to the addition of
timeoutsupport inqueue.iterator(), introduced in version 6.7.0:This change caused a decrease in performance compared to version 6.6.1 and subsequent versions inherited this gap.
PR can hopefully resolve this.
Regards!