Checklist

• I have verified that the issue exists against the master branch of Celery.
• This has already been asked to the discussion group first.
• I have read the relevant section in the
  contribution guide
  on reporting bugs.
• I have checked the issues list
  for similar or identical bug reports.
• I have checked the pull requests list
  for existing proposed fixes.
• I have checked the commit log
  to find out if the bug was already fixed in the master branch.
• I have included all related issues and possible duplicate issues
  in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

• I have included the output of celery -A proj report in the issue.
  (if you are not able to do this, then at least specify the Celery
  version affected).
• I have verified that the issue exists against the master branch of Celery.
• I have included the contents of pip freeze in the issue.
• I have included all the versions of all the external dependencies required
  to reproduce this bug.

Optional Debugging Information

• I have tried reproducing the issue on more than one Python version
  and/or implementation.
• I have tried reproducing the issue on more than one message broker and/or
  result backend.
• I have tried reproducing the issue on more than one version of the message
  broker and/or result backend.
• I have tried reproducing the issue on more than one operating system.
• I have tried reproducing the issue on more than one workers pool.
• I have tried reproducing the issue with autoscaling, retries,
  ETA/Countdown & rate limits disabled.
• I have tried reproducing the issue after downgrading
  and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

• None

Possible Duplicates

• None

Environment & Settings

Celery version:

software -> celery:4.4.3 (cliffs) kombu:4.6.9 py:3.7.6
            billiard:3.6.3.0 py-amqp:2.6.0
platform -> system:Linux arch:64bit, ELF
            kernel version:4.15.0-101-generic imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:amqp results:cassandra

Queue: <class 'kombu.entity.Queue'>
RoundRobinPolicy: <class 'cassandra.policies.RoundRobinPolicy'>
broker_url: 'amqp://guest:********@localhost:5672//'
cassandra_auth_kwargs: {
 'password': '********', 'username': 'cassandra'}
cassandra_auth_provider: 'PlainTextAuthProvider'
cassandra_entry_ttl: 86400
cassandra_keyspace: '********'
cassandra_options: {
    'load_balancing_policy': <cassandra.policies.RoundRobinPolicy object at 0x7f09de9a34d0>}
cassandra_servers: ['127.0.0.1']
cassandra_table: 'celery_jobs_result'
ignore_result: False
result_backend: 'cassandra'
worker_concurrency: 4

Steps to Reproduce

Required Dependencies

• Minimal Python Version: N/A or Unknown
• Minimal Celery Version: N/A or Unknown
• Minimal Kombu Version: N/A or Unknown
• Minimal Broker Version: N/A or Unknown
• Minimal Result Backend Version: N/A or Unknown
• Minimal OS and/or Kernel Version: N/A or Unknown
• Minimal Broker Client Version: N/A or Unknown
• Minimal Result Backend Client Version: N/A or Unknown

Python Packages

amqp==2.6.0
attrs==19.3.0
billiard==3.6.3.0
case==1.5.3
cassandra-driver==3.20.2
future==0.18.2
gevent==20.5.2
greenlet==0.4.15
importlib-metadata==1.6.0
kombu==4.6.9
linecache2==1.0.0
mock==4.0.2
more-itertools==8.3.0
nose==1.3.7
packaging==20.4
pluggy==0.13.1
py==1.8.1
pyparsing==2.4.7
pytest==5.4.2
pytz==2020.1
redis==3.5.2
six==1.15.0
traceback2==1.4.0
unittest2==1.1.0
vine==1.3.0
wcwidth==0.1.9
zipp==3.1.0
zope.event==4.4
zope.interface==5.1.0

Other Dependencies

Minimally Reproducible Test Case

Expected Behavior

CassandraBackend work fine

Actual Behavior

in prefork pool or solo:

it can work, just not good, i think

in threads or gevent pool:

get these error frequently

[2020-06-02 21:52:19,592: ERROR/MainProcess] Task add[4bd528a0-c1ba-456c-b74e-7dee054616bb] raised unexpected: AttributeError("'NoneType' object has no attribute 'execute'")
Traceback (most recent call last):
  File "/home/bx/project/celery/celery/app/trace.py", line 480, in trace_task
    uuid, retval, task_request, publish_result,
  File "/home/bx/project/celery/celery/backends/base.py", line 158, in mark_as_done
    self.store_result(task_id, result, state, request=request)
  File "/home/bx/project/celery/celery/backends/base.py", line 443, in store_result
    request=request, **kwargs)
  File "/home/bx/project/celery/celery/backends/cassandra.py", line 201, in _store_result
    self._session.execute(self._write_stmt, (
AttributeError: 'NoneType' object has no attribute 'execute'

and

[2020-06-02 21:52:29,773: ERROR/MainProcess] Task add[27c980d3-126a-430d-a8e0-92d749ceb735] raised unexpected: OperationTimedOut("errors=errors={'Connection defunct by heartbeat': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.1:9042, last_host=None")
Traceback (most recent call last):
  File "/home/bx/project/celery/celery/app/trace.py", line 480, in trace_task
    uuid, retval, task_request, publish_result,
  File "/home/bx/project/celery/celery/backends/base.py", line 158, in mark_as_done
    self.store_result(task_id, result, state, request=request)
  File "/home/bx/project/celery/celery/backends/base.py", line 443, in store_result
    request=request, **kwargs)
  File "/home/bx/project/celery/celery/backends/cassandra.py", line 207, in _store_result
    buf_t(self.encode(self.current_task_children(request)))
  File "cassandra/cluster.py", line 2345, in cassandra.cluster.Session.execute
  File "cassandra/cluster.py", line 4304, in cassandra.cluster.ResponseFuture.result
cassandra.OperationTimedOut: errors=errors={'Connection defunct by heartbeat': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.1:9042, last_host=None

Fix it

remove method(CassandraBackend.process_cleanup), it works fine.

because.

all tasks should share one instance of cassandra.cluster.Cluster and session, not create one ervery time. that is the correct way to use cassandra-driver