memory leak when doing requests against AWS #3010
Description
Long story short
while investigating a leak in botocore (boto/botocore#1464), I decided to check to see if this also affected aiobotocore. I found that even the get_object case was leaking in aiobotocore. So I tried doing a the underlying GET aiohttp request and found it triggers a leak in aiohttp.
Expected behaviour
no leak
Actual behaviour
leaking ~800kB over 300 requests
Steps to reproduce
update the below script with your credentials and run like so: PYTHONMALLOC=malloc
python3 `which mprof` run --interval=1 /tmp/test_leak.py -test aiohttp_test
you should get a memory usage graph like so:
Your environment
bash-4.2# /valgrind/bin/python3 -m pip freeze
aiobotocore==0.8.0
aiohttp==3.1.3
APScheduler==3.3.1
async-timeout==2.0.1
asynctest==0.12.0
attrs==18.1.0
aws-xray-sdk==1.0
botocore==1.10.12
cachetools==2.1.0
certifi==2018.4.16
chardet==3.0.4
cycler==0.10.0
datadog==0.20.0
decorator==4.3.0
dill==0.2.7.1
docutils==0.14
editdistance==0.3.1
future==0.16.0
gapic-google-cloud-pubsub-v1==0.15.4
google-api-python-client==1.6.7
google-auth==1.4.1
google-auth-httplib2==0.0.3
google-cloud-core==0.25.0
google-cloud-pubsub==0.26.0
google-gax==0.15.16
googleapis-common-protos==1.5.3
graphviz==0.8.3
grpc-google-iam-v1==0.11.4
grpcio==1.12.0
httplib2==0.11.3
idna==2.6
idna-ssl==1.0.1
iso8601==0.1.12
jmespath==0.9.3
jsonpickle==0.9.6
kiwisolver==1.0.1
matplotlib==2.2.2
memory-profiler==0.52.0
multidict==4.3.1
nameparser==0.5.6
numpy==1.14.3
oauth2client==4.1.2
objgraph==3.4.0
packaging==17.1
ply==3.8
proto-google-cloud-pubsub-v1==0.15.4
protobuf==3.5.2.post1
psutil==5.4.5
pyasn1==0.4.2
pyasn1-modules==0.2.1
PyGithub==1.39
PyJWT==1.6.1
pyparsing==2.2.0
python-dateutil==2.7.3
pytz==2018.4
requests==2.18.4
rsa==3.4.2
setproctitle==1.1.10
simplejson==3.15.0
six==1.11.0
tzlocal==1.5.1
uritemplate==3.0.0
urllib3==1.22
wrapt==1.10.11
yarl==1.2.4