Repository navigation
Description
https.request leaks in this example:
'use strict';
var https = require('https');
var options = {
hostname: 'google.com'
};
function next() {
setTimeout(iterate, 10);
}
function onError(error) {
console.log(error);
}
function noop() {}
function onResponse(res) {
console.log('statusCode: ', res.statusCode);
res.on('data', noop);
res.on('end', next);
}
function iterate() {
console.log(JSON.stringify(process.memoryUsage()));
console.log('iterating...');
https.request(options, onResponse).
on('error', onError).
end();
};
iterate();rss grows over time, heapUsed remains about constant (~30m).
This could be reproduced with all released iojs versions that I tried v1.0.1 and v1.8.1.
Node v0.12.2 behaves better in this testcase: rss grows to about 100m, but goes down periodically (I wouldn't call that a perfect behaviour, let's call that «fine»).
53ba494, dad73f6 and 8a83eba are three sequental commits.
53ba494 is fine (as 0.12), dad73f6 fails to build, 8a83eba leaks.
dad73f6 is «upgrade v8 to 3.31.74.1» with 831 files changed, 104394 insertions(+), 21427 deletions(-).
Node v0.12.2 could be fine because it still uses v8 3.28.73.
io.js 1.0.0/1.1.0 + shared v8 3.30.33.16 behaves fine (the same as 0.12).
All the testcases and massif data are at http://oserv.org/bugs/iojs/memory0/
Activity
ChALkeR commentedon Apr 25, 2015
Could be reproduced with
wreck:ChALkeR commentedon Apr 25, 2015
I found a work-around for that particular testcase.
It's not applicable in real world, i think, but it could give some ideas on where the bug is.
In this example, if
agent.maxSocketsis less than the number ofhttps.requests being executed at the same time, then RSS stops growing.If only one
https.requestis being executed at every moment, then it can't help.agent.maxSockets = 10 and 20 calls to https.request is ok
agent.maxSockets = 10 and 10 calls to https.request is not ok
http://oserv.org/bugs/iojs/memory0/test-agent.js — testcase.
http://oserv.org/bugs/iojs/memory0/test-agent.txt — results.
http://oserv.org/bugs/iojs/memory0/test-fast.js could be used to reproduce this a bit faster (20 requests at the same time).
ChALkeR commentedon Apr 25, 2015
cc @indutny
ChALkeR commentedon Apr 25, 2015
http://oserv.org/bugs/iojs/memory0/test-fast.js could be used to reproduce this a bit faster (20 requests at the same time).
ChALkeR commentedon Apr 25, 2015
io.js 1.8.1:

node 0.12.2:

The time that those two were running for could be different, but you could clearly see the difference still.
ChALkeR commentedon Apr 25, 2015
Actually, it goes down after manually calling
gc().If I call
gc()manually every 100 requests, RSS does not go above 70 MiB.Could it be that
gc()is never called?ChALkeR commentedon Apr 25, 2015
Until a full gc run (calling
gc()),TLSWrapnative objects get never deleted.Full gc is not called automatically, probably because v8 thinks that incremental gc works fine (well,
heapUsedis ~30 MiB most of the time).Full gc is not called automatically until ~500MiB rss (without valgrind/massif) is reached.
ChALkeR commentedon Apr 25, 2015
But even when rss reaches ~500MiB and full gc is called (deleting
TLSWrapobjects), rss memory usage does not go down.indutny commentedon Apr 25, 2015
@ChALkeR I have modified the test case to work with localhost server (partly because I'm on 3G internet, partly because I'm not so patient). And it appears to me that it grows up to 1.1gb very fast, then drops to 800mb and then stays around 600mb for 50 parallel connections.
The breaking point (1.1gb) happens at 21373 iteration for me. Have you tried waiting that long?
indutny commentedon Apr 25, 2015
I have a fix for lowering 1.1gb, btw. Will push a PR tomorrow.
ChALkeR commentedon Apr 26, 2015
Lowering to which point? If
agent.maxSocketsis lower than the number of parallel connections, rss somehow stabilizes at ~40 MiB. What's the difference in this case?Just checked — in this case
TLSWrapobjects are reused.For 20 parallel connections and 1000 total requests:
agent.maxSockets= 10 — only 10TLSWrapobjects are created,agent.maxSockets= 17 — only 17TLSWrapobjects are created,agent.maxSockets= 18 — 21TLSWrapobjects are created,agent.maxSockets= 19 — 83TLSWrapobjects are created,agent.maxSockets= 20 — 1000TLSWrapobjects are created.indutny commentedon Apr 26, 2015
@ChALkeR to 400mb as far as I remember. Anyway, do you see that everything is getting collected after growing to 1gb?
ChALkeR commentedon Apr 26, 2015
No.
For me, full gc kicks in at 500 MiB. But it does not lower rss memory usage, it just (almost) stops growing.
indutny commentedon Apr 26, 2015
@ChALkeR I wasn't talking about GC in general. I was talking about the point where all accumulated TLSWraps are collected and RSS lowers significantly.
53 remaining items
tls: destroy SSL once it is out of use
crypto: track external memory for SSL structures
tls: zero SSL_CTX freelist for a singleUse socket
tls: destroy singleUse context immediately
tls: destroy SSL once it is out of use
crypto: track external memory for SSL structures
tls: zero SSL_CTX freelist for a singleUse socket
tls: destroy singleUse context immediately
tls: destroy SSL once it is out of use
crypto: track external memory for SSL structures
tls: zero SSL_CTX freelist for a singleUse socket
tls: destroy singleUse context immediately
silverwind commentedon Apr 30, 2015
Fixed by 2d241b3 ... 1787416