Skip to content

[perf] httputil: Memory optimization for ResponseChain buffer management #699

@dwisiswant0

Description

@dwisiswant0

Problem

ResponseChain memory usage causes OOM in nuclei when scanning critical severity templates (projectdiscovery/nuclei#6567). Profile analysis shows 6.73GB allocs during 300 reqs scan.

Analysis

3 alloc hotspots identified:

  • fullResponse duplication (1.04GB, 15%): Headers+body duplicated in Fill() and held until Close() even when unused.

  • Pool pollution: Buffers >2MB returned to the pool, causing small requests to reuse 8MB buffers allocated for large responses.

  • String conversions (3.78GB, 55%): (*ResponseChain).Body().String() and (*ResponseChain).FullResponse().String() creates copies that outlive Close().

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions