-
Notifications
You must be signed in to change notification settings - Fork 151
Memory leak in plugin creation/destruction cycle - extism_plugin_free does not fully release memory #890
Description
Description
Creating and destroying extism.Plugin instances in a loop causes unbounded memory growth. Memory is not reclaimed even after explicitly calling del() (which invokes extism_plugin_free) and
running Python garbage collection.
This prevents using extism in long-running services that need to create fresh plugin instances per request.
Reproduction
"""
Minimal reproduction of memory leak in extism-py plugin creation/destruction.
"""
import gc
import resource
import platform
import extism
def get_memory_mb():
"""Get current RSS memory in MB."""
rss = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
if platform.system() == "Darwin":
return rss / 1024 / 1024 # bytes to MB
else:
return rss / 1024 # KB to MB
# Minimal WASM module - exports a single empty function
MINIMAL_WASM = bytes([
0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00,
0x01, 0x04, 0x01, 0x60, 0x00, 0x00,
0x03, 0x02, 0x01, 0x00,
0x07, 0x08, 0x01, 0x04, 0x74, 0x65, 0x73, 0x74, 0x00, 0x00,
0x0a, 0x04, 0x01, 0x02, 0x00, 0x0b,
])
def test_memory_leak():
gc.collect()
mem_start = get_memory_mb()
print(f"extism runtime version: {extism.extism_version()}")
print(f"Memory at start: {mem_start:.2f} MB\n")
for batch in range(5):
for _ in range(100):
plugin = extism.Plugin(MINIMAL_WASM, wasi=False)
plugin.__del__() # Explicitly free
del plugin
gc.collect()
mem_now = get_memory_mb()
iterations = (batch + 1) * 100
delta = mem_now - mem_start
print(f"After {iterations} iterations: {mem_now:.2f} MB (delta: {delta:.2f} MB)")
print("\nExpected: Memory stable after gc")
print("Actual: Memory grows ~0.2-0.5 MB per iteration")
if __name__ == "__main__":
test_memory_leak()
Output
extism runtime version: 1.9.1
Memory at start: 14.20 MB
After 100 iterations: 41.62 MB (delta: 27.42 MB)
After 200 iterations: 61.81 MB (delta: 47.61 MB)
After 300 iterations: 82.98 MB (delta: 68.78 MB)
After 400 iterations: 103.45 MB (delta: 89.25 MB)
After 500 iterations: 123.73 MB (delta: 109.53 MB)
Expected: Memory stable after gc
Actual: Memory grows ~0.2-0.5 MB per iteration
Environment
• extism (Python SDK): 1.0.4
• extism-sys: 1.9.1 (also reproduced with 1.13.0)
• Python: 3.11
• OS: macOS 14.x (also reproduced on Linux)
Workaround
Reusing a single plugin instance avoids the leak. However, this workaround is not viable for use cases requiring isolation between invocations, as interpreter state (e.g., modified globals, patched modules) persists across calls even when using extism_plugin_reset.