Reduce runtime memory use by SamCarlberg · Pull Request #699 · WPIRoboticsProjects/GRIP

SamCarlberg · 2016-11-02T19:29:11Z

Reduces memory use by watershed (release temp objects and reuse memory)

I can run in headless mode with -Xmx15m and UI mode with -Xmx150m (was previously seeing up to 890MB heap in UI with watershed)

Reduces memory use by watershed (release temp objects and reuse memory) Schedule GC to run every 15 seconds I can run in headless mode with -Xmx15m and UI mode with -Xmx150m (was previously seeing up to 890MB heap in UI with watershed)

codecov-io · 2016-11-02T20:57:07Z

Current coverage is 53.54% (diff: 20.00%)

Merging #699 into master will decrease coverage by 0.13%

@@             master       #699   diff @@
==========================================
  Files           225        226     +1   
  Lines          7329       7364    +35   
  Methods           0          0          
  Messages          0          0          
  Branches        721        724     +3   
==========================================
+ Hits           3934       3943     +9   
- Misses         3221       3247    +26   
  Partials        174        174

Powered by Codecov. Last update 0af249a...25abdd5

Sets -Xmx200m in native app Also fixes watershed output

SamCarlberg · 2016-11-04T16:24:12Z

So just reducing the max heap size will cause more garbage collections on the young generation, but doesn't seem to run full GC's any more often.

The biggest problem with memory is that if the GC leaves dead native objects (Mats, Rects, etc.) floating around, the native memory won't get freed up and causes the process's memory use to explode -- 290MB of used heap space can correspond to 10GB of memory used by the process (this could go even higher, but my 16GB of RAM was used up and swapping was getting nasty). This was just running the core jar.

Calling System.gc() after every pipeline run hugely alleviates this problem. With the same pipeline and settings as above, I'm seeing heap use between 5MB and 7MB and a total process use of ~200MB. The downside to this is that a full GC runs about 20 times per second.

Calling System.gc periodically (I tested with 15 second intervals) helped, but heap use was peaking around 50MB and the process was using ~1.6GB of memory. Not great.

Calling System.gc() every 10 pipeline runs, however, seems to be a good solution. Heap use is between 6MB and 8MB, process seems to be steady at about 230-240MB, and a full GC about 2-3 times per second.

Heap and CPU profiles

`-Xmx30m`, no GC calls

`-Xmx2G`, no GC calls

`-Xmx2G`, `System.gc()` after every pipeline run

`-Xmx2G`, `System.gc()` every 15 seconds

`-Xmx2G`, `System.gc()` after every 10 pipeline runs

Currently set to run after every 15 pipeline runs

JLLeitschuh · 2016-11-04T17:37:21Z

I'd like to get some other people's thoughts on this before we merge it.

@saudet Is calling release on these objects okay or should we be calling something else?

SamCarlberg · 2016-11-04T17:54:26Z

deallocate() only seems to work if the Deallocator object hasn't been set (which it seems to be) or on garbage collection. But it looks like calling release() on a mat will immediately free up the native memory if there's no longer a reference to it

saudet · 2016-11-05T01:02:46Z

deallocate() should also free memory immediately, it doesn't? To free the Pointer object itself from the Java heap, we need to wait after the GC though...

Conflicts: build.gradle

SamCarlberg · 2016-11-10T20:54:47Z

So it looks like the only way to address this is to periodically call System.gc(), or to make Eden space small to force more GC runs to clean up unused mat objects and deallocate their native memory.

The first one has the advantage of not putting any limits on the heap size, but it will trigger a major stop-the-world GC, which can take a while. It also keeps memory use very low in both UI and headless modes.

The second option will only trigger minor GCs in Eden and S1/S2 that don't take much time, but that will impose limits on other parts of the program. For example, previewing large images in the UI takes a ton of memory on the heap -- a 12MP picture can take upwards of 150MB to preview -- and won't be able to fit in new gen, making the app hang while the JVM keeps trying to unsuccessfully allocate enough memory for it. #711 addresses this problem, but there may be other parts of the program or future features that would be limited by a tiny new gen space.

I'm not happy with either of these options, but I don't see any other alternatives.

saudet · 2016-11-11T02:04:52Z

For cases like these, it's useful to reuse (pre)allocated memory as much as possible, to avoid constantly allocating and deallocating. I am not familiar with the details of this project, but that doesn't look possible?

SamCarlberg · 2016-11-22T20:15:17Z

I don't think it's possible for us to use preallocated memory. We also can't make eden small because that puts limits on the size of the images we can preview in the UI (big images = big buffers that need to be allocated). So the only solution I see is to periodically call System.gc() to clean up the old Mats leftover from runs of the pipeline.

SamCarlberg · 2016-11-29T05:29:59Z

@JLLeitschuh I'd like you to try this out and see if you can find any serious problems

saudet · 2016-11-29T05:41:52Z

@SamCarlberg BTW, recent versions of JavaCPP check before each allocation the amount of physical memory used and call System.gc() only when that amount is crossed, so it's possible to limit these calls to a minimum that way. We can adjust that amount by setting the org.bytedeco.javacpp.maxphysicalbytes system property to the desired value:
http://bytedeco.org/javacpp/apidocs/org/bytedeco/javacpp/Pointer.html#maxPhysicalBytes

SamCarlberg · 2016-11-29T06:01:44Z

That's good to know. I'll try it out, see how it compares to running System.gc() every few passes.

JLLeitschuh · 2016-12-01T03:18:43Z

core/src/main/java/edu/wpi/grip/core/sources/IPCameraFrameGrabber.java

      CvMat mat = cvMat(1, b.length, CV_8UC1, new BytePointer(b));
-      if (decoded != null) {
-        cvReleaseImage(decoded);
-      }


JLLeitschuh · 2016-12-01T03:19:42Z

core/src/main/java/edu/wpi/grip/core/operations/composite/WatershedOperation.java

  private final InputSocket<ContoursReport> contoursSocket;
  private final OutputSocket<ContoursReport> outputSocket;

+  private static final int maxMarkers = 253;


Shouldn't this be MAX_MARKERS?

JLLeitschuh · 2016-12-01T03:19:57Z

core/src/main/java/edu/wpi/grip/core/operations/composite/WatershedOperation.java

  private final OutputSocket<ContoursReport> outputSocket;

+  private static final int maxMarkers = 253;
+  private final List<Mat> markerPool = new ArrayList<>(maxMarkers);


Can this be an immutable list?

JLLeitschuh · 2016-12-01T03:20:35Z

core/src/main/java/edu/wpi/grip/core/Cleaner.java

+      runsSinceLastGc = 0;
+      lastRun = System.nanoTime();
+      System.gc();
+    }


Use StopWatch?

JLLeitschuh · 2016-12-01T03:47:48Z

core/src/main/java/edu/wpi/grip/core/operations/composite/WatershedOperation.java

+    for (int i = 0; i < MAX_MARKERS; i++) {
+      tmpPool.add(new Mat());
+    }
+    markerPool = ImmutableList.copyOf(tmpPool);


Use the ImmutableListBuilder

That's no better than what's currently happening.

If we use Guava 21.0:
markerPool = Stream.generate(Mat::new).limit(MAX_MARKERS).collect(ImmutableList.toImmutableList());

AustinShalit · 2016-12-01T16:31:36Z

core/src/main/java/edu/wpi/grip/core/Cleaner.java

+  /**
+   * The maximum number of runs allowed before calling System.gc().
+   */
+  private static final int MAX_RUNS_BEFORE_GC = 5;


Should this be called minimum instead?

Yeah, I should change that

AustinShalit · 2016-12-01T16:42:39Z

core/src/main/java/edu/wpi/grip/core/operations/composite/WatershedOperation.java

+    for (int i = 0; i < MAX_MARKERS; i++) {
+      tmpPool.add(new Mat());
+    }
+    markerPool = ImmutableList.copyOf(tmpPool);


If we use Guava 21.0:
markerPool = Stream.generate(Mat::new).limit(MAX_MARKERS).collect(ImmutableList.toImmutableList());

Reduce runtime memory use by ~70%

69de1b3

Reduces memory use by watershed (release temp objects and reuse memory) Schedule GC to run every 15 seconds I can run in headless mode with -Xmx15m and UI mode with -Xmx150m (was previously seeing up to 890MB heap in UI with watershed)

Remove Cleaner; allow JVM args to be specified as gradle properties

79d5c86

Sets -Xmx200m in native app Also fixes watershed output

SamCarlberg force-pushed the reduce-memory-footprint branch from dc07b19 to 79d5c86 Compare November 2, 2016 21:00

SamCarlberg added 2 commits November 2, 2016 17:03

Check for external-only contours in watershed

d132017

Release temp Mats and use preallocated objects in WatershedOperation

e476d02

SamCarlberg changed the title ~~Reduce runtime memory use by ~70%~~ Reduce runtime memory use Nov 3, 2016

Add Cleaner class to run GC periodically

daa0dfc

Currently set to run after every 15 pipeline runs

AustinShalit modified the milestone: v1.5.0 Nov 7, 2016

SamCarlberg added 4 commits November 8, 2016 12:37

Limit eden size in deployed and native apps

76c60d8

Remove cleaner, be more aggressive about releasing unneeded mats

4703249

Merge branch 'master'

4c0ec12

Conflicts: build.gradle

Too aggressive

05ef2eb

SamCarlberg added 2 commits November 16, 2016 16:04

Merged branch master into reduce-memory-footprint

7272af0

Re-add cleaner and make it run every 5 pipeline runs. Bump MaxNewSize

efb209d

Wait at least 1 second between gc calls. Improve Cleaner documentation

4c51b51

JLLeitschuh reviewed Dec 1, 2016

View reviewed changes

Cleanup and fixes

71745ef

JLLeitschuh reviewed Dec 1, 2016

View reviewed changes

Move args and jvmArgs to common configuration block

63112dd

AustinShalit requested changes Dec 1, 2016

View reviewed changes

SamCarlberg added 2 commits December 1, 2016 12:14

Improve watershed markerpool assignment

f77955b

Change constant name

25abdd5

AustinShalit approved these changes Dec 1, 2016

View reviewed changes

SamCarlberg merged commit 92c589c into WPIRoboticsProjects:master Dec 2, 2016

Comments

Conversation

SamCarlberg commented Nov 2, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-io commented Nov 2, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Current coverage is 53.54% (diff: 20.00%)

Uh oh!

SamCarlberg commented Nov 4, 2016

Heap and CPU profiles

-Xmx30m, no GC calls

-Xmx2G, no GC calls

-Xmx2G, System.gc() after every pipeline run

-Xmx2G, System.gc() every 15 seconds

-Xmx2G, System.gc() after every 10 pipeline runs

Uh oh!

JLLeitschuh commented Nov 4, 2016

Uh oh!

SamCarlberg commented Nov 4, 2016

Uh oh!

saudet commented Nov 5, 2016 via email

Uh oh!

SamCarlberg commented Nov 10, 2016

Uh oh!

saudet commented Nov 11, 2016

Uh oh!

SamCarlberg commented Nov 22, 2016

Uh oh!

SamCarlberg commented Nov 29, 2016

Uh oh!

saudet commented Nov 29, 2016

Uh oh!

SamCarlberg commented Nov 29, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

SamCarlberg commented Nov 2, 2016 •

edited

Loading

codecov-io commented Nov 2, 2016 •

edited

Loading

`-Xmx30m`, no GC calls

`-Xmx2G`, no GC calls

`-Xmx2G`, `System.gc()` after every pipeline run

`-Xmx2G`, `System.gc()` every 15 seconds

`-Xmx2G`, `System.gc()` after every 10 pipeline runs