[android] Test application for profiling, CMake params for debug symbols #28406

IvanKobzarev · 2019-10-22T04:33:55Z

Reason:
To have one-step build for test android application based on the current code state that is ready for profiling with simpleperf, systrace etc. to profile performance inside the application.

Parameters to control debug symbols stripping

Introducing /CMakeLists parameter ANDROID_DEBUG_SYMBOLS to be able not to strip symbols for pytorch (not add linker flag -s)
which is checked in scripts/build_android.sh

On gradle side stripping happens by default, and to prevent it we have to specify

android {
  packagingOptions {
       doNotStrip "**/*.so"
  }
}

which is now controlled by new gradle property nativeLibsDoNotStrip

Test_App

android/test_app - android app with one MainActivity that does inference in cycle

android/build_test_app.sh - script to build libtorch with debug symbols for specified android abis and adds NDK_DEBUG=1 and -PnativeLibsDoNotStrip=true to keep all debug symbols for profiling.
Script assembles all debug flavors:

└─ $ find . -type f -name *apk
./test_app/app/build/outputs/apk/mobilenetQuant/debug/test_app-mobilenetQuant-debug.apk
./test_app/app/build/outputs/apk/resnet/debug/test_app-resnet-debug.apk

Different build configurations

Module for inference can be set in android/test_app/app/build.gradle as a BuildConfig parameters:

    productFlavors {
        mobilenetQuant {
            dimension "model"
            applicationIdSuffix ".mobilenetQuant"
            buildConfigField ("String", "MODULE_ASSET_NAME", buildConfigProps('MODULE_ASSET_NAME_MOBILENET_QUANT'))
            addManifestPlaceholders([APP_NAME: "PyMobileNetQuant"])
            buildConfigField ("String", "LOGCAT_TAG", "\"pytorch-mobilenet\"")
        }
        resnet {
            dimension "model"
            applicationIdSuffix ".resnet"
            buildConfigField ("String", "MODULE_ASSET_NAME", buildConfigProps('MODULE_ASSET_NAME_RESNET18'))
            addManifestPlaceholders([APP_NAME: "PyResnet"])
            buildConfigField ("String", "LOGCAT_TAG", "\"pytorch-resnet\"")
        }

In that case we can setup several apps on the same device for comparison, to separate packages applicationIdSuffix: 'org.pytorch.testapp.mobilenetQuant' and different application names and logcat tags as manifestPlaceholder and another BuildConfig parameter:

─ $ adb shell pm list packages | grep pytorch
package:org.pytorch.testapp.mobilenetQuant
package:org.pytorch.testapp.resnet

In future we can add another BuildConfig params e.g. single/multi threads and other configuration for profiling.

At the moment 2 flavors - for resnet18 and for mobilenetQuantized
which can be installed on connected device:

cd android

gradle test_app:installMobilenetQuantDebug

gradle test_app:installResnetDebug

Testing:

cd android
sh build_test_app.sh
adb install -r test_app/app/build/outputs/apk/mobilenetQuant/debug/test_app-mobilenetQuant-debug.apk

cd $ANDROID_NDK
python simpleperf/run_simpleperf_on_device.py record --app org.pytorch.testapp.mobilenetQuant -g --duration 10 -o /data/local/tmp/perf.data
adb pull /data/local/tmp/perf.data
python simpleperf/report_html.py

Simpleperf report has all symbols:

AshkanAliabadi

Awesome, thank you so much for doing this. You and Tao are life savers.

AshkanAliabadi · 2019-10-23T17:50:24Z

android/test_app/app/src/main/java/org/pytorch/testapp/MainActivity.java

Am I correct to assume that since I don't spot a loop here, we are calling PyTorch forward only one time? Or is this function called from a loop somewhere?

I am mainly asking this question to understand whether we are profiling one or multiple runs of a network here. I think it is worthwhile to be able to have the ability to run a network multiple times through a configurable parameter to be able to study thermal behavior.

P.S. It can be the same image sent to the network over and over again to avoid image acquisition and decoding overhead.

private final Runnable mModuleForwardRunnable = new Runnable() { @Override public void run() { runOnUiThread(new Runnable() { @Override public void run() { handleResult(doModuleForward()); if (mBackgroundHandler != null) { mBackgroundHandler.post(mModuleForwardRunnable); } } }); } };

We call doModuleForward() in a loop, but its organized with scheduling the same implementation of Runnable interface mModuleForwardRunnable on mBackgroundHandler which does queue
of Runnableprocessing.
So it will put it on queue of the thread mBackgroundThread = new HandlerThread(TAG + "_bg"); the name depends what we put in build.gradle as LOG_TAG

P.S. It can be the same image sent to the network over and over again to avoid image acquisition and decoding overhead.

At the moment here we even do not have image - just allocating buffer of proper size and creating tensor from it - so we should not see any overhead from camera, image decoding etc.

supriyar · 2019-10-23T22:31:54Z

android/test_app/app/build.gradle

nit: can you change the model names to mobilenet_v2 and resnet18 where it is used?

dreiss · 2019-11-05T23:14:23Z

I don't think we should check in the models, since they are quite big. That puts a tax on everyone using this repo. Maybe have the build script download them from somewhere? Also, we don't need the hdpi and x*hdpi icons. Scaling up mdpi is fine. Other than that, I think this is fine as long as it works for you and Ashkan.

dreiss · 2019-11-07T23:49:54Z

android/settings.gradle

Why isn't this ':test_app'?

dreiss · 2019-11-07T23:54:07Z

android/test_app/build.gradle

Do we need this, since the top-level build.gradle already does it?

facebook-github-bot

@IvanKobzarev has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

… symbols

facebook-github-bot

@IvanKobzarev has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot

@IvanKobzarev has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2019-11-08T23:13:48Z

@IvanKobzarev merged this pull request in 92b9de1.

IvanKobzarev changed the title ~~[WI{][android] Test application for profiling + CMake params for debug symbols~~ [WIP][android] Test application for profiling + CMake params for debug symbols Oct 22, 2019

IvanKobzarev force-pushed the android_test_app_for_profiling branch 5 times, most recently from 73f2038 to da13dbf Compare October 22, 2019 17:51

IvanKobzarev changed the title ~~[WIP][android] Test application for profiling + CMake params for debug symbols~~ [android] Test application for profiling, CMake params for debug symbols Oct 22, 2019

IvanKobzarev requested review from AshkanAliabadi, dreiss, ljk53 and supriyar October 22, 2019 18:21

AshkanAliabadi reviewed Oct 23, 2019

View reviewed changes

IvanKobzarev force-pushed the android_test_app_for_profiling branch from da13dbf to 33af80c Compare October 23, 2019 18:48

supriyar reviewed Oct 23, 2019

View reviewed changes

IvanKobzarev force-pushed the android_test_app_for_profiling branch from 33af80c to 0b010d1 Compare October 24, 2019 16:40

dreiss removed their request for review November 5, 2019 23:14

IvanKobzarev force-pushed the android_test_app_for_profiling branch 2 times, most recently from d449f0f to 1a86fbf Compare November 7, 2019 21:47

IvanKobzarev requested a review from dreiss November 7, 2019 21:58

dreiss approved these changes Nov 7, 2019

View reviewed changes

IvanKobzarev force-pushed the android_test_app_for_profiling branch from 1a86fbf to c9da33d Compare November 8, 2019 00:59

facebook-github-bot reviewed Nov 8, 2019

View reviewed changes

IvanKobzarev force-pushed the android_test_app_for_profiling branch from c9da33d to adee5e7 Compare November 8, 2019 01:40

[android] Test application for profiling + CMake params to keep debug…

c0f5360

… symbols

IvanKobzarev force-pushed the android_test_app_for_profiling branch from adee5e7 to c0f5360 Compare November 8, 2019 20:02

facebook-github-bot reviewed Nov 8, 2019

View reviewed changes

facebook-github-bot closed this in 92b9de1 Nov 8, 2019

facebook-github-bot added the merged label Nov 8, 2019

mruberry added the Merged label Oct 28, 2020

[android] Test application for profiling, CMake params for debug symbols #28406

[android] Test application for profiling, CMake params for debug symbols #28406

Uh oh!

Conversation

IvanKobzarev commented Oct 22, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Parameters to control debug symbols stripping

Test_App

Different build configurations

Testing:

Uh oh!

AshkanAliabadi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dreiss commented Nov 5, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Nov 8, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

IvanKobzarev commented Oct 22, 2019 •

edited

Loading