[tflite] export SetNumThreads to TFLite Python API by freedomtan · Pull Request #25748 · tensorflow/tensorflow

freedomtan · 2019-02-14T07:59:01Z

export SetNumThreads() and add an option to set the number of threads in the label_image.py for TFLite. On PYNQ-Z1, a board with 2xCA9,

$ python tensorflow/lite/examples/python/label_image.py --num_threads 1
0.415686: 653:military uniform
0.352941: 907:Windsor tie
0.058824: 668:mortarboard
0.035294: 458:bow tie, bow-tie, bowtie
0.035294: 835:suit, suit of clothes
time:  0.691580057144

$ python tensorflow/lite/examples/python/label_image.py --num_threads 2
0.415686: 653:military uniform
0.352941: 907:Windsor tie
0.058824: 668:mortarboard
0.035294: 458:bow tie, bow-tie, bowtie
0.035294: 835:suit, suit of clothes
time:  0.412344932556

jianlijianli · 2019-02-15T21:49:36Z

added @miaout17 to the reviewer list since it involves interpreter change.

jdinkel88 · 2019-02-18T02:57:46Z

Mine built well, but I get an object has no attribute
Traceback (most recent call last):
File "./label_image.py", line 74, in
interpreter.set_num_threads(int(args.num_threads))
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/lite/python/interpreter.py", line 260, in set_num_threads
return self._interpreter.SetNumThreads(i)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/lite/python/interpreter_wrapper/tensorflow_wrap_interpreter_wrapper.py", line 97, in
getattr = lambda self, name: _swig_getattr(self, InterpreterWrapper, name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/lite/python/interpreter_wrapper/tensorflow_wrap_interpreter_wrapper.py", line 74, in _swig_getattr
return _swig_getattr_nondynamic(self, class_type, name, 0)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/lite/python/interpreter_wrapper/tensorflow_wrap_interpreter_wrapper.py", line 69, in _swig_getattr_nondynamic
return object.getattr(self, name)
AttributeError: type object 'object' has no attribute 'getattr'

freedomtan · 2019-02-18T03:07:01Z

@jdinkel88 maybe you forgot to install the newly built pip wheel (remember to pip3 install -U YOUR_NEWLY_BUILT_PIP_WHEEL) or ran a wrong python interpreter (make sure that the python interpreter uses the newly installed one)?

dpsm · 2019-03-08T16:18:43Z

@petewarden can you review this? RPI right now runs the apps using the python binding on single core :(

dpsm · 2019-03-08T17:57:03Z

👍

freedomtan · 2019-03-09T01:26:35Z

@miaout17 can you help review this? it seems some guys using RPI really want this.

dpsm · 2019-03-12T03:50:19Z

@miaout17 is there anyone else you can nominate as reviewer in case you don't have cycles?

dpsm

Verified on rpi that it indeed uses more than a single core

tensorflow/lite/python/interpreter.py

export SetNumThreadS() and add an option to set the number of threads in the `label_image.py` for TFLite.

tensorflow/lite/examples/python/label_image.py

teijeong · 2020-05-20T05:07:09Z

tensorflow/lite/examples/python/label_image.py

      help='input standard deviation')
+  parser.add_argument(
+      '--num_threads',
+      default=1,


How about adding type=int here, so that line 66 can be just num_threads=args.num_threads?

teijeong · 2020-05-20T05:11:17Z

tensorflow/lite/examples/python/label_image.py

    else:
      print('{:08.6f}: {}'.format(float(results[i] / 255.0), labels[i]))
+
+  print("time: ", stop_time - start_time)


nit: wrap string with single quote
remove extra space after ":", the print statement adds space between arguments.

Preferably would be nicer to format this as something like 'time: {:.3f}ms'.format((stop_time - start_time) * 1000)

teijeong · 2020-05-20T05:15:22Z

tensorflow/lite/python/interpreter.py

  def reset_all_variables(self):
    return self._interpreter.ResetVariableTensors()

-


re-add newline

teijeong · 2020-05-20T05:16:49Z

tensorflow/lite/python/interpreter_wrapper/interpreter_wrapper.cc

  Py_RETURN_NONE;
 }

+PyObject* InterpreterWrapper::SetNumThreads(int i) {


give this more descriptive name :)

teijeong · 2020-05-20T05:16:59Z

tensorflow/lite/python/interpreter_wrapper/interpreter_wrapper.h

  // should be the interpreter object providing the memory.
  PyObject* tensor(PyObject* base_object, int i);

+  PyObject* SetNumThreads(int i);


teijeong · 2020-05-20T05:17:07Z

tensorflow/lite/python/interpreter_wrapper/interpreter_wrapper_pybind11.cc

+          )pbdoc")
+      .def(
+          "SetNumThreads",
+          [](InterpreterWrapper& self, int i) {


tensorflow/lite/python/interpreter.py

teijeong

Looks good, just one nit ;) Thanks!

teijeong · 2020-05-21T01:43:01Z

tensorflow/lite/examples/python/label_image.py

      print('{:08.6f}: {}'.format(float(results[i] / 255.0), labels[i]))

-  print("time: ", stop_time - start_time)
+  #print("time: ", stop_time - start_time)


please remove :)

oops, that's embarrassing. Thanks.

No worries, it happens all the time for me too ;)

jdduke · 2020-05-26T17:10:31Z

tensorflow/lite/python/interpreter.py

        [TfLiteDelegate](https://www.tensorflow.org/lite/performance/delegates)
          objects returned by lite.load_delegate().
+      num_threads: Set the number of threads used by TFLite kernels.
+        If not set, kernels are running single-threaded. Note that currently,


This should really be:

"Sets the number of threads used by the interpreter and available to CPU kernels. If not set, the interpreter will use an implementation-dependent default number of threads. Currently, only a subset of kernels, such as conv, support multi-threading."

jdduke · 2020-05-26T17:11:16Z

tensorflow/lite/examples/python/label_image.py

      help='input standard deviation')
+  parser.add_argument(
+      '--num_threads',
+      default=1, type=int,


Would prefer that the default be None, if possible?

rthadur · 2020-06-05T17:21:54Z

@jdduke gentle ping if we need more changes here.

jdduke

Can you add a test or two in interpreter_test.py? Validate that it throws an error for invalid input, and that 2 threads works as expected?

srjoglekar246 · 2020-06-11T21:41:13Z

I can add the tests internally, so that we don't have to reimport the changes. Will do it by tomorrow and get this in.

metanav · 2020-06-13T01:56:22Z

How can I make a pip package to test it on RPI 4?

teijeong · 2020-06-15T13:27:02Z

@metanav Would you try this? https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/pip_package

googlebot added the cla: yes label Feb 14, 2019

pragyaak self-assigned this Feb 14, 2019

pragyaak added the size:S CL Change Size: Small label Feb 14, 2019

pragyaak requested a review from jianlijianli February 14, 2019 18:19

pragyaak added the awaiting review Pull request awaiting review label Feb 14, 2019

jianlijianli requested a review from miaout17 February 15, 2019 21:47

tensorflowbutler removed the awaiting review Pull request awaiting review label Feb 16, 2019

freedomtan mentioned this pull request Feb 16, 2019

Tensorflow Lite, python API does not work #21574

Closed

This was referenced Feb 16, 2019

it hangs after a while (RPI3B) PINTO0309/Tensorflow-bin#2

Closed

TF lite has link error when I try to build with build_rpi_lib.sh #25120

Closed

pragyaak added the awaiting review Pull request awaiting review label Feb 19, 2019

pragyaak assigned rthadur and unassigned pragyaak Feb 23, 2019

dpsm previously approved these changes Mar 18, 2019

View reviewed changes

rthadur requested review from jdduke and removed request for jianlijianli and miaout17 March 18, 2019 20:31

tensorflowbutler removed the awaiting review Pull request awaiting review label Mar 19, 2019

jdduke requested changes Mar 19, 2019

View reviewed changes

tensorflow/lite/python/interpreter.py Show resolved Hide resolved

jdduke requested a review from miaout17 March 19, 2019 18:32

freedomtan and others added 2 commits March 21, 2019 14:47

[tflite] export SetNumThreads to TFLite Python API

61d8cb0

export SetNumThreadS() and add an option to set the number of threads in the `label_image.py` for TFLite.

add simple description of set_num_threads()

7dba4b1

freedomtan dismissed dpsm’s stale review via 7dba4b1 March 21, 2019 06:49

googlebot added the cla: yes label Apr 21, 2020

gbaned requested a review from teijeong May 19, 2020 14:54

teijeong reviewed May 20, 2020

View reviewed changes

gbaned removed the awaiting review Pull request awaiting review label May 20, 2020

freedomtan added 3 commits May 20, 2020 16:25

Merge branch 'master' into export_SetNumThreads_to_tflite_python

969b77d

address review commments

30e5e29

more modifications

a7c3bc9

teijeong previously approved these changes May 21, 2020

View reviewed changes

google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels May 21, 2020

kokoro-team removed kokoro:force-run Tests on submitted change labels May 21, 2020

clean up

b0c869a

freedomtan dismissed teijeong’s stale review via b0c869a May 21, 2020 01:46

teijeong previously approved these changes May 21, 2020

View reviewed changes

PINTO0309 mentioned this pull request May 22, 2020

GLIBC_2.28' not found PINTO0309/Tensorflow-bin#22

Closed

gbaned previously approved these changes May 22, 2020

View reviewed changes

jdduke requested changes May 26, 2020

View reviewed changes

address review comments

fd546da

rthadur approved these changes May 27, 2020

View reviewed changes

rthadur approved these changes Jun 1, 2020

View reviewed changes

jdduke approved these changes Jun 1, 2020

View reviewed changes

jdduke requested changes Jun 11, 2020

View reviewed changes

ArktinenKarpalo mentioned this pull request Jun 22, 2020

tensorflow lite käyttää vain 1 thread, thread määrä konfiguroitavaksi CACDAR-Project/Ohtuprojekti-kesa2020#98

Open

bafu mentioned this pull request Jul 19, 2020

Failed to manually run bnpipeline on x86_64 DT42/AIKEA#4

Open

TylerADavis mentioned this pull request Nov 3, 2020

Add tflite_runtime 2.3.1 wheels to Python quickstart #44100

Closed

		def reset_all_variables(self):
		return self._interpreter.ResetVariableTensors()

Conversation

freedomtan commented Feb 14, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jianlijianli commented Feb 15, 2019

Uh oh!

jdinkel88 commented Feb 18, 2019

Uh oh!

freedomtan commented Feb 18, 2019

Uh oh!

dpsm commented Mar 8, 2019

Uh oh!

dpsm commented Mar 8, 2019

Uh oh!

freedomtan commented Mar 9, 2019

Uh oh!

dpsm commented Mar 12, 2019

Uh oh!

dpsm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

teijeong left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rthadur commented Jun 5, 2020

Uh oh!

jdduke left a comment

Choose a reason for hiding this comment

Uh oh!

srjoglekar246 commented Jun 11, 2020

Uh oh!

metanav commented Jun 13, 2020

Uh oh!

teijeong commented Jun 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

19 participants

freedomtan commented Feb 14, 2019 •

edited

Loading