The convolutional.py example is directly taken from the tensorflow/models/image example folder. It will train a neural net based on the NIST handwriting data set. More specifically a LeNet-5 like convolutional neural network. I suspect the ancient ancestors of tensorflow were used by Google Books. First we need to detect the language, then we can approach the text by recognizing each graph and converting it into language specific text and finally we can try to make sense out of it and finally play god by interpreting and censoring thoughts (the government version maybe called CensorFlow). Well potentially. Right?

The Google TensorFlow tutorials contain beginners and expert code for better understanding of the machinery behind. Before we can start we need to download the python file from the repository or best clone the repository locally.

git clone https://tensorflow.googlesource.com/release

or

git clone https://github.com/tensorflow/tensorflow.git

After the installation or download of the tensorflow repository a new subdirectory with the name data will be created. This is were four files will be installed. The data is taken from http://yann.lecun.com/exdb/mnist/ which also nicely explains the details behind this data set.

-rw-r--r-- 1 vm vm 1648877 Nov 20 18:59 t10k-images-idx3-ubyte.gz
-rw-r--r-- 1 vm vm    4542 Nov 20 18:59 t10k-labels-idx1-ubyte.gz
-rw-r--r-- 1 vm vm 9912422 Nov 20 18:59 train-images-idx3-ubyte.gz
-rw-r--r-- 1 vm vm   28881 Nov 20 18:59 train-labels-idx1-ubyte.gz

Then we can change to the tensorflow directory and call

python tensorflow/models/image/mnist/convolutional.py

vm@ubuntu:~/tensorflow$python tensorflow/models/image/mnist/convolutional.py
Extracting data/train-images-idx3-ubyte.gz
Extracting data/train-labels-idx1-ubyte.gz
Extracting data/t10k-images-idx3-ubyte.gz
Extracting data/t10k-labels-idx1-ubyte.gz
I tensorflow/core/common_runtime/local_device.cc:25] Local device intra op parallelism threads: 8
I tensorflow/core/common_runtime/local_session.cc:45] Local session inter op parallelism threads: 8
Initialized!
Epoch 0.00
Minibatch loss: 12.054, learning rate: 0.010000
Minibatch error: 90.6%
Validation error: 84.6%

We can see now that tensorflow starts training and that the CPU use is quite interesting with 50% use of the 8 threads (4 CPU cores) and a sudden up-shoot to maximum utilization. Because the program is running in a Oracle VM VirtualBox using Ubuntu 13 this could be an artifact and can only be confirmed when running natively without a virtual machine. Usually such behavior can hint to inefficient threading within the algorithm, or disk or memory bottlenecks.

Epoch 9.77
Minibatch loss: 1.596, learning rate: 0.006302
Minibatch error: 0.0%
Validation error: 0.9%
Epoch 9.89
Minibatch loss: 1.602, learning rate: 0.006302
Minibatch error: 0.0%
Validation error: 0.9%
Test error: 0.8%

real	34m16.645s
user	104m20.376s
sys     33m22.867s
vm@ubuntu:~/tensorflow$ 

We can see that benchmarking the convolutional.py using a three year old Core i7-2600K CPU (4.2 Ghz) and some random DDR memory with tensorflow in CPU-only mode (no CUDA or OPENCL) will take around 34 minutes to finalize the example.

Links

MNIST data set - THE MNIST DATABASE of handwritten digits

LeNet 5 NN - Short explanation of a LeNet 5 neural network

convolutional.py - The above discussed example hosted at Google Git

MNIST For ML Beginners - Introduction to the MNIST hand writing problem

Deep MNIST for Experts - TensorFlow tutorial for the MNIST expert

H2O Flow and TensorFlow - H20 flow calls Tensorflow