Getting Started with Training a Caffe Object Detection Inference Network

Applicable products

Firefly-DL

Application note description

This application note describes how to install SSD-Caffe on Ubuntu and how to train and test the files needed to create a compatible network inference file for Firefly-DL.

Installing SSD-Caffe on Ubuntu 18.04/16.04

We strongly recommend you start with a fresh Ubuntu 18.04/16.04 system.

1. Install NCSDK 2.05.00.02 (https://github.com/movidius/ncsdk/tree/v2.05.00.02).

a. To download this version, in terminal use the following:

git clone -b v2.05.00.02 --single-branch https://github.com/movidius/ncsdk.git

b. To install, navigate to the ncsdk-2.05.00.02 folder and use the following:

$ make install

Note: If using Ubuntu 18.04, open install.sh file, on Line 32 replace 1604 with 1804.

c. By default, caffe is installed under the directory “/opt/movidius/caffe”. Set this as the $CAFFE_ROOT:

$ echo "export CAFFE_ROOT=/opt/movidius/caffe/" >> ~/.profile
$ source ~/.profile

1. d. Set PYTHONPATH: 

$ export PYTHONPATH="${PYTHONPATH}:/opt/movidius/caffe/python"

2. Install Boost:

$ sudo apt-get install -y --no-install-recommends libboost-all-dev

3. Install all the necessary packages for Caffe:

$ sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev libopenblas-dev \
libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler

4. Install all the necessary Python packages:

$ pip3 install scikit-image protobuf
$ cd $CAFFE_ROOT/python
$ pip3 install -r requirements.txt

5. Make a copy of the file Makefile.config.example:

$ cd $CAFFE_ROOT/
$ cp Makefile.config.example Makefile.config
$ sudo gedit Makefile.config

Note: You can also use vim or Text Editor to edit.

6. Modify Makefile.config as below:

a. Uncomment Line 8: CPU_ONLY := 1

b. Uncomment Line 21: OPENCV_VERSION := 3

c. On our test setup, we used python 3.6. To ensure that the makefile is configurated to use python 3.6, uncomment and modify Line 76 - 78 to below:

PYTHON_LIBRARIES := boost_python3 python3.6m
PYTHON_INCLUDE := /usr/include/python3.6m \
/usr/lib/python3.6/list-packages/numpy/core/include

d. Save and Exit.

7. Check the name of the hdf5 library and hdf5_hl library. It may vary depending on the Ubuntu version.

$ cd /usr/lib/x86_64-linux-gnu/
$ ls -al | grep libhdf5_serial
...
libhdf5_serial.so.100.0.1
libhdf5_serial_hl.so.100.0.0
...

8. Note the libraries that are present on your system. Note that the postfixes of hdf5 and hdf5_hl are not always the same. Then make a link to them:

$ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial.so.100.0.1 /usr/lib/x86_64-linux-gnu/libhdf5.so
$ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial_hl.so.100.0.0 /usr/lib/x86_64-linux-gnu/libhdf5_hl.so

9. In MakeFile.config, modify Line 92 and 93 to the following:

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/

10. Navigate to $CAFFE_ROOT folder and make:

$ cd $CAFFE_ROOT
$ make all -j8
$ make test -j8
$ make runtest -j8

Each make command might take a few minutes to finish.

11. Open /opt/movidius/caffe/src/caffe/util/math_functions.cpp

$ sudo gedit $CAFFE_ROOT/src/caffe/util/math_functions.cpp

12. Replace the function caffe_rng_uniform() at Line 247 to:

void caffe_rng_uniform(const int n, Dtype a, Dtype b, Dtype* r) {
CHECK_GE(n, 0);
CHECK(r);
if(a > b) {
Dtype c = a;
a = b;
b = c;
}
CHECK_LE(a, b);
boost::uniform_real<Dtype> random_distribution(a, caffe_nextafter<Dtype>(b));
boost::variate_generator<caffe::rng_t*, boost::uniform_real<Dtype> >
variate_generator(caffe_rng(), random_distribution);
for (int i = 0; i < n; ++i) {
r[i] = variate_generator();
}
}

 Reference: https://github.com/weiliu89/caffe/issues/669

13. Run the command below:

$ make pycaffe

14. To confirm the installation, import caffe in python3.6:

$ python3.6
>>> import caffe

Training and testing

Once ssd-caffe is properly set up, you can train your data to generate the .caffemodel and .prototxt files necessary to create a compatible network inference file for Firefly-DL.

Preparing data

1. Follow Preparation - Step 2 and 3 at https://github.com/weiliu89/caffe/tree/ssd to generate the LMDB files at:

$HOME/data/VOCdevkit/VOC0712/lmdb/VOC0712_trainval_lmdb (Size: around 1.8GB)
$HOME/data/VOCdevkit/VOC0712/lmdb/VOC0712_test_lmdb (Size: around 446MB)

2. Since we are using python3, in /opt/movidius/caffe/data/VOC0712/create_data.sh:

• On Line 24 change “python” to “python3”

3. If you get the error “python3 can’t open file create_annoset.py No such file or directory”:

a. Make the following changes to create_data.sh:

• Change line 1 to:

cur_dir=$(pwd)

• Change line 2 to:

root_dir=$cur_dir

b. Run the script again from the $CAFFE_ROOT directory.

The labelmap_voc.prototxt file is generated in the directory “/opt/movidius/caffe/data/VOC0712”. The labelmap_voc.prototxt file lists the objects to be detected, and can be used to create the label file to be uploaded to the FFY_DL camera.

The file voc0712_detection_labels.txt for this example can downloaded from: https://flir.box.com/s/szr8p4n9f5ucoydb3l0jz28mp2pqcdd6

Preparing training scripts and other files

1. Download training scripts (for MobileNet-SSD model):

$ cd $CAFFE_ROOT/examples
$ git clone https://github.com/chuanqi305/MobileNet-SSD

A 'MobileNet-SSD' folder is created in '/opt/movidius/caffe/examples' with the code from the original MobileNet-SSD repo for retraining and testing.

2. Generate your own training (train/test/deploy) prototxt files:

$ cd MobileNet-SSD
$ ./gen_model.sh CLASSNUM

CLASSNUM is the number of classes in your dataset. It is also reflected in labelmap_voc.prototxt file. In this example for VOC0712 dataset, CLASSNUM = 21.

Three files are generated:

• MobileNetSSD_train.prototxt
• MobileNetSSD_test.prototxt
• MobileNetSSD_deploy.prototxt

These are saved in the "/opt/movidius/caffe/examples/MobileNet-SSD/example/" directory.

3. Create symlinks to lmdb data:

$ cd $CAFFE_ROOT/examples/MobileNet-SSD
$ ln -s PATH_TO_YOUR_TRAIN_LMDB trainval_lmdb
$ ln -s PATH_TO_YOUR_TEST_LMDB test_lmdb

In this example:

PATH_TO_YOUR_TRAIN_LMDB is: $HOME/data/VOCdevkit/VOC0712/lmdb/VOC0712_trainval_lmdb

PATH_TO_YOUR_TEST_LMDB is: $HOME/data/VOCdevkit/VOC0712/lmdb/VOC0712_test_lmdb

Under the “/opt/movidius/caffe/examples/MobileNet-SSD" directory, there are two symlink folders named “trainval_lmdb” and “test_lmdb” that point to your training data folder and test data folder, respectively.

Training

Note: You can also use vim or Text Editor to edit.

1. Modify training scripts for CPU-ONLY training:

$ sudo gedit solver_train.prototxt

a. Change line 13 to:

solver_mode: CPU

b. Save.

2. Edit train.sh:

$ sudo gedit train.sh

a. Comment out the last line:

#-gpu 0

b. Save.

3. Run train.sh to train your own model:

$ ./train.sh

Training can take a long time, but you can temporarily pause the training by pressing “Ctrl+C”.

Two output files are generated:

• mobilenet_iter_*.caffemodel
• mobilenet_iter_*.solverstate

These are saved in the "/opt/movidius/caffe/examples/MobileNet-SSD/snapshot" directory.

The more iterations you train for, the higher accuracy can be achieved. We have tested mobilenet_iter_1000 (1000 iterations), and have achieved good results.

The mobilenet_iter_*.caffemodel and MobileNetSSD_deploy.prototxt files are needed for conversion. These are in the "/opt/movidius/caffe/examples/MobileNet-SSD/example/" directory.For conversion, follow the steps in Example 2 in Getting Started with Firefly-DL in Linux.

Testing (optional)

1. Modify testing scripts for CPU-ONLY testing:

$ sudo gedit solver_test.prototxt

a. Change line 1 to:

train_net: "example/MobileNetSSD_train.prototxt"

b. Change line 2 to:

test_net: "example/MobileNetSSD_test.prototxt"

c. Change line 13 to:

solver_mode: CPU

d. Save.

2. Edit train.sh:

$ sudo gedit train.sh

a. Comment out the last line:

#-gpu 0

b. Save.

3. Run test.sh to test your own model, which can take a long time as well:

$ ./test.sh