[Note] Learning to See by Moving

这是一篇用CNN做特征学习的文章,题目比较格调。目标是训练得到可以用来做场景识别、物体识别等等任务的特征。
和之前的工作相比,训练时需要的监督信息不是图片所属的类别,而是两张图片之间的位置变换。准确地说,是拍摄两张相关图片的相机的相对位置变换。

Screen Shot 2015-05-25 at 12.29.27 PM

具体的网络结构如上图。输入两张图片,训练这个网络来判断两张图片之间的变换(Transformation)。每两个BaseCNN取得的特征(Lk)连接起来作为TopCNN的输入,TopCNN输出变换向量。训练结束之后TopCNN就不用了,直接用BaseCNN对输入图片取特征。

在使用相同数量的有标记数据的情况下,用这种相对更容易得到的监督信息训练出来的网络来做分类网络的初始化,比随机初始化分类效果更好。

虽然这个方法并不是完全的无监督学习,但是在移动机器人这种平台上,两张照片的相对位置变换确实是很容易得到的信息。按照这种方法确实可以很方便的给机器人增加场景识别功能。理想情况下,只需要在所在环境下移动的时间采集一些图片来学习特征,简单的通过人机交互标定几个场景,这种用CNN得到的特征应该会有很不错的表现吧。

Reference:

Pulkit Agrawal, Joao Carreira, Jitendra Malik, Learning to See by Moving

Convert the saved checkpoint of cuda-convenet to the proto of Caffe

*Credits of the scripts go to jamt9000 on github, thanks for the helpful work!

You are a deep learning consumer, you have a saved check point you trained with the cuda-convenet. You want to use Caffe to evaluate the trained neural networks on new data possiblely due to the absent of GPU. Hence you need to convert the saved checkpoint to the format compatible to Caffe.

1. check out this version of caffe: jamt9000/caffe:convert-cuda-convnet or my copy.

$git clone https://github.com/jamt9000/caffe/tree/convert-cuda-convnet convert-cuda-convnet

2. compile it following the Caffe installation instruction

## Misc. Setup 
$make all
$make pycaffe

3. convert the check-point with the following codesnippet

import sys
sys.path.append('./python/')
sys.path.append('./tools/extra/')

from convert_net import *

if (len(sys.argv) < 2):
    print "Usage:- path-to-saved-convnet"
    sys.exit(-1)

arg_convnet = sys.argv[1]
output_prototxt = arg_convnet + "-prototxt"
output_proto = arg_convnet + "-proto"

netpt = cudaconv_to_prototxt(arg_convnet)
fh=open(output_prototxt, 'w')
fh.write(netpt)
fh.close()
print "Wrote {0}".format(output_prototxt)

netpb = cudaconv_to_proto(arg_convnet)
fh=open(output_proto, 'wb')
fh.write(netpb.SerializeToString())
fh.close()
print "Wrote {0}".format(output_proto)
$python convert-convnet2caffe.py saved-convnet-checkpoint

4. Now you should find two files: saved-convnet-checkpoint-proto and saved-convnet-checkpoint-prototxt, they are in an older format, you need some tools provided by Caffe to upgrade them.

5. checkout and compile the up-to-date (or any more recent stable) version of Caffe

6. Under build/tools/ you will find tools named upgrade_net_proto_text.bin and upgrade_net_proto_binary.bin, use them to upgrade the saved-convnet-checkpoint-proto and saved-convnet-checkpoint-prototxt accordingly.

7. Almost done, you may need to manually fix the prototext file a little bit for example, add the input layer following any example proto text files provided by the Caffe.

Good luck!