Blog

mxnet里im2rec的坑

用了一下mxnet,感觉速度上确实比Tensorflow快不少。之前试过部署到Moible端[LINK],也挺方便的。
mxnet训练的时候可以通过ImageDataIter从预处理过的rec文件里读图。rec基本上就是一个打包了label和image的大文件。用rec文件配合DataIter做训练,GPU利用率很高。

遇到两个问题,废了些时间。

1.用im2rec.py生成rec的时候,输入文件格式是,图片序号+Label+图片路径。之间必须用\t分隔,不然会有奇怪的报错。
2.im2rec.py生成图片的时候是用OpenCV读的图,然后从BGR转RGB。图读出来是HxWxC,需要转成CxHxW。不然结果自然会差很多。

Crazyflie 2.0 + Camera

最近给Crazyflie加上了Camera,挺有趣的。

之前在网上看过不少帖子讨论各种解决方案,但是始终没有发现一个简单易行的方法。

IMG_6237

IMG_6239

IMG_6238

最后发现是上面这种带无线传输的一体式的摄像头最方便,重量也小,Crazyflie带着飞无压力。

Crazyflie 2.0 + Flow Deck + Camera!

从VCOM的引脚直接给摄像头供电可以利用Crazyflie本身的开关来控制Camera的开和关。摄像头除了供电以外和Crazyflie是独立的。

接下来的问题就是怎么样在另一台机器上得到Crazyflie传回的图像。因为是摄像头输出的其实是模拟信号,所以需要一个模转数的模块。这部分在很早之前做 智能轮椅 (^_^) 的时候就发现很麻烦。通常是去找一个叫Easycap的小东西,但似乎因为仿制Easycap或者是Ez-Cap的厂家太多了,所以很难找到一个真的可以兼容信号和系统的Easycap。而且Easycap只是一个模转数的模块,在此之前还需要一个多频道的无线接收器。因为这个无线接收器通常需要12V的电源,所以整套接收图像的设备非常“笨重”,而且还不一定好用…

不过所幸最近发现了一个从USB供电的一体的无线接收模转数的模块,非常棒。随便用一个带摄像头的drone试了一下:

可以直接用OpenCV读到图像!可以做CV乐!

IMG_6268

Crazyflie 2.0 + Flow Deck

Crazyflie是一个小型四旋翼无人机平台。开源并且文档丰富,很适合做相关的研究。最新的型号是Crazyflie 2.0。

整体很小,自然续航时间和载重也有限。具体参数可以参见官网

很多做机器人控制的实验室在用这个平台,于是很容易看到有意思的Demo。比如这个Swarm

还有这个奇怪的控制盒子:

这些精准的控制大多依赖于准确并且贵并且贵的Motion Capture系统。Crazyflie本身Sensor有限,在没有外部反馈的情况下是很难控制的。
不像是大疆那些消费级的无人机,很容易就可以飞的挺好。如果用官方的手机App来飞,想做到悬停就要稳定的给它适合的Thrust,基本上很难。

所以,当官方出了这个Flow Deck的时候,立马买了。据说一发布就卖断货了,除了说明产量小以外,也说明刚需旺盛。

Flow Deck可以为Crazyflie提供简单的位置信息,让它可以在没有外部Motion Capture系统的情况下稳定的悬停。

很明显的可以看见板子上有一个向下的摄像头。Flow也就是指Optical Flow。

这就是小飞机和Flow Deck:

UNADJUSTEDNONRAW_thumb_2b1b

可以用原装的卡电池的架子透出板子的部分,把Flow Deck固定在小飞机下面。仔细看板子上有一个小箭头表明正确的安装方向。

UNADJUSTEDNONRAW_thumb_2b1d

然后需要做的是升级小飞机的固件

这个demo.py可以让crazyflie悬停在0.4米高的位置半分钟


import logging
import time

import cflib.crtp
from cflib.crazyflie.syncCrazyflie import SyncCrazyflie

URI = 'radio://0/80/250k'

# Only output errors from the logging framework
logging.basicConfig(level=logging.ERROR)


if __name__ == '__main__':
    # Initialize the low-level drivers (don't list the debug drivers)
    cflib.crtp.init_drivers(enable_debug_driver=False)

    with SyncCrazyflie(URI) as scf:
        cf = scf.cf

        cf.param.set_value('kalman.resetEstimation', '1')
        time.sleep(0.1)
        cf.param.set_value('kalman.resetEstimation', '0')
        time.sleep(2)

        for y in range(300):
            cf.commander.send_hover_setpoint(0, 0, 0, 0.4)
            time.sleep(0.1)

效果还不错。

送几本书

第一次来美国的时候带过来了几本书,都是当年本科在读的时候奉为经典的大部头。这些年,搬家毕业带来带去,敝帚自珍,却也没有真的好好读过。

如果你在湾区,请联系我,找个方法我把书给你。


Code Complete: 豆瓣介绍


算法导论: 豆瓣介绍


深入理解计算机系统: 豆瓣介绍


深入理解LINUX内核(第三版): 豆瓣介绍


UNIX编程艺术: 豆瓣介绍

小火车

家里小朋友最近大爱小火车。不久前去Santa Clara的一个火车博物馆,被里面的模型惊艳到了。一个大房间里面是细节丰富的轨道模型,铁轨经过城市、山洞、桥梁、农场,好几辆不同样式的模型小火车在其间穿行。隐隐有清明上河图的感觉。

DSC02195

DSC02196

DSC02197

DSC02199

DSC02200

DSC02201

DSC02202

DSC02203

DSC02204

DSC02205

DSC02206

DSC02207

DSC02208

DSC02209

DSC02210

DSC02211

DSC02212

DSC02213

DSC02214

DSC02215

DSC02216

湾区这边的通勤利器 — Caltrain

DSC02217

灰狗巴士

DSC02218

DSC02219

DSC02220

DSC02222

DSC02223

略有点纽约Penn Station的感觉

DSC02224

地址在:

“Distribute” an under development app without enrolling iOS Developer Program

You are developing an iOS App.

Prior to Xcode 7, you have to enroll into Apple’s iOS Developer Program (IDP) to install an App to a device, either yours or your friend’s.

Now in Xcode 7, you can compile and deploy an under development App to your device.
The problem is: How do you “distribute” your App to your friend?

I quoted the “distribute” here, because I am talking about a casual situation that it is not a serious distribution but only within a small scope.

If your friend lives nearby, this is easy. You do the something you did for yourself. You connect his/her device to your Mac/Macbook and use Xcode to install the App.

If you need to make this happen remotely, we need some work. This solution only works if your friend has a Mac with Xcode 7 installed.

There are two options.

1. You send the whole project to your friend and ask him/her to compile it and deploy it to the device. This requires advanced knowledge if your project is somehow complex.

2. After you compiled your App, you can get the actually “AppName.app” file after compiling it for device from Xcode. You send this “AppName.app” to your friend. He/she can open Xcode and follow “Window->Device”, then drag-n-drop the “AppName.app” to his/her device.

However, it is very likely that the second option will show an error “a valid provisioning profile for this executable was not found” on your friend’ screen.

This is because without IDP, the provisioning file we get from Xcode contains the exact UDID (unique device ID) of the device connected to Xcode during compilation. Any other device would not accept the App unless its UDID is listed in the provisioning file. But without IDP, we can not explicitly add UDID to an provisioning file.

Here is the workaround.

1. Register a new Apple ID if you don’t want to share your Apple ID with your friend (for good reason).
2. Ask your friend to add this new Apple ID to your Xcode Account (Preference -> Account -> + button).
3. Your friend will need to create an iOS project (simply choose the single-view Application template), compile it and install it to the device. During this process, Xcode will prompt to ask to “fix issues”. Confirm that and make sure the right “Apple ID” is selected in the “Team” field.
4. Now the UDID of the remote device is associated with the Apple ID.
5. On your local machine, use the same Apple ID to compile the App for device and send out the “AppName.app” file. Now because the UDID of both your device and the remote device are all listed in the provisioning file, the “AppName.app” can be installed into the remote device by either iTunes or Xcode.

This solution is not perfect but it works well without paying $99 each year. Good luck!

Jonas过后

来美国第四个年头了。新泽西这个地方每年都有不小的雪。看雪的心情也从刚来时的兴奋变成了爱恨交织。一片白茫茫是不错的景致,但是想到白茫茫一片之下还有自己的车心情也就复杂起来了。

本来以为今年是一个暖冬。本该是雪花纷飞的圣诞节前后可以穿着单衣出门。所以当知道气象台预报有暴风雪要来的时候,我一开始是怀疑的。天气预报总是这样的小题大作。第一年来的时候,学校常常因为预报有飓风而停课。可是飓风常常失约,又或是还未到地方地就失去了活力。

只有Sandy那次,是准的。那时候住在学校附近。天气已经转凉,整整一周的停热水停电停暖气还真是难捱。所幸的是因为离学校不远。那时候每天早早的起床去实验室,再晚晚的回去。中午学校食堂还有免费的救济餐。

按照惯例,这次的暴风雪也有名字,叫Jonas。似乎是拉丁文,不知道有什么含义。如果不是如预报所说的猛烈,恐怕也没有记住的必要了。

气象台也算是是幸运的,这次他们又说对了。

周六早上起来,外边就一片白茫茫了。门口积的雪已经影响到开门了。

https://goo.gl/photos/k8fgeU24NKViZn5k8

https://goo.gl/photos/ByNS4hcQRPNJRcfYA

从家里往窗外望去,窗前小树上堆起的雪已经很高了。

https://goo.gl/photos/vc46b9nWn2E3xL8d7

可是雪还在一直下。这时我才感觉到天气预报这回又立功了。
果然,一夜之后,前门彻底打不开了。

https://goo.gl/photos/VHRwMV3GU2MdwRLNA

不得已,还得出门铲雪挖车。只好绕道从后门出去。计划要从后面挖出一条小道回到前门,以解前门之围。

后院的情况也不太乐观,之前院子里还有房东家小朋友的玩具小车和一些杂物。看起来它们要重见天日还需要耐心等待了。

https://goo.gl/photos/5rnPpbMwkU6pcGtt8

大多数地方的雪都没过膝盖了。吭哧吭哧地劳作了一段时间,总算是从后院挪动到了停车道上。这才看见了车被埋成了什么样。

https://goo.gl/photos/z1ZSkvNhRJ5hR61b7

要把车给挖出来,这个基本上很难。

https://goo.gl/photos/67zUQVsfGUP6es4R7

挖啊挖,终于挖到前门了。抬头一看,门打不开也是讲道理的。

https://goo.gl/photos/HMGezgjix2zfwtCZ9

https://goo.gl/photos/N1DfHZ9zBJJhSgvn7

之前是台阶的地方已经无迹可寻。没有办法,只有继续挖。
成功的挖开门之后,是这个样子的。

https://goo.gl/photos/2XJCaJ6BKogUuzBk6

从上往下看

https://goo.gl/photos/FDiowu3GnXDsPFkv7

下午和小伙伴们一起在停车道上劳作了两三个小时。
Sandy之后,这下又记住Jonas的大名了。

希望这个冬天快点过去吧。

iOS上mxnet的一个演示App

mxnet是最近火的不行的一个深度学习的框架,支持好多好多语言,有好多好多大牛在写。

之前也有想过把同样很牛很牛的caffe跑到iOS上看看速度怎么样,但是caffe有一大堆文件,感觉做起来很麻烦。

最近看到mxnet居然有一个单文件的版本
就做了一个简单的图像识别的演示App。跑在6上速度还可以,大概4秒一张图。

代码在这里:

WhatsThis-iOS on Github

Cardboard VR

上次开会的时候排队尝试了Google的Cardboard和Oculus,虚拟现实(VR)的体验很真实。于是也不难理解现在VR/AR的火热程度了。Facebook有Oculus,Google有Cardboard,微软有HoloLens,还有Magic Leap的神秘兮兮又牛叉哄哄的演示,也是醉了。大家都看好这个市场,虚拟现实(VR)和增强现实(AR)的市场规模可能会达到1,500亿美元…

为了紧跟社会潮流,果断去大Amazon淘了一个Cardboard来玩,就长这样。
http://www.amazon.com/Pro-Compatible-Instructions-Construction-Experience/dp/B00Q1FITMO

Amazon上有各种Cardboard,大都支持Google Cardboard的Apps。Cardboard主体就一张纸板,还有两个镜片,折了几折就成了前面图上的样子。把你的手机放上去做屏幕就可以体验VR了!

这基本上应该是最经济方便的体验VR的方式了。Cardboard支持很多大屏幕手机,我用的iPhone 6没有问题。用的时候需要凑近镜片去看屏幕。因为近视的关系,我起初比较担心戴着眼镜看不是很方便,实际上把眼镜摘了也看得很清楚。因为其实看到的不是真的3D…屏幕就在眼前不远,不是上千度的近视的话,当然应该看得清楚。

目前各种VR设备面临的一个重要问题是晕动病 (VR motion sickness)。wiki上的对晕动病的解释是:

晕动病或运动病,生活中通常被称为晕车、暈機和晕船,是一种平衡失调的疾病。当人眼所见到的运动与前庭系統感觉到的运动不相符时,就会有昏厥、恶心、食欲减退等症状出现。严重者会呕吐。

这个和有些同学打CS会头晕或者开极品飞车就也能开吐了是相同的原理。可能的一个原因是显示屏的刷新率不够高,我自己用的时候也能不时地感觉到屏幕的刷新率跟不上头的移动。这个时候就会有点“跳戏”的感觉 🙂 据说只有一小部分人是很幸运的天生没有晕动症这个问题的。所以,如何解决这个问题也是VR厂商们面前的一个难题。当然大家也在积极的寻找对策。简单的解决方法据说是喝点酒… 也有研究科学家表示,在画面中增加一个虚拟的鼻子也可以缓解晕动症 ._.

我自己用Cardboard的时候也没有感觉到这个问题,可能是因为看的时间不是很久。毕竟Cardboard只是用来体验VR的,需要用手托住Cardboard来看,不适合长时间的看。不知道是不是iPhone 6屏幕的问题,看的时候感觉画面不是特别的锐利,有一种显示的分辨率不够高的感觉。但是整体的体验还是很赞的。iOS上的VR Apps不如Andriod上的多。在这里推荐几个值得试试的应用。

Google Carboard
这个是Google官方的应用,可以先通过这个应用体验一下基本的功能。其中有一个街景的功能,可以体验站在各个著名景点现场环视四周的感觉。

InMind VR
这是一个VR的游戏,画面不错,挺有代入感。用头部的动作来完成射击任务,时间不长,值得一试。

VRSE
这个App里提供了不少很有意思的VR电影,影片时间长短合适,非常推荐。有些是记录片,有些是软广,但是通过VR来看,都挺有代入感。
类似的App还有:RYOT

暂时就发现这么几个,试试看吧 🙂

补充:

GermBuster
一个非常有趣的游戏,用泡泡攻击细菌。最有趣的是细菌也会吐泡泡攻击你,快被打到的时候3D感特别明显,有种被直接打到脸上的感觉…

Build a Robot From A Power Wheelchair (1/2)

I have been working on a robotic project for a while. In this project, we build a robotic wheelchair controlled by head motions.

IMG_2245

It starts from a commercial powered wheelchair (Titan Front Wheel Drive Power Chair).

We took over the control signals, added a number of sensors and wrote some computer vision softwares. It is a lot of fun. At this time point, we can drive it with head motions, a little slow but very easy to control. A short video is posted.

Robotic Wheelchair
[KGVID width=”640″ height=”360″]http://personal.stevens.edu/~hli18/video/robot.mp4[/KGVID]

The most challenging part so far, to our surprise, is how to take over the driving signals. In fact, once the powered wheelchair can be controlled with digital signals, it is somehow a robot.

We can find quite a few posts discussing how to hack the joysticks with Arduino. If you have a VR2 joystick, it would be much easier. This post (http://myrobotnstuff.blogspot.com.au/2012/07/circuit-for-wheelchair-robot.html) explains the process well. Basically, you can reproduce the digital signals triggered by the joystick with the Arduino.

But unfortunately, the joystick with the Titan power chair is with a different design and it is much more difficult to hack.
It is Dynamic Shark controller. If your remote (joystick) looks like this one , you probably have this kind of joystick.

The difficulty in reproducing the control signals generated by this kind of joystick is that it directly outputs 25V differential signals from the remote to the power module, which is a big black box under the seat. The signals are encoded and there is probably no way to get the encoding protocol.

We did quite a few rounds of trial and error and finally figured out a plan after burning out a joystick…

IMG_1587

This kind of joystick works in a special way. At the end of the handle of joystick, there is with a small coil. 4 small coils are on the board under it. Because of the electromagnetic induction, the coil on the handle of the joystick produces different signals at different positions in the 4 small coils under it.

Roughly speaking, the (analog) signals produced by the 4 small coils are then processed by an on-board micro-controller to produce the driving signal. Because the signals produced by the micro-controller are encoded in an unknown protocol, we choose to get in by reproducing the outputs from the 4 small coils.

If you take away all 4 small coils (DO NOT do that…), the board looks like this

A

After some processing circuits, finally the outputs of the 4 coils can be found as two signals passed the two resistances in the red circuits. If you use an oscilloscope to check the signals, you will find that they are sine waves!

With this observation, the plan is clear. Soldering two wires at the right end of these two resistances (be careful!), we need to use an Arduino to reproduce these two sine waves to drive the wheelchair.

One of the two sine waves is for the X-direction and the other one is controlling the driving in the Y-direction.
The phase the sine wave indicates the sign (forward in X-direction or backward in X-direction) and the amplitude indicates the speed. The two sine waves should be sort-of aligned, which means the phase difference should be either 0 or 180.

With an Arduino, we will produce these sine waves signals to the two wires.