Category: Article

[OpenCV] detectMultiScale: output detection score

OpenCV provides quite decent implementation of the Viola-Jones Face detector.

A quick example looks like this (OpenCV 2.4.5 tested):

// File: main.cc
#include 

using namespace cv;

int main(int argc, char **argv) {

    CascadeClassifier cascade;
    const float scale_factor(1.2f);
    const int min_neighbors(3);

    if (cascade.load("./lbpcascade_frontalface.xml")) {

        for (int i = 1; i < argc; i++) {

            Mat img = imread(argv[i], CV_LOAD_IMAGE_GRAYSCALE);
            equalizeHist(img, img);
            vector objs;
            cascade.detectMultiScale(img, objs, scale_factor, min_neighbors);

            Mat img_color = imread(argv[i], CV_LOAD_IMAGE_COLOR);
            for (int n = 0; n < objs.size(); n++) {
                rectangle(img_color, objs[n], Scalar(255,0,0), 8);
            }
            imshow("VJ Face Detector", img_color);
            waitKey(0);
        }
    }

    return 0;
}
g++ -std=c++0x -I/usr/local/include `pkg-config --libs opencv` main.cc -o main

The detection results are as shown below:
result

For more serious user, it would be nice to have a detection result for each detected face.
The OpenCV provides a overloaded function designed for this usage which is lack of detailed documentation:

vector reject_levels;
vector level_weights;
cascade.detectMultiScale(img, objs, reject_levels, level_weights, scale_factor, min_neighbors);

The reject_levels and level_weights will keep being empty until you write it like this (The whole file):

// File: main.cc
#include 

using namespace cv;

int main(int argc, char **argv) {

    CascadeClassifier cascade;
    const float scale_factor(1.2f);
    const int min_neighbors(3);

    if (cascade.load("./lbpcascade_frontalface.xml")) {

        for (int i = 1; i < argc; i++) {

            Mat img = imread(argv[i], CV_LOAD_IMAGE_GRAYSCALE);
            equalizeHist(img, img);
            vector objs;
            vector reject_levels;
            vector level_weights;
            cascade.detectMultiScale(img, objs, reject_levels, level_weights, scale_factor, min_neighbors, 0, Size(), Size(), true);

            Mat img_color = imread(argv[i], CV_LOAD_IMAGE_COLOR);
            for (int n = 0; n < objs.size(); n++) {
                rectangle(img_color, objs[n], Scalar(255,0,0), 8);
                putText(img_color, std::to_string(level_weights[n]),
                        Point(objs[n].x, objs[n].y), 1, 1, Scalar(0,0,255));
            }
            imshow("VJ Face Detector", img_color);
            waitKey(0);
        }
    }

    return 0;
}

However, this will give you a large number of detected rectangles:
result-org

This is because OpenCV skips the step of filtering out the overlapped small rectangles. I have no idea whether this is by design. But output likes this would not be helpful at least in my own case.

So we would need to make our own changes in the OpenCV's source code.
There are different ways to design detection score, such as
"In the OpenCV implementation, stage_sum is computed and compared against the i stage_threshold for each stage to accept/reject a candidate window. We define the detection score for a candidate window as K*stage_when_rejected + stage_sum_for_stage_when_rejected. If a window is accepted by the cascade, we just K*last_stage + stage_sum_for_last_stage. Choosing K as a large value e.g., 1000, we ensure that windows rejected at stage i have higher score than those rejected at stage i-1." from http://vis-www.cs.umass.edu/fddb/faq.html

Actually, I found a straightforward design of detection score works well in my own work. In the last stage of the face detector in OpenCV, detection rectangles are grouped into clustered to eliminated small overlapped rectangles while keeping the most potential rectangles. The number of final detected faces is at most same as the number of clusters. So we can simply use the number of rectangles grouped into the cluster as the detection score of the associated final rectangle, which may not be accurate but could work.

To make this change, in OpenCV-2.4.5, find the file modules/objdetect/src/cascadedetect.cpp (line 200)

// modules/objdetect/src/cascadedetect.cpp (line 200)
// int n1 = levelWeights ? rejectLevels[i] : rweights[i]; //< comment out this line
int n1 = rweights[i]; //< the change

We then modify the file main.cc accordingly:

// File: main.cc
#include 

using namespace cv;

int main(int argc, char **argv) {

    CascadeClassifier cascade;
    const float scale_factor(1.2f);
    const int min_neighbors(3);

    if (cascade.load("./lbpcascade_frontalface.xml")) {

        for (int i = 1; i < argc; i++) {

            Mat img = imread(argv[i], CV_LOAD_IMAGE_GRAYSCALE);
            equalizeHist(img, img);
            vector objs;
            vector reject_levels;
            vector level_weights;
            cascade.detectMultiScale(img, objs, reject_levels, level_weights, scale_factor, min_neighbors, 0, Size(), Size(), true);

            Mat img_color = imread(argv[i], CV_LOAD_IMAGE_COLOR);
            for (int n = 0; n < objs.size(); n++) {
                rectangle(img_color, objs[n], Scalar(255,0,0), 8);
                putText(img_color, std::to_string(reject_levels[n]),
                        Point(objs[n].x, objs[n].y), 1, 1, Scalar(0,0,255));
            }
            imshow("VJ Face Detector", img_color);
            waitKey(0);
        }
    }

    return 0;
}

And we can have the detection scores like this:
result-final

Linux Swap文件

想象一下,两个实验进程跑了两天,还有一天就跑完了,这个时候你发现如果再跑一会儿内存就要爆了…怎么办? (好惊险的感觉 XD)

好吧,其实用到的只是很基本的操作系统知识,不过还真难得用到一回。

程序面对的都是虚拟内存。64位的操作系统下,虚拟内存非常大,但是实际物理内存相对而言小得多。所以,操作系统对内存分页 (就是分成一块一块的,每一块儿叫做一页) 物理内存一旦满了,把暂时不需要的页写到硬盘里。过了一会儿程序又要访问被写到硬盘里的那部分内容,操作系统就在物理内存中选一个页 (怎么选很讲究的),把硬盘里的那个给换回来。程序不停的运行,操作系统就换来换去…

所以,上面我们遇到的情形就可以解决了。把其中一个进程挂起 (suspend),Linux下可以用

Ctrl+Z

,然后这部分内存就是暂时不用的了。这个时候用

$top

查看内存使用情况,可以看到一个CPU占用率为0的进程占用的内存越来越少,另一个越来越多。这样就行了,等一个进程跑完,再用

$fg

命令把挂起的进程调到前台就可以了。

但是等等。虚拟内存具体是在哪里呢?数据终究是写在内存/硬盘上的,Linux下被换到硬盘上的内存在Swap分区 (交换分区) 里。安装系统的时候需要格式化一个分区为Swap格式,就是这个分区。

$swapon -s

可以查看交换分区的大小。

糟糕!刚才那个被挂起的进程占用了24G的内存,但是现在看到我的交换分区只用12G,怎么办?一旦交换分区和内存都满了,会发生神马事情,我也没有体验过,估计应该是系统卡死或者卡而不死吧。

所以,应该赶紧增加交换分区的大小才是。可是如果你和我一样,很悲催的没有Root权限 (Root权限貌似是必须的…),而且也根本没有多余的分区可以挂载了,怎么办?

可以用Swap文件 (点题) ! 就是把一个文件用做swap分区,Linux下什么都是文件,分区应该也是吧。要增加系统可用的虚拟内存,当然这要求你硬盘剩余空间够大…

$dd if=/dev/zero of=~/swapfile bs=1024 count=41943040

会在HOME下创建一个40G的文件”~/swapfile”,命令要执行一会儿,需要写一段时间硬盘,执行完了会显示写硬盘的速度,可以用来做测速的。
然后告诉系统用这个文件做交换文件

mkswap ~/swapfile

就没问题了。
(实际上不行,还需要这个命令, –!)

sudo swapon ~/swapfile

其实最一开始那个情形下,如果两个进程继续跑下去,操作系统仍然会把一部分内容换出来的。只是大量换页操作会让程序执行的时间更长,而且如果交换空间不够大,系统最终仍然可能会被卡死。

感叹一下,一个实验要跑三天的同学伤不起啊..

Draw ROC Curve

A piece of fairly simple Matlab script to draw the ROC Curve from an array of scores and an array of labels.

function [Tps, Fps] = ROC(scores, labels)
 
%% Sort Labels and Scores by Scores
sl = [scores; labels];
[d1 d2] = sort(sl(1,:));
 
sorted_sl = sl(:,d2);
s_scores = sorted_sl(1,:);
s_labels = round(sorted_sl(2,:));
 
%% Constants
counts = histc(s_labels, unique(s_labels));
 
Tps = zeros(1, size(s_labels,2) + 1);
Fps = zeros(1,  size(s_labels,2) + 1);
 
negCount = counts(1);
posCount = counts(2);
 
%% Shift threshold to find the ROC
for thresIdx = 1:(size(s_scores,2)+1)
 
    % for each Threshold Index
    tpCount = 0;
    fpCount = 0;
 
    for i = [1:size(s_scores,2)]
 
        if (i >= thresIdx)           % We think it is positive
            if (s_labels(i) == 1)   % Right!
                tpCount = tpCount + 1;
            else                    % Wrong!
                fpCount = fpCount + 1;
            end
        end
 
    end
 
    Tps(thresIdx) = tpCount/posCount;
    Fps(thresIdx) = fpCount/negCount;
 
end
 
%% Draw the Curve

% Sort [Tps;Fps]
x = Tps;
y = Fps;

% Interplotion to draw spline line
count = 100;
dist = (x(1) - x(size(x,2)))/100;
xx = [x(1):-dist:x(size(x,2))];

% In order to get the interpolations, we remove all the unique numbers
[d1 d2] = unique(x);
uni_x = x(1,d2);
uni_y = y(1,d2);
yy = spline(uni_x,uni_y,xx);

% No value should exceed 1
yy = min(yy, 1);

plot(x,y,'x',xx,yy);

Hope it helps.


Some improvements were added.

For a sample input:

>> scores = rand(1,20)*100

scores =

  Columns 1 through 7

   43.8744   38.1558   76.5517   79.5200   18.6873   48.9764   44.5586

  Columns 8 through 14

   64.6313   70.9365   75.4687   27.6025   67.9703   65.5098   16.2612

  Columns 15 through 20

   11.8998   49.8364   95.9744   34.0386   58.5268   22.3812

>> labels = round(rand(1,20))

labels =

  Columns 1 through 12

     1     0     1     1     1     1     1     0     0     0     1     0

  Columns 13 through 20

     1     0     1     0     0     0     1     0

>> ROC(scores,labels);

Gives an output like:
ROC

Fork it on Github: DrawROC on Github