[OpenCV] detectMultiScale: output detection score

OpenCV provides quite decent implementation of the Viola-Jones Face detector.

A quick example looks like this (OpenCV 2.4.5 tested):

// File: main.cc
#include 

using namespace cv;

int main(int argc, char **argv) {

    CascadeClassifier cascade;
    const float scale_factor(1.2f);
    const int min_neighbors(3);

    if (cascade.load("./lbpcascade_frontalface.xml")) {

        for (int i = 1; i < argc; i++) {

            Mat img = imread(argv[i], CV_LOAD_IMAGE_GRAYSCALE);
            equalizeHist(img, img);
            vector objs;
            cascade.detectMultiScale(img, objs, scale_factor, min_neighbors);

            Mat img_color = imread(argv[i], CV_LOAD_IMAGE_COLOR);
            for (int n = 0; n < objs.size(); n++) {
                rectangle(img_color, objs[n], Scalar(255,0,0), 8);
            }
            imshow("VJ Face Detector", img_color);
            waitKey(0);
        }
    }

    return 0;
}
g++ -std=c++0x -I/usr/local/include `pkg-config --libs opencv` main.cc -o main

The detection results are as shown below:
result

For more serious user, it would be nice to have a detection result for each detected face.
The OpenCV provides a overloaded function designed for this usage which is lack of detailed documentation:

vector reject_levels;
vector level_weights;
cascade.detectMultiScale(img, objs, reject_levels, level_weights, scale_factor, min_neighbors);

The reject_levels and level_weights will keep being empty until you write it like this (The whole file):

// File: main.cc
#include 

using namespace cv;

int main(int argc, char **argv) {

    CascadeClassifier cascade;
    const float scale_factor(1.2f);
    const int min_neighbors(3);

    if (cascade.load("./lbpcascade_frontalface.xml")) {

        for (int i = 1; i < argc; i++) {

            Mat img = imread(argv[i], CV_LOAD_IMAGE_GRAYSCALE);
            equalizeHist(img, img);
            vector objs;
            vector reject_levels;
            vector level_weights;
            cascade.detectMultiScale(img, objs, reject_levels, level_weights, scale_factor, min_neighbors, 0, Size(), Size(), true);

            Mat img_color = imread(argv[i], CV_LOAD_IMAGE_COLOR);
            for (int n = 0; n < objs.size(); n++) {
                rectangle(img_color, objs[n], Scalar(255,0,0), 8);
                putText(img_color, std::to_string(level_weights[n]),
                        Point(objs[n].x, objs[n].y), 1, 1, Scalar(0,0,255));
            }
            imshow("VJ Face Detector", img_color);
            waitKey(0);
        }
    }

    return 0;
}

However, this will give you a large number of detected rectangles:
result-org

This is because OpenCV skips the step of filtering out the overlapped small rectangles. I have no idea whether this is by design. But output likes this would not be helpful at least in my own case.

So we would need to make our own changes in the OpenCV's source code.
There are different ways to design detection score, such as
"In the OpenCV implementation, stage_sum is computed and compared against the i stage_threshold for each stage to accept/reject a candidate window. We define the detection score for a candidate window as K*stage_when_rejected + stage_sum_for_stage_when_rejected. If a window is accepted by the cascade, we just K*last_stage + stage_sum_for_last_stage. Choosing K as a large value e.g., 1000, we ensure that windows rejected at stage i have higher score than those rejected at stage i-1." from http://vis-www.cs.umass.edu/fddb/faq.html

Actually, I found a straightforward design of detection score works well in my own work. In the last stage of the face detector in OpenCV, detection rectangles are grouped into clustered to eliminated small overlapped rectangles while keeping the most potential rectangles. The number of final detected faces is at most same as the number of clusters. So we can simply use the number of rectangles grouped into the cluster as the detection score of the associated final rectangle, which may not be accurate but could work.

To make this change, in OpenCV-2.4.5, find the file modules/objdetect/src/cascadedetect.cpp (line 200)

// modules/objdetect/src/cascadedetect.cpp (line 200)
// int n1 = levelWeights ? rejectLevels[i] : rweights[i]; //< comment out this line
int n1 = rweights[i]; //< the change

We then modify the file main.cc accordingly:

// File: main.cc
#include 

using namespace cv;

int main(int argc, char **argv) {

    CascadeClassifier cascade;
    const float scale_factor(1.2f);
    const int min_neighbors(3);

    if (cascade.load("./lbpcascade_frontalface.xml")) {

        for (int i = 1; i < argc; i++) {

            Mat img = imread(argv[i], CV_LOAD_IMAGE_GRAYSCALE);
            equalizeHist(img, img);
            vector objs;
            vector reject_levels;
            vector level_weights;
            cascade.detectMultiScale(img, objs, reject_levels, level_weights, scale_factor, min_neighbors, 0, Size(), Size(), true);

            Mat img_color = imread(argv[i], CV_LOAD_IMAGE_COLOR);
            for (int n = 0; n < objs.size(); n++) {
                rectangle(img_color, objs[n], Scalar(255,0,0), 8);
                putText(img_color, std::to_string(reject_levels[n]),
                        Point(objs[n].x, objs[n].y), 1, 1, Scalar(0,0,255));
            }
            imshow("VJ Face Detector", img_color);
            waitKey(0);
        }
    }

    return 0;
}

And we can have the detection scores like this:
result-final

iOS逐帧处理录像-MPVideoProcessor

一般在iOS上做录像都可以直接使用UIImagePickerController。
但有时候难免需要做逐帧的处理,比如实时的滤镜之类的。

参照这个帖子: A (quasi-) real-time video processing on iOS 把使用AVFoundation做录像的代码,做了一个简单的封装。

Delegate可以得到逐帧的彩色或者灰度图,然后就可以加上自己需要的处理了。

代码放在Gihub上:MPVideoProcessor
具体使用请参见Github上的Readme.

Draw ROC Curve

A piece of fairly simple Matlab script to draw the ROC Curve from an array of scores and an array of labels.

function [Tps, Fps] = ROC(scores, labels)
 
%% Sort Labels and Scores by Scores
sl = [scores; labels];
[d1 d2] = sort(sl(1,:));
 
sorted_sl = sl(:,d2);
s_scores = sorted_sl(1,:);
s_labels = round(sorted_sl(2,:));
 
%% Constants
counts = histc(s_labels, unique(s_labels));
 
Tps = zeros(1, size(s_labels,2) + 1);
Fps = zeros(1,  size(s_labels,2) + 1);
 
negCount = counts(1);
posCount = counts(2);
 
%% Shift threshold to find the ROC
for thresIdx = 1:(size(s_scores,2)+1)
 
    % for each Threshold Index
    tpCount = 0;
    fpCount = 0;
 
    for i = [1:size(s_scores,2)]
 
        if (i >= thresIdx)           % We think it is positive
            if (s_labels(i) == 1)   % Right!
                tpCount = tpCount + 1;
            else                    % Wrong!
                fpCount = fpCount + 1;
            end
        end
 
    end
 
    Tps(thresIdx) = tpCount/posCount;
    Fps(thresIdx) = fpCount/negCount;
 
end
 
%% Draw the Curve

% Sort [Tps;Fps]
x = Tps;
y = Fps;

% Interplotion to draw spline line
count = 100;
dist = (x(1) - x(size(x,2)))/100;
xx = [x(1):-dist:x(size(x,2))];

% In order to get the interpolations, we remove all the unique numbers
[d1 d2] = unique(x);
uni_x = x(1,d2);
uni_y = y(1,d2);
yy = spline(uni_x,uni_y,xx);

% No value should exceed 1
yy = min(yy, 1);

plot(x,y,'x',xx,yy);

Hope it helps.


Some improvements were added.

For a sample input:

>> scores = rand(1,20)*100

scores =

  Columns 1 through 7

   43.8744   38.1558   76.5517   79.5200   18.6873   48.9764   44.5586

  Columns 8 through 14

   64.6313   70.9365   75.4687   27.6025   67.9703   65.5098   16.2612

  Columns 15 through 20

   11.8998   49.8364   95.9744   34.0386   58.5268   22.3812

>> labels = round(rand(1,20))

labels =

  Columns 1 through 12

     1     0     1     1     1     1     1     0     0     0     1     0

  Columns 13 through 20

     1     0     1     0     0     0     1     0

>> ROC(scores,labels);

Gives an output like:
ROC

Fork it on Github: DrawROC on Github

[Demo]OmniGridView

OmniGridView Code (GitHub)

OmniGridView

This is a half-finished grid view for iOS.
Cause Apple’s UITableView only allows us to add and reuse vertical cells, we need to write our own
scroll view when we want to have horizontal cells.

HOW TO USE

Drag the OmniGridView Folder to your project. Use the OmniGridView like any other UIView.
Assign a delegate to OmniGridView and implement the OmniGridViewDelegate.

Please refer to the OmniGridTestViewController.m to see the information you need to provide.

这是一个半成品的Grid View。 因为Apple的UITableView只提供垂直滚动的表格,我们可能会需要水平滚动的表格。
这里提供一个全方向滚动包含单元格重用机制的Grid View。

如何使用

把OmniGridView文件夹加入你的工程。就像使用UIView一样使用OmniGridView。
设置OminiGridView的delegate实现OmniGridViewDelegate。

请参考OmniGridTestViewController.m的实现.

#pragma mark OmniGridViewDelegate

@implementation OmniGridTestViewController (Private)

- (OmniGridCell *)gridCellAt:(OmniGridLoc *)gridLoc inGridView:(OmniGridView *)gridView {
    OmniGridCell *gridCell = [gridView dequeueReusableGridCell];
    if (!gridCell)
    {
        gridCell = [[OmniGridCell alloc] init];
        gridCell.layer.borderColor = [UIColor blackColor].CGColor;
        gridCell.layer.borderWidth = 1.0f;
        gridCell.textLabel.textAlignment = UITextAlignmentCenter;
    }
    
    gridCell.textLabel.text = [NSString stringWithFormat:@"(%d,%d)", gridLoc.row, gridLoc.col];
    
    return gridCell;
}

- (float)gridCellHeightInGridView:(OmniGridView *)gridView {
    return 200.0f;
}

- (float)gridCellWidthInGridView:(OmniGridView *)gridView {
    return 200.0f;
}

- (int)numberOfGridCellsInRow:(int)row inGridView:(OmniGridView *)gridView {
    return 12;
}

- (int)numberOfRowsInGridView:(OmniGridView *)gridView {
    return 12;
}

@end