OpenCV (检测兴趣点)
Eva.Q Lv9
2021-08-10 11:15:09

兴趣点（关键点、特征点），目标识别、图像配准、视觉跟踪、三维重建领域热词之一

检测图像中的角点

角点：两条边缘线的接合点，是一种二维特征，可以被精确地检测（即使是亚像素级精度）

Harris角点：https://blog.csdn.net/qq_41598072/article/details/83651629

OpenCV 中检测 Harris 角点的基本函数是 cornerHarris

基本用法

1
2
3
4
5
6
7
8
9
10
11
// 检测Harris角点
Mat cornerStrength;
cornerHarris(image, // 输入图像 
			 cornerStrength, // 角点强度的图像
			 3, // 邻域尺寸
			 3, // 口径尺寸
			 0.01); // Harris 参数
// 对角点强度阈值化
Mat harrisCorners;
double threshold = 0.0001;
threshold(cornerStrength, harrisCorners, threshold, 255, THRESH_BINARY_INV);

创建类

为了使用方法便于调节，和结果更好，定义一个检测 Harris 角点的类。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
#if !defined HARRISD
#define HARRISD

#include <vector>
#include <opencv2/core.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/features2d.hpp>
#include <opencv2/imgproc.hpp>

class HarrisDetector {

  private:

	  // 32-bit float image of corner strength
	  cv::Mat cornerStrength;
	  // 32-bit float image of thresholded corners
	  cv::Mat cornerTh;
	  // image of local maxima (internal)
	  cv::Mat localMax;
	  // size of neighbourhood for derivatives smoothing
	  int neighborhood; 
	  // aperture for gradient computation
	  int aperture; 
	  // Harris parameter
	  double k;
	  // maximum strength for threshold computation
	  double maxStrength;
	  // calculated threshold (internal)
	  double threshold;
	  // size of neighbourhood for non-max suppression
	  int nonMaxSize; 
	  // kernel for non-max suppression
	  cv::Mat kernel;

  public:

	  HarrisDetector() : neighborhood(3), aperture(3), k(0.1), maxStrength(0.0), threshold(0.01), nonMaxSize(3) {
	  // 用于非最大值抑制的内核
		  setLocalMaxWindowSize(nonMaxSize);
	  }

	  // Create kernel used in non-maxima suppression
	  void setLocalMaxWindowSize(int size) {

		  nonMaxSize= size;
		  kernel.create(nonMaxSize,nonMaxSize,CV_8U);
	  }

      // 计算每个像素的 Harris 值
    
	  // Compute Harris corners
	  void detect(const cv::Mat& image) {
	
		  // Harris computation
		  cv::cornerHarris(image,cornerStrength,
		             neighborhood,// neighborhood size
					 aperture,     // aperture size
					 k);           // Harris parameter
	
		  // internal threshold computation
		  cv::minMaxLoc(cornerStrength,0,&maxStrength);

		  // local maxima detection
		  cv::Mat dilated;  // temporary image
		  cv::dilate(cornerStrength,dilated,cv::Mat());
		  cv::compare(cornerStrength,dilated,localMax,cv::CMP_EQ);
	  }
    
      // 用指定阈值获取特征点。阈值的可选范围取决于选择的参数，所以阈值被作为质量等级，用最大Harris值的一个比例值表示。

	  // Get the corner map from the computed Harris values
	  cv::Mat getCornerMap(double qualityLevel) {

		  cv::Mat cornerMap;

		  // thresholding the corner strength
		  threshold= qualityLevel*maxStrength;
		  cv::threshold(cornerStrength,cornerTh,threshold,255,cv::THRESH_BINARY);

		  // convert to 8-bit image
		  cornerTh.convertTo(cornerMap,CV_8U);
	
		  // non-maxima suppression
		  cv::bitwise_and(cornerMap,localMax,cornerMap);

		  return cornerMap;
	  }

	  // Get the feature points vector from the computed Harris values
	  void getCorners(std::vector<cv::Point> &points, double qualityLevel) {

		  // Get the corner map
		  cv::Mat cornerMap= getCornerMap(qualityLevel);
		  // Get the corners
		  getCorners(points, cornerMap);
	  }

	  // Get the feature points vector from the computed corner map
	  void getCorners(std::vector<cv::Point> &points, const cv::Mat& cornerMap) {
			  
		  // Iterate over the pixels to obtain all feature points
		  for( int y = 0; y < cornerMap.rows; y++ ) {
    
			  const uchar* rowPtr = cornerMap.ptr<uchar>(y);
    
			  for( int x = 0; x < cornerMap.cols; x++ ) {

				  // if it is a feature point
				  if (rowPtr[x]) {

					  points.push_back(cv::Point(x,y));
				  }
			  } 
		  }
	  }

	  // Draw circles at feature point locations on an image
	  void drawOnImage(cv::Mat &image, const std::vector<cv::Point> &points, cv::Scalar color= cv::Scalar(255,255,255), int radius=3, int thickness=1) {

		  std::vector<cv::Point>::const_iterator it= points.begin();

		  // for all corners
		  while (it!=points.end()) {

			  // draw a circle at each corner location
			  cv::circle(image,*it,radius,color,thickness);
			  ++it;
		  }
	  }
};

#endif

类的使用

基本使用
1
2
3
4
5
6
7
8
9
// Create Harris detector instance
HarrisDetector harris;
// Compute Harris values
harris.detect(image);
// Detect Harris corners
std::vector<cv::Point> pts;
harris.getCorners(pts,0.02);
// Draw Harris corners
harris.drawOnImage(image,pts);
适合跟踪的特征 GFTT
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
// GFTT:

// Read input image
image= cv::imread("church01.jpg",0);
// rotate the image (to produce a horizontal image)
cv::transpose(image, image);
cv::flip(image, image, 0);

// Compute good features to track

std::vector<cv::KeyPoint> keypoints;
// GFTT detector
cv::Ptr<cv::GFTTDetector> ptrGFTT = cv::GFTTDetector::create(
		500,	// maximum number of keypoints to be returned
		0.01,	// quality level
		10);	// minimum allowed distance between points	  
// detect the GFTT
ptrGFTT->detect(image,keypoints);

// for all keypoints
std::vector<cv::KeyPoint>::const_iterator it= keypoints.begin();
while (it!=keypoints.end()) {
	// draw a circle at each corner location
	cv::circle(image,it->pt,3,cv::Scalar(255,255,255),1);
	++it;
}

// Display the keypoints
cv::namedWindow("GFTT");
cv::imshow("GFTT",image);

快速检测特征

Fast (加速分割测试获得特征)

基本使用
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// FAST feature:

// Read input image
image= cv::imread("church01.jpg",0);
// rotate the image (to produce a horizontal image)
cv::transpose(image, image);
cv::flip(image, image, 0);
keypoints.clear();
// FAST detector
cv::Ptr<cv::FastFeatureDetector> ptrFAST = cv::FastFeatureDetector::create(40);
// detect the keypoints
ptrFAST->detect(image,keypoints);
// draw the keypoints
	cv::drawKeypoints(image,keypoints,image,cv::Scalar(255,255,255),cv::DrawMatchesFlags::DRAW_OVER_OUTIMG);
std::cout << "Number of keypoints (FAST): " << keypoints.size() << std::endl; 

// Display the keypoints
cv::namedWindow("FAST");
cv::imshow("FAST",image);
将图像分割成网格状，对每个小图像进行单独检测。使特征点分布均匀
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
// The final vector of keypoints
keypoints.clear();
// detect on each grid
for (int i = 0; i < vstep; i++)
	for (int j = 0; j < hstep; j++) {

		// create ROI over current grid
		imageROI = image(cv::Rect(j*hsize, i*vsize, hsize, vsize));
		// detect the keypoints in grid
		gridpoints.clear();
		ptrFAST->detect(imageROI, gridpoints);
		std::cout << "Number of FAST in grid " << i << "," << j << ": " << gridpoints.size() << std::endl;
		if (gridpoints.size() > subtotal) {
			for (auto it = gridpoints.begin(); it != gridpoints.begin() + subtotal; ++it) {
				std::cout << "  " << it->response << std::endl;
			}
		}

        // get the strongest FAST features
		auto itEnd(gridpoints.end());
		if (gridpoints.size() > subtotal) { // select the strongest features
			std::nth_element(gridpoints.begin(), gridpoints.begin() + subtotal, gridpoints.end(), [](cv::KeyPoint& a, cv::KeyPoint& b) {return a.response > b.response; });
			itEnd = gridpoints.begin() + subtotal;
		}

		// add them to the global keypoint vector
		for (auto it = gridpoints.begin(); it != itEnd; ++it) {
            it->pt += cv::Point2f(j*hsize, i*vsize); // convert to image coordinates
			keypoints.push_back(*it);
			std::cout << "  " <<it->response << std::endl;
		}
	}

// draw the keypoints
cv::drawKeypoints(image, keypoints, image, cv::Scalar(255, 255, 255), cv::DrawMatchesFlags::DRAW_OVER_OUTIMG);

// Display the keypoints
cv::namedWindow("FAST Features (grid)");
cv::imshow("FAST Features (grid)", image);

尺度不变特征的检测

SURF & SIFT

https://evaqwq.github.io/2021/08/02/OPENCV1-5/

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
// SURF:

// Read input image
image = cv::imread("church01.jpg", 0);
// rotate the image (to produce a horizontal image)
cv::transpose(image, image);
cv::flip(image, image, 0);

keypoints.clear();
// Construct the SURF feature detector object
cv::Ptr<cv::xfeatures2d::SurfFeatureDetector> ptrSURF = cv::xfeatures2d::SurfFeatureDetector::create(2000.0);
// detect the keypoints
ptrSURF->detect(image, keypoints);
	
// Detect the SURF features
ptrSURF->detect(image,keypoints);
	
cv::Mat featureImage;
	cv::drawKeypoints(image,keypoints,featureImage,cv::Scalar(255,255,255),cv::DrawMatchesFlags::DRAW_RICH_KEYPOINTS);

// Display the keypoints
cv::namedWindow("SURF");
cv::imshow("SURF",featureImage);

std::cout << "Number of SURF keypoints: " << keypoints.size() << std::endl; 

// Read a second input image
image= cv::imread("church03.jpg", cv::IMREAD_GRAYSCALE);	
// rotate the image (to produce a horizontal image)
cv::transpose(image, image);
cv::flip(image, image, 0);

// Detect the SURF features
ptrSURF->detect(image,keypoints);
	
	cv::drawKeypoints(image,keypoints,featureImage,cv::Scalar(255,255,255),cv::DrawMatchesFlags::DRAW_RICH_KEYPOINTS);

// Display the keypoints
cv::namedWindow("SURF (2)");
cv::imshow("SURF (2)",featureImage);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// SIFT:

// Read input image
image= cv::imread("church01.jpg", cv::IMREAD_GRAYSCALE);
// rotate the image (to produce a horizontal image)
cv::transpose(image, image);
cv::flip(image, image, 0);

keypoints.clear();
// Construct the SIFT feature detector object
cv::Ptr<cv::xfeatures2d::SiftFeatureDetector> ptrSIFT = cv::xfeatures2d::SiftFeatureDetector::create();
// detect the keypoints
ptrSIFT->detect(image, keypoints);

// Detect the SIFT features
ptrSIFT->detect(image,keypoints);
	
	cv::drawKeypoints(image,keypoints,featureImage,cv::Scalar(255,255,255),cv::DrawMatchesFlags::DRAW_RICH_KEYPOINTS);

// Display the keypoints
cv::namedWindow("SIFT");
cv::imshow("SIFT",featureImage);

std::cout << "Number of SIFT keypoints: " << keypoints.size() << std::endl;

多尺度FAST特征的检测

BRISK (二元稳健恒定可拓展关键点)

代码实现
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
// BRISK:

// Read input image
image= cv::imread("church01.jpg",CV_LOAD_IMAGE_GRAYSCALE);
// rotate the image (to produce a horizontal image)
cv::transpose(image, image);
cv::flip(image, image, 0);

// Display the keypoints
cv::namedWindow("BRISK");
cv::imshow("BRISK",featureImage);

keypoints.clear();
// Construct another BRISK feature detector object
cv::Ptr<cv::BRISK> ptrBRISK = cv::BRISK::create(
		60,  // threshold for BRISK points to be accepted
		5);  // number of octaves

// Detect the BRISK features
ptrBRISK->detect(image,keypoints);
	
	cv::drawKeypoints(image,keypoints,featureImage,cv::Scalar(255,255,255),cv::DrawMatchesFlags::DRAW_RICH_KEYPOINTS);

// Display the keypoints
cv::namedWindow("BRISK");
cv::imshow("BRISK", featureImage);

std::cout << "Number of BRISK keypoints: " << keypoints.size() << std::endl;
原理

BRISK 不仅是一个特征点检测器，它还包含了描述每个被检测关键点的邻域的过程。

为了在不同尺度下检测兴趣点，该算法首先通过两个下采样过程后见一个图像金字塔。

第一个过程从原始图像尺寸开始，然后每一图层（八度）减少一半。第二个过程先将原始图像的尺寸除以1.5得到第一幅图像，然后在这副图像的基础上每一层减少一半，两个过程产生的图层交替在一起。

然后在该金字塔的所有图像上应用FAST特征检测器，提取关键点的条件与SIFT算法类似。首先，将一个像素与相邻的八个像素之一进行强度值的比较，只有是局部最大值的像素才可能成为关键点。这个条件满足后，比较这个点与上下两层的相邻像素的评分；如果它的评分在尺度上也更高，那么就认为它是一个兴趣点。

该算法的关键在于，不同图层具有不同的分辨率。为了精确定位每个关键点，算法需要在尺度和空间两个方面进行插值。

插值基于FAST关键点评分。在空间方面，在3*3的邻域上进行插值；在尺度方面，计算要符合一个一维抛物线，该抛物线在尺度坐标轴上，穿过当前点和上下两层的两个局部关键点，这个关键点在尺度上的位置见前面的图片。

ORB (定向FAST 和 旋转BRIEF)

代码实现
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
// ORB:

// Read input image
image= cv::imread("church01.jpg",CV_LOAD_IMAGE_GRAYSCALE);
// rotate the image (to produce a horizontal image)
cv::transpose(image, image);
cv::flip(image, image, 0);

keypoints.clear();
// Construct the BRISK feature detector object
cv::Ptr<cv::ORB> ptrORB = cv::ORB::create(75, // total number of keypoints
										  1.2, // scale factor between layers
		                                  8);  // number of layers in pyramid
// detect the keypoints
ptrORB->detect(image, keypoints);
	
	cv::drawKeypoints(image,keypoints,featureImage,cv::Scalar(255,255,255),cv::DrawMatchesFlags::DRAW_RICH_KEYPOINTS);

// Display the keypoints
cv::namedWindow("ORB");
cv::imshow("ORB",featureImage);

std::cout << "Number of ORB keypoints: " << keypoints.size() << std::endl;
  • Post title：OpenCV (检测兴趣点)
  • Post author：Eva.Q
  • Create time：2021-08-10 11:15:09
  • Post link：https://qyy/2021/08/10/OPENCV/OPENCV1-11/
  • Copyright Notice：All articles in this blog are licensed under BY-NC-SA unless stating additionally.