3

在对象检测算法中，非最大抑制（NMS）用于丢弃对象（例如车辆）的额外检测结果。

通常，水平边界框用于对象检测算法，水平 NMS 的 GPU 实现已经存在，但我希望 GPU 实现旋转边界框。

CPU 实现已经完成，但我正在努力使用 CuPy 包将 CPU 版本转换为 GPU 版本。这是我写的代码。在代码部分之后，您可以看到错误。

我的问题是 TypeError 的原因是什么：列表索引必须是整数或切片，而不是 cupy.core.core.ndarray？

    from shapely.geometry import Polygon as shpoly
    import time
    
    #### CPU implementation
    import numpy as np   
    
    def polygon_iou(poly1, poly2):
      """
      Intersection over union between two shapely polygons.
      """
      if not poly1.intersects(poly2): # this test is fast and can accelerate calculation
        iou = 0
      else:
        try:
          inter_area = poly1.intersection(poly2).area
          union_area = poly1.area + poly2.area - inter_area
          iou = float(inter_area) / float(union_area)
        except shapely.geos.TopologicalError:
          warnings.warn("'shapely.geos.TopologicalError occured, iou set to 0'", UserWarning)
          iou = 0
        except ZeroDivisionError:
          iou = 0
      return iou
    
    def polygon_from_array(poly_):
      """
      Create a shapely polygon object from gt or dt line.
      """
      polygon_points = np.array(poly_).reshape(4, 2)
      polygon = shpoly(polygon_points).convex_hull
      return polygon
    
    def nms(dets, thresh):
        scores = dets[:, 8]
        order = scores.argsort()[::-1]
        polys = []
        areas = []
        for i in range(len(dets)):
            tm_polygon = polygon_from_array(dets[i,:8])
            polys.append(tm_polygon)
        keep = []
        while order.size > 0:
            ovr = []
            i = order[0]
            keep.append(i)
            for j in range(order.size - 1):
                iou = polygon_iou(polys[i], polys[order[j + 1]])
                ovr.append(iou)
            ovr = np.array(ovr)
            inds = np.where(ovr <= thresh)[0]
            order = order[inds + 1]
        return keep
    
    
    #### GPU implementation
    import cupy as cp  
      
    def polygon_iou_gpu(poly1, poly2):
      """
      Intersection over union between two shapely polygons.
      """
      if not poly1.intersects(poly2): # this test is fast and can accelerate calculation
        iou = 0
      else:
        try:
          inter_area = poly1.intersection(poly2).area
          union_area = poly1.area + poly2.area - inter_area
          iou = float(inter_area) / float(union_area)
        except shapely.geos.TopologicalError:
          warnings.warn("'shapely.geos.TopologicalError occured, iou set to 0'", UserWarning)
          iou = 0
        except ZeroDivisionError:
          iou = 0
      return iou
    
    def polygon_from_array_gpu(poly_):
      """
      Create a shapely polygon object from gt or dt line.
      """
      polygon_points = cp.array(poly_).reshape(4, 2)
      polygon = shpoly(polygon_points).convex_hull
      return polygon
    
    def nms_gpu(dets, thresh):
        scores = dets[:, 8]
        order = scores.argsort()[::-1]
        polys = []
        areas = []
        for i in range(len(dets)):
            tm_polygon = polygon_from_array_gpu(dets[i,:8])
            polys.append(tm_polygon)
        keep = []
        while order.size > 0:
            ovr = []
            i = order[0]
            keep.append(i)
            for j in range(order.size - 1):   
                iou = polygon_iou_gpu(polys[i], polys[order[j + 1]])
                ovr.append(iou)
            ovr = np.array(ovr)
            inds = np.where(ovr <= thresh)[0]
            order = order[inds + 1]
        return keep
    
    
    if __name__ == '__main__':
        import random
        boxes = np.random.randint(0,100,(1000,8))
        scores = np.random.rand(1000, 1)
        dets = np.hstack((boxes, scores[:])).astype(np.float32)

    
        thresh = 0.1
        start = time.time()
        keep = nms(dets, thresh)
        print("CPU implementation took: {}".format(time.time() - start))
    
        cp.cuda.Device(1)
        dets_gpu = cp.array(dets)
        start = time.time()
        keep = nms_gpu(dets_gpu, thresh)
        print("GPU implementation took: {}".format(time.time() - start))

错误是

CPU 实现占用：0.3672311305999756

回溯（最近一次通话最后）：

文件“nms_rotated.py”，第 117 行，在

keep = nms_gpu(dets_gpu, thresh)

文件“nms_rotated.py”，第 97 行，在 nms_gpu 中

iou = polygon_iou_gpu(polys[i], polys[order[j + 1]])

TypeError：列表索引必须是整数或切片，而不是 cupy.core.core.ndarray

更新：13.02.2019 我试过@Yuki Hashimoto 的回答

通过替换iou = polygon_iou_gpu(polys[i], polys[order[j + 1]])为 iou = polygon_iou_gpu(polys[i.get()], polys[order[j + 1].get()]). 它不会抛出任何错误，但 GPU 版本比 CPU 版本慢几倍。

通过使用 100000 次随机检测：

      CPU implementation took: 47.125494956970215
      GPU implementation took: 142.08464860916138
4

2 回答 2

2

简而言之：使用PFN官方的非最大抑制。

详细信息： Use cp.where，它返回一个list匹配某个条件的对象。

corochann不推荐 ' 的答案，因为它polys是一个列表，list也不应该被np.ndarray任何一个切片。（并且不建议注入另一个依赖项......）

>>> polys[order.get()]  # get method returns np.ndarray
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: only integer scalar arrays can be converted to a scalar index
>>> polys[order[j + 1].get()]
### some result in some case, but this may fails depending on your env.###
于 2019-02-12T02:00:02.203 回答
0

[更新 2019/2/13]

请参考@yuki-hashimoto 的回答，比较合适。

如错误消息中所述

TypeError：列表索引必须是整数或切片，而不是 cupy.core.core.ndarray

我猜order是cupy数组？在这种情况下polys[order[j + 1]]，使用索引order[j+1]作为可能会导致问题的 Cupy 数组。如何尝试通过cuda.to_cpu(array)方法将它们转换为 numpy 数组？

from chainer import cuda
iou = polygon_iou_gpu(polys[i], polys[cuda.to_cpu(order[j + 1])])
于 2019-02-11T01:52:26.060 回答