8

我正在尝试让一个简单的矩形跟踪控制器运行，并且我可以很好地进行矩形检测，但是跟踪请求总是以我无法找到的原因而失败。

有时跟踪请求会在失败之前触发它的回调几次，有时它会在单个回调发生之前立即失败。我觉得这与我提交请求的方式有关，但我无法深入了解它。

这是视图控制器的代码

class TestController: UIViewController, AVCaptureVideoDataOutputSampleBufferDelegate {

    // Video capture
    private var videoSession = AVCaptureSession()
    private var videoLayer: AVCaptureVideoPreviewLayer!

    // Detection
    private var detectionRequest: VNDetectRectanglesRequest?
    private let sequenceHandler = VNSequenceRequestHandler()

    // Tracking
    private var trackingRequest: VNTrackRectangleRequest?
    private var shape: Detection?
    private var pixelBuffer: CVImageBuffer?




    // MARK: Setup
    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)

        startVideoFeed()
    }

    override func viewDidLayoutSubviews() {
        videoLayer.frame = view.layer.bounds
    }

    override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
        startDetectingRectangles()
    }



    private func startDetectingRectangles() {
        let request = VNDetectRectanglesRequest(completionHandler: didDetectRectangle)
        request.maximumObservations = 1
        request.minimumSize = 0.07
        request.minimumConfidence = 0.9
        request.minimumAspectRatio = 0.5
        request.maximumAspectRatio = 2
        request.quadratureTolerance = 10
        detectionRequest = request
    }

    private func didDetectRectangle(request: VNRequest, error: Error?) {

        // Fetch results of the correct type
        guard let observations = request.results, observations.count > 0 else { return }
        let results = observations.map { $0 as? VNRectangleObservation }

        for case let rectangle? in results {
            detectionRequest = nil

            let request = VNTrackRectangleRequest(rectangleObservation: rectangle, completionHandler: didTrackRectangle)
            trackingRequest = request
        }
    }


    private func didTrackRectangle(request: VNRequest, error: Error?) {

        // Fetch results of the correct type
        guard let observation = request.results?.first as? VNRectangleObservation else { return }

        // Create or update UI
    }




    // Start capturing video frames
    private func startVideoFeed() {

        // Session config
        videoSession.sessionPreset = .photo

        // Create device and input to device
        guard
            let captureDevice = AVCaptureDevice.default(for: .video),
            let deviceInput = try? AVCaptureDeviceInput(device: captureDevice)
            else {
                fatalError("Error setting up capture device.")
        }

        // Setup device output
        let deviceOutput = AVCaptureVideoDataOutput()
        deviceOutput.setSampleBufferDelegate(self, queue: DispatchQueue.global(qos: .default))

        // Set input and output
        videoSession.addInput(deviceInput)
        videoSession.addOutput(deviceOutput)

        // Setup video display layer
        videoLayer = AVCaptureVideoPreviewLayer(session: videoSession)
        videoLayer.frame = view.bounds
        videoLayer.videoGravity = .resizeAspectFill
        view.layer.addSublayer(videoLayer)

        videoSession.startRunning()
    }

    func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {

        pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)

        guard let pixelBuffer = pixelBuffer else { return }
        do {

            var requests: [VNRequest] = []

            if let detectionRequest = detectionRequest {
                requests.append(detectionRequest) }
            if let trackingRequest = trackingRequest {
                requests.append(trackingRequest) }

            try sequenceHandler.perform(requests, on: pixelBuffer, orientation: .right)
        } catch {
            print(error)
        }
    }
}

错误正在打印：

错误域 = com.apple.vis 代码 = 9“内部错误：跟踪其中一个角失败，置信度 = 0.000000；阈值 = 0.650000” UserInfo={NSLocalizedDescription=内部错误：跟踪其中一个角失败，置信度 = 0.000000 ; 阈值 = 0.650000}

这是在跟踪请求第一次失败后的每一帧输出的，并且请求永远不会自行恢复。这try sequenceHandler.perform(requests, on: pixelBuffer, orientation: .right)是引发错误的行（然后我打印）。

我查看了散布在网络上的对象观察示例，它们似乎需要在每一帧都发出新的跟踪请求来跟踪对象。我试过VNTrackRectangleRequest在didTrackRectangle函数中创建一个新的，所以每个帧都有一个新的请求，但我遇到了同样的问题。

非常感谢您对此的任何帮助。

4

2 回答 2

0

我每帧都看到这个错误，为了修复它，我发现我需要performRequests:onCVPixelBuffer:orientation:error:使用正确的orientation.

（对于使用后置摄像头的纵向，我相信方向是kCGImagePropertyOrientationRight）。

从文档中：

对于每个这样的请求，调用序列请求处理程序的 performRequests:onCVPixelBuffer:orientation:error: 方法，确保传入视频阅读器的方向以确保垂直跟踪。

于 2019-09-07T01:34:55.330 回答
0

至少在跟踪清晰可见的矩形时，我能够毫无问题地运行您的代码。

VNSequenceRequestHandler当跟踪请求失败时，您可能应该创建一个新的。似乎当跟踪请求失去对对象的跟踪时间过长时，它将无法恢复并再次开始跟踪它。这也意味着它们永远不会被释放，并且会不断抛出错误。如果您继续向请求处理程序添加新的跟踪请求，您将很快超过同时跟踪请求的限制。

如果您能够预料到跟踪序列的结束，您可以将跟踪请求的isLastFrametrue属性设置为，当当前帧完成处理时，它将被释放到可用跟踪器池中。

于 2019-01-15T19:59:37.353 回答