ios - 动作 ML 分类器未给出预期结果

Question

我正在创建一个检测练习的应用程序。我使用 create ML 训练了模型。我在创建 ML 应用程序时得到了 100% 的结果。但是当我使用Vision框架集成到应用程序中时，它总是只显示一个练习。我完全按照Build an Action Classifier with Create ML中的代码来创建 ml 和请求VNHumanBodyPoseObservation。按照此转换VNHumanBodyPoseObservation为MLMultiArray.

这是我所做的代码：

func didOutput(pixelBuffer: CVPixelBuffer) {
    self.extractPoses(pixelBuffer)
}
func extractPoses(_ pixelBuffer: CVPixelBuffer) {
        let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer)
    let request = VNDetectHumanBodyPoseRequest { (request, err) in
        if err == nil {
            if let observations =
                request.results as? [VNRecognizedPointsObservation], observations.count > 0 {
                if let prediction = try? self.makePrediction(observations) {
                    print("\(prediction.label), confidence: \(prediction.confidence)")
                }
            }
        }
    }
      do {
        // Perform the body pose-detection request.
        try handler.perform([request])
      } catch {
        print("Unable to perform the request: \(error).\n")
      }
}


func makePrediction(_ observations: [VNRecognizedPointsObservation]) throws -> (label: String, confidence: Double) {
    let fitnessClassifier = try PlayerExcercise(configuration: MLModelConfiguration())

        let numAvailableFrames = observations.count
        let observationsNeeded = 60
        var multiArrayBuffer = [MLMultiArray]()

        for frameIndex in 0 ..< min(numAvailableFrames, observationsNeeded) {
            let pose = observations[frameIndex]
            do {
                let oneFrameMultiArray = try pose.keypointsMultiArray()
                multiArrayBuffer.append(oneFrameMultiArray)
            } catch {
                continue
            }
        }
        
        // If poseWindow does not have enough frames (45) yet, we need to pad 0s
        if numAvailableFrames < observationsNeeded {
            for _ in 0 ..< (observationsNeeded - numAvailableFrames) {
                do {
                    let oneFrameMultiArray = try MLMultiArray(shape: [1, 3, 18], dataType: .double)
                    try resetMultiArray(oneFrameMultiArray)
                    multiArrayBuffer.append(oneFrameMultiArray)
                } catch {
                    continue
                }
            }
        }
    let modelInput = MLMultiArray(concatenating: [MLMultiArray](multiArrayBuffer), axis: 0, dataType: .float)
//
//
let predictions = try fitnessClassifier.prediction(poses: modelInput)

return (label: predictions.label, confidence: predictions.labelProbabilities[predictions.label]!)

}

func resetMultiArray(_ predictionWindow: MLMultiArray, with value: Double = 0.0) throws {
    let pointer = try UnsafeMutableBufferPointer<Double>(predictionWindow)
    pointer.initialize(repeating: value)
}

我怀疑将 VNRecognizedPointsObservation 转换为 MLMultiArray 时会出现问题，请帮助我，我正在努力实现这一目标。提前致谢。

Answer 1

您是否在模拟器上运行您的应用程序？因为当我在 iPhone 12 模拟器上运行图像分类器应用程序时，我遇到了模型预测错误结果的问题。但是当我尝试在真实设备上运行该应用程序时，问题就解决了。因此，也许您的模型或代码没有任何问题，请尝试在真实设备上运行它，看看是否获得了预期的结果。