unity3d - Hololens 2 上的 LocatableCamera (PhotoCapture)

Question

在 Hololens 2 上使用 PhotoCapture 时出现 2 个问题，它们可能已连接。它们没有出现在 Hololens 1 上。

• 唯一可获得的分辨率是 3904x2196
• 获取 CameraToWorldMatrix 总是失败

从文档中可以看出，此分辨率没有与之关联的帧速率。我的假设是 CameraToWorldMatrix 仅适用于分辨率较低的相机配置文件。

如何更改分辨率并在 Unity 中获取矩阵？

---

最小可重现示例

我正在使用 Unity 2019.2.19f1 和 Visual Studio 2019 Community (16.4.5)

1. 按照此处的步骤创建一个新的 Unity 项目，除了使用 IL2CPP 作为脚本后端而不是 .NET
2. 在Player Settings: Capabilities enable InternetClient , InternetClientServer , PrivateNetworkClientServer , Webcam , Microphone and SpatialPerception , 在Player Settings: Supported Device Families中选择Holographic

3. 创建一个新的空游戏对象并添加以下脚本：

   using System.Collections;
   using System.Collections.Generic;
   using System.Linq;
   using UnityEngine;
   using UnityEngine.Windows.WebCam;
   
   public class PhotoCaptureController : MonoBehaviour
   {
       PhotoCapture photoCaptureObject = null;
   
       bool isRunning = false;
   
       void Start()
       {
           StartCoroutine(StartCameraCapture());
       }
   
       private IEnumerator StartCameraCapture()
       {
           if (!Application.HasUserAuthorization(UserAuthorization.WebCam))
           {
               yield return Application.RequestUserAuthorization(UserAuthorization.WebCam);
           }
           if (Application.HasUserAuthorization(UserAuthorization.WebCam))
           {
               Debug.Log("Creating PhotoCapture");
               PhotoCapture.CreateAsync(false, OnPhotoCaptureCreated);
           }
           else
           {
               Debug.Log("Webcam Permission not granted");
           }
       }
   
       private void Update()
       {
           if (isRunning)
           {
               photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
           }
       }
   
       void OnPhotoCaptureCreated(PhotoCapture captureObject)
       {
           photoCaptureObject = captureObject;
   
           IEnumerable<Resolution> availableResolutions = PhotoCapture.SupportedResolutions;
   
           foreach (var res in availableResolutions)
           {
               Debug.Log("PhotoCapture Resolution: " + res.width + "x" + res.height);
           }
   
           Resolution cameraResolution = availableResolutions.OrderByDescending((res) => res.width * res.height).First();
   
           CameraParameters c = new CameraParameters();
           c.hologramOpacity = 0.0f;
           c.cameraResolutionWidth = cameraResolution.width;
           c.cameraResolutionHeight = cameraResolution.height;
           c.pixelFormat = CapturePixelFormat.BGRA32;
   
           captureObject.StartPhotoModeAsync(c, OnPhotoModeStarted);
       }
   
       private void OnPhotoModeStarted(PhotoCapture.PhotoCaptureResult result)
       {
           if (result.success)
           {
               isRunning = true;
               photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
           }
       }
   
       void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame frame)
       {
           if (result.success)
           {
               if (frame.TryGetCameraToWorldMatrix(out Matrix4x4 cameraToWorldMatrix))
               {
                   Debug.Log("Successfully obtained CameraToWorldMatrix: " + cameraToWorldMatrix.ToString());
   
               }
               else
               {
                   Debug.Log("Failed to obtain CameraToWorldMatrix");
               }
           }
           frame.Dispose();
       }
   }
   

4. 更改构建设置：
5. 构建，打开 VS 解决方案，将构建目标设置为ARM64，调试并部署到设备

Answer 1

这个CameraCapture插件可能适合你。它是 Unity 的原生插件。readme.txt 包含有关构建它的信息。基本上，在尝试打开 Unity 示例之前，您需要构建原生插件变体。如果您尚未构建 C++/WinRT 组件，您可能需要完成此处的设置步骤。SpatialCameraTracker.cs 文件显示了如何接收相机矩阵。

对此的支持很少，但所有代码都在 repo 中