javascript - Posenet P5.js 的视频输入（而不是网络摄像头）

Question

我正在尝试创建一个网络程序，该程序使用 P5.js 和 ML5.js 中的 Posenet 使用身体的特定部分进行绘制。我已经成功创建了网络模型，该模型在设置中使用 createCapture(VIDEO) 使用实时摄像机源，如下所示

function setup() {
  socket = io();
  cnv = createCanvas(windowWidth,windowHeight);
  capture = createCapture(VIDEO);
  background(255);
  
  PoseZero.init();
  
  local = new Agent();

    socket.emit('game-start', local.data)
    socket.on('heartbeat', function(data){
        world = data;
    })
  
}

var localPreviousPose = {};
var remotePreviousPose = {};

function draw() {
  
  image(capture, 0, 0, 220, 140);

（完整的代码可以在https://glitch.com/edit/#!/makeyourmark-final看到）

但是，为了更容易测试，我想在本地使用本地视频，我尝试以这种方式实现它：

function setup() {
  socket = io();

  cnv = createCanvas(windowWidth, windowHeight);
  capture = createVideo(
    [
      "https://cdn.glitch.com/e36b62ca-cf57-47f9-a16d-caa525a0d130%2FScreen%20Recording%202021-05-25%20at%2013.06.25.MOV?v=1621952189772"
    ],
    onVideoLoad
  );
  background(255);
  PoseZero.init();

  local = new Agent();

  socket.emit("game-start", local.data);
  socket.on("heartbeat", function(data) {
    world = data;
  });
}

var localPreviousPose = {};
var remotePreviousPose = {};

function onVideoLoad() {
  // The media will play as soon as it is loaded.
  capture.autoplay();
  capture.loop();
}

function draw() {
  image(capture, 0, 0, 220, 140);

（完整的代码可以在https://glitch.com/edit/#!/makeyourmark-video看到）

该错误似乎是不相关的（例如“未捕获的类型错误：无法读取未定义的属性‘自动播放’”，当它已经在自动播放时）

有什么办法可以解决这个问题吗？任何见解/解决方案表示赞赏。

谢谢！

score 1 · Accepted Answer

您的代码有几个问题，在您的帖子中包含的内容中没有一个问题很明显：

缺少.bind(this)对pose.js中回调函数的调用

在pose.js中声明的类的init函数中，您将onVideoLoad函数作为回调传递给createVideo。但是 onVideoLoad 函数引用this. 任何时候你this要从作为回调传递的函数中使用，你需要调用.bind(this)：

  this.init = function() {
    this.video = createVideo(
      ['https://cdn.glitch.com/e36b62ca-cf57-47f9-a16d-caa525a0d130%2FScreen%20Recording%202021-05-25%20at%2013.06.25.MOV?v=1621952189772'],
      // !!! Here, you need to use onVideoLoad.bind(this)
      onVideoLoad
    );

    // ...
  }
  
  function onVideoLoad() {
    // The media will play as soon as it is loaded.
    this.video.autoplay();
    this.video.loop();
  }

i因未声明而损坏的代码

在 client.js 中的 draw() 函数中，您多次引用world[i].data.pose但尚未声明i.

调用到pop()没有匹配的调用到push()

在pose.js中，您必须遵循以下代码：


    push();
    blendMode(MULTIPLY);
    // line(pose.leftWrist.x, pose.leftWrist.y, previousPose.data.leftWrist.x, previousPose.data.leftWrist.y);

    colorMode(HSB, 255);
    fill.apply(this, args.color);
    noStroke();
    beginShape();
    vertex(previousPose.data.leftWrist.x, previousPose.data.leftWrist.y);
    vertex(pose.leftWrist.x, pose.leftWrist.y);
    vertex(pose.leftElbow.x, pose.leftElbow.y);
    vertex(previousPose.data.leftElbow.x, previousPose.data.leftElbow.y);
    endShape(CLOSE);
    pop();
    
    // Presumably there should be another call to push() here
    colorMode(HSB, 255);
    fill.apply(this, args.color);
    noStroke();
    beginShape();
    vertex(previousPose.data.rightWrist.x, previousPose.data.rightWrist.y);
    vertex(pose.rightWrist.x, pose.rightWrist.y);
    vertex(pose.rightElbow.x, pose.rightElbow.y);
    vertex(previousPose.data.rightElbow.x, previousPose.data.rightElbow.y);
    endShape(CLOSE);
    pop();

视频播放需要在用户交互时启动

加载视频后立即调用loop()并非在所有情况下都有效。相反，您应该提示用户单击以启动播放。

工作混音

这是我认为“有效”的混音，尽管我不是 100% 了解您的预期行为：https ://glitch.com/edit/#!/stackoverflow-67694586 。我做了一些可能不是您想要的更改，例如使其创建单个视频元素并将其传递给 PoseZero.init（而不是 PoseZero 拥有自己的视频），并隐藏视频元素，因为您正在绘图无论如何，它到画布上。

结论

我认为您应该注意以下建议：

将/* globals ... */注释添加到使用 p5.js 的 javascript 文件（下面的完整示例）中，以便在您使用 p5.js 全局变量而不声明它们的情况下，故障不会显示错误。这样，您确实会看到错误，它们将是有意义的。
单击常规上的格式化此文件按钮。这将使您的代码更易于阅读。
注意javascript控制台。当它开始发出警告时，不要忽略它，修复它们。

这是 p5.js 的示例全局注释（可能并不详尽）：

/* globals ADD, ALT, ARROW, AUDIO, AUTO, AXES, BACKSPACE, BASELINE, BEVEL, BEZIER, BLEND, BLUR, BOLD, BOLDITALIC, BOTTOM, BURN, CENTER, CHORD, CLAMP, CLOSE, CONTROL, CORNER, CORNERS, CROSS, CURVE, DARKEST, DEGREES, DEG_TO_RAD, DELETE, DIFFERENCE, DILATE, DODGE, DOWN_ARROW, ENTER, ERODE, ESCAPE, EXCLUSION, FALLBACK, FILL, GRAY, GRID, HALF_PI, HAND, HARD_LIGHT, HSB, HSL, IMAGE, IMMEDIATE, INVERT, ITALIC, LABEL, LANDSCAPE, LEFT, LEFT_ARROW, LIGHTEST, LINEAR, LINES, LINE_LOOP, LINE_STRIP, MIRROR, MITER, MOVE, MULTIPLY, NEAREST, NORMAL, OPAQUE, OPEN, OPTION, OVERLAY, P2D, PI, PIE, POINTS, PORTRAIT, POSTERIZE, PROJECT, QUADRATIC, QUADS, QUAD_STRIP, QUARTER_PI, RADIANS, RADIUS, RAD_TO_DEG, REMOVE, REPEAT, REPLACE, RETURN, RGB, RIGHT, RIGHT_ARROW, ROUND, SCREEN, SHIFT, SOFT_LIGHT, SQUARE, STROKE, SUBTRACT, TAB, TAU, TESS, TEXT, TEXTURE, THRESHOLD, TOP, TRIANGLES, TRIANGLE_FAN, TRIANGLE_STRIP, TWO_PI, UP_ARROW, VIDEO, WAIT, WEBGL, accelerationX, accelerationY, accelerationZ, deltaTime, deviceOrientation, displayHeight, displayWidth, focused, frameCount, height, isKeyPressed, key, keyCode, keyIsPressed, mouseButton, mouseIsPressed, mouseX, mouseY, movedX, movedY, pAccelerationX, pAccelerationY, pAccelerationZ, pRotateDirectionX, pRotateDirectionY, pRotateDirectionZ, pRotationX, pRotationY, pRotationZ, pixels, pmouseX, pmouseY, pwinMouseX, pwinMouseY, rotationX, rotationY, rotationZ, touches, turnAxis, width, winMouseX, winMouseY, windowHeight, windowWidth, abs, acos, alpha, ambientLight, ambientMaterial, angleMode, append, applyMatrix, arc, arrayCopy, asin, atan, atan2, background, beginContour, beginShape, bezier, bezierDetail, bezierPoint, bezierTangent, bezierVertex, blend, blendMode, blue, boolean, box, brightness, byte, camera, ceil, char, circle, clear, clearStorage, color, colorMode, concat, cone, constrain, copy, cos, createA, createAudio, createButton, createCamera, createCanvas, createCapture, createCheckbox, createColorPicker, createDiv, createElement, createFileInput, createGraphics, createImage, createImg, createInput, createNumberDict, createP, createRadio, createSelect, createShader, createSlider, createSpan, createStringDict, createVector, createVideo, createWriter, cursor, curve, curveDetail, curvePoint, curveTangent, curveTightness, curveVertex, cylinder, day, debugMode, degrees, describe, describeElement, directionalLight, displayDensity, dist, downloadFile, ellipse, ellipseMode, ellipsoid, emissiveMaterial, endContour, endShape, erase, exitPointerLock, exp, fill, filter, float, floor, fract, frameRate, frustum, fullscreen, get, getFrameRate, getItem, getURL, getURLParams, getURLPath, green, gridOutput, hex, hour, httpDo, httpGet, httpPost, hue, image, imageMode, int, isLooping, join, keyIsDown, lerp, lerpColor, lightFalloff, lightness, lights, line, loadBytes, loadFont, loadImage, loadJSON, loadModel, loadPixels, loadShader, loadStrings, loadTable, loadXML, log, loop, mag, map, match, matchAll, max, millis, min, minute, model, month, nf, nfc, nfp, nfs, noCanvas, noCursor, noDebugMode, noErase, noFill, noLights, noLoop, noSmooth, noStroke, noTint, noise, noiseDetail, noiseSeed, norm, normalMaterial, orbitControl, ortho, perspective, pixelDensity, plane, point, pointLight, pop, popMatrix, popStyle, pow, print, push, pushMatrix, pushStyle, quad, quadraticVertex, radians, random, randomGaussian, randomSeed, rect, rectMode, red, redraw, registerPromisePreload, removeElements, removeItem, requestPointerLock, resetMatrix, resetShader, resizeCanvas, reverse, rotate, rotateX, rotateY, rotateZ, round, saturation, save, saveCanvas, saveFrames, saveGif, saveJSON, saveJSONArray, saveJSONObject, saveStrings, saveTable, scale, second, select, selectAll, set, setAttributes, setCamera, setFrameRate, setMoveThreshold, setShakeThreshold, shader, shearX, shearY, shininess, shorten, shuffle, sin, smooth, sort, specularColor, specularMaterial, sphere, splice, split, splitTokens, spotLight, sq, sqrt, square, storeItem, str, stroke, strokeCap, strokeJoin, strokeWeight, subset, tan, text, textAlign, textAscent, textDescent, textFont, textLeading, textOutput, textSize, textStyle, textWidth, texture, textureMode, textureWrap, tint, torus, translate, triangle, trim, unchar, unhex, updatePixels, vertex, writeFile, year */

javascript - Posenet P5.js 的视频输入（而不是网络摄像头）

1 回答 1

工作混音

结论

Related

Reference