17

是否可以<audio/>通过-element 加载音频文件createMediaElementSource，然后将音频数据加载到AudioBufferSourceNode 中？

使用音频元素作为源 (MediaElementSource) 似乎不是一种选择，因为我想使用 Buffer 方法noteOn和noteGrain.

不幸的是，通过 XHR 将音频文件直接加载到缓冲区也不是一种选择 （请参阅Open stream_url of a Soundcloud Track via Client-Side XHR?）

不过，从音频元素加载缓冲区内容似乎是可能的：

http://www.w3.org/2011/audio/wiki/Spec_Differences#Reading_Data_from_a_Media_Element

或者甚至可以直接使用 -element 的缓冲区<audio/>作为 sourceNode 吗？

4

4 回答 4

9

似乎无法从 MediaElementSourceNode 中提取音频缓冲区。

见https://groups.google.com/a/chromium.org/forum/?fromgroups#!topic/chromium-html5/HkX1sP8ONKs

非常欢迎任何证明我错的回复！

于 2012-07-14T22:45:24.237 回答
6

这个有可能。请参阅我在http://updates.html5rocks.com/2012/02/HTML5-audio-and-the-Web-Audio-API-are-BFFs上的帖子。那里还有一个代码片段和示例。有一些突出的错误，但将一个加载<audio>到 Web Audio API 应该可以按您的意愿工作。

// Create an <audio> element dynamically.
var audio = new Audio();
audio.src = 'myfile.mp3';
audio.controls = true;
audio.autoplay = true;
document.body.appendChild(audio);

var context = new webkitAudioContext();
var analyser = context.createAnalyser();

// Wait for window.onload to fire. See crbug.com/112368
window.addEventListener('load', function(e) {
  // Our <audio> element will be the audio source.
  var source = context.createMediaElementSource(audio);
  source.connect(analyser);
  analyser.connect(context.destination);

  // ...call requestAnimationFrame() and render the analyser's output to canvas.
}, false);
于 2012-07-08T13:52:38.503 回答
1

今天 2020+ 可以通过 audioWorklet 节点

https://developer.mozilla.org/en-US/docs/Web/API/AudioWorkletProcessor/AudioWorkletProcessor

在 AudioWorkletContext 中运行，只有这样你才能通过消息传递二进制原始数据

// test-processor.js
class RecorderWorkletProcessor extends AudioWorkletProcessor {
  constructor (options) {
    super()
    console.log(options.numberOfInputs)
    console.log(options.processorOptions.someUsefulVariable)
  }
  // @ts-ignore 
  process(inputs, output, parameters) {
      /**
      * @type {Float32Array} length 128 Float32Array(128)
      * non-interleaved IEEE754 32-bit linear PCM 
      * with a nominal range between -1 and +1, 
      * with each sample between -1.0 and 1.0.
      * the sample rate depends on the audioContext and is variable
      */
      const inputChannel = inputs[0][0];  //inputChannel Float32Array(128)
      const { postMessage } = this.port;
      postMessage(inputChannel)  // float32Array sent as byte[512] 
      return true; // always do this!
  }
}

主要代码

const audioContext = new AudioContext()

const audioMediaElement = audioContext.createMediaElementSource(
  /** @type {HTMLAudioElement} */ audio
);

await audioContext.audioWorklet.addModule('test-processor.js')
const recorder = new AudioWorkletNode(audioContext, 'test-processor', 
  {
     processorOptions: {
     someUsefulVariable: new Map([[1, 'one'], [2, 'two']])
  }
});

/**
 *   
 * Objects of these types are designed to hold small audio snippets, 
 * typically less than 45 s. For longer sounds, objects implementing 
 * the MediaElementAudioSourceNode are more suitable. 
 * The buffer contains data in the following format: 
 * non-interleaved IEEE754 32-bit linear PCM (LPCM)
 * with a nominal range between -1 and +1, that is, a 32-bit floating point buffer, 
 * with each sample between -1.0 and 1.0.  
 * @param {ArrayBufferLike|Float32Array} data 
 */
 const convertFloatToAudioBuffer = (data) => {
    const sampleRate = 8000 | audioContext.sampleRate
    const channels = 1;
    const sampleLength = 128 | data.length; // 1sec = sampleRate * 1
    const audioBuffer = audioContext.createBuffer(channels, sampleLength, sampleRate); // Empty Audio
    audioBuffer.copyToChannel(new Float32Array(data), 0); // depending on your processing this could be already a float32array
    return audioBuffer;
}
let startAt = 0
const streamDestination = audioContext.createMediaStreamDestination();
/**
 * Note this is a minimum example it plays only the first sound 
 * it uses the main audio context if it would use a
 * streamDestination = context.createMediaStreamDestination();
 * 
 * @param {ArrayBufferLike|Float32Array} data 
 */
const play = (data) => {
    const audioBufferSourceNoce = audioContext.createBufferSource();  
    audioBufferSourceNoce.buffer = convertFloatToAudioBuffer(data);

    const context = audioContext; // streamDestination; // creates a MediaStream on streamDestination.stream property
    audioBufferSourceNoce.connect(context);    

    // here you will need a hugh enqueue algo that is out of scope for this answer
    startAt = Math.max(context.currentTime, startAt);
    source.start(startAt);
    startAt += buffer.duration;
    audioBufferSourceNoce.start(startAt);
}

// Here is your raw arrayBuffer ev.data
recorder.port.onmessage = (ev) => play(ev.data);

// connect the processor with the source
audioMediaElement.connect(recorder);

笔记

这只是 console.log 记录来自记录器处理器的数据的最低要求，当您真的想要处理该数据时，我们只采用 1 个通道，您应该考虑在工作人员中再次注册一个将数据直接发布到的处理程序如果您进行大量处理，该工作人员以及您的主进程可能会变得无响应。

于 2022-01-18T10:00:04.863 回答
0

我不确定您是否找到了更好的解决方案，我还检查了您发布的 W3C 链接：http: //www.w3.org/2011/audio/wiki/Spec_Differences#Reading_Data_from_a_Media_Element

但为了让它真正起作用，你必须使用AudioContext.createScriptProcessor(). 我还没有尝试过，但基本上你将源节点（音频元素）连接到脚本处理器，但如果你不需要它甚至不输出音频。在onaudioprocess回调中，您可以直接访问音频缓冲区数据（当然是指定大小的块）。上面的链接中有例子。

此外，我认为您可以以某种方式调整播放速度，以便更快地获得更多缓冲区数组。

于 2016-03-13T09:52:55.483 回答