0

我正在尝试将用户的声音与音乐混合并将其保存到文件中。

我创建了 2 个解码器 - 1 个用于语音，1 个用于音乐，并将它们放入混音器的输入中。我解码每一帧并使用 FILE/createWAV/fwrite 将其保存到文件中。

当我的歌曲是 .wav 并且具有与录制的语音 (48000/1024) 相同的 sampleRate 和 samplesPerFrame 时，一切正常。

但是，当我想使用具有不同参数 (44100/1152) 的 .mp3 文件时，最终文件不正确 - 它被拉伸或有一些噼啪声。我认为这是因为我们为每个解码器获得了不同的 sampledDecoded，当它被放入 Mixer 或保存到文件时 - 这些样本之间的差异丢失了。

就我而言，当我们这样做时，它voiceDecoder->decode(buffer, &samplesDecoded)会移动。samplePositionsamplesDecoded

我试图做的是使用两个解码器的最小值。但是根据上面的句子，每次循环迭代歌曲都会丢失 (1152 - 1024 = 128) 128 个样本，所以我也尝试将 songDecoder 寻求与 voiceDecoder 相同：songDecoder->seek(voiceDecoder->samplePosition, true)但它导致文件完全不正确。

总结一下：当每个解码器都有不同的 sampleRate 和 samplesPerFrame 时，我应该如何使用 2 个解码器处理混合器/离线处理？

代码：

void AudioProcessor::startProcessing() {
    SuperpoweredStereoMixer *mixer = new SuperpoweredStereoMixer();
    float *mixerInputs_[] = {0,0,0,0};
    float *mixerOutputs_[] = {0,0};
    float inputLevels_[]= {0.5f, 0.5f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f};
    float outputLevels_[] = { 1.0f, 1.0f };

    SuperpoweredDecoder *voiceDecoder = new SuperpoweredDecoder();
    SuperpoweredDecoder *songDecoder = new SuperpoweredDecoder();

    if (voiceDecoder->open(voiceInputPath, false) || songDecoder->open(songInputPath, false, songOffset, songLength)) {
        delete voiceDecoder;
        delete songDecoder;
        delete mixer;
        callJavaVoidMethodWithBoolParam(jvm, jObject, processingFinishedMethodId, false);
        return;
    };

    FILE *fd = createWAV(outputPath, songDecoder->samplerate, 2);
    if (!fd) {
        delete voiceDecoder;
        delete songDecoder;
        delete mixer;
        callJavaVoidMethodWithBoolParam(jvm, jObject, processingFinishedMethodId, false);
        return;
    };

    // Create a buffer for the 16-bit integer samples coming from the decoder.
    short int *voiceIntBuffer = (short int *)malloc(voiceDecoder->samplesPerFrame * 4 * sizeof(short int) + 32768);
    short int *songIntBuffer = (short int *)malloc(songDecoder->samplesPerFrame * 4 * sizeof(short int) + 32768);
    short int *outputIntBuffer = (short int *)malloc(voiceDecoder->samplesPerFrame * 4 * sizeof(short int) + 32768);

    // Create a buffer for the 32-bit floating point samples required by the effect.
    float *voiceFloatBuffer = (float *)malloc(voiceDecoder->samplesPerFrame * 4 * sizeof(float) + 32768);
    float *songFloatBuffer = (float *)malloc(songDecoder->samplesPerFrame * 4 * sizeof(float) + 32768);
    float *outputFloatBuffer = (float *)malloc(voiceDecoder->samplesPerFrame * 4 * sizeof(float) + 32768);

    bool isError = false;

    // Processing.
    while (true) {
        if (isCanceled) {
            isError = true;
            break;
        }

        // Decode one frame. samplesDecoded will be overwritten with the actual decoded number of samples.
        unsigned int voiceSamplesDecoded = voiceDecoder->samplesPerFrame;
        if (voiceDecoder->decode(voiceIntBuffer, &voiceSamplesDecoded) == SUPERPOWEREDDECODER_ERROR) {
            break;
        }
        if (voiceSamplesDecoded < 1) {
            break;
        }

        //
        // Decode one frame. samplesDecoded will be overwritten with the actual decoded number of samples.
        unsigned int songSamplesDecoded = songDecoder->samplesPerFrame;
        if (songDecoder->decode(songIntBuffer, &songSamplesDecoded) == SUPERPOWEREDDECODER_ERROR) {
            break;
        }
        if (songSamplesDecoded < 1) {
            break;
        }

        unsigned int samplesDecoded = static_cast<unsigned int>(fmin(voiceSamplesDecoded, songSamplesDecoded));

        // Convert the decoded PCM samples from 16-bit integer to 32-bit floating point.
        SuperpoweredShortIntToFloat(voiceIntBuffer, voiceFloatBuffer, samplesDecoded);
        SuperpoweredShortIntToFloat(songIntBuffer, songFloatBuffer, samplesDecoded);

        //setup mixer inputs
        mixerInputs_[0] = voiceFloatBuffer;
        mixerInputs_[1] = songFloatBuffer;
        mixerInputs_[2] = NULL;
        mixerInputs_[3] = NULL;

        // setup mixer outputs, might have two separate outputs (L/R) if second not null
        mixerOutputs_[0] = outputFloatBuffer;
        mixerOutputs_[1] = NULL;

        mixer->process(mixerInputs_, mixerOutputs_, inputLevels_, outputLevels_, NULL, NULL, samplesDecoded);

        // Convert the PCM samples from 32-bit floating point to 16-bit integer.
        SuperpoweredFloatToShortInt(outputFloatBuffer, outputIntBuffer, samplesDecoded);

        // Write the audio to disk.
        fwrite(outputIntBuffer, 1, samplesDecoded * 4, fd);

        // songDecoder->seek(voiceDecoder->samplePosition, true);
    }

    // Cleanup.
    closeWAV(fd);
    delete voiceDecoder;
    delete songDecoder;
    delete mixer;
    free(voiceIntBuffer);
    free(voiceFloatBuffer);
    free(songIntBuffer);
    free(songFloatBuffer);
    free(outputFloatBuffer);
    free(outputIntBuffer);
}

提前致谢！

4

2 回答 2

0

您需要使用 SuperpoweredResampler 类匹配采样率。您还需要为两个输入提供一些循环缓冲区，因为在许多情况下可用的样本数不匹配。

于 2019-02-12T12:23:48.687 回答
0

好的，所以我设法让它工作。我按照@Gabor 的建议做了，但它并没有完全发挥作用。我缺少的是频道——我必须将它包含在我的缓冲/移位操作中，现在很好了！

于 2019-03-05T17:18:21.443 回答