iphone - 根据声音频率移动图像？

Question

我正在尝试制作类似的东西，我正在录制声音，并且基于声音（音高，频率，不确定）图像应该移动。

我能够实现记录，我也可以进行图像序列，但是是分开的。我不知道如何链接它，只是为了提供信息，我正在尝试实现类似嘴巴移动应用程序： app url here

我的问题是，如何根据声音频率移动/动画图像。

谢谢

Answer 1

我完成了解决方案。使用狄拉克并解决了问题。

编辑：

它是什么？

DiracAudioPlayer 是一组新的 Cocoa 类，以一种方便的方式包装了整个 Dirac 功能，公开了一个类似于 AVAudioPlayer 提供的 API。请注意，这不是 AVAudioPlayer 子类。以下是 API 的核心功能和描述。

DiracAudioPlayer 核心功能

DiracAudioPlayer 是一组类，允许基于文件播放各种音频格式（包括 MPMediaItems），同时实时更改音频文件的速度和音高。3.6 版由 DiracAudioPlayerBase（负责文件 IO 和播放的基类）、DiracAudioPlayer（包装 Dirac Core API）和 DiracFxAudioPlayer（包装 DiracFx API）组成。

确保在项目中包含所有 3 个类以及“ExtAudioFile”和“util”文件夹，并将 Accelerate.framework 和 CoreAudio.framework 添加到项目中。在 MacOS X 上，您还必须添加 AudioUnit.framework，在 iOS 上，您必须添加 AudioToolbox.framework、AVFoundation.framework、MediaPlayer.framework 和 CoreMedia.framework。

DiracAudioPlayer 是……</p>

...一个与 Apple 兼容的类，用于播放时间延长的音频，适用于 iOS（版本 4 及更高版本）和 MacOS X（版本 10.6 及更高版本）......非常易于使用......完全兼容 ARC......提供给您，包括完整的源代码

DiracAudioPlayer API

2012 年 11 月发布的 3.6 版提供以下调用：

- (id) initWithContentsOfURL:(NSURL*)inUrl channels:(int)channels error: (NSError **)error;

Initializes and returns an audio player for playing a designated sound file. A URL identifying the sound file to play. The audio data must be in a format supported by Core Audio. Pass in the address of a nil-initialized NSError object. If an error occurs, upon return the NSError object describes the error. To use an item from the user's iPod library supply the URL that you get via MPMediaItem's MPMediaItemPropertyAssetURL property as inUrl. Note that FairPlay protected content can NOT be processed.


- (void) setDelegate:(id)delegate;
- (id) delegate;

设置/获取班级的代表。如果您实现委托协议，DiracAudioPlayer 将调用您的实现

- (void)diracPlayerDidFinishPlaying:(DiracAudioPlayerBase *)player successfully:(BOOL)flag

播放完毕后

- (void) changeDuration:(float)duration;
- (void) changePitch:(float)pitch;

更改播放速度和音高

- (NSInteger) numberOfLoops;
- (void) setNumberOfLoops:(NSInteger)loops;

A value of 0, which is the default, means to play the sound once. Set a positive integer value to specify the number of times to return to the start and play again. For example, specifying a value of 1 results in a total of two plays of the sound. Set any negative integer value to loop the sound indefinitely until you call the stop method.

- (void) updateMeters;

必须在调用 -peakPowerForChannel 之前调用以更新其内部测量值

- (float) peakPowerForChannel:(NSUInteger)channelNumber;

A floating-point representation, in decibels, of a given audio channel’s current peak power. A return value of 0 dB indicates full scale, or maximum power; a return value of -160 dB indicates minimum power (that is, near silence). If the signal provided to the audio player exceeds ±full scale, then the return value may exceed 0 (that is, it may enter the positive range). To obtain a current peak power value, you must call the updateMeters method before calling this method.


- (BOOL) prepareToPlay;

Starts the Dirac processing thread and prepares the sound file for playback. If you don't call this explicitly it will be called when calling -play


- (NSUInteger) numberOfChannels;

与音频播放器关联的声音中的音频通道数。（只读）

- (NSTimeInterval) fileDuration;

返回与音频播放器关联的声音的总持续时间（以秒为单位）。（只读）

- (NSTimeInterval) currentTime;
- (void) setCurrentTime:(NSTimeInterval)time

Returns the current play time in the input file. Note that if you apply time stretching, -currentTime will reflect the slowed down time depending on the time stretch factor. 

IMPORTANT CHANGE: In previous versions this value returned the total play time independent of the position in the file. Please update your code accordingly to reflect the change

Setting this property causes playback to fast forward or rewind to the specified play time.


- (void) play;

Plays a sound asynchronously. Returns YES on success, or NO on failure. Calling this method implicitly calls the -prepareToPlay method if the audio player is not already prepared to play.

- (NSURL*) url;

与音频播放器关联的声音的 URL。（只读）

- (void) setVolume:(float)volume;
- (float) volume;

音频播放器的播放增益，范围从 0.0 到 1.0。

- (BOOL) playing;

A Boolean value that indicates whether the audio player is playing (YES) or not (NO). (read-only). To find out when playback has stopped, use the diracPlayerDidFinishPlaying:successfully: delegate method.

- (void) pause;

Pauses playback; sound remains ready to resume playback from where it left off. Calling pause leaves the audio player prepared to play; it does not release the audio hardware that was acquired upon calling -play or -prepareToPlay.

- (void) stop;

Stops playback and undoes the setup needed for playback. Calling this method, or allowing a sound to finish playing, undoes the setup performed upon calling the -play or -prepareToPlay methods.

Answer 2

大多数文本到语音系统将允许您注册一个回调函数，该函数将向您发送正在产生的音素（用外行的话来说是声音）。看看下面的链接。单击左侧的回调。往下看 SpeechPhonemeProcPtr ，它允许您注册一个函数，当发出的噪音是“uh”、“th”、“ah”或任何噪音时将调用该函数。然后，您将更新您的图像，使其看起来像一个人在发出特定声音时的嘴巴的样子。这在 IBM 的 ViaVoice 中非常容易，而且我从未在 iPhone 上编写过这样的应用程序，但我认为这比尝试匹配音频要好。

如果这是您尝试匹配的真正未经过滤的音频，那么您可以将其传递给语音识别系统，并将识别的文本传递给 TTS 系统并获取音素。