android - 如何在 Android 中获取视频文件的关键帧（同步帧）时间戳列表？

Question

是否有任何 API 可以为我提供 Android 中给定视频文件的关键帧的时间戳列表？

Answer 1

For an elegant solution, it seems as if you'd need to use ffmpeg for android to achieve this. Someone else tried it that way: How to get the frame from video file in android ; and was suggested to use ffmpeg.

I checked the reference manual, but could only find a way to get a keyframe itself (without timestamp) relative to a given position in time using the MediaMetadataRetriever() as explained here: http://developer.android.com/reference/android/media/MediaMetadataRetriever.html#extractMetadata%28int%29

If you don't want to use ffmpeg you could try the following:

private ArrayList<int> getKeyFrameTimestamps(String filename) {
  ArrayList<int> timeStamps = new ArrayList<int>();

  try {
    MediaMetadataRetriever mRetriever = new MediaMetadataRetriever();
    mRetriever.setDataSource(getDataSource(filename));
    mRetriever.getFrameAtTime(i, MediaMetadataRetriever.OPTION_CLOSEST_SYNC);

    Bitmap lastFame = mRetriever.getFrameAtTime(0, MediaMetadataRetriever.OPTION_NEXT_SYNC);
    result.add(0);

    for(int time=1; frame ; ++time) {
      Bitmap frame = mRetriever.getFrameAtTime(time, MediaMetadataRetriever.OPTION_PREVIOUS_SYNC);
      if (frame && !frame.sameAs(lastFrame)) {
        result.add(time);
        lastFrame = frame;
      }
    }
  } catch (Exception e) {  /* TODO: HANDLE THIS */ }

  return timeStamps;
}

This is of course not very elegant, but would probably work without hassle to install ffmpeg... It's probably also quite slow (I don't know). But maybe it's just what you need.

PS: An ffmpeg tutorial that fits your problem can be found here: http://dranger.com/ffmpeg/tutorial07.html

Answer 2

我相信正确的答案是http://code.google.com/p/mp4parser/

isoparser API 可以读取和写入 MP4 文件结构。它是一个处理所谓的盒子的低级工具，但它也处理像轨道和电影这样的结构。

不错的 API 和比 ffmpeg 庞然大物快得多的方法

Answer 3

4.1+ - 比 mp4parser 更快，因为在本机下完成并且代码更干净

mp4parser 你必须从 getsyncsamples + getsampledurations 计算时间戳，这也很烦人，它们与 ffprobe 不匹配。这会得到准确的结果

MediaExtractor extractor = new MediaExtractor();
extractor.setDataSource(getVideoPath());

int trackindex = MediaExtractorUtil.selectVideoTrack(extractor);
extractor.selectTrack(trackindex);

while (extractor.getSampleTime() != -1) {
    long sampleTime = extractor.getSampleTime();

    // check not really necessary but JIC
    if ((extractor.getSampleFlags() & MediaExtractor.SAMPLE_FLAG_SYNC) > 0) {
        mKeyframeTimestampsMS.add(sampleTime / 1000);
    }

    extractor.seekTo(sampleTime + 1, MediaExtractor.SEEK_TO_NEXT_SYNC);
}

Answer 4

以@ryan gordon 的解决方案为基础。seekTo可能会寻找同一帧，即使SEEK_TO_NEXT_SYNC已指定。为避免这种情况，您可以在帧开始重复时终止。

例如：

    private fun getSyncFrameTimestamps(source: MediaExtractor): List<Long> {
        val result = mutableListOf<Long>()

        var lastSampleTime = -1L
        var sampleTime = source.sampleTime

        while (sampleTime >= 0L && sampleTime != lastSampleTime) {
            if (source.sampleFlags and MediaExtractor.SAMPLE_FLAG_SYNC != 0) {
                result.add(sampleTime)
            }
            source.seekTo(sampleTime + 1L, MediaExtractor.SEEK_TO_NEXT_SYNC)
            lastSampleTime = sampleTime
            sampleTime = source.sampleTime
        }

        return result
    }

请注意，在调用此之前，source应设置数据源并选择视频轨道。

Answer 5

使用MediaExtractor，您可以在任何所需时间拥有所有关键（同步）帧时间和/或任何特定关键帧。+一些更有用的数据。

然后通过 MediaMetadataRetriever -> getFrameAtTime 可以获得帧位图。

这是使用 MediaExtractor 的示例代码：

    mediaExtractor = new MediaExtractor() ;
    mediaExtractor.setDataSource(mediaPath);
    int numTracks = mediaExtractor.getTrackCount();
    for (int i = 0; i < numTracks; ++i) {
        MediaFormat format = mediaExtractor.getTrackFormat(i);
        String mime = format.getString(MediaFormat.KEY_MIME);
        if ( mime.startsWith("video") ) {
            mediaExtractor.selectTrack(i);
        }
        Log.i( "LOG_Extractor" , "track "+i+" "+mime );
    }

    currentTime = mediaPlayer.getCurrentPosition();
    mediaExtractor.seekTo( (long)currentTime*1000 , MediaExtractor.SEEK_TO_PREVIOUS_SYNC );
    previousKeyFrameTimeMilis = mediaExtractor.getSampleTime() /1000 ;