(Go: >> BACK << -|- >> HOME <<)
Page MenuHomePhabricator
Create Task
Maniphest T135671

Ogg theora .ogv video transcodes from webm sometimes 1000fps
Closed, ResolvedPublic
Actions

Description

Ogg theora video transcodes sometimes come out at 1000 fps, with lots of duplicate frame markers to maintain the actual frame rate.

https://commons.wikimedia.org/wiki/File:Latourell_Falls,_Oregon.webm
https://commons.wikimedia.org/wiki/File:Calligraphy_demonstration_at_the_Ukrainian_Cultural_Center_in_Tallinn,_Estonia.webm
Note both of these were uploaded through videoconvert on tool labs, so the webms produced there might be funky. The Latourell Falls video was shot on my Nexus 5x Android phone and uploaded from the .mp4 source.

Note also that webm/vp8 doesn't specify a frame rate the same way ogg theora does; frames are simply positioned in time. So there may be something funky about the webm files, and there may be something funky with the ogg conversion.

Details

SubjectRepoBranchLines +/-
Cap .ogv transcode frame rate to 60 fps (1000 fps bug)mediawiki/extensions/TimedMediaHandlerwmf/1.29.0-wmf.7+11 -6
Cap .ogv transcode frame rate to 60 fps (1000 fps bug)mediawiki/extensions/TimedMediaHandlermaster+11 -6
Customize query in gerrit

Related Objects
Search...

Event Timeline

TheDJ created this task.May 18 2016, 7:04 PM
Restricted Application added subscribers: Zppix, Base, Aklapper. · View Herald TranscriptMay 18 2016, 7:04 PM
Paladox subscribed.May 18 2016, 7:07 PM
zhuyifei1999 subscribed.May 31 2016, 3:42 PM
• brooke subscribed.Jun 2 2016, 7:10 PM

This seems to be an issue with ffmpeg's detection of the incoming frame rate; "1k fps" or "30k fps" shows up via ffprobe on offending source files.

Note that while playback can be ok with proper handling of dupe frame markers, there are bandwidth/quality issues with the resulting files because the max keyframe interval applies to the total frame count including those dropped frames! So a keyframe very 128 frames, on a ~30fps file at "1k fps" means roughly 1/4 of frames are going to be keyframes. That means either low quality (CBR) or inflated bandwidth (VBR).

WebM/Matroska container format and VP8/VP9 codecs are frame rate agnostic (packets specify a timestamp for each frame in absolute time units from stream start), so I can only assume that getting a frame rate *at all* is based on some heuristic, such as reading a few packets in and checking the timestamp deltas, which is failing on some files.

... looking at ffmpeg2theora's code I believe it's using the time_base (which is usually 1000 in WebM) because the avg_frame_rate reported by ffmpeg/avlib is even bigger (30k) instead of smaller (say, 30).

from ffprobe -show_streams on Latourell_Falls,_Oregon.webm:

r_frame_rate=30/1
avg_frame_rate=30000/1
time_base=1/1000

So the question is why is the avg_frame_rate super high?

The ff_rfps_calculate function that sets the avg_frame_rate property appears to take the overall estimate of frame rate from a portion of the file w/ known frame count & duration, and then tries to reduce it if necessary to a nicer fraction, including normalizing "very close to" some standard frame rates to the standard rates.

Passing -log_level 64 to ffprobe to force debug output, we see some of its output:

Probing matroska,webm score:100 size:2048
[matroska,webm @ 0x7ffa61800000] Format matroska,webm probed with size=2048 and score=100
st:0 removing common factor 1000000 from timebase
st:1 removing common factor 1000000 from timebase
[matroska,webm @ 0x7ffa61800000] Before avformat_find_stream_info() pos: 4313 bytes read:32768 seeks:0
[matroska,webm @ 0x7ffa61800000] All info found
[matroska,webm @ 0x7ffa61800000] rfps: 29.666667 0.016027
[matroska,webm @ 0x7ffa61800000] rfps: 29.750000 0.008954
[matroska,webm @ 0x7ffa61800000] rfps: 29.833333 0.003938
[matroska,webm @ 0x7ffa61800000] rfps: 29.916667 0.000979
[matroska,webm @ 0x7ffa61800000] rfps: 30.000000 0.000078
    Last message repeated 1 times
[matroska,webm @ 0x7ffa61800000] rfps: 60.000000 0.000311
    Last message repeated 1 times
[matroska,webm @ 0x7ffa61800000] rfps: 120.000000 0.001244
    Last message repeated 1 times
[matroska,webm @ 0x7ffa61800000] rfps: 240.000000 0.004976
[matroska,webm @ 0x7ffa61800000] rfps: 29.970030 0.000165
    Last message repeated 1 times
[matroska,webm @ 0x7ffa61800000] rfps: 59.940060 0.000661
    Last message repeated 1 times
[matroska,webm @ 0x7ffa61800000] 0: start_time: 0.000 duration: -9223372036854.775
[matroska,webm @ 0x7ffa61800000] 1: start_time: 0.000 duration: -9223372036854.775
[matroska,webm @ 0x7ffa61800000] stream: start_time: 0.000 duration: 17.813 bitrate=12132 kb/s
[matroska,webm @ 0x7ffa61800000] After avformat_find_stream_info() pos: 1985285 bytes read:1986183 seeks:0 frames:113
Input #0, matroska,webm, from 'Latourell_Falls,_Oregon.webm':

and

[matroska,webm @ 0x7f8e03008e00] Format matroska,webm probed with size=2048 and score=100
st:0 removing common factor 1000000 from timebase
st:1 removing common factor 1000000 from timebase
[matroska,webm @ 0x7f8e03008e00] Before avformat_find_stream_info() pos: 4299 bytes read:32768 seeks:0
[matroska,webm @ 0x7f8e03008e00] All info found
[matroska,webm @ 0x7f8e03008e00] rfps: 29.666667 0.016552
[matroska,webm @ 0x7f8e03008e00] rfps: 29.750000 0.009347
[matroska,webm @ 0x7f8e03008e00] rfps: 29.833333 0.004197
[matroska,webm @ 0x7f8e03008e00] rfps: 29.916667 0.001104
[matroska,webm @ 0x7f8e03008e00] rfps: 30.000000 0.000067
    Last message repeated 1 times
[matroska,webm @ 0x7f8e03008e00] rfps: 60.000000 0.000270
    Last message repeated 1 times
[matroska,webm @ 0x7f8e03008e00] rfps: 120.000000 0.001079
    Last message repeated 1 times
[matroska,webm @ 0x7f8e03008e00] rfps: 240.000000 0.004316
    Last message repeated 1 times
[matroska,webm @ 0x7f8e03008e00] rfps: 29.970030 0.000204
    Last message repeated 1 times
[matroska,webm @ 0x7f8e03008e00] rfps: 59.940060 0.000814
    Last message repeated 1 times
[matroska,webm @ 0x7f8e03008e00] 0: start_time: 0.000 duration: -9223372036854.775
[matroska,webm @ 0x7f8e03008e00] 1: start_time: 0.000 duration: -9223372036854.775
[matroska,webm @ 0x7f8e03008e00] stream: start_time: 0.000 duration: 56.640 bitrate=5105 kb/s
[matroska,webm @ 0x7f8e03008e00] After avformat_find_stream_info() pos: 792201 bytes read:819200 seeks:0 frames:112
Input #0, matroska,webm, from 'Calligraphy_demonstration_at_the_Ukrainian_Cultural_Center_in_Tallinn,_Estonia.webm':

This appears to be comparing the measured frame rate from demuxing the file against several standard-ish frame rates, of which 30.0 is closest, but it's still not saving the reduced value. So something's off that I can't quite see, will have to debug in detail later.

• brooke mentioned this in T155307: Transcode failed.Jan 14 2017, 5:03 PM
• brooke added a comment.Jan 14 2017, 5:29 PM

So it looks like a partial improvement may be to add -F 60 to ffmpeg2theora options; along with --no-upscaling this will ensure that input frame rates higher than 60 get reduced *to* 60, while lower framerates are kept as-is. That'll HUGELY reduce the number of dupe frames processed, which should make both encoding faster and playback more reliable and give a better keyframe distance (which may improve picture quality or file size compared to the 1000fps mode).

• brooke added a subtask: T155307: Transcode failed.Jan 14 2017, 5:35 PM
gerritbot subscribed.Jan 14 2017, 5:41 PM

Change 332066 had a related patch set uploaded (by Brion VIBBER):
Cap .ogv transcode frame rate to 60 fps (1000 fps bug)

https://gerrit.wikimedia.org/r/332066

gerritbot added a project: Patch-For-Review.Jan 14 2017, 5:41 PM
• brooke claimed this task.Jan 14 2017, 6:56 PM
gerritbot added a comment.Jan 14 2017, 7:28 PM

Change 332066 merged by jenkins-bot:
Cap .ogv transcode frame rate to 60 fps (1000 fps bug)

https://gerrit.wikimedia.org/r/332066

Yann subscribed.Jan 14 2017, 7:51 PM
ReleaseTaggerBot added a project: MW-1.29-release (WMF-deploy-2017-01-17_(1.29.0-wmf.8)).Jan 14 2017, 8:00 PM
TheDJ closed this task as Resolved.Jan 14 2017, 8:50 PM
TheDJ moved this task from To sort to Done on the TimedMediaHandler board.
gerritbot added a comment.Jan 14 2017, 10:46 PM

Change 332081 had a related patch set uploaded (by Paladox):
Cap .ogv transcode frame rate to 60 fps (1000 fps bug)

https://gerrit.wikimedia.org/r/332081

gerritbot added a comment.Jan 20 2017, 7:27 PM

Change 332081 abandoned by Chad:
Cap .ogv transcode frame rate to 60 fps (1000 fps bug)

Reason:
Already included in wmf.8, which is live everywhere.

https://gerrit.wikimedia.org/r/332081

TheDJ closed subtask T155307: Transcode failed as Resolved.Jan 26 2017, 12:50 PM
Aklapper removed a project: TimedMediaHandler.Nov 7 2019, 4:02 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits