"vop not coded" - Xvid problem?

[Rheinhard]

I believe something is seriously broken in the latest builds of XVID. I have experienced the same problem with the latest episode of Galaxy Railways from LiveEvil/AnimeOne and the latest episode of Planetes from AnimeEmpire. Both are encoded with XVID, and I noticed that both look much more jerky than any previous episode of either series from either group (played in VLC on Mac OS X). I just saw a post on a mailing list I run from another member who had the latest Galaxy Railways cause Windows Media Player to crash.

When running these I have noted "vop not coded" warnings. Most irritating to me personally is that I have for some time been using the open source software ffmpeg, mpeg2enc and y4mscaler to export the fansubs to mpeg2 in order to build normal video DVDs for set-top players, so I can share these series with friends who don't have powerful enough computers to watch them, and who, like me, prefer to watch anime on TV from the comfort of their living room rather than huddled around a CRT. I have been doing this for some time and never before these 2 series episodes encountered anything that caused ffmpeg and y4mscaler to outright crash.

I would appreciate any insight on this. If indeed this is a problem with the latest build of XVID, how can it be corrected? If not, what are possible alternative explanations?

[Cruzz]

Originally Posted by Rheinhard

I would appreciate any insight on this. If indeed this is a problem with the latest build of XVID, how can it be corrected? If not, what are possible alternative explanations?

The problem stems from an encoding option called "packed bitstream". For some moronic reason, it was made the default setting in the newest XVID build. Okay, so packed bitstream is not a bad thing per se, but the problem is that right now all libavcodec based decoding (ffmpeg, ffdshow, VideoLan to name the most well known) has problems with it when it's coupled with 2 or more consecutive B-frames (also a default setting).

Unfortunately there isn't a way to fix this right now apart from a total re-encode of the file. You can bypass the problem by using the xvid codec to decode xvid, but I'm not sure if your encoding applications permit that.

[Rheinhard]

Some more info from a friend of mine...

I did a Google search for "vop not closed" and ended up at the ffmpeg bug forum.

As I understand it, you can encode XviD packed or not. This may or may not
be a new feature; I think problems first showed up with stuff encoded by
XviD 1.0beta2 or thereabouts. Anyway, evidently because of an API constraint on Windows, there's no way to give the second VOP a header, and that's what causes the
errors with ffmpeg (and anything based off it). Some programs (like VLC)
keep ffmpeg quiet and just show the resulting stream, but it looks "jerky".
That's because every headerless VOP is a dropped frame. Yuck.

Windows users can probably just get the newest version of the XviD codec and
they'll be fine as long as the rest of their system doesn't get upset. The
rest of us will be better off once the ffmpeg guys get around to releasing
0.4.9.


...and once again, the root of the problem can be traced to the evil Empire in Redmond...

[outlaw55]

Gee, i don't see the problem with you ppl not using the XviD Direct Show Decoder for XviD 1.0.....it is what it was DESIGNED for......altho most ppl think ffdshow is GOD of decoders....ffdishow is good, but for BEST support of xvid it is ALWAYS better to use the xvid decoder ;p

[Cruzz]

Originally Posted by outlaw55

Gee, i don't see the problem with you ppl not using the XviD Direct Show Decoder for XviD 1.0.....it is what it was DESIGNED for......altho most ppl think ffdshow is GOD of decoders....ffdishow is good, but for BEST support of xvid it is ALWAYS better to use the xvid decoder ;p

If you fail to see the problem then I suggest you should get your eyes checked . More seriously, packed bitstream gives very little benefit (if any, it seems that it doesn't even help make an xvid stream more compatible with stand-alones) but stops people from using arguably the best decoder around right now. It also stops people from using players such as VideoLan, which have libavcodec integrated.

Also, ffdshow still works a bit faster than xvid's own decoder and offers a lot more options too. I fail to see any real reason to use xvid's own decoder just because an encoder is too lazy to unselect a single setting before encoding.

[flammie]

Originally Posted by outlaw55

Gee, i don't see the problem with you ppl not using the XviD Direct Show Decoder for XviD 1.0.....it is what it was DESIGNED for......altho most ppl think ffdshow is GOD of decoders....ffdishow is good, but for BEST support of xvid it is ALWAYS better to use the xvid decoder ;p

With the newer versions of xvid, the flaws in ffdshow are more apparent, but as a simple 1 time installation to solve 95% of people's "wtf my file doesn't play!@@!!11" it works pretty well. Also the postprocessing features are nice, and for people with slow comptuers, its practically necessary for any kind of somewhat decent playback.

[Cruzz]

Originally Posted by Rheinhard

Windows users can probably just get the newest version of the XviD codec and they'll be fine as long as the rest of their system doesn't get upset. The rest of us will be better off once the ffmpeg guys get around to releasing 0.4.9.

...and once again, the root of the problem can be traced to the evil Empire in Redmond...

Don't be so sure that a new version of ffmpeg will fix the problem. This isn't a new problem in any way, it has been around since god knows how long, it's just that it has suddenly become a lot more common because Packed Bitstream was made a default option in Xvid 1.0.

There's no point in blaming Redmond because they have very little to do with this. Packed Bistream was originally a hack by Divx Networks to add support for b-frames in an avi container. Xvid's packed bitstream simply stores B-Frames in a different way in order to (atleast in theory) increase DIVX interoperability. In practice, this doesn't work in most cases. Divx5 itself has the exact same problem as libavcodec with PB Xvid files, and while the effect on stand-alone players varies, it's not a positive one overall.

[Rheinhard]

Originally Posted by Cruzz

There's no point in blaming Redmond because they have very little to do with this. Packed Bistream was originally a hack by Divx Networks to add support for b-frames in an avi container. Xvid's packed bitstream simply stores B-Frames in a different way in order to (atleast in theory) increase DIVX interoperability. In practice, this doesn't work in most cases. Divx5 itself has the exact same problem as libavcodec with PB Xvid files, and while the effect on stand-alone players varies, it's not a positive one overall.

Thanks for the info. That's the most concise explanation I've seen of this yet. is there some way to get the word out to fansubbers to please not use the default Packed Bitstream setting? It'll really suck if more and more subs start coming out that exhibit this behavior that will make it hard for a lot of people to watch them. Another friend on a mailing list I run just experienced this same issue when trying to watch the latest galaxy railways.

[meingts]

Well...if you're willing to get your hands dirty, it appears it is possible to convert packed bitstream videos to videos that are not packed. (see http://forum.doom9.org/showthread.php?threadid=70689)

One way to do this (it's kind of long - there might be an easier way, I don't know) is to use graphedit (which comes with the DirectX 9 SDK) to convert the video to an mp4 using the 3ivx muxer (www.3ivx.com), then convert the mp4 back to avi. (I think the 3ivx muxer does the unpacking?) After that, change the fourcc of the new avi to XVID (or DX50 or DIVX, whatever you prefer) and mux the audio back in with VDub or something. I'd say it takes 10-15 minutes including setting it up and actually running it.

This might not be terribly clear; it's been a while since I've done this so I'd have to work through it slowly myself once again before I prepare a step-by-step guide.

I have done this under Windows successfully with Kraze's Hundred Stories ep 8. And it doesn't drop b-frames under ffdshow (yay!) I don't know about other OS's though.

[panianna]

And I just recompressed Krazes's 8,9 and 10 episodes of 100 stories. Still it was a painful job on my slow computer. Now the latest episodes of Galaxy Raiways are waiting to be recompressed. I don't even want to start about Miyuki fansubs. ALL of them are plain unwatchable.
Dear encoders, please be considerate. We know that encoding is a hard work and you do it for free and we appreciate it very much, but still don't use the f*** default settings of XVid! Avoid packed bitstream, GMC and Qpel. It doesn't improve the quality that much, but it still prevents a lot of people from watching your excellent funsubs. You see, not all of us have the newest computers. Some of us watch our files on DVD, Xbox and other home theater devices.
We just don't want to reencode every second episode we download.

[lavalyn]

The default setting of Packed Bitstream, while causing inconvenience for some, is used because the alternative is worse. Not everybody uses XviD / ffdshow codecs, and DivX uses PB to get around the AVI demuxer (which by default works in pipeline fashion, 1 frame in, 1 frame out. Not nice to deal with if you have forward prediction. So in the end, you can still blame Microsoft), breaking audio badly... but that was when I played with DivX, times may have changed.

GMC and QPel are not default options in any codec, it's understood that set-tops will have difficulty with them. If these were turned on, the encoder had a reason (idiocy may be one of them).

[Cruzz]

Originally Posted by lavalyn

The default setting of Packed Bitstream, while causing inconvenience for some, is used because the alternative is worse. Not everybody uses XviD / ffdshow codecs, and DivX uses PB to get around the AVI demuxer (which by default works in pipeline fashion, 1 frame in, 1 frame out. Not nice to deal with if you have forward prediction. So in the end, you can still blame Microsoft), breaking audio badly... but that was when I played with DivX, times may have changed.

GMC and QPel are not default options in any codec, it's understood that set-tops will have difficulty with them. If these were turned on, the encoder had a reason (idiocy may be one of them).

Xvid Packed Bitstream != DivX Packed Bitstream.

How exactly is using Packed Bitstream preferrable? What other (reasonably wide-spread) choices apart from Xvid, libavcodec and Divx are there (hmmm, I guess there's 3ivx but I see very little reason to use that)? Out of those three, xvid is the only one which can play Packed Bitstream files, Divx5 exhibits the exact same behaviour as libavcodec*. Stand-alones are another thing entirely, but on average packed bitstream won't make an xvid play on a player which doesn't support normal xvid and makes the file play worse on the ones which do decode xvid normally.

It's still not the fault of Microsoft that people continue to use the avi container in a way for which it was never meant for.

*Apparently, the latest (CVS?) release of libavcodec has finally fixed this.

[panianna]

Well, I've checked the aforementioned file Galaxy Railways ep.7 and I found that it contains all those little highly inconvenient features like QPel, GMC AND packed bitstream that reduces the compatability with any modern stand alone player to zero. Maybe somebody can explain why do encoders prefer to use these features anyway? May be there is a slight improvement considering the quality but it is almost unnoticeble.