[TEST] Rv9, Divx 51, H.264 R-D Plots

Ok guys, I did some tests using the VQEG 'Mobile & Calendar' sequence. I deinterlaced it using Tom's MoComp, converted to YV12, and cropped it to 720x480. I encoded using all the slowest settings with all pre/post processing disabled. Each video followed the GOP structure IBPBPBP... Target Bitrates (k): 256, 512, 768, 1500, 2000, 3000, 4000,6000 Divx 5.1 2-Pass , slowest, no PyschoVisual RealVideo 9 - 2-Pass, CBR, encoderComplexity=100, noisyEdgeFilter=false, MSL=60, patternAdaptivity=1, scalingFactor=20 H.264 - JM 6.1e, 5 Reference frames, CABAC, R-D on, Hadamard on. Since no rate control exists, Iframe QP = Pframe = Bframe QP - 2 (about 20% larger step size) I would love to include WM9, but I don't know of an easy to way to perform command line encoding to ease batch encoding. I am trying to keep this a fair comparison as possible, so please feel free to posts any questions, comments, and suggestions you might have. I will be posting more data as soon as I have it.

I think this is an excellent testing methodology and for once I can't offer any constructive criticism. All I can suggest is to try the same test but with some different sequences. Also I'd like to see WMV9 in the comparison, but I don't know any way to run it from command line other than to launch vdub with a pre-prepared job file.

2-pass CBR on RV9 ??? What's 2-pass CBR? You probably meant VBR :) And can you add XVID to it? I don't see the need to way for dev-api-4 to come out... :) Bilu

I aggree with Bilu, why CBR??? Also, why doesn't all codecs begin where H264 begins? Also #2, use EHQ=85, it will be faster and have the same Q.

I guess you're right, although VBR is more related to reality. But I think tests should use clips with 5 minutes at least, not 10 seconds. After all, VBR decisions should be a part of the benchmark as well. Bilu

@FastMike: interesting data, but I am afraid something appears to go quite wrong with your RV9 measurements. MobCal is one of the sequences I test with, incl measurements for H264 and DivX. Looking at your parameters, please do not use scalingFactor or patternAdaptivity for RV9. Leave these at default, i.e. adaptive as recommended. scalingFactor = 20 is very suboptimal for MobCal, with B frame QP almost the same as P frame QP. RV9 step sizes are not the same as in MPEG-4. Also, for a PSNR plot, I would recommend to use fixed Quality (QP) for all codecs, since rate control for a 10 second clip can be quite hazardous, and 1-pass, since 2-pass makes no sense for fixed Q, or for such a short clip, and then just plot the points on the curve as the bitrate turns out. One last datapoint, how did you obtain RV9 PSNR numbers? If with Avisynth compare, did you make sure to take into account the one missing frame?

Sorry, I forgot one item: Please specify maxPacketSize 16000 as well for RV9. This is required for good performance for such high bitrates that are needed for MobCal. If you use the latest Producer Milestone, it automatically sets large packetsizes, but I think only for VBR high bitrates. If a slightly older Producer is used, maxPacketSize always needs to be specified, so it is best to make sure 16000 is used by explicitly specifying it.

Karl, how did I figure you would have something to say about RV9? :) * I was trying to keep the GOP structure the same for all codecs. I did test RV9 with scalingFactor and patternAdaptivity set to default values. I was shooting for a 20% higher step sizes for the B-frames, which I thought scalingFactor = 20 was achieving that? * I obtained PSNR numbers from the rv9log.txt file. I haven't yet verified myself the numbers to be accurate. :) I will look into using constant quality and setting the larger packet size. Oh, and all the graphs don't start in the same place because Divx & RV9 don't seem to follow the intended bitrate at those low of levels. Hopefully using constant quality will help fix that. All of the standard test sequences used in video coding research are all short. I am trying to utilize a method similiar to the MPEG Call for Proposals method (CfP) which does not allow rate control. I am trying to remove as many variables as possible, and rate control is a huge variable.

Why not add tests using one of the visual quality-based metrics in addition to PSNR. I would guess that PSNR is even more useless than normal when encoding at very low bitrates.

Wow, I would have never expected the curves to be so smooth...