shithub: libvpx

Download patch

ref: efe1982e63b73680ac0190ccd5e747e1992a4394
parent: 0aa3677d9df2d732fcf49b545e2a3c5aa4a29267
author: paulwilkins <[email protected]>
date: Wed Jun 21 13:26:20 EDT 2017

Fix int overflow in rate control for high bit rates.

Fix misplaced cast that caused an overflow and incorrect rate adaptation
behavior for high data rates. This in particular will have affected 4k encodes
but could also have come into play for some higher rate 1080p cases.

In our standard test sets the quality impact is small though several high rate
clips show improved rate accuracy. This can also impact the number of recode
loop hits and on one problem 4k  clip the encode time for speeds 0 and 1 was
reduced by >25%

Change-Id: I108da7ca42f3bc95c5825dd33c9d84583227dac1

--- a/vp9/encoder/vp9_ratectrl.c

+++ b/vp9/encoder/vp9_ratectrl.c

@@ -209,7 +209,7 @@

   const int bpm =

       (int)(vp9_rc_bits_per_mb(frame_type, q, correction_factor, bit_depth));

   return VPXMAX(FRAME_OVERHEAD_BITS,

-                (int)((uint64_t)bpm * mbs) >> BPER_MB_NORMBITS);

+                (int)(((uint64_t)bpm * mbs) >> BPER_MB_NORMBITS));

 }

 int vp9_rc_clamp_pframe_target_size(const VP9_COMP *const cpi, int target) {