shithub: libvpx

Info  •  Files  •  Log  •  Branches

Change to use proper type in vp{9,10}_token_state

"qc" in vp{9,10}_token_state is used to save quantized coefficients, this
commit changes the type from short to tran_low_t to properly reflect
the value range for highbitdepth build.

This fixes an out-of-range bug when optimize_b is used in highbitdepth
build.

Change-Id: Ibf330879e6ac6ae8f099e085caa9d3d9a889fde8

--- a/vp10/encoder/encodemb.c

+++ b/vp10/encoder/encodemb.c

@@ -57,7 +57,7 @@

   int           error;

   int           next;

   int16_t       token;

-  short         qc;

+  tran_low_t    qc;

 } vp10_token_state;

 // TODO(jimbankoski): experiment to find optimal RD numbers.

--- a/vp9/encoder/vp9_encodemb.c

+++ b/vp9/encoder/vp9_encodemb.c

@@ -59,7 +59,7 @@

   int           error;

   int           next;

   int16_t       token;

-  int16_t       qc;

+  tran_low_t    qc;

 } vp9_token_state;

 static const int plane_rd_mult[REF_TYPES][PLANE_TYPES] ={ {10, 6}, {8, 7}, };