Quantization

Libavcodec

Trellis searched quantization

This will find the optimal encoding for each 8x8 block.

Trellis searched quantization is quite simple a optimal quantization in the PSNR vs bitrate sense (assuming that there would be no rounding errors introduced by the IDCT, which is obviously not the case) it simply finds a block for the minimum of error + lambda*bits. Lambda is a qp dependant constant Bits is the amount of bits needed to encode the block Error is simple the sum of squared errors of the quantization

Inter/intra quantizer bias

Lowering these values may raise PSNR

Quantizer noise shaping

Reduces ringing artefacts, larger values are slower but may not result in better quality this can and should be used together with trellis quantization, in which case the trellis quantization (optimal for constant weight) will be used as startpoint for the iterative search.