Texture Compression Using Mipmaps

    A mipmap is a sequence of textures, each of which is a progressively lower resolution, prefiltered representation of the same image. Mipmapping is a computationally low-cost way of improving the quality of rendered textures. Each pre-filtered image, or level, in the mipmap is a power of two smaller than the previous level. 

Mipmap compression approach  is based on our observation that the difference between level surfaces levels of mipmapped images for smooth areas of textures is very small. This allows to use bilinear interpolation of lower resolution mipmap level for the representation of next levels of mipmap.  However, for noisy texture areas  we need to apply texture compression approaches better suited to handle such situations. Mipmap compression is a combination of "smooth" and "noisy" approaches. Better compression ratios are achieved because less data is to be stored to represent smooth texture areas comparing to other texture compression approaches.

Encoding/Decoding  Scheme

Basic encoding scheme looks as follows:

1. Create mipmap surface of 1st or 2nd level ( basic level ), i.e. filtering highest resolution (level 0 ) texture blocks sized 2-by-2 or 4-by-4;
   
   
   
2. Calculate bilinear interpolation for the highest resolution texture cells using basic level;
   
   
   
3. Calculate mean squared error ( MSE ) values for each cell of interpolation;
4. If cell's MSE is less than specified threshold value (smooth cell) represent the original texture within the cell by bilinear interpolation of basic level;
5. Otherwise (noisy cell) use any other texture compression approach to represent original texture.

Mipmap compression approach uses hybrid compression scheme, that is currently under development.  In our research we used distributed colors algorithm for encoding noisy cells. We could propose the following scheme:

• Compressed data elements are in one-to-one correspondence with basic level texture blocks. They contain either 15 bits basic mipmap color for smooth areas or 15 bits pointer for noisy areas. We also use 1 bit flag to distinguish between smooth and noisy areas, making total of 2 bytes per block.  In the first case we interpolate basic colors, in the second one we need to retrieve further information about noisy area. This is sequential access, but it is needed for part of image and for large blocks.
• To determine the contents of  a compressed data element, consider four texture cells adjacent at the corresponding  texture block's center. Assume that we have precalculated filtered mipmap values for all texture blocks. Then if  all these four cells are smooth (i.e. could be approximated using bilinear interpolation of filtered values), set compressed data element to  mipmap color of the corresponding texture block. Otherwise set  compressed data element to the offset of compressed data for this block obtained using any other texture compression method.
  
  
  

Analysis

We found that mipmap surface of 2nd level ( 4-by-4 block ) could be efficiently applied to represent 30-60 percents of area of real textures. Compression ratio for these smooth areas is 24x (4-by-4 block = 16 pixels*3 bytes = 48 bytes are compressed to 2 bytes).  For the surface of first level (2-by-2 mipmap) this value is 50-80 percents, but compression ratio is 6x only ( 2-by-2 block = 4 pixels*3 bytes = 12 bytes are compressed to 2 bytes). Our study shows that 2nd level of mipmap is optimal. For the noisy blocks we applied S3TC-like block decomposition approach which results in compression ratio for the entire texture about 8-12x, with PSNR value about 35 dB for MSE threshold 5. These compression ratios  are valid for true-color images, however, for reduced color space images compression ratios are 3-6x.

Figure. Original grayscale Lena 512x512 pixels.

Figure: Lena with bilinear interpolated smooth cells (62% of total) and noisy cells marked red. MSE threshold = 5.0

Figure: Lena with bilinear interpolated smooth cells and noisy cells compressed/decompressed using S3TC-like algorithm.  MSE threshold = 5.0, PSNR = 34.2 dB, compression ratio =  4.55x (1.76 bpp).

This approach handles smooth color graduation excellently.  Visual quality of smooth parts is the best among all tested compression methods.

Mipmap texture compression allows to avoid 3-linear interpolation for smooth areas, because we know that our basic level mipmap surface represent this interpolation with required quality. That reduces AGP traffic. Another good feature of this approach is that we have good quality estimation for the compressed image/texture.  

This type of compression is very suitable for photo textures and images. Therefore, it could handle the textures with precalculated lighting/shadowing effects. However, this method is not so good for artificial textures with reduced color space.