Adding "lion" benchmark files

This commit is contained in:
Rich Geldreich
2019-05-14 17:45:03 -07:00
parent 8c7be6702a
commit ef14749387
59 changed files with 187 additions and 0 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.0 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.9 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.0 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.9 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.7 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.0 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.7 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.6 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.2 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.5 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.8 MiB

35
bench/bc1/stats.txt Normal file
View File

@@ -0,0 +1,35 @@
C:\dev\basis_universal-master\bin\bc1>C:\dev\basis_repo2\bin\ssim lion_photo-1470848051974-964b789cb6fa_PNG_BC1_3_view.png C:\dev\test_images\lion_photo-1470848051974-964b789cb6fa.png
OpenCV SSIM:
R: 0.976879
G: 0.981297
B: 0.97449
Avg: 0.977555
709 L: 0.982236
basislib:
RGB Total Error: Max: 51, Mean: 5.742, MSE: 33.319, RMSE: 5.772, PSNR: 32.904, CRCA: 0x877115A1 CRCB: 0x51B2976A
RGB Average Error: Max: 51, Mean: 1.914, MSE: 11.106, RMSE: 3.333, PSNR: 37.675, SSIM: 0.996128, CRCA: 0x877115A1 CRCB: 0x51B2976A
Luma Error: Max: 41, Mean: 1.708, MSE: 8.857, RMSE: 2.976, PSNR: 38.658, SSIM: 0.998707, CRCA: 0xB446B025 CRCB: 0x02E146BE
Red Error: Max: 47, Mean: 2.018, MSE: 12.229, RMSE: 3.497, PSNR: 37.257, SSIM: 0.995928, CRCA: 0x94030431 CRCB: 0x136610E3
Green Error: Max: 40, Mean: 1.750, MSE: 9.341, RMSE: 3.056, PSNR: 38.427, SSIM: 0.998345, CRCA: 0x1B3E55F4 CRCB: 0x0544399F
Blue Error: Max: 51, Mean: 1.974, MSE: 11.749, RMSE: 3.428, PSNR: 37.431, SSIM: 0.994112, CRCA: 0x877115A1 CRCB: 0x51B2976A
PSNR-HVSM Y : 46.205879
PSNR-HVSM Red : 40.615753
PSNR-HVSM Green: 44.716896
PSNR-HVSM Blue : 40.497608
PSNR-HVSM Avg : 41.943420
C:\dev\basis_universal-master\bin\bc1>

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.9 MiB

21
bench/bc7/stats.txt Normal file
View File

@@ -0,0 +1,21 @@
C:\dev\basis_universal-master\bin\bc7>C:\dev\basis_repo2\bin\ssim lion_photo-1470848051974-964b789cb6fa_PNG_BC7_1_view.png c:\dev\test_images\lion_photo-1470848051974-964b789cb6fa.png
OpenCV SSIM:
R: 0.9971
G: 0.997848
B: 0.996443
Avg: 0.997131
709 L: 0.998305
basislib:
RGB Total Error: Max: 25, Mean: 1.891, MSE: 4.422, RMSE: 2.103, PSNR: 41.675, CRCA: 0x92C75040 CRCB: 0x51B2976A
RGB Average Error: Max: 25, Mean: 0.630, MSE: 1.474, RMSE: 1.214, PSNR: 46.446, SSIM: 0.999999, CRCA: 0x92C75040 CRCB: 0x51B2976A
Luma Error: Max: 13, Mean: 0.446, MSE: 0.781, RMSE: 0.884, PSNR: 49.204, SSIM: 1.000000, CRCA: 0x282AB56B CRCB: 0x02E146BE
Red Error: Max: 25, Mean: 0.666, MSE: 1.604, RMSE: 1.267, PSNR: 46.078, SSIM: 1.000000, CRCA: 0xC7D2AB95 CRCB: 0x136610E3
Green Error: Max: 14, Mean: 0.545, MSE: 1.109, RMSE: 1.053, PSNR: 47.680, SSIM: 1.000000, CRCA: 0xF6EB599F CRCB: 0x0544399F
Blue Error: Max: 22, Mean: 0.680, MSE: 1.708, RMSE: 1.307, PSNR: 45.805, SSIM: 0.999998, CRCA: 0x92C75040 CRCB: 0x51B2976A
PSNR-HVSM Y : 60.192661
PSNR-HVSM Red : 52.682346
PSNR-HVSM Green: 55.557362
PSNR-HVSM Blue : 50.865295
PSNR-HVSM Avg : 53.035004

89
bench/bench.txt Normal file
View File

@@ -0,0 +1,89 @@
- URL/license of image
https://unsplash.com/photos/sAGXVK6bNFc
Photo by Amar Yashlaha
License:
"All photos published on Unsplash can be used for free. You can use them for commercial and noncommercial purposes. You do not need to ask permission from or provide credit to the photographer or Unsplash, although it is appreciated when possible.
More precisely, Unsplash grants you an irrevocable, nonexclusive, worldwide copyright license to download, copy, modify, distribute, perform, and use photos from Unsplash for free, including for commercial purposes, without permission from or attributing the photographer or Unsplash. This license does not include the right to compile photos from Unsplash to replicate a similar or competing service."
- Size of uncompressed file and the original image
Original JPEG image: 1951x1300, 891,721 bytes (1.7 bits/pixel)
We cropped it to 2048x2048 using Paint Shop Pro, due to PVRTC1 power of 2 texture limitations, and saved to PNG: 4,896,491 bytes (9.34 bits/pixel)
- Size of compressed file using GPU format + lossless, for each GPU format, the resulting image, and PSNR/SSIM quality rating
Error metric details: All error metrics are computed in REC 709 Y (Luma).
SSIM computed using OpenCV and http://mehdi.rabah.free.fr/SSIM/SSIM.cpp
PSNR-HVSM computed using code from https://github.com/Rolinh/VQMT
LZMA is 7-zip Ultra LZMA
ETC1 (Mali Texture Tool, "fast" setting):
KTX: 2,097,220 bytes (4 bits/texel)
KTX+LZMA: 1,001,797 bytes (1.911 bits/texel)
PSNR: 38.989 PSNR-HVSM: 45.396 SSIM: .982527
PVRTC1 4bpp (PVRTexTool, normal/good setting):
PVR: 2,097,219 bytes (4 bits/texel)
PVR+LZMA: 890,812 bytes (1.7 bits/texel)
PSNR: 37.262 PSNR-HVSM: 41.833 SSIM: .976158
BC1 (AMD Compressonator default settings):
KTX: 2,097,220 bytes (4 bits/texel)
KTX+LZMA: 959,831 bytes (1.831 bits/texel)
PSNR: 38.658 PSNR-HVSM: 46.206 SSIM: .982236
BC7 (AMD Compressonator default settings):
KTX: 4,194,372 bytes (8 bits/texel)
KTX+LZMA: 2,161,666 bytes (4.123 bits/texel)
PSNR: 49.204 PSNR-HVSM: 60.193 SSIM: .998305
- Basis Universal Baseline at 4 different quality settings, transcoded to each major texture format:
---- .basis quality 16 (very low quality):
.basis file size: 190,706 (.364 bits/texel)
.basis losslessly unpacked to ETC1S .KTX, then compressed with LZMA: 278,826 (.532 bits/texel)
PSNR PSNR-HVSM SSIM
ETC1: 28.825 30.956 .786465
PVRTC1 4bpp: 28.347 29.596 .770552
BC1: 28.825 30.954 .787366
BC7: 28.823 30.944 .786056
---- .basis quality 64 (low quality):
.basis file size: 238,919 (.456 bits/texel)
.basis losslessly unpacked to ETC1S .KTX, then compressed with LZMA: 333,106 (.635 bits/texel)
PSNR PSNR-HVSM SSIM
ETC1: 29.526 31.972 .822336
PVRTC1 4bpp: 28.873 30.195 .804267
BC1: 29.517 31.936 .822921
BC7: 29.526 31.968 .82164
---- .basis quality 128 (default quality):
.basis file size: 307,307 (.586 bits/texel)
.basis losslessly unpacked to ETC1S .KTX, then compressed with LZMA: 406,677 (.776 bits/texel)
PSNR PSNR-HVSM SSIM
ETC1: 30.557 33.168 .863934
PVRTC1 4bpp: 29.682 30.927 .843242
BC1: 30.533 33.087 .86426
BC7: 30.555 33.159 .86302
---- .basis highest achievable quality:
basisu -slower -max_selectors 16128 -max_endpoints 16128 -endpoint_rdo_thresh 1.00 -selector_rdo_thresh 1.00
.basis file size: 469,119 (.895 bits/texel)
.basis losslessly unpacked to ETC1S .KTX, then compressed with LZMA: 542,338 (1.034 bits/texel)
PSNR PSNR-HVSM SSIM
ETC1: 32.878 36.865 .918791
PVRTC1 4bpp: 31.196 32.571 .892816
BC1: 32.782 36.616 .918009
BC7: 32.876 36.855 .918128
- Compare VRAM consumed by the original uncompressed and then the Basis-encoded-and-transcoded into a compressed texture
The 2K texture stored uncompressed in VRAM would be 16,777,216 bytes (16MB) using 32-bit RGBA (few if any GPU's support 24-bpp uncompressed textures).
Uncompressed 16-bpp 565: 8,388,608 bytes (8MB).
BC1/ETC1/PVRTC1: 2,097,152 bytes (2MB)
BC7: 4,194,304 bytes (4MB)

Binary file not shown.

Binary file not shown.

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 MiB

21
bench/etc1/stats.txt Normal file
View File

@@ -0,0 +1,21 @@
C:\dev\basis_universal-master\bin\mali_etc1>C:\dev\basis_repo2\bin\ssim mali_etc1_fast.png C:\dev\test_images\lion_photo-1470848051974-964b789cb6fa.png
OpenCV SSIM:
R: 0.978464
G: 0.981031
B: 0.972616
Avg: 0.97737
709 L: 0.982527
basislib:
RGB Total Error: Max: 62, Mean: 5.746, MSE: 32.545, RMSE: 5.705, PSNR: 33.006, CRCA: 0x7014DCD3 CRCB: 0x51B2976A
RGB Average Error: Max: 62, Mean: 1.915, MSE: 10.848, RMSE: 3.294, PSNR: 37.777, SSIM: 0.995340, CRCA: 0x7014DCD3 CRCB: 0x51B2976A
Luma Error: Max: 45, Mean: 1.648, MSE: 8.206, RMSE: 2.865, PSNR: 38.989, SSIM: 0.998359, CRCA: 0x6F0BD68E CRCB: 0x02E146BE
Red Error: Max: 51, Mean: 1.931, MSE: 10.948, RMSE: 3.309, PSNR: 37.738, SSIM: 0.996199, CRCA: 0xFB958721 CRCB: 0x136610E3
Green Error: Max: 50, Mean: 1.738, MSE: 8.988, RMSE: 2.998, PSNR: 38.594, SSIM: 0.997763, CRCA: 0x47B3296A CRCB: 0x0544399F
Blue Error: Max: 62, Mean: 2.078, MSE: 12.609, RMSE: 3.551, PSNR: 37.124, SSIM: 0.992057, CRCA: 0x7014DCD3 CRCB: 0x51B2976A
PSNR-HVSM Y : 45.395554
PSNR-HVSM Red : 40.206581
PSNR-HVSM Green: 43.017883
PSNR-HVSM Blue : 37.988522
PSNR-HVSM Avg : 40.404327

Binary file not shown.

After

Width:  |  Height:  |  Size: 871 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.7 MiB

Binary file not shown.

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.0 MiB

Binary file not shown.

21
bench/pvrtc/stats.txt Normal file
View File

@@ -0,0 +1,21 @@
C:\dev\basis_universal-master\bin\pvrtc>C:\dev\basis_repo2\bin\ssim lion_pvrtextool_good.png C:\dev\test_images\lion_photo-1470848051974-964b789cb6fa.png
OpenCV SSIM:
R: 0.969437
G: 0.974552
B: 0.973104
Avg: 0.972365
709 L: 0.976158
basislib:
RGB Total Error: Max: 66, Mean: 6.337, MSE: 43.434, RMSE: 6.590, PSNR: 31.752, CRCA: 0x3AB1868C CRCB: 0x51B2976A
RGB Average Error: Max: 66, Mean: 2.112, MSE: 14.478, RMSE: 3.805, PSNR: 36.524, SSIM: 0.991711, CRCA: 0x3AB1868C CRCB: 0x51B2976A
Luma Error: Max: 64, Mean: 1.934, MSE: 12.216, RMSE: 3.495, PSNR: 37.262, SSIM: 0.995279, CRCA: 0xCF4FF6F7 CRCB: 0x02E146BE
Red Error: Max: 64, Mean: 2.308, MSE: 17.298, RMSE: 4.159, PSNR: 35.751, SSIM: 0.988385, CRCA: 0xF880E24C CRCB: 0x136610E3
Green Error: Max: 65, Mean: 2.008, MSE: 13.012, RMSE: 3.607, PSNR: 36.987, SSIM: 0.993933, CRCA: 0xCB34FED9 CRCB: 0x0544399F
Blue Error: Max: 66, Mean: 2.021, MSE: 13.124, RMSE: 3.623, PSNR: 36.950, SSIM: 0.992816, CRCA: 0x3AB1868C CRCB: 0x51B2976A
PSNR-HVSM Y : 41.833385
PSNR-HVSM Red : 37.257824
PSNR-HVSM Green: 40.689716
PSNR-HVSM Blue : 41.202507
PSNR-HVSM Avg : 39.716682