jessikitty/unity
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: jessikitty/unity:linux64
Branches Tags
jessikitty/unity:master jessikitty/unity:unity jessikitty/unity:wiki jessikitty/unity:linux64
jessikitty/unity:v104 jessikitty/unity:v103 jessikitty/unity:v101 jessikitty/unity:v100
..
pull from: jessikitty/unity:unity
Branches Tags
jessikitty/unity:unity jessikitty/unity:master jessikitty/unity:wiki jessikitty/unity:linux64
jessikitty/unity:v104 jessikitty/unity:v103 jessikitty/unity:v101 jessikitty/unity:v100

88 Commits
linux64 .. unity

Author SHA1 Message Date
Alexander Suvorov 8708900eca Implement ETC1S/ETC2AS image compression 
Explanation:

ETC1S encoding is a subset of ETC1, which is using only one color endpoint per 4x4 block (modifier indices are identical for both subblocks, base color is encoded differentially as RGB555 with the differential RGB333 part always set to zero, flip bit is always set to zero).
Usage: crunch_x64.exe -ETC1S input.png -out output.ktx

ETC2AS encoding is a subset of ETC2A encoding which is using ETC1S encoding for color and ETC2A encoding for alpha.
Usage: crunch_x64.exe -ETC2AS input.png -out output.ktx
 2018-10-23 23:16:00 +02:00
Alexander Suvorov 4bb735f796 Make Crunch compression work correctly on CPU supporting 16 or more threads 
Explanation:

Crunch library has been designed to work correctly when using up to 16 threads. Considering that one of those threads is the main thread, the maximum number of helper threads should be set to 15.
 2018-10-23 20:23:56 +02:00
Alexander Suvorov 660322d3a6 Add compression support for ETC1S/ETC2AS encodings 
Explanation:

ETC1S encoding is a subset of ETC1, which is using only one color endpoint per 4x4 block. The base color is therefore is always encoded as RGB555 and there is no need to encode block flips. ETC2AS encoding is a subset of ETC2A encoding which is using ETC1S encoding for color and default ETC2A encoding for alpha.

ETC1S/ETC2AS Crunch compression and decompression is based on ETC and DXT Crunch compression and decompression algorithms:
- ETC1S/ETC2AS tiling is performed within the area of 8x8 pixels using DXT1/DXT5 tiling scheme
- ETC1S color endpoints are generated using standard ETC1 optimization
- ETC1S color codebook encoding is equivalent to ETC1 codebook encoding
- ETC1S level encoding is equivalent to DXT1 level encoding
- ETC2AS alpha codebook encoding is equivalent to ETC2A alpha codebook encoding
- ETC2AS level encoding is equivalent to DXT5 level encoding

Testing results suggest that ETC1S/ETC2AS encodings can be used to achieve lower bitrates than ETC1/ETC2A on the Kodak test set while providing equivalent image quality (estimated using PSNR).

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.854 sec
Modified: 1468204 bytes / 5.473 sec
Improvement: 7.21% (compression ratio) / 81.03% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.925 sec
Modified: 1914805 bytes / 7.297 sec
Improvement: 7.28% (compression ratio) / 80.24% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 12.842 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB

ETCS Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1S quantization parameters have been selected in such a way, so that ETC1S compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1S encoding]
Total size: 1363676 bytes
Total time: 15.586 sec
Average bitrate: 1.156 bpp
Average Luma PSNR: 34.047 dB
 2018-06-07 19:20:30 +02:00
Alexander Suvorov c1d8e8da71 Optimize vector quantization step 
This change improves the compression speed for both DXT and ETC encodings.

Explanation:

The vector quantization algorithm takes floating point vectors as input and performs vector preprocessing right before the quantization. At the same time, selector training vectors are generated directly from integer selector values, packed into a single uint64. It would therefore be more efficient to perform preprocessing of the selector training vectors (which includes sorting and deduplication) while still having them in a packed form. Additional performance boost is achieved by using multiple threads for sorting the training vectors.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.869 sec
Modified: 1468204 bytes / 5.477 sec
Improvement: 7.21% (compression ratio) / 81.03% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.961 sec
Modified: 1914805 bytes / 7.322 sec
Improvement: 7.28% (compression ratio) / 80.19% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 12.766 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-27 16:50:44 +02:00
Alexander Suvorov 21eb70bc10 Optimize tile computation step 
This change improves the compression speed for both DXT and ETC encodings.

Explanation:

In the tile computation step, pixels within the tiling area are palettized using a general purpose tree clusterization algorithm. At the same time, clusterization of the tile pixels is always performed with the following restrictions: the maximum number of palettized pixels is 64, the maximum number of clusters is 2. The performance can therefore be improved by solving the palettizing task with a specialized version of the tree clusterizer, which does not maintain the tree structure and uses constant memory.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.863 sec
Modified: 1468204 bytes / 5.726 sec
Improvement: 7.21% (compression ratio) / 80.16% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.950 sec
Modified: 1914805 bytes / 7.683 sec
Improvement: 7.28% (compression ratio) / 79.21% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 13.071 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-25 19:15:36 +02:00
Alexander Suvorov b8b456d48c Optimize endpoint reordering algorithm 
This change improves the compression speed for both DXT and ETC encodings.

Explanation:
The main ideas used for optimization of the endpoint reordering algorithm:
- On each iteration, the list of the chosen endpoints is updated only from one side, so all the computations performed for the unchanged side of the list can be cached and reused on the next iteration.
- The list of the chosen endpoints can be build using an array of double size, growing from the middle of the array. This eliminates unnecessary memory reallocations and movements.
- When an element is removed from the list of remaining endpoints, instead of moving all the elements with higher indices, just a single last element of the list can be moved into the position of the removed element (the original indices of the remaining endpoints should be stored within the list elements to maintain proper indexing).

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.848 sec
Modified: 1468204 bytes / 5.875 sec
Improvement: 7.21% (compression ratio) / 79.63% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.952 sec
Modified: 1914805 bytes / 7.834 sec
Improvement: 7.28% (compression ratio) / 78.80% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 13.261 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-25 16:17:06 +02:00
Alexander Suvorov 7143913032 Optimize DXT endpoints computation 
This change improves the compression speed for DXT encoding.

Explanation:

When performing per-component endpoint optimization, the trial solutions are generated using all possible combinations of the component values. Then the error boundary computation is performed for each block color of the trial solution in order to check the possibility of early out. The important observation here is that some component values are present in several trial solutions and therefore are processed multiple times. The overall performance can therefore be improved by computing and caching the errors for all the possible component values in advance.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.843 sec
Modified: 1468204 bytes / 6.067 sec
Improvement: 7.21% (compression ratio) / 78.97% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.983 sec
Modified: 1914805 bytes / 8.080 sec
Improvement: 7.28% (compression ratio) / 78.15% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 13.421 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-24 19:48:37 +02:00
Alexander Suvorov dbbef6a21f Perform multithreaded node preprocessing for faster vector quantization 
This change significantly improves the compression speed for both DXT and ETC encodings.

Explanation:

On each iteration of the vector quantization algorithm, the leaf with the highest variance is selected for splitting. If the leaf gets split, then two new leaves are created (while the leaves that can not be split will be ignored on the future iterations). There does not seem to be any simple way to compute or reliably predict the variances of the future leaves in advance, which means that there is no simple way to efficiently perform split operations in parallel.

And still, there is an interesting observation. Even though the order of the split operations depends on the previous iterations, the split operations performed in different subtrees are completely independent. So what if instead of solving the main quantization task we will first solve an alternative quantization task, which has a lot in common with the main task, but at the same time can be efficiently parallelized. Then the intermediate computation results of the alternative solution can be reused when solving the main task. Specifically, the idea is to efficiently compute an alternative split tree, which is more or less balanced, and has approximately the same number of nodes as the main tree. Then the overlapping part of the main and alternative trees can be reused while solving the main quantization task.

In order to achieve this, the initial root is first split normally until the number of splittable leaves reaches the number of available threads. Then each leaf is split in a separate thread, while the maximum number of split iterations for each subtree is defined as the maximum number of split iterations for the whole main tree divided by the number of used threads. This way the total number of nodes in the alternative tree will be approximately the same as the number of nodes in the main tree.

Note that in general, the alternative tree does not match the main tree, so some nodes of the alternative tree will never be reused. In practice however, the portion of such unnecessarily precomputed nodes is not very big. And considering that the nodes of the alternative tree are precomputed in parallel using multiple threads, in most cases the overall performance is significantly improved.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.840 sec
Modified: 1468204 bytes / 6.303 sec
Improvement: 7.21% (compression ratio) / 78.14% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.955 sec
Modified: 1914805 bytes / 8.342 sec
Improvement: 7.28% (compression ratio) / 77.43% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 13.322 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-20 19:18:08 +02:00
Alexander Suvorov 1028520280 Use multiple threads for node split in vector quantization 
This change improves the compression speed for both DXT and ETC encodings.

Explanation:

During the node split iteration, identical computations are performed for all the vectors of the split node. The overall performance can be improved by performing independent computations in separate threads. In order to avoid possible performance overhead, on each iteration the number of threads is selected in such a way so that each thread processes at least 512 vectors.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.892 sec
Modified: 1468204 bytes / 7.578 sec
Improvement: 7.21% (compression ratio) / 73.77% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.943 sec
Modified: 1914805 bytes / 9.993 sec
Improvement: 7.28% (compression ratio) / 72.95% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 14.753 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-20 14:23:47 +02:00
Alexander Suvorov fbe3f6ca10 Optimize vector quantization algorithm 
This change improves the compression speed for both DXT and ETC encodings.

Explanation:

On each iteration of the vector quantization algorithm, the leaf with the highest variance is selected for splitting. At the same time, each split operation adds at most 2 new leaves. Considering this, the search of the leaf with the highest variance can be performed more efficiently if all the leaves are stored in a priority queue (in order to guarantee that texture decompression gives identical result to the original version of Crunch, the node comparison operation also takes the node index into account).

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.844 sec
Modified: 1468204 bytes / 7.883 sec
Improvement: 7.21% (compression ratio) / 72.67% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.978 sec
Modified: 1914805 bytes / 10.490 sec
Improvement: 7.28% (compression ratio) / 71.63% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 15.165 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-19 18:58:05 +02:00
Alexander Suvorov 11a89d25ed Optimize vector quantization algorithm 
This change improves the compression speed for both DXT and ETC encodings.

Explanation:

When a node is split during the quantization step, all of its vectors are split between the child nodes, and new memory is allocated to store each new set of vectors. At the same time, the set of vectors of the parent node is no longer accessed after the split. Considering that the sets of vectors of the child nodes do not intersect, it is possible to reuse the memory allocated for the parent set of vectors, to store the child sets of vectors. This can be achieved in the following way. All the source vectors are initially stored in an array. Let's assume that it is possible to reorder this common array of vectors in such a way, so that vectors of each node would form a continuous block within this array. Then it would be sufficient to store only two indices for each node (pointing to the first and to the last node vectors in the common array of vectors) in order to describe the complete set of vectors of this node. This assumption is correct for the root node, which has initial vector indices pointing to the first and to the last elements of the complete vector array. When a node is split, let's reorder its vectors (stored in a continuous block within the common array of vectors) in such a way, so that vectors of the left child node are put in front, and then followed by the vectors of the right child node (the indices of the first and last vectors of the child nodes should be set accordingly). This way each child node will also have its vectors stored in a continuous block within the common array of vectors, defined by two indices, and the split iteration can be repeated. Note that the memory, which is used to store the sets of vectors for all the nodes, now needs to be allocated only once.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.872 sec
Modified: 1468204 bytes / 8.276 sec
Improvement: 7.21% (compression ratio) / 71.34% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.971 sec
Modified: 1914805 bytes / 10.944 sec
Improvement: 7.28% (compression ratio) / 70.40% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 15.364 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-17 15:12:55 +02:00
Alexander Suvorov a14a313361 Optimize color endpoint solution evaluation 
This change improves the compression speed for DXT encoding.

Explanation:

In order to evaluate an endpoint solution, it is necessary to compute the sum of the squared distances from the source pixels to their nearest block colors, defined by the evaluated endpoint solution. Such computation is quite complicated, so before it is performed, we can compute the sum of the squared distances from the source pixels to the axis-aligned bounding box enclosing all the evaluated block colors (if the source pixel appears to be inside the AABB of the evaluated solution, then the distance is considered to be 0). If the sum of the squared distances to the AABB of the current solution is already bigger than the sum of the squared distances computed for the previously found best solution, then the current solution does not need to be evaluated.

The actual trick here is that the sum of the squared distances to the AABB of the current solution can be computed in constant time using the following approach. The sums of the squared distances for each color component can be computed separately. For each color component the AABB determines 2 planes: the "lower" plane, defined by the lower boundary of the AABB, and the "upper" plane, defined by the upper boundary of the AABB. The sum for each color component is combined from two parts: the sum of the squared distances from the lower plane to all the source pixels which are below the lower plane, and the sum of the squared distances from the upper plane to all the source pixels which are above the upper plane. Considering that the endpoints of the evaluated solution are encoded as RGB565, there are 32 possible planes for the red and blue components, and 64 possible planes for the green component. For each plane it is sufficient to precompute the following two values: the sum of the squared distances from the plane to all the source pixels which are "below" this plane, and the sum of the squared distances from the plane to all the source pixels which are "above" this plane. The total sum of the squared distances from the source pixels to any evaluated AABB can then be represented as a sum of 6 precomputed values, while all the used values can be precomputed in linear time with dynamic programming.

Note: The AABB check seems to work faster than inserting a solution into the hash map. For this reason the AABB check is performed first.

Additional improvements: A few minor adjustments have been made in order to make sure that the texture decompression gives identical result to the original version of Crunch also for 32-bit builds (original Crunch library uses different floating point models for 32-bit and 64-bit builds).

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.861 sec
Modified: 1468204 bytes / 8.622 sec
Improvement: 7.21% (compression ratio) / 70.13% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.980 sec
Modified: 1914805 bytes / 11.294 sec
Improvement: 7.28% (compression ratio) / 69.46% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 15.529 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-13 17:20:31 +02:00
Alexander Suvorov 65f44319c0 Optimize computation of the endpoint cluster indices 
This change improves the compression speed for both DXT and ETC encodings.

Explanation:

The vectors which are processed in the cluster indices computation step, are the very same vectors which were used in the vector quantization step. This means that every processed vector already has a specific centroid associated with it. Even though the associated centroid is not necessarily the closest one to the processed vector, the distance to the associated centroid can be used as an upper boundary of the distance to the closest centroid. This allows to efficiently perform early out while computing the distances to the other centroids.

Note: The modified algorithm is supposed to generate decompression result identical to the original version of Crunch. For this reason the centroid associated with a specific training vector is not used as an initial best solution, because it could potentially change the decompression result in cases when the processed training vector is equidistant from multiple centroids (selection of the closest centroid in such cases depends on the processing order).

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.847 sec
Modified: 1468204 bytes / 8.929 sec
Improvement: 7.21% (compression ratio) / 69.05% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.953 sec
Modified: 1914805 bytes / 11.651 sec
Improvement: 7.28% (compression ratio) / 68.47% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 15.695 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-10 17:13:41 +02:00
Alexander Suvorov 51f73fdfed Optimize vector quantization algorithm 
This change improves the compression speed for both DXT and ETC encodings.

Explanation:
The main ideas used for optimization of the vector quantization algorithm:
- intermediate structures can store vector indices instead of the vector data, which minimizes the total amount of copied data when splitting a node (this is especially important for selector quantization, where processed vectors have 16 components)
- weighted vectors and weighted dot products can be cached

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.893 sec
Modified: 1468204 bytes / 9.310 sec
Improvement: 7.21% (compression ratio) / 67.78% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.942 sec
Modified: 1914805 bytes / 12.232 sec
Improvement: 7.28% (compression ratio) / 66.89% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 16.121 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-06 19:38:02 +02:00
Alexander Suvorov 205829da99 Optimize DXT color endpoint solution evaluation 
This change improves the compression speed for DXT encoding.

Explanation:

In order to evaluate an endpoint solution, it is necessary to compute the sum of the squared distances from the source pixels to their nearest block colors, defined by the evaluated endpoint solution. Considering that we are looking for a solution with a minimal sum, the computation can be stopped as soon as the current sum is higher or equal than the previously found best sum. An interesting observation here is that the performance improvement, achieved by such early out approach, depends on the order in which the source pixels are processed. It makes sense to process the pixels with the highest introduced errors first, as this significantly increases the chances to exit the computation earlier.

On the one hand, equal source pixels are grouped together, so the computed distance from each unique source pixel is multiplied by its weight. For this reason it makes sense to first process the pixels with the highest weights, as their errors have the highest multipliers. On the other hand, the pixels which project onto the middle part of the endpoint interval, have higher chances of being close to one of the block colors. For this reason it makes sense to first process the pixels, which projections are the most distant from the middle of the endpoint interval, as those pixels will normally introduce the highest errors. In order to combine those two aspects, it is proposed to sort the pixels according to the multiplication of the weight and the distance from the projected pixel to the center of the endpoint interval.

Of course, reordering the pixels on each iteration would be very expensive and is not considered. However, there is a high chance that most endpoint intervals will be aligned in a similar way as the principle axis, as well as have their centers close to the mean color. As soon as the principle axis is computed, it can be used for approximation of all the future endpoint intervals. So the projection and reordering of the source pixels is performed only once.

Two approaches have been considered. In the first approach, the pixels have been sorted by the multiplication of the weight and the absolute distance in decreasing order. In the second approach, the pixels have been sorted by the multiplication of the weight and the signed distance, and then interleaved starting from the opposite sides of the ordered sequence. When tested on the Kodak image set, the interleaving approach shows better results.

Additional optimization: perceptual and uniform versions of the evaluation function are now implemented separately, which slightly improves the performance.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.845 sec
Modified: 1468204 bytes / 10.071 sec
Improvement: 7.21% (compression ratio) / 65.09% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.929 sec
Modified: 1914805 bytes / 13.248 sec
Improvement: 7.28% (compression ratio) / 64.13% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 17.126 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-10-04 17:33:32 +02:00
Alexander Suvorov 4bd4355683 Optimize DXT endpoints computation 
This change improves the compression speed for DXT encoding.

Explanation:

When performing per-component endpoint optimization, it is not necessary to go through all the source pixels on every iteration in order to calculate the total weighted squared error for a specific trial endpoint. The computation can be optimized in the following way:

sum(w(i) * (x - p(i)) * (x - p(i))) = sum(w(i) * x * x) - sum(w(i) * 2 * x * p(i)) + sum(w(i) * p(i) * p(i)) = sum(w(i)) * x * x - sum(2 * w(i) * p(i)) * x + sum(w(i) * p(i) * p(i))

The values of sum(w(i)), sum(2 * w(i) * p(i)), sum(w(i) * p(i) * p(i)) can be precalculated for each of 4 selectors, and only have to be updated when the solution improves. This way the error computation on each iteration can be performed using 12 multiplications instead of 2 * N (where N is the number of pixels in the processed cluster).

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.811 sec
Modified: 1468204 bytes / 10.520 sec
Improvement: 7.21% (compression ratio) / 63.49% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.936 sec
Modified: 1914805 bytes / 13.902 sec
Improvement: 7.28% (compression ratio) / 62.36% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 17.121 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-09-13 14:10:00 +02:00
Alexander Suvorov 335f0ee056 Optimize DXT endpoints refinement 
This change improves the compression speed for DXT encoding.

Explanation:

When creating the array of trial alpha endpoints, there is no need to use bit array for tracking duplicate entries. Instead, the uniqueness of the endpoint pair can be determined using simple comparison operations. Moreover, it is not necessary to go through all the source pixels on every iteration in order to calculate the total squared error for a specific trial endpoint. Considering that selector values are not modified during the refinement step, each selector has a fixed set of pixels associated with it during optimization. This means that calculation of the total squared error can be optimized in the following way:

sum((x - p(i)) * (x - p(i))) = sum(x * x) + sum(2 * x * p(i)) + sum(p(i) * p(i)) = N * x * x + sum(2 * p(i)) * x + sum(p(i) * p(i))

As the set of pixels, associated with a specific selector is fixed, the sum(2 * p(i)) and sum(p(i) * p(i)) values can be precalculated in advance. This means that error computation for each component now requires only (3 * S) multiplications instead of N (where N is the number of pixels in the processed cluster, and S is the number of selectors, equal to 4 for color components and 8 for alpha components).

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.864 sec
Modified: 1468204 bytes / 10.794 sec
Improvement: 7.21% (compression ratio) / 62.60% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.912 sec
Modified: 1914805 bytes / 14.244 sec
Improvement: 7.28% (compression ratio) / 61.41% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 17.125 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-09-12 16:50:03 +02:00
Alexander Suvorov 3053c9dd93 Optimize DXT endpoints computation 
This change improves the compression speed for DXT encoding.

Explanation:

The main ideas used for the DXT endpoints computation optimization:
- Instead of using map in tree clusterizer, the source vectors can be stored in an array and sorted before the quantization. This might increase the amount of used memory, but is much more efficient in terms of memory reallocation.
- Endpoint caching can be used throughout the color endpoint computation, and not just within the optimize_endpoints function. The only place where endpoint caching can not be used is the final step of the try_combinatorial_encoding function, where alternate rounding is used.
- When computing endpoint codebooks, endpoint optimizer and endpoint refiner can be reused, which eliminates unnecessary memory reallocations.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.879 sec
Modified: 1468204 bytes / 11.099 sec
Improvement: 7.21% (compression ratio) / 61.57% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.919 sec
Modified: 1914805 bytes / 14.621 sec
Improvement: 7.28% (compression ratio) / 60.40% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 17.108 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-09-12 13:03:56 +02:00
Alexander Suvorov 3e12aff909 Fix miscellaneous compiler warnings 
DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.866 sec
Modified: 1468204 bytes / 11.858 sec
Improvement: 7.21% (compression ratio) / 58.92% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.878 sec
Modified: 1914805 bytes / 15.625 sec
Improvement: 7.28% (compression ratio) / 57.63% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 17.181 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-09-11 13:52:21 +02:00
Alexander Suvorov 6b3172f793 Optimize DXT color endpoints computation 
This change significantly improves the compression speed for DXT encoding.

Explanation:

The main ideas used for the DXT color endpoints computation optimization:
- When the DXT endpoint computation function is called from the qunatization algorithm, almost all of its input parameters (except the color metrics) are hardcoded in the quantization code. This allows to optimize the endpoint evaluation function (which is the bottleneck of the endpoint computation algorithm) for this specific set of parameters.
- In the original version of the evaluation function, selectors are computed each time when a new endpoint is evaluated. While in fact, this is not necessary, because some selector values are never used, so they can be computed lazily, based on the previously determined optimal endpoint values. This approach significantly reduces the amount of computations.

Other improvements:
- The original version of Crunch has a minor bug: the counter for the cached endpoint values does not get initialized. This results in nondeterministic DXT conversion of large textures, as the counter overflow can occur at a random moment. The issue is now fixed in the current branch.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.893 sec
Modified: 1468204 bytes / 11.882 sec
Improvement: 7.21% (compression ratio) / 58.88% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.946 sec
Modified: 1914805 bytes / 15.628 sec
Improvement: 7.28% (compression ratio) / 57.70% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 17.352 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-08-11 13:12:44 +02:00
Alexander Suvorov bec4114bea Add compression support for ETC2A textures 
This change makes it possible to use Crunch algorithms for ETC textures with Alpha channel.

Explanation:

For simplicity, Crunch algorithms currently do not use ETC2 specific modes (T, H or P). For this reason, the currently used ETC2A compression format is technically equivalent to ETC1 + Alpha. Note that ETC2 encoding is a superset of ETC1, so any texture, which consists of ETC1 color blocks and ETC2 Alpha blocks, can be correctly decoded by an ETC2A (ETC2_RGBA8) decoder.

Compression scheme for ETC2 Alpha blocks is equivalent to the compression scheme for DXT5 Alpha blocks. ETC2 Alpha endpoint clusterization is performed based on the very same output of the Alpha palettizer which is used for DXT5 Alpha. The only part which is actually different is the Alpha endpoint optimization step.

In order to perform ETC2 Alpha encoding, we can first run the already existing algorithm for DXT5 Alpha endpoint optimization, in order to obtain the initial approximate solution. Then the approximate solution is refined based on the ETC2 Alpha modifier table. When performing raw ETC2A encoding, all the 16 ETC2 Alpha modifiers are used during optimization. However, when performing ETC2A quantization, for performance reasons, only 2 Alpha modifiers are currently used (modifier 13, which allows to perform precise approximation on short Alpha intervals, and modifier 11, which has more or less regularly distributed values, and is used for large Alpha intervals).

For compatibility reasons, ETC2 color compression wrappers have also been added to the code, though, as has been mentioned before, at the current moment ETC2 specific modes are not used, so ETC2 color compression is currently equivalent to ETC1 compression.

The ETC decoder functionality has been significantly extended, Crunch is now capable to decode ETC2 and ETC2A textures (input ETC2 textures can have T, H or P blocks).

In order to use ETC2A compression, use the -ETC2A command line option (i.e. "crunch_x64.exe -ETC2A input.png"). By default, compressed ETC2A textures will be decompressed into KTX file format.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.880 sec
Modified: 1468204 bytes / 13.288 sec
Improvement: 7.21% (compression ratio) / 53.99% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.936 sec
Modified: 1914805 bytes / 18.044 sec
Improvement: 7.28% (compression ratio) / 51.15% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 17.361 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-08-04 16:56:10 +02:00
Alexander Suvorov 54d4084c68 Use XOR-deltas for selector codebook encoding 
This change improves compression ratio for both DXT and ETC encodings.

Explanation:

When encoding the deltas between two pixel selectors, it is possible to use XOR-deltas instead of modulo-deltas. At first it might seem counterintuitive that XOR-delta can perform better than modulo-delta, as it does not reflect the continuity properties of the data that well. The actual trick here is that the encoded selectors are first sorted according to the used delta operation and the corresponding metric. The initial distance maps for the XOR-deltas have been obtained experimentally, using bitrate optimization on the test set of images. Additionally, ETC1 decoding has been optimized for speed: all the normal and flipped ETC1 selectors are now computed in advance.

Note: This modification alters the output file format and makes it incompatible with the previous revisions.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.899 sec
Modified: 1468204 bytes / 13.353 sec
Improvement: 7.21% (compression ratio) / 53.79% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.985 sec
Modified: 1914805 bytes / 18.111 sec
Improvement: 7.28% (compression ratio) / 51.03% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1607858 bytes
Total time: 17.356 sec
Average bitrate: 1.363 bpp
Average Luma PSNR: 34.050 dB
 2017-07-19 12:33:48 +02:00
Alexander Suvorov e972e0b480 Improve selector weight computation for ETC1 encoding 
This change improves compression ratio for ETC1 encoding.

Explanation:

When computing endpoint weights for ETC1 encoding, it is possible to use delta luma instead of the Euclidean distance between the outer endpoint colors, as it gives approximately the same result.

When computing selector weight, it is important to take into account the following factors:
- The bigger is the difference between the outer endpoint colors, the bigger error can be introduced by the corresponding selector, therefore the bigger should be the weight of that selector. In the original Crunch algorithm, the selector weight is proportional to the squared distance between the outer endpoint colors. Such optimization improves PSNR, but it might also introduce significant distortion in smooth areas of the image. In order to mitigate this effect, it is proposed to limit the maximum difference between the endpoint colors (currently delta luma is limited by 100).
- Blocks with low difference between the outer endpoint colors introduce relatively small error, so their selectors should have smaller weights. In the original algorithm it is achieved by using squared distance between the outer endpoint colors, though the effect can be amplified further by using powers higher than 2 (currently it is set to 2.7), which improves PSNR.

In the original Crunch algorithm the encoding weights are initialized non-symmetrically (and are set to math::lerp(1.15f, 1.0f, 1.0f / 7.0f) for horizontal split and to math::lerp(1.15f, 1.0f, 2.0f / 7.0f) for vertical split). It is proposed to use the same encoding weight for both splits in case of ETC1 (the used coefficient 0.972 has been computed as math::lerp(1.15f, 1.0f, 1.5f / 7.0f) / 1.15f).

The ETC1 quantization parameters have been adjusted accordingly to preserve the average Luma PSNR.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.843 sec
Modified: 1473711 bytes / 13.312 sec
Improvement: 6.86% (compression ratio) / 53.85% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.962 sec
Modified: 1920600 bytes / 18.122 sec
Improvement: 7.00% (compression ratio) / 50.97% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1612083 bytes
Total time: 17.351 sec
Average bitrate: 1.367 bpp
Average Luma PSNR: 34.050 dB
 2017-07-17 18:07:42 +02:00
Alexander Suvorov a0044903aa Use diagonal endpoint references for ETC1 encoding 
This change slightly improves compression ratio for ETC1 encoding, and also demonstrates how to adjust the endpoint reference configuration for a specific texture format.

Note: This modification alters the output file format for ETC1 encoding and makes it incompatible with the previous revisions.

Explanation:

In addition to the standard endpoint references (to the top and to the left ETC1 blocks), it is also possible to use an endpoint reference to the top-left diagonal neighbour ETC1 block. Specifically, the first ETC1 subblock will now have the reference value of 3 if the endpoint is copied from the second subblock of the top-left neighbour ETC1 block.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.895 sec
Modified: 1473711 bytes / 13.356 sec
Improvement: 6.86% (compression ratio) / 53.78% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.979 sec
Modified: 1920600 bytes / 18.083 sec
Improvement: 7.00% (compression ratio) / 51.10% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1633207 bytes
Total time: 17.434 sec
Average bitrate: 1.384 bpp
Average Luma PSNR: 34.057 dB
 2017-07-12 18:17:29 +02:00
Alexander Suvorov 7402f3d4f3 Use endpoint references for all the ETC1 subblocks 
This change significantly improves compression ratio for ETC1 encoding.

Note: This modification alters the output file format for ETC1 encoding and makes it incompatible with the previous revisions.

Explanation:

Previously, for simplicity, endpoint references for ETC1 encoding have been only computed withing the tiling area. Now endpoint references are computed for all the ETC1 subblocks. This means that endpoints can now be inherited from the surrounding ETC1 blocks, which significantly improves the compression ratio.

Endpoint references for ETC1 subblocks are encoded in the following way:
- The first ETC1 subblock has the reference value of 0 if the endpoint is decoded from the input stream, the value of 1 if the endpoint is copied from the second subblock of the left neighbour ETC1 block, and the value of 2 if the endpoint is copied from the first subblock of the top neighbour ETC1 block.
- The second ETC1 subblock has the reference value of 0 if the endpoint is copied from the first subblock, the value of 1 if the endpoint is decoded from the input stream and the corresponding ETC1 block is split horizontally, and the value of 2 if the endpoint is decoded from the input stream and the corresponding ETC1 block is split vertically.

DXT Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (revision ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.901 sec
Modified: 1473711 bytes / 13.353 sec
Improvement: 6.86% (compression ratio) / 53.80% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.997 sec
Modified: 1920600 bytes / 18.096 sec
Improvement: 7.00% (compression ratio) / 51.09% (compression time)

ETC Testing:

The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1639063 bytes
Total time: 17.421 sec
Average bitrate: 1.389 bpp
Average Luma PSNR: 34.057 dB
 2017-07-11 15:36:48 +02:00
Alexander Suvorov e3c1c6baf6 Use modulo deltas for selector codebook encoding 
This change improves compression ratio for both DXT and ETC encodings.

Explanation:

In the original version of Crunch, selector codebook is encoded with Huffman coding applied to the raw deltas between corresponding pixel selectors of the neighbour codebook elements. However, using Huffman coding for raw deltas has a downside. Specifically, for each individual pixel selector, only about a half of all the possible raw deltas are valid. Indeed, once the value of the current selector is determined, the selector delta depends only on the next selector value, so only N out of 2 * N - 1 total raw delta values are possible at any specific point. And yet, the impossible raw delta values are encoded with a non-zero probability, as the probability table is calculated throughout the whole codebook.

The situation can be improved by using modulo deltas instead of raw deltas (modulo 4 for color selectors and modulo 8 for alpha selectors). This eliminates the mentioned implicit restriction on the value of selector delta, and therefore improves the compression ratio. The distance maps are initialized using squared distances between the selector values (the distances are calculated on a wrapped interval, according to the modulo arithmetics).

DXT Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch (rev ea9b8d8).

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.870 sec
Modified: 1473711 bytes / 13.286 sec
Improvement: 6.86% (compression ratio) / 53.98% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.991 sec
Modified: 1920600 bytes / 18.035 sec
Improvement: 7.00% (compression ratio) / 51.24% (compression time)

ETC Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1681327 bytes
Total time: 17.403 sec
Average bitrate: 1.425 bpp
Average Luma PSNR: 34.057 dB
 2017-07-10 13:05:10 +02:00
Alexander Suvorov 205f8a171d Use 4x4 selector dictionary for ETC1 compression 
This change significantly improves the ETC1 compression ratio.

Explanation:

As has been shown in the previous commit, each element of the ETC1 endpoint dictionary should correspond to a single ETC1 base color. In order to achieve near-lossless compression with unlimited dictionary, it has been proposed to use 4x2 or 2x4 ETC1 subblocks as building elements, defined by a single endpoint and selector. This scheme is equivalent to the original DXT compression scheme, expect the different size of the block, defined by the dictionary elements.

Now let's pay attention to the following interesting observation. Even though in the original DXT compression scheme the dictionaries are defined in such a way, so that both endpoints and selectors from the dictionaries correspond to the same size of the decoded block (in case of DXT it is 4x4), there is no requirement for this implied by the Crunch algorithms. In fact, selector dictionary and indices are defined after the endpoint optimization is complete. At this point each image pixel is already associated with a specific endpoint. At the same time, the selector computation step is only using those per-pixel endpoint associations as an input information, so the size and the shape of the blocks, defined by selector dictionary elements, does not depend in any way on the size or shape of the blocks, defined by endpoint dictionary elements.

In other words, the endpoint space of the texture can be split into one set of blocks, defined by endpoint dictionary and endpoint indices. And the selector space of the texture can be split into absolutely different set of blocks, defined by selector dictionary and selector indices. Endpoint blocks can be different in size from the selector blocks, as well as endpoint blocks can overlap in arbitrary way with the selector blocks, and such setup will still be fully compatible with the existing Crunch algorithms.

In the current commit, the size of the block, defined by an ETC1 selector dictionary element, has been set to 4x4, which significantly improves the compression ratio (the ETC1 quantization parameters have been adjusted to preserve the average Luma PSNR).

Future research:
The discovered property of the Crunch algorithms opens another dimension for optimization of the compression ratio. Specifically, the quality of the compressed selectors can now be adjusted in two ways: by changing the size of the selector dictionary and by changing the size of the selector block. Note that both DXT and ETC formats have selectors encoded as plain bits in the output format, so there is no technical limitation on the size or shape of the selector block (though, for performance reasons, non-power-of-two selector blocks might require some specific optimizations in the decoder).

DXT Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.859 sec
Modified: 1482780 bytes / 13.326 sec
Improvement: 6.28% (compression ratio) / 53.82% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.996 sec
Modified: 1931586 bytes / 18.121 sec
Improvement: 6.47% (compression ratio) / 51.02% (compression time)

ETC Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1692204 bytes
Total time: 17.528 sec
Average bitrate: 1.434 bpp
Average Luma PSNR: 34.057 dB
 2017-07-07 17:36:30 +02:00
Alexander Suvorov f284523b15 Add compression support for ETC1 textures 
Explanation:

Crunch algorithms are normally used for compression of DXTn textures. However, Crunch algorithms are much more powerful, and with some minor adjustments, those algorithms can be directly used to compress other texture formats. For example, the current commit demonstrates how to use the existing Crunch algorithms to compress ETC1 textures.

Basics:

In general, Crunch is performing the following steps:
- tiling (determines block encodings)
- quantization of the tile endpoints (determines endpoint indices)
- optimization of the endpoints for each tile group (determines endpoint dictionary)
- quantization of the selectors (determines selector indices)
- selector refinement for each selector group (determines selector dictionary)
- compression of the previously determined block encodings, dictionaries and indices

Dictionary element:

When applying Crunch algorithms to a new texture format, it is necessary to first define the dictionary element. In context of Crunch, this means thats the whole image consists of smaller non-overlapping blocks, while the contents of each individual block is determined by an endpoint and a selector from the corresponding dictionaries. For example, in case of DXT format, each endpoint and selector codebook element corresponds to a 4x4 pixel block. In general, the size of the blocks, which form the encoded image, depends on the texture format and quality considerations.

It is proposed to define the dictionaries according to the following limitations:
- The dictionary elements should be compatible with the existing Crunch algorithms, while the image blocks defined by those dictionary elements should be compatible with the texture encoding format.
- It should be possible to cover a wide range of image quality and bitrates by just changing the size of the endpoint and selector dictionaries. If there is no limitation on the dictionary size, the encoding should preferably become lossless or near-lossless (not considering the quality loss implied by the texture format itself).

In case of ETC1, the texture format itself determines the minimal size of the image block, defined by endpoint and selector: it can be either 2x4 or 4x2 rectangle, aligned to the borders of the 4x4 grid. It is not possible to use higher granularity, because each of those rectangles can have only one base color, according to the ETC1 format. For the same reason, any image block, defined by an endpoint and a selector from the dictionary, should be combined from those aligned 2x4 or 4x2 rectangles.

Let's investigate the possibilities for the endpoint dictionary. According to the ETC1 format, each 4x4 ETC1 block is split in half, while each ETC1 subblock has it's own base color and a modifier table index. In fact, the base color and the modifier table index simply define the high and the low colors for the subblock (while there are some limitations on the position of those high and low colors, implied by the ETC1 encoding). If we define the endpoint dictionary element in such a way that it contains information about more than one ETC1 base color, then such a dictionary will become incompatible with the existing tile quantization algorithm, and the reason for this is the following. The Crunch tiling algorithm first performs quantization of all the tile pixel colors, down to just 2 colors. Then it quantizes all those color pairs, coming from different tiles. This approach works quite well for 4x4 DXT blocks, as those 2 colors approximately represent the principle component of the tile pixel colors. In case of ETC1 however, mixing together pixels, which correspond to different base colors, does not make much sense, as each group of those pixels has it's own low and high color values, independent from other groups. When those pixels are mixed together, the information about the original principle components of each subblock gets lost.

For the mentioned reason, each endpoint dictionary element should correspond to a single ETC1 base color. In such case, the tile quantization algorithm will work almost the same way as for DXT format. Each pair of colors, generated by the tile palletizer, will normally have the subblock base color value somewhere in the middle between those 2 colors, so quantizing those color pairs should also automatically quantize the corresponding base colors. Moreover, each color pair implicitly contains information about the modifier table index (which corresponds to the distance between the high and the low colors), and therefore the corresponding table index will also get automatically quantized.

Endpoint and selector dictionary elements, which define a single 2x4 or 4x2 ETC1 subblock, are fully compatible with the existing Crunch algorithms (because each ETC1 subblock is associated with a single base color and a single modifier table index). At the same time, those subblocks are minimal possible blocks, which can be defined by a dictionary element for ETC1 format (as has been shown earlier). Of course, it is also possible to use blocks larger than 2x4 or 4x2 (assuming that all the ETC1 subblocks, which form such a block, will have the same base color and the same modifier table index), however, with a larger block area it would be not possible to achieve near-lossless quality when the dictionary size is not limited.

As the result, it is proposed to define the dictionaries in the following way:
- Each element of the endpoint dictionary defines a single base color and a single modifier table index of a 2x4 or a 4x2 pixel block (which represents an ETC1 subblock).
- Each endpoint is encoded as 3555 (3 bits for the table index and 5 bits for each component of the base color).
- Each element of the selector dictionary defines selectors for a 2x4 or a 4x2 block.
- Each selector is encoded using 16 bits.

ETC1-specific adjustments:

In case of DXT, the size of the encoded block is 4x4, while the tiling is performed in a 8x8 area (4 blocks). In case of ETC1, the tiling can be performed either in a 4x4 area (2 blocks), or in a 8x8 area (8 blocks), while other possibilities are either not symmetrical or too complex. For simplicity it is proposed to use 4x4 area for tiling. There are therefore 3 possible encodings: the 4x4 block is not split (encoded with a single endpoint), the 4x4 block is split horizontally, the 4x4 block is split vertically.

For simplicity, endpoint references are currently determined only within the tiling area, while the encoding of the endpoint references has been adjusted in the following way:
- The first ETC1 subblock will always have the reference value of 0
- The second ETC1 subblock can have the reference value of 0 if it has the same endpoint as the first subblock (note that in such case the flip of the ETC1 block does not need to be defined), the value of 1 if the corresponding ETC1 block is split horizontally, and the value of 2 if the corresponding ETC1 block is split vertically

According to the ETC1 format, the base colors within an ETC1 block can be encoded either as 444 and 444, or differentially as 555 and 333. For simplicity, this aspect is currently not taken into account (all the endpoints are encoded as 3555 in the codebook). If it appears that the base colors in the resulting ETC1 block can not be encoded differentially, the decoder will convert both base colors from 555 to 444.

At first, it might look like the ETC1 block flipping can bring some complications for Crunch, as the subblock structure might not look like a grid. This can be easily resolved by mirroring all the vertical ETC1 blocks across the main diagonal of the block after the tiling step (so that all the ETC1 subblocks will become 4x2 and form a regular grid). The decoder can mirror the ETC1 selector back according to the decoded block flip.

The code adjustments for the ETC1 compression support are pretty straightforward and mostly trivial. Just note that when format-specific adjustments affect performance critical code, it makes sense to duplicate the body of the affected function and perform format-specific optimizations in each copy of the function individually. For performance reasons, the following 4 functions now got both ETC and DTX specific versions:
- determine_tiles_task_etc() is an ETC-optimized version of the determine_tiles_task(), where dxt_fast class has been replaced with the etc1_optimizer class.
- determine_color_endpoint_codebook_task_etc() is an ETC-optimized version of the determine_color_endpoint_codebook_task(), where dxt1_endpoint_optimizer class has been replaced with the etc1_optimizer class.
- pack_color_endpoints_etc() is an ETC-optimized version of the pack_color_endpoints(), where 565565 DXT color endpoint encoding has been replaced with 3555 ETC color endpoint encoding.
- unpack_etc1() is an ETC version of the unpack_dxt1() function.

The color_quality_power_mul and m_adaptive_tile_color_psnr_derating parameters for ETC1 format have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the equivalent DXT1 compression, when compressing the Kodak test set without mipmaps using default quality.

In order to use ETC1 compression, use the -ETC1 command line option (i.e. "crunch_x64.exe -ETC1 input.png"). By default, compressed ETC1 textures will be decompressed into KTX file format.

DXT Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps using DXT1 encoding]
Original: 1582222 bytes / 28.876 sec
Modified: 1482780 bytes / 13.255 sec
Improvement: 6.28% (compression ratio) / 54.10% (compression time)

[Compressing Kodak set with mipmaps using DXT1 encoding]
Original: 2065243 bytes / 36.987 sec
Modified: 1931586 bytes / 18.068 sec
Improvement: 6.47% (compression ratio) / 51.15% (compression time)

ETC Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). The ETC1 quantization parameters have been selected in such a way, so that ETC1 compression gives approximately the same average Luma PSNR as the corresponding DXT1 compression (which is equal to 34.044 dB for the Kodak test set compressed without mipmaps using DXT1 encoding and default quality settings).

[Compressing Kodak set without mipmaps using ETC1 encoding]
Total size: 1887265 bytes
Total time: 14.954 sec
Average bitrate: 1.600 bpp
Average Luma PSNR: 34.049 dB
 2017-07-05 18:19:23 +02:00
Alexander Suvorov 39b85b74c2 Optimize selector codebook creation algorithm 
This change significantly improves compression speed.

Explanation:
When generating selector codebook, pixel selectors can be processed in groups, while the intermediate error results for those groups can be precalculated.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.865 sec
Modified: 1482780 bytes / 13.340 sec
Improvement: 6.28% (compression ratio) / 53.78% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.988 sec
Modified: 1931586 bytes / 18.087 sec
Improvement: 6.47% (compression ratio) / 51.10% (compression time)
 2017-06-16 14:55:32 +02:00
Alexander Suvorov eee6b26e5d Optimize endpoint and selector sorting algorithms 
This change significantly improves compression speed.

Explanation:
The main ideas used for the endpoint and selector sorting optimization:
- unpacked color and alpha endpoints can be cached
- pixel selectors can be processed in groups, while the intermediate error results for those groups can be precalculated
- instead of maintaining the mask of the processed elements, the remaining elements can be reorganized to form a continuous block on each iteration (the last remaining element is moved into the position of the processed element)
- after optimization, endpoint sorting works significantly faster than endpoint reordering, so the overall performance can be improved by moving selector optimization into the endpoint sorting thread

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.863 sec
Modified: 1482780 bytes / 14.564 sec
Improvement: 6.28% (compression ratio) / 49.54% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.968 sec
Modified: 1931586 bytes / 19.717 sec
Improvement: 6.47% (compression ratio) / 46.66% (compression time)
 2017-06-14 14:56:41 +02:00
Alexander Suvorov f1d6a5a735 Improve and optimize the endpoint reordering algorithm 
This change significantly improves the compression ratio and compression speed.

Explanation:
After the endpoint codebook has been determined, the endpoints can be reordered in order to improve the compression ratio. On the one hand, endpoint indices of the neighbor blocks should be similar, as the encoder compresses the deltas between those neighbour indices. On the other hand, the neighbor endpoints in the codebook should be also similar, as the encoder compresses the deltas between the color components of those neighbor endpoints. The optimization is based on the Zeng's technique, using a weighted function which takes into account both similarity of the endpoint indices for the neighbor blocks and similarity of the neighbor endpoints in the codebook.

The similarity of the endpoint indices is optimized using the combined neighborhood frequency of the candidate endpoint and all the currently selected endpoints in the list. The similarity of the neighbor endpoints in the codebook is optimized using euclidian distance from the candidate endpoint to the extremity of selected endpoints list. The original optimization function for the endpoint candidate (i) can be represented as:

F(i) = (total_neighborhood_frequency(i) + 1) * (endpoint_similarity(i) + 1)

The problem with this approach is the following. While the endpoint_similarity(i) has a limited range of values, the total_neighborhood_frequency(i) grows rapidly with the increasing size of the selected endpoints list. With each iteration this introduces additional disbalance for the weighted function. In order to minimize this effect, is it proposed to normalize the total_neighborhood_frequency(i) on each iteration. For computational simplicity, the normalizer is computed as the optimal total_neighborhood_frequency value from the previous iteration, multiplied by a constant. The modified optimization function can be represented as:

F(i) = (total_neighborhood_frequency(i) + total_neighborhood_frequency_normalizer) * (endpoint_similarity(i) + 1)

The main ideas used for endpoint reordering optimization:
- all the computations, which are common for the endpoint reordering threads, have been moved outside of the threads
- the ordering histogram offsets, which point to the neighborhood frequency values for a specific endpoint, are now cached, which reduces the number of multiplications when accessing the histogram
- floating point operations have been replaced with integer operations

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.873 sec
Modified: 1482726 bytes / 15.791 sec
Improvement: 6.29% (compression ratio) / 45.31% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.925 sec
Modified: 1931475 bytes / 20.970 sec
Improvement: 6.48% (compression ratio) / 43.21% (compression time)
 2017-06-09 19:14:41 +02:00
Alexander Suvorov 5822475b22 Completely remove all the chunk related code from the encoder and decoder 
This change slightly improves compression speed and simplifies further modification of the code.

Explanation:
Additional performance boost is achieved by using linear representation for selectors and storing block selectors in a single uint32/uint64.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.927 sec
Modified: 1494501 bytes / 17.301 sec
Improvement: 5.54% (compression ratio) / 40.19% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.992 sec
Modified: 1945365 bytes / 22.548 sec
Improvement: 5.80% (compression ratio) / 39.05% (compression time)
 2017-06-07 16:55:41 +02:00
Alexander Suvorov e7d458aa22 Switch from chunk encoding to block encoding while performing image quantization 
This change improves compression speed and simplifies further modification of the code.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.947 sec
Modified: 1494501 bytes / 17.642 sec
Improvement: 5.54% (compression ratio) / 39.05% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.965 sec
Modified: 1945365 bytes / 22.989 sec
Improvement: 5.80% (compression ratio) / 37.81% (compression time)
 2017-06-02 18:13:49 +02:00
Alexander Suvorov cd9ba9b615 Switch from chunk encoding to block encoding after the tile computation 
This change improves compression speed and simplifies further modification of the code.

Explanation:
This change is required for further optimization of the tile computation code. Additional performance boost is achieved by moving the tile palettizing into the tile computation thread.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.928 sec
Modified: 1494501 bytes / 18.259 sec
Improvement: 5.54% (compression ratio) / 36.88% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.978 sec
Modified: 1945365 bytes / 23.857 sec
Improvement: 5.80% (compression ratio) / 35.48% (compression time)
 2017-05-31 15:05:32 +02:00
Alexander Suvorov 7b6f456399 Optimize selector quantization, assignment and refinement 
This change significantly improves compression speed.

Explanation:
The main ideas used for selector computations optimization:
- possible pixel values for each endpoint can be cached
- the distances between the possible pixel values and the actual pixels values within a block can be cached for fast error computation during selector assignment
- selector refinement can be efficiently integrated with the selector assignment, as it is based on the same set of cached error values
- using block encoding instead of chunk encoding for both endpoints and selectors eliminates extra levels of indirection

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.953 sec
Modified: 1494501 bytes / 19.667 sec
Improvement: 5.54% (compression ratio) / 32.07% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.998 sec
Modified: 1945365 bytes / 25.642 sec
Improvement: 5.80% (compression ratio) / 30.69% (compression time)
 2017-05-19 19:55:10 +02:00
Alexander Suvorov b8349dfac8 Use block encoding to store intermediate selectors after endpoint quantization 
This change simplifies further modification of the code.

Explanation:
This change is required for further optimization of the quantization code.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.935 sec
Modified: 1494501 bytes / 24.528 sec
Improvement: 5.54% (compression ratio) / 15.23% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.982 sec
Modified: 1945365 bytes / 32.308 sec
Improvement: 5.80% (compression ratio) / 12.64% (compression time)
 2017-05-18 13:44:04 +02:00
Alexander Suvorov 1ef829ed6f Move alpha endpoint refinement into the alpha endpoint optimization thread 
This change improves compression speed when using alpha channel.

Explanation:
As the alpha endpoint refinement does not depend on the alpha selector codebook computation, it can be safely moved into the alpha endpoint optimization thread.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.912 sec
Modified: 1494501 bytes / 24.128 sec
Improvement: 5.54% (compression ratio) / 16.55% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.985 sec
Modified: 1945365 bytes / 31.741 sec
Improvement: 5.80% (compression ratio) / 14.18% (compression time)
 2017-05-12 14:06:53 +02:00
Alexander Suvorov 9c289fc621 Move color endpoint refinement into the color endpoint optimization thread 
This change significantly improves compression speed.

Explanation:
If we take a closer look at the color endpoint refinement, we can see that the input for the color refinement comes directly from the color endpoint optimization step, while the selector codebook computation does not affect the color endpoint refinement at all. Therefore color endpoint refinement can be safely moved into the endpoint optimization thread.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.899 sec
Modified: 1494501 bytes / 24.043 sec
Improvement: 5.54% (compression ratio) / 16.80% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.884 sec
Modified: 1945365 bytes / 31.586 sec
Improvement: 5.80% (compression ratio) / 14.36% (compression time)
 2017-05-11 16:04:51 +02:00
Alexander Suvorov c9fd4dca75 Compute compressed endpoints size without pack simulation 
This change improves compression speed.

Explanation:
While trying different remappings for the endpoint indices, there is no need to perform full pack simulation when using Huffman coding. Once the delta index histogram is generated, it is sufficient to simply multiply the code sizes by the corresponding frequences in order to get the total size of the compressed endpoint indices stream. There is also no need to compute the rest of the compressed stream, as its size does not depend on the endpoint remapping and therefore is always constant, so it will not affect the size comparison during endpoint optimization.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.864 sec
Modified: 1494501 bytes / 25.317 sec
Improvement: 5.54% (compression ratio) / 12.29% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.927 sec
Modified: 1945365 bytes / 33.151 sec
Improvement: 5.80% (compression ratio) / 10.23% (compression time)
 2017-05-10 11:32:01 +02:00
Alexander Suvorov d0b6f5759b Switch from chunk encoding to block encoding after quantization 
This change simplifies further modification of the code.

Explanation:
Considering that chunks are no longer used in the output format, it makes sense to also remove chunk related code from the intermediate processing. This modification also allows to use endpoint references from the leftmost block to the rightmost block in the previous scanline (wrapped reference to the left).

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.846 sec
Modified: 1494501 bytes / 25.628 sec
Improvement: 5.54% (compression ratio) / 11.16% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.869 sec
Modified: 1945365 bytes / 33.497 sec
Improvement: 5.80% (compression ratio) / 9.15% (compression time)
 2017-05-09 17:34:21 +02:00
Alexander Suvorov 5258727545 Remove duplicate endpoints and selectors from the codebooks 
This change significantly improves the compression ratio.

Explanation:
By default, the size of the endpoint and selector codebooks is calculated based on the number of blocks in the image and the quality parameter, while the actual complexity of the image does not affect the initial codebook size. So the target codebook size is selected in such a way, that even complex images can be approximated well enough. At the same time, normally, the lower is the complexity of the image, the higher is the density of the quantized vectors. Considering that vector quantization is performed using floating point computations, and the quantized endpoints have integer components, high density of quantized vectors will result in large number of duplicate endpoints. As the result, some identical endpoints are being represented by multiple different indices, which significantly affects the compression ratio. Note that this is not the case for selectors, as their corresponding vector components are rounded after quantization, but instead it leads to some duplicate selectors in the codebook being not used. In the modified version of the algorithm all the duplicate codebook entries are merged together, unused entries are removed from the codebooks, the endpoint and selector indices are updated accordingly.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.835 sec
Modified: 1494630 bytes / 25.637 sec
Improvement: 5.54% (compression ratio) / 11.09% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.875 sec
Modified: 1946533 bytes / 33.546 sec
Improvement: 5.75% (compression ratio) / 9.03% (compression time)
 2017-05-05 20:06:00 +02:00
Alexander Suvorov ef540e54de Encode raw selector indices instead of selector indices deltas 
This change significantly improves compression ratio and compression speed.

Explanation:
The original version of Crunch encodes the differences between the neighbour indices in order to get advantage of the neighbour indices similarity. The efficiency of such approach highly depends on the continuity of the encoded data. While neighbour color and alpha endpoints are usualy similar, this is usually not the case for selectors. Of course, in some situations, encoding deltas for selector indices makes sense, for example, when the image contains a lot of regular patterns (except the special case of completely flat areas, where using selector deltas does not bring much advantage). In any case, such situations are relatively rare, so it usually appears to be more efficient to encode raw selector indices. Note that when not using deltas for selector indices, the remapping of the selector indices no longer affects the size of the encoded selector indices stream (at least when using Huffman coding). This makes the Zeng optimization step unnecessary, and it is sufficient to simply optimize the size of the packed selector codebook.

Note:
This modification alters the output file format and makes it incompatible with the previous revisions.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.845 sec
Modified: 1521167 bytes / 26.048 sec
Improvement: 3.86% (compression ratio) / 9.70% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.949 sec
Modified: 1977373 bytes / 33.889 sec
Improvement: 4.25% (compression ratio) / 8.28% (compression time)
 2017-05-05 11:26:52 +02:00
Alexander Suvorov 974fab40a5 Switch from the chunk encoding concept to the reference encoding concept 
This change improves the compression ratio.

Explanation:
In the original version of Crunch all the blocks are grouped into chunks of 2x2 blocks. Each chunk can have one of 8 different types. The type of the chunk determines which blocks inside the chunk share the same endpoints (for example, all the blocks inside the chunk share the same endpoints, or blocks in the right column share the same endpoints, or all the blocks have different endpoints, etc.). Encoding of endpoints equality is usually cheaper than encoding of duplicate endpoint indices. The used 8 chunk types do not cover all the possibilities, but they can be efficiently encoded using 0.75 bits per block (uncompressed).

The modified algorithm no longer uses the concept of chunks in the output file format and is based on an alternative approach. Endpoints for each block can be either copied from the left nearest block (reference to the left), copied from the upper nearest block (reference to the top), or decoded from the stream (reference to itself). Note that this is a superset of the original encoding, so all the images previously encoded with the original algorithm can be losslessly transcoded into the new format, but not vice versa. Even though the new endpoint equality encoding is more expensive (about 1.58 bits per block, uncompressed), it provides more flexibility for endpoint matching inside the former "chunks", and more importantly, it allows to inherit endpoints from outside the former "chunks" (which is not possible when using the original chunk encoding). The blocks are no longer grouped together and are encoded in the same order as they appear on the image.

Note:
This modification alters the output file format and makes it incompatible with the previous revisions.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.903 sec
Modified: 1548791 bytes / 28.818 sec
Improvement: 2.11% (compression ratio) / 0.29% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.978 sec
Modified: 2017245 bytes / 36.846 sec
Improvement: 2.32% (compression ratio) / 0.36% (compression time)
 2017-05-04 18:41:24 +02:00
Alexander Suvorov 178742ca6f Remove linear lists of endpoint and selector indices 
Explanation:
After switching to ordering histograms, the linear lists of endpoint and selector indices are no longer used in Zeng function, and therefore can be removed.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.872 sec
Modified: 1561622 bytes / 28.434 sec
Improvement: 1.30% (compression ratio) / 1.52% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.910 sec
Modified: 2033151 bytes / 36.369 sec
Improvement: 1.55% (compression ratio) / 1.47% (compression time)
 2017-05-02 13:03:11 +02:00
Alexander Suvorov 125536a3b5 Use left nearest block for selector index prediction 
This change improves compression ratio.

Explanation:
In the original algorithm the relative position of the block, used for prediction of the selector index for the currently decoded block, depends on the position of the current block in the chunk. It can be a horizontal neighbour or a diagonal neighbour. Using left nearest neighbour for selector index prediction for each block (except the blocks at the image borders) minimizes the average distance to the prediction block and therefore usually improves the selector index prediction. Similarly to the endpoint index processing, the selector ordering histogram in now generated based on the selector index prediction order.

Note:
This modification alters the output file format and makes it incompatible with the previous revisions.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.869 sec
Modified: 1561622 bytes / 28.522 sec
Improvement: 1.30% (compression ratio) / 1.20% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 37.038 sec
Modified: 2033151 bytes / 36.407 sec
Improvement: 1.55% (compression ratio) / 1.70% (compression time)
 2017-04-28 16:55:54 +02:00
Alexander Suvorov a4ab9fedee Generate ordering histogram for endpoint indexes based on the prediction order 
This change improves compression ratio.

Explanation:
The original histogram has been generated based on the linear order of encoded endpoint indexes. In the modified version of the algorithm, endpoint indexes are predicted using the nearest left block on the image, which is not necessarily the preceding block in the encoded sequence. Using the same block ordering both for prediction and Zeng optimization normally improves the compression ratio.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.905 sec
Modified: 1566133 bytes / 28.457 sec
Improvement: 1.02% (compression ratio) / 1.55% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 37.021 sec
Modified: 2040086 bytes / 36.300 sec
Improvement: 1.22% (compression ratio) / 1.95% (compression time)
 2017-04-28 13:49:47 +02:00
Alexander Suvorov 19f05aadbc Prepare for encoding of endpoint and selector indexes in non-linear order 
This change makes the compression scheme more flexible.

Explanation:
In the original scheme, indexes are encoded in linear order, which means that each index uses the previously encoded index for prediction. However, more sophisticated schemes might require arbitrary references into the stream of already encoded indexes. For this reason, Zeng function has been modified to accept the ordering histogram as an input, instead of the linear array of indexes. Note that Zeng function itself does not rely on the indexes being encoded in linear order.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.867 sec
Modified: 1570534 bytes / 28.524 sec
Improvement: 0.74% (compression ratio) / 1.19% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 37.001 sec
Modified: 2051509 bytes / 36.388 sec
Improvement: 0.67% (compression ratio) / 1.66% (compression time)
 2017-04-28 11:32:14 +02:00
Alexander Suvorov 8cc5f19ae5 Use left nearest block for endpoint index prediction 
This change improves compression ratio.

Explanation:
In the original algorithm the relative position of the block, used for prediction of the endpoint index for the currently decoded block, depends on the chunk encoding type. It can be a horizontal neighbour, a vertical neighbour, a diagonal neighbour, or in some rare cases even a block at relative position (-2, 0) or (-3, 0). Using left nearest neighbour for endpoint index prediction for each block (except the blocks at the image borders) minimizes the average distance to the prediction block and therefore usually improves the endpoint index prediction.

Note:
This modification alters the output file format and makes it incompatible with the previous revisions.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.838 sec
Modified: 1570534 bytes / 28.629 sec
Improvement: 0.74% (compression ratio) / 0.72% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.977 sec
Modified: 2051509 bytes / 36.568 sec
Improvement: 0.67% (compression ratio) / 1.11% (compression time)
 2017-04-27 15:49:48 +02:00
Alexander Suvorov 13b1faa48d Reorder chunks in each scanline in the left-to-right manner 
This change slightly improves compression ratio and compression time.

Explanation:
The efficiency of the Crunch encoding scheme depends on the similarity between the neighbour chunks. For this reason in original version of Crunch the order of chunks is reversed after each scanline, so that there is no jump from one side of the image to another at the image borders. The problem here is that inside of each chunk, the blocks are normally ordered in a usual up-to-down-left-to-right manner, regardless of the chunk scanning order. While on the forward scan we normally need to perform diagonal jumps (+1, +1) in order to get to the next chunk, on the reverse scan we normally need to perform much larger (-3, +1) jumps, which usually defeats the advantage of not having discontinuity at the image borders.

Note:
This modification alters the output format and makes it incompatible with the previous revisions.

Testing:
The modified algorithm has been tested on the Kodak test set using 64-bit build with default settings (running on Windows 10, i7-4790, 3.6GHz). All the decompressed test images are identical to the images being compressed and decompressed using original version of Crunch.

[Compressing Kodak set without mipmaps]
Original: 1582222 bytes / 28.882 sec
Modified: 1579618 bytes / 28.743 sec
Improvement: 0.16% (compression ratio) / 0.48% (compression time)

[Compressing Kodak set with mipmaps]
Original: 2065243 bytes / 36.920 sec
Modified: 2061499 bytes / 36.833 sec
Improvement: 0.18% (compression ratio) / 0.24% (compression time)
 2017-04-27 11:08:16 +02:00
Alexander Suvorov 5d09a511d5 Update .gitignore 2017-04-26 15:54:16 +02:00
Alexander Suvorov 1df47a4250 Remove big endian support, write barriers, byte streams and dxt1 decoding optimization from the decompression code 
This change makes the code more simple to modify. The removed functionality might be reintroduced in the future if necessary.
 2017-04-26 15:09:07 +02:00
Alexander Suvorov d34192aa07 Split the header block from the crn_decomp.h into a separate crn_defs.h file. This change makes the used CRND_HEADER_FILE_ONLY macro unneccesary. 2017-04-26 13:16:13 +02:00
Alexander Suvorov 7c02055d05 Reformat the source files. The source files have been reformatted using: clang-format.exe -style="{BasedOnStyle: Google, AllowAllParametersOfDeclarationOnNextLine: false, AllowShortFunctionsOnASingleLine: Inline, AllowShortIfStatementsOnASingleLine: false, AllowShortLoopsOnASingleLine: false, ColumnLimit: 0, DerivePointerAlignment: false, SortIncludes: false}" 2017-04-26 11:41:07 +02:00
Alexander Suvorov 41d7b962b0 Update solution to use Visual C++ 2010 compiler and libraries. When compiled with Visual Studio 2010, the code will produce the same results as the originally distributed Crunch binaries. 2017-04-26 10:59:07 +02:00
Rich Geldreich ea9b8d8c00 Fixing emscripten docs 2017-01-09 14:38:39 -08:00
Rich Geldreich 66892464f6 Merged file 2017-01-09 14:37:37 -08:00
Rich Geldreich a1281e5b2a Improve default documentation 
Conflicts:
	README.md
 2017-01-09 14:17:29 -08:00
Rich Geldreich 7ae16dda71 clang is also on board (reverted from commit 0c95eb14d6) 2017-01-09 14:12:47 -08:00
Rich Geldreich d64212091e Change data type to match assigned one. (reverted from commit 133a78398f) 2017-01-09 13:57:00 -08:00
Rich Geldreich bf5e9d9c3b Linux fixes 2017-01-09 13:54:05 -08:00
Rich Geldreich 9f20c8c318 Merge pull request #8 from nwnk/master 
readme: Update link to Compressonator
 2017-01-09 13:44:40 -08:00
Rich Geldreich 0990cc8341 Merge pull request #9 from AnalyticalGraphicsInc/emcc 
Compile to Javascript with Emscripten.
 2017-01-09 13:42:09 -08:00
Dan Bagnell 1d3fcb59cd Update README with instructions for using Emscripten. 2017-01-09 15:04:52 -05:00
Dan Bagnell 0c22f98142 Rename crn.cpp -> crunch_lib.cpp. 2017-01-09 14:57:52 -05:00
Dan Bagnell a60ed487c5 Update crn.cpp license. 2017-01-09 14:17:31 -05:00
Dan Bagnell 8fbb07dc57 Add functions to export with emscripten. 2017-01-09 14:13:39 -05:00
Adam Jackson 3cae875fda readme: Update link to Compressonator 
Signed-off-by: Adam Jackson <ajax@redhat.com>
 2016-11-15 15:12:47 -05:00
Tomer Barletz e974687ae6 Update with crunch executable. 2016-07-11 18:32:16 -07:00
Tomer Barletz 75632ff47e Fix gcc name-lookup, which requires to use the 'this' qualifier. 2016-07-11 18:31:20 -07:00
Tomer Barletz 133a78398f Change data type to match assigned one. 2016-07-11 18:28:27 -07:00
Tomer Barletz 732cd33239 Fix return types from boolean/false to pointer/NULL. 2016-07-11 15:38:49 -07:00
Tomer Barletz a9653b88e7 Add build support for 64-bit with g++. 2016-07-11 15:37:15 -07:00
Tomer Barletz 001ba71348 Add .gitignore file. 2016-07-11 15:34:36 -07:00
Arvid Gerstmann 0c95eb14d6 clang is also on board 2016-06-29 23:07:41 +02:00
Arvid Gerstmann 1faffa9b1b fixes building 2016-06-29 22:56:37 +02:00
Jake 2ef76b99e5 docs(readme): Update copyright 2016-06-20 11:53:18 +02:00
Jake bf683690a9 Merge remote-tracking branch 'refs/remotes/upstream/master' 
Conflicts:
	readme.txt
 2016-06-20 11:51:19 +02:00
Rich Geldreich 0aea5beeb2 Fixing copyright 2016-06-16 20:08:40 -07:00
Jake 8f29f0fd75 docs: Add changelog 
Moved the version info in top of readme to a changelog. Converted the
version identifiers to more semantic versioning, but it is easy to
change if semantic versioning is not interesting, or my conversion
doesn't make sense.
 2016-06-16 09:22:38 +02:00
Jake 9e7a5e80c7 docs(readme): Convert to Markdown 
Includes small tweaks to the original text for a nicer look.
 2016-06-16 09:20:43 +02:00
Rich Geldreich 97b8233acc Adding URL 2016-06-15 23:25:15 -07:00
Rich Geldreich e647680ef9 Updating github URL 2016-06-15 23:21:39 -07:00
Rich Geldreich eeeeba7e40 Updating license/copyright 2016-06-15 23:03:05 -07:00
Rich Geldreich d2a3948ab9 Updating license/copyright/email contact info 2016-06-15 22:59:25 -07:00
Rich Geldreich dbc9ddb2cd updating readme 2015-11-19 19:34:40 -08:00
Rich Geldreich 6bde5a95b1 checking in new windows execs 2015-11-19 19:14:06 -08:00
Rich Geldreich 0fddd12519 Removing linux/mingw executables 2015-11-19 19:13:44 -08:00
Rich Geldreich 91fbf1fcc4 Fixing integer overflow problem, which can rarely cause serious artifacts. 2015-11-19 18:55:22 -08:00
227 changed files with 68825 additions and 76711 deletions
Expand all files Collapse all files
.gitignore
+17
View File
@@ -0,0 +1,17 @@
*.o
*.2010.vcxproj.user
*.2010.suo
/crnlib/crunch
/crnlib/Win32
/crnlib/x64
/crunch/Win32
/crunch/x64
/example1/Win32
/example1/x64
/example2/Win32
/example2/x64
/example3/Win32
/example3/x64
/lib
/bin/*
!bin/crunch_x64.exe
CHANGELOG.md
+17
View File
@@ -0,0 +1,17 @@
# Change Log
## [0.1.4] - 2012-11-24
### Added
* KTX file format
* Basic ETC1 support
* Simple makefile
### Fixed
* Various DDS format fixes
## [0.1.3] - 2012-04-26
### Added
* Ported to Linux (tested on Ubuntu x86 w/Codeblocks). Note that a few features of the cmd line tool don't work yet (eg. -timestamp)
[0.1.4]: https://github.com/BinomialLLC/crunch
[0.1.3]: https://github.com/BinomialLLC/crunch
readme.txt → README.md
+307 -307
View File
@@ -1,307 +1,307 @@
crunch/crnlib v1.04 - Advanced DXTn texture compression library crunch/crnlib v1.04 - Advanced DXTn texture compression library
Copyright (C) 2010-2012 Rich Geldreich and Tenacious Software LLC Copyright (C) 2010-2017 Richard Geldreich, Jr. and Binomial LLC http://binomial.info
For bugs or support contact Rich Geldreich <richgel99@gmail.com>. For bugs or support contact Binomial <info@binomial.info>.
This software uses the ZLIB license, which is located in license.txt. This software uses the ZLIB license, which is located in license.txt.
http://opensource.org/licenses/Zlib http://opensource.org/licenses/Zlib
Portions of this software make use of public domain code originally Portions of this software make use of public domain code originally
written by Igor Pavlov (LZMA), RYG (crn_ryg_dxt*), and Sean Barrett (stb_image.c). written by Igor Pavlov (LZMA), RYG (crn_ryg_dxt*), and Sean Barrett (stb_image.c).
If you use this software in a product, an acknowledgment in the product If you use this software in a product, an acknowledgment in the product
documentation would be highly appreciated but is not required. documentation would be highly appreciated but is not required.
New for v1.04 [11/24/12]: KTX file format, basic ETC1 support, DDS format fixes, simple makefile Note: crunch originally used to live on Google Code: https://code.google.com/p/crunch/
------------------------------------------------
## Overview
Lots of higher level changes to get crnlib into a state where I can carry it forward to
support other file and texture compression formats. No major codec-level changes. crnlib is a lossy texture compression library for developers that ship
I've regression tested the codec writing .CRN and RDO .DDS files at various bitrates. content using the DXT1/5/N or 3DC compressed color/normal map/cubemap
Everything seems OK, but with all the changes I made to support KTX and other formats like ETC1 mipmapped texture formats. It was written by the same author as the open
I'm still worried about bugs. source [LZHAM compression library](http://code.google.com/p/lzham/).
New for v1.03 [4/26/12]: crnlib more portable, Linux Port It can compress mipmapped 2D textures, normal maps, and cubemaps to
------------------------------------------------ approx. 1-1.25 bits/texel, and normal maps to 1.75-2 bits/texel. The
actual bitrate depends on the complexity of the texture itself, the
v1.03 has been ported to Linux. It's still a work in progress. A few features specified quality factor/target bitrate, and ultimately on the desired
of the command line tool don't work under Linux yet (such as -timestamp), quality needed for a particular texture.
but the core functionality (compression/decompression/transcoding) should work
OK. I'm currently testing crnlib/crunch with Codeblocks 10.05 under Ubuntu x86. crnlib's differs significantly from other approaches because its
compressed texture data format was carefully designed to be quickly
Overview transcodable directly to DXTn with no intermediate recompression step.
-------- The typical (single threaded) transcode to DXTn rate is generally
between 100-250 megatexels/sec. The current library supports PC
crnlib is a lossy texture compression library for developers that ship (Win32/x64) and Xbox 360. Fast random access to individual mipmap levels
content using the DXT1/5/N or 3DC compressed color/normal map/cubemap is supported.
mipmapped texture formats. It was written by the same author as the open
source LZHAM lossless data compression library: crnlib can also generates standard .DDS files at specified quality
http://code.google.com/p/lzham/ setting, which results in files that are much more compressible by
LZMA/Deflate/etc. compared to files generated by standard DXTn texture
It can compress mipmapped 2D textures, normal maps, and cubemaps to tools (see below). This feature allows easy integration into any engine
approx. 1-1.25 bits/texel, and normal maps to 1.75-2 bits/texel. The or graphics library that already supports .DDS files.
actual bitrate depends on the complexity of the texture itself, the
specified quality factor/target bitrate, and ultimately on the desired The .CRN file format supports the following core DXTn texture formats:
quality needed for a particular texture. DXT1 (but not DXT1A), DXT5, DXT5A, and DXN/3DC
crnlib's differs significantly from other approaches because its It also supports several popular swizzled variants (several are
compressed texture data format was carefully designed to be quickly also supported by AMD's Compressonator):
transcodable directly to DXTn with no intermediate recompression step. DXT5_XGBR, DXT5_xGxR, DXT5_AGBR, and DXT5_CCxY (experimental luma-chroma YCoCg).
The typical (single threaded) transcode to DXTn rate is generally
between 100-250 megatexels/sec. The current library supports PC ## Recommended Software
(Win32/x64) and Xbox 360. Fast random access to individual mipmap levels
is supported. AMD's [Compressonator tool](https://github.com/GPUOpen-Tools/Compressonator)
is recommended to view the .DDS files created by the crunch tool and the included example projects.
crnlib can also generates standard .DDS files at specified quality
setting, which results in files that are much more compressible by Note: Some of the swizzled DXTn .DDS output formats (such as DXT5_xGBR)
LZMA/Deflate/etc. compared to files generated by standard DXTn texture read/written by the crunch tool or examples deviate from the DX9 DDS
tools (see below). This feature allows easy integration into any engine standard, so DXSDK tools such as DXTEX.EXE won't load them at all or
or graphics library that already supports .DDS files. they won't be properly displayed.
The .CRN file format supports the following core DXTn texture formats: ## Compression Algorithm Details
DXT1 (but not DXT1A), DXT5, DXT5A, and DXN/3DC
The compression process employed in creating both .CRN and
It also supports several popular swizzled variants (several are clustered .DDS files utilizes a very high quality, scalable DXTn
also supported by AMD's Compressonator): endpoint optimizer capable of processing any number of pixels (instead
DXT5_XGBR, DXT5_xGxR, DXT5_AGBR, and DXT5_CCxY (experimental luma-chroma YCoCg). of the typical hard coded 16), optional adaptive switching between
several macroblock sizes/configurations (currently any combination of
Recommended Software 4x4, 8x4, 4x8, and 8x8 pixel blocks), endpoint clusterization using
-------------------- top-down cluster analysis, vector quantization (VQ) of the selector
indices, and several custom algorithms for compressing the resulting
AMD's Compressonator tool is recommended to view the .DDS files created by endpoint/selector codebooks and macroblock indices. Multiple feedback
the crunch tool and the included example projects: passes are performed between the clusterization and VQ steps to optimize
quality, and several steps use a brute force refinement approach to improve
http://developer.amd.com/gpu/compressonator/pages/default.aspx quality. The majority of compression steps are multithreaded.
Note: Some of the swizzled DXTn .DDS output formats (such as DXT5_xGBR) The .CRN format currently utilizes canonical Huffman coding for speed
read/written by the crunch tool or examples deviate from the DX9 DDS (similar to Deflate but with much larger tables), but the next major
standard, so DXSDK tools such as DXTEX.EXE won't load them at all or version will also utilize adaptive binary arithmetic coding and higher
they won't be properly displayed. order context modeling using already developed tech from the my LZHAM
compression library.
Compression Algorithm Details
----------------------------- ## Supported File Formats
The compression process employed in creating both .CRN and crnlib supports two compressed texture file formats. The first
clustered .DDS files utilizes a very high quality, scalable DXTn format (clustered .DDS) is simple to integrate into an existing project
endpoint optimizer capable of processing any number of pixels (instead (typically, no code changes are required), but it doesn't offer the
of the typical hard coded 16), optional adaptive switching between highest quality/compression ratio that crnlib is capable of. Integrating
several macroblock sizes/configurations (currently any combination of the second, higher quality custom format (.CRN) requires a few
4x4, 8x4, 4x8, and 8x8 pixel blocks), endpoint clusterization using typically straightforward engine modifications to integrate the
top-down cluster analysis, vector quantization (VQ) of the selector .CRN->DXTn transcoder header file library into your tools/engine.
indices, and several custom algorithms for compressing the resulting
endpoint/selector codebooks and macroblock indices. Multiple feedback ### .DDS
passes are performed between the clusterization and VQ steps to optimize
quality, and several steps use a brute force refinement approach to improve crnlib can compress textures to standard DX9-style .DDS files using
quality. The majority of compression steps are multithreaded. clustered DXTn compression, which is a subset of the approach used to
create .CRN files.(For completeness, crnlib also supports vanilla, block
The .CRN format currently utilizes canonical Huffman coding for speed by block DXTn compression too, but that's not very interesting.)
(similar to Deflate but with much larger tables), but the next major Clustered DXTn compressed .DDS files are much more compressible than
version will also utilize adaptive binary arithmetic coding and higher files created by other libraries/tools. Apart from increased
order context modeling using already developed tech from the my LZHAM compressibility, the .DDS files generated by this process are completely
compression library. standard so they should be fairly easy to add to a project with little
to no code changes.
Supported File Formats
---------------------- To actually benefit from clustered DXTn .DDS files, your engine needs to
further losslessly compress the .DDS data generated by crnlib using a
crnlib supports two compressed texture file formats. The first lossless codec such as zlib, lzo, LZMA, LZHAM, etc. Most likely, your
format (clustered .DDS) is simple to integrate into an existing project engine does this already. (If not, you definitely should because DXTn
(typically, no code changes are required), but it doesn't offer the compressed textures generally contain a large amount of highly redundant
highest quality/compression ratio that crnlib is capable of. Integrating data.)
the second, higher quality custom format (.CRN) requires a few
typically straightforward engine modifications to integrate the Clustered .DDS files are intended to be the simplest/fastest way to
.CRN->DXTn transcoder header file library into your tools/engine. integrate crnlib's tech into a project.
.DDS ### .CRN
crnlib can compress textures to standard DX9-style .DDS files using
clustered DXTn compression, which is a subset of the approach used to The second, better, option is to compress your textures to .CRN files
create .CRN files.(For completeness, crnlib also supports vanilla, block using crnlib. To read the resulting .CRN data, you must add the .CRN
by block DXTn compression too, but that's not very interesting.) transcoder library (located in the included single file, stand-alone
Clustered DXTn compressed .DDS files are much more compressible than header file library inc/crn_decomp.h) into your application. .CRN files
files created by other libraries/tools. Apart from increased provide noticeably higher quality at the same effective bitrate compared
compressibility, the .DDS files generated by this process are completely to clustered DXTn compressed .DDS files. Also, .CRN files don't require
standard so they should be fairly easy to add to a project with little further lossless compression because they're already highly compressed.
to no code changes.
.CRN files are a bit more difficult/risky to integrate into a project, but
To actually benefit from clustered DXTn .DDS files, your engine needs to the resulting compression ratio and quality is superior vs. clustered .DDS files.
further losslessly compress the .DDS data generated by crnlib using a
lossless codec such as zlib, lzo, LZMA, LZHAM, etc. Most likely, your ### .KTX
engine does this already. (If not, you definitely should because DXTn
compressed textures generally contain a large amount of highly redundant crnlib and crunch can read/write the .KTX file format in various pixel formats.
data.) Rate distortion optimization (clustered DXTc compression) is not yet supported
when writing .KTX files.
Clustered .DDS files are intended to be the simplest/fastest way to
integrate crnlib's tech into a project. The .KTX file format is just like .DDS, except it's a fairly well specified
standard created by the Khronos Group. Unfortunately, almost all of the tools I've
.CRN found that support .KTX are fairly (to very) buggy, or are limited to only a handful
The second, better, option is to compress your textures to .CRN files of pixel formats, so there's no guarantee that the .KTX files written by crnlib can
using crnlib. To read the resulting .CRN data, you must add the .CRN be reliably read by other tools.
transcoder library (located in the included single file, stand-alone
header file library inc/crn_decomp.h) into your application. .CRN files ## Building the Examples
provide noticeably higher quality at the same effective bitrate compared
to clustered DXTn compressed .DDS files. Also, .CRN files don't require This release contains the source code and projects for three simple
further lossless compression because they're already highly compressed. example projects:
.CRN files are a bit more difficult/risky to integrate into a project, but crn_examples.2010.sln is a Visual Studio 2010 (VC10) solution file
the resulting compression ratio and quality is superior vs. clustered .DDS files. containing projects for Win32 and x64. crnlib itself also builds with
VS2005, VS2008, and gcc 4.5.0 (TDM GCC+MinGW). A codeblocks 10.05
.KTX workspace and project file is also included, but compiling crnlib this
way hasn't been tested much.
crnlib and crunch can read/write the .KTX file format in various pixel formats.
Rate distortion optimization (clustered DXTc compression) is not yet supported ### example1
when writing .KTX files.
Demonstrates how to use crnlib's high-level C-helper
The .KTX file format is just like .DDS, except it's a fairly well specified compression/decompression/transcoding functions in inc/crnlib.h. It's a
standard created by the Khronos Group. Unfortunately, almost all of the tools I've fairly complete example of crnlib's functionality.
found that support .KTX are fairly (to very) buggy, or are limited to only a handful
of pixel formats, so there's no guarantee that the .KTX files written by crnlib can ### example2
be reliably read by other tools. Shows how to transcodec .CRN files to .DDS using **only**
the functionality in inc/crn_decomp.h. It does not link against against
Building the Examples crnlib.lib or depend on it in any way. (Note: The complete source code,
--------------------- approx. 4800 lines, to the CRN transcoder is included in inc/crn_decomp.h.)
This release contains the source code and projects for three simple example2 is intended to show how simple it is to integrate CRN textures
example projects: into your application.
crn_examples.2008.sln is a Visual Studio 2008 (VC9) solution file ### example3
containing projects for Win32 and x64. crnlib itself also builds with Shows how to use the regular, low-level DXTn block compressor
VS2005, VS2010, and gcc 4.5.0 (TDM GCC+MinGW). A codeblocks 10.05 functions in inc/crnlib.h. This functionality is included for
workspace and project file is also included, but compiling crnlib this completeness. (Your engine or toolchain most likely already has its own
way hasn't been tested much. DXTn compressor. crnlib's compressor is typically very competitive or
superior to most available closed and open source CPU-based
example1: Demonstrates how to use crnlib's high-level C-helper compressors.)
compression/decompression/transcoding functions in inc/crnlib.h. It's a
fairly complete example of crnlib's functionality. ## Creating Compressed Textures from the Command Line (crunch.exe)
example2: Shows how to transcodec .CRN files to .DDS using *only* The simplest way to create compressed textures using crnlib is to
the functionality in inc/crn_decomp.h. It does not link against against integrate the bin\crunch.exe or bin\crunch_x64.exe) command line tool
crnlib.lib or depend on it in any way. (Note: The complete source code, into your texture build toolchain or export process. It can write DXTn
approx. 4800 lines, to the CRN transcoder is included in inc/crn_decomp.h.) compressed 2D/cubemap textures to regular DXTn compressed .DDS,
clustered (or reduced entropy) DXTn compressed .DDS, or .CRN files. It
example2 is intended to show how simple it is to integrate CRN textures can also transcode or decompress files to several standard image
into your application. formats, such as TGA or BMP. Run crunch.exe with no options for help.
example3: Shows how to use the regular, low-level DXTn block compressor The .CRN files created by crunch.exe can be efficiently transcoded to
functions in inc/crnlib.h. This functionality is included for DXTn using the included CRN transcoding library, located in full source
completeness. (Your engine or toolchain most likely already has its own form under inc/crn_decomp.h.
DXTn compressor. crnlib's compressor is typically very competitive or
superior to most available closed and open source CPU-based Here are a few example crunch.exe command lines:
compressors.)
1. Compress blah.tga to blah.dds using normal DXT1 compression:
Creating Compressed Textures from the Command Line (crunch.exe) * `crunch -file blah.tga -fileformat dds -dxt1`
---------------------------------------------------------------
2. Compress blah.tga to blah.dds using clustered DXT1 at an effective bitrate of 1.5 bits/texel, display image statistic:
The simplest way to create compressed textures using crnlib is to * `crunch -file blah.tga -fileformat dds -dxt1 -bitrate 1.5 -imagestats`
integrate the bin\crunch.exe or bin\crunch_x64.exe) command line tool
into your texture build toolchain or export process. It can write DXTn 3. Compress blah.tga to blah.dds using clustered DXT1 at quality level 100 (from [0,255]), with no mipmaps, display LZMA statistics:
compressed 2D/cubemap textures to regular DXTn compressed .DDS, * `crunch -file blah.tga -fileformat dds -dxt1 -quality 100 -mipmode none -lzmastats`
clustered (or reduced entropy) DXTn compressed .DDS, or .CRN files. It
can also transcode or decompress files to several standard image 3. Compress blah.tga to blah.crn using clustered DXT1 at a bitrate of 1.2 bits/texel, no mipmaps:
formats, such as TGA or BMP. Run crunch.exe with no options for help. * `crunch -file blah.tga -dxt1 -bitrate 1.2 -mipmode none`
The .CRN files created by crunch.exe can be efficiently transcoded to 4. Decompress blah.dds to a .tga file:
DXTn using the included CRN transcoding library, located in full source * `crunch -file blah.dds -fileformat tga`
form under inc/crn_decomp.h.
5. Transcode blah.crn to a .dds file:
Here are a few example crunch.exe command lines: * `crunch -file blah.crn`
1. Compress blah.tga to blah.dds using normal DXT1 compression: 6. Decompress blah.crn, writing each mipmap level to a separate .tga file:
crunch -file blah.tga -fileformat dds -dxt1 * `crunch -split -file blah.crn -fileformat tga`
2. Compress blah.tga to blah.dds using clustered DXT1 at an effective bitrate of 1.5 bits/texel, display image statistic: crunch.exe can do a lot more, like rescale/crop images before
crunch -file blah.tga -fileformat dds -dxt1 -bitrate 1.5 -imagestats compression, convert images from one file format to another, compare
images, process multiple images, etc.
3. Compress blah.tga to blah.dds using clustered DXT1 at quality level 100 (from [0,255]), with no mipmaps, display LZMA statistics:
crunch -file blah.tga -fileformat dds -dxt1 -quality 100 -mipmode none -lzmastats Note: I would have included the full source to crunch.exe, but it still
has some low-level dependencies to crnlib internals which I didn't have
3. Compress blah.tga to blah.crn using clustered DXT1 at a bitrate of 1.2 bits/texel, no mipmaps: time to address. This version of crunch.exe has some reduced
crunch -file blah.tga -dxt1 -bitrate 1.2 -mipmode none functionality compared to an earlier eval release. For example, XML file
support is not included in this version.
4. Decompress blah.dds to a .tga file:
crunch -file blah.dds -fileformat tga ## Using crnlib
5. Transcode blah.crn to a .dds file: The most flexible and powerful way of using crnlib is to integrate the
crunch -file blah.crn library into your editor/toolchain/etc. and directly supply it your
raw/source texture bits. See the C-style API's and comments in
6. Decompress blah.crn, writing each mipmap level to a separate .tga file: inc/crnlib.h.
crunch -split -file blah.crn -fileformat tga
To compress, you basically fill in a few structs in and call one function:
crunch.exe can do a lot more, like rescale/crop images before
compression, convert images from one file format to another, compare ```c
images, process multiple images, etc. void *crn_compress( const crn_comp_params &comp_params,
crn_uint32 &compressed_size,
Note: I would have included the full source to crunch.exe, but it still crn_uint32 *pActual_quality_level = NULL,
has some low-level dependencies to crnlib internals which I didn't have float *pActual_bitrate = NULL);
time to address. This version of crunch.exe has some reduced ```
functionality compared to an earlier eval release. For example, XML file
support is not included in this version. Or, if you want crnlib to also generate mipmaps, you call this function:
Using crnlib ```c
------------ void *crn_compress( const crn_comp_params &comp_params,
const crn_mipmap_params &mip_params,
The most flexible and powerful way of using crnlib is to integrate the crn_uint32 &compressed_size,
library into your editor/toolchain/etc. and directly supply it your crn_uint32 *pActual_quality_level = NULL,
raw/source texture bits. See the C-style API's and comments in float *pActual_bitrate = NULL);
inc/crnlib.h. ```
To compress, you basically fill in a few structs in and call one function: You can also transcode/uncompress .DDS/.CRN files to raw 32bpp images
using `crn_decompress_crn_to_dds()` and `crn_decompress_dds_to_images()`.
void *crn_compress(const crn_comp_params &comp_params, crn_uint32 &compressed_size, crn_uint32 *pActual_quality_level = NULL, float *pActual_bitrate = NULL);
Internally, crnlib just uses inc/crn_decomp.h to transcode textures to
Or, if you want crnlib to also generate mipmaps, you call this function: DXTn. If you only need to transcode .CRN format files to raw DXTn bits
at runtime (and not compress), you don't actually need to compile or
void *crn_compress(const crn_comp_params &comp_params, const crn_mipmap_params &mip_params, crn_uint32 &compressed_size, crn_uint32 *pActual_quality_level = NULL, float *pActual_bitrate = NULL); link against crnlib at all. Just include inc/crn_decomp.h, which
contains a completely self-contained CRN transcoder in the "crnd"
You can also transcode/uncompress .DDS/.CRN files to raw 32bpp images namespace. The `crnd_get_texture_info()`, `crnd_unpack_begin()`,
using crn_decompress_crn_to_dds() and crn_decompress_dds_to_images(). `crnd_unpack_level()`, etc. functions are all you need to efficiently get
at the raw DXTn bits, which can be directly supplied to whatever API or
Internally, crnlib just uses inc/crn_decomp.h to transcode textures to GPU you're using. (See example2.)
DXTn. If you only need to transcode .CRN format files to raw DXTn bits
at runtime (and not compress), you don't actually need to compile or Important note: When compiling under native client, be sure to define
link against crnlib at all. Just include inc/crn_decomp.h, which the `PLATFORM_NACL` macro before including the `inc/crn_decomp.h` header file library.
contains a completely self-contained CRN transcoder in the "crnd"
namespace. The crnd_get_texture_info(), crnd_unpack_begin(), ## Known Issues/Bugs
crnd_unpack_level(), etc. functions are all you need to efficiently get
at the raw DXTn bits, which can be directly supplied to whatever API or * crnlib currently assumes you'll be further losslessly compressing its
GPU you're using. (See example2.) output .DDS files using LZMA. However, some engines use weaker codecs
such as LZO, zlib, or custom codecs, so crnlib's bitrate measurements
Important note: When compiling under native client, be sure to define will be inaccurate. It should be easy to allow the caller to plug-in
the PLATFORM_NACL macro before including the inc/crn_decomp.h header file library. custom lossless compressors for bitrate measurement.
Known Issues/Bugs * Compressing to a desired bitrate can be time consuming, especially when
----------------- processing large (2k or 4k) images to the .CRN format. There are several
high-level optimizations employed when compressing to clustered DXTn .DDS
* crnlib currently assumes you'll be further losslessly compressing its files using multiple trials, but not so for .CRN.
output .DDS files using LZMA. However, some engines use weaker codecs
such as LZO, zlib, or custom codecs, so crnlib's bitrate measurements * The .CRN compressor does not currently use 3 color (transparent) DXT1
will be inaccurate. It should be easy to allow the caller to plug-in blocks at all, only 4 color blocks. So it doesn't support DXT1A
custom lossless compressors for bitrate measurement. transparency, and its output quality suffers a little due to this
limitation. (Note that the clustered DXTn compressor used when
* Compressing to a desired bitrate can be time consuming, especially when writing clustered .DDS files does *not* have this limitation.)
processing large (2k or 4k) images to the .CRN format. There are several
high-level optimizations employed when compressing to clustered DXTn .DDS * Clustered DXT5/DXT5A compressor is able to group DXT5A blocks into
files using multiple trials, but not so for .CRN. clusters only if they use absolute (black/white) selector indices. This
hurts performance at very low bitrates, because too many bits are
* The .CRN compressor does not currently use 3 color (transparent) DXT1 effectively given to alpha.
blocks at all, only 4 color blocks. So it doesn't support DXT1A
transparency, and its output quality suffers a little due to this * DXT3 is not supported when writing .CRN or clustered DXTn DDS files.
limitation. (Note that the clustered DXTn compressor used when (DXT3 is supported by crnlib's when compressing to regular DXTn DDS
writing clustered .DDS files does *not* have this limitation.) files.) You'll get DXT5 files if you request DXT3. However, DXT3 is
supported by the regular DXTn block compressor. (DXT3's 4bpp fixed alpha
* Clustered DXT5/DXT5A compressor is able to group DXT5A blocks into sucks verses DXT5 alpha blocks, so I don't see this as a bug deal.)
clusters only if they use absolute (black/white) selector indices. This
hurts performance at very low bitrates, because too many bits are * The DXT5_CCXY format uses a simple YCoCg encoding that is workable but
effectively given to alpha. hasn't been tuned for max. quality yet.
* DXT3 is not supported when writing .CRN or clustered DXTn DDS files. * Clustered (or rate distortion optimized) DXTc compression is only
(DXT3 is supported by crnlib's when compressing to regular DXTn DDS supported when writing to .DDS, not .KTX. Also, only plain block by block
files.) You'll get DXT5 files if you request DXT3. However, DXT3 is compression is supported when writing to ETC1, and .CRN does not support ETC1.
supported by the regular DXTn block compressor. (DXT3's 4bpp fixed alpha
sucks verses DXT5 alpha blocks, so I don't see this as a bug deal.) ## Compile to Javascript with Emscripten
* The DXT5_CCXY format uses a simple YCoCg encoding that is workable but Download and install Emscripten:
hasn't been tuned for max. quality yet. http://kripken.github.io/emscripten-site/docs/getting_started/downloads.html
* Clustered (or rate distortion optimized) DXTc compression is only From the root directory, run:
supported when writing to .DDS, not .KTX. Also, only plain block by block ```c
compression is supported when writing to ETC1, and .CRN does not support ETC1. emcc -O3 emscripten/crn.cpp -I./inc -s EXPORTED_FUNCTIONS="['_malloc', '_free', '_crn_get_width', '_crn_get_height', '_crn_get_levels', '_crn_get_dxt_format', '_crn_get_bytes_per_block', '_crn_get_uncompressed_size', '_crn_decompress']" -s NO_EXIT_RUNTIME=1 -s NO_FILESYSTEM=1 -s ELIMINATE_DUPLICATE_FUNCTIONS=1 -s ALLOW_MEMORY_GROWTH=1 --memory-init-file 0 -o crunch.js
```
bin/crunch.exe
BIN
View File
Binary file not shown.
bin/crunch.pdb
BIN
View File
Binary file not shown.
bin/crunch_x64.exe
BIN
View File
Binary file not shown.
bin/crunch_x64.pdb
BIN
View File
Binary file not shown.
bin_linux/crunch
BIN
View File
Binary file not shown.
bin_mingw/crunch.exe
BIN
View File
Binary file not shown.
crn.2008.sln → crn.2010.sln
+4 -7
View File
@@ -1,12 +1,9 @@
 
Microsoft Visual Studio Solution File, Format Version 10.00 Microsoft Visual Studio Solution File, Format Version 11.00
# Visual Studio 2008 # Visual Studio 2010
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "crunch", "crunch\crunch.2008.vcproj", "{8F645BA1-B996-49EB-859B-970A671DE05D}" Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "crunch", "crunch\crunch.2010.vcxproj", "{8F645BA1-B996-49EB-859B-970A671DE05D}"
ProjectSection(ProjectDependencies) = postProject
{CF2E70E8-7133-4D96-92C7-68BB406C0664} = {CF2E70E8-7133-4D96-92C7-68BB406C0664}
EndProjectSection
EndProject EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "crnlib", "crnlib\crnlib.2008.vcproj", "{CF2E70E8-7133-4D96-92C7-68BB406C0664}" Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "crnlib", "crnlib\crnlib.2010.vcxproj", "{CF2E70E8-7133-4D96-92C7-68BB406C0664}"
EndProject EndProject
Global Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution GlobalSection(SolutionConfigurationPlatforms) = preSolution
crn_examples.2008.sln → crn_examples.2010.sln
+5 -5
View File
@@ -1,11 +1,11 @@
 
Microsoft Visual Studio Solution File, Format Version 10.00 Microsoft Visual Studio Solution File, Format Version 11.00
# Visual Studio 2008 # Visual Studio 2010
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "example1", "example1\example1.2008.vcproj", "{8F745B42-F996-49EB-859B-970A671DE05D}" Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "example1", "example1\example1.2010.vcxproj", "{8F745B42-F996-49EB-859B-970A671DE05D}"
EndProject EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "example2", "example2\example2.2008.vcproj", "{AF745B42-F996-49EB-859B-970A671DEF5E}" Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "example2", "example2\example2.2010.vcxproj", "{AF745B42-F996-49EB-859B-970A671DEF5E}"
EndProject EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "example3", "example3\example3.2008.vcproj", "{AF745B42-E296-46EB-859B-970A671DEF5E}" Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "example3", "example3\example3.2010.vcxproj", "{AF745B42-E296-46EB-859B-970A671DEF5E}"
EndProject EndProject
Global Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution GlobalSection(SolutionConfigurationPlatforms) = preSolution
crnlib/Makefile
+3 -10
View File
@@ -1,8 +1,6 @@
COMPILE_OPTIONS = -O3 -fomit-frame-pointer -ffast-math -fno-math-errno -g -fPIC -fno-strict-aliasing -Wall -Wno-unused-value -Wno-unused -march=core2 COMPILE_OPTIONS = -O3 -fomit-frame-pointer -ffast-math -fno-math-errno -g -fno-strict-aliasing -Wall -Wno-unused-value -Wno-unused -march=core2
LINKER_OPTIONS = -lpthread -g LINKER_OPTIONS = -lpthread -g
LIBCRUNCH = libcrunch.a
OBJECTS = \ OBJECTS = \
crn_arealist.o \ crn_arealist.o \
crn_assert.o \ crn_assert.o \
@@ -84,9 +82,6 @@ all: crunch
%.o: %.cpp %.o: %.cpp
g++ $< -o $@ -c $(COMPILE_OPTIONS) g++ $< -o $@ -c $(COMPILE_OPTIONS)
$(LIBCRUNCH): $(OBJECTS)
ar rcs $(LIBCRUNCH) $(OBJECTS)
crunch.o: ../crunch/crunch.cpp crunch.o: ../crunch/crunch.cpp
g++ $< -o $@ -c -I../inc -I../crnlib $(COMPILE_OPTIONS) g++ $< -o $@ -c -I../inc -I../crnlib $(COMPILE_OPTIONS)
@@ -96,8 +91,6 @@ corpus_gen.o: ../crunch/corpus_gen.cpp
corpus_test.o: ../crunch/corpus_test.cpp corpus_test.o: ../crunch/corpus_test.cpp
g++ $< -o $@ -c -I../inc -I../crnlib $(COMPILE_OPTIONS) g++ $< -o $@ -c -I../inc -I../crnlib $(COMPILE_OPTIONS)
crunch: $(LIBCRUNCH) crunch.o corpus_gen.o corpus_test.o crunch: $(OBJECTS) crunch.o corpus_gen.o corpus_test.o
g++ crunch.o corpus_gen.o corpus_test.o -o crunch $(LIBCRUNCH) $(LINKER_OPTIONS) g++ $(OBJECTS) crunch.o corpus_gen.o corpus_test.o -o crunch $(LINKER_OPTIONS)
clean:
rm -rf $(LIBCRUNCH) crunch *.o
crnlib/crn_arealist.cpp
+455 -528
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_arealist.h
+49 -52
View File
@@ -2,73 +2,70 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
namespace crnlib namespace crnlib {
{ struct Area {
struct Area struct Area *Pprev, *Pnext;
{
struct Area *Pprev, *Pnext;
int x1, y1, x2, y2; int x1, y1, x2, y2;
uint get_width() const { return x2 - x1 + 1; }
uint get_height() const { return y2 - y1 + 1; }
uint get_area() const { return get_width() * get_height(); }
};
typedef Area * Area_Ptr; uint get_width() const { return x2 - x1 + 1; }
uint get_height() const { return y2 - y1 + 1; }
uint get_area() const { return get_width() * get_height(); }
};
struct Area_List typedef Area* Area_Ptr;
{
int total_areas;
int next_free;
Area *Phead, *Ptail, *Pfree; struct Area_List {
}; int total_areas;
int next_free;
typedef Area_List * Area_List_Ptr; Area *Phead, *Ptail, *Pfree;
};
Area_List * Area_List_init(int max_areas); typedef Area_List* Area_List_Ptr;
void Area_List_deinit(Area_List* Pobj_base);
void Area_List_print(Area_List *Plist); Area_List* Area_List_init(int max_areas);
void Area_List_deinit(Area_List* Pobj_base);
Area_List * Area_List_dup_new(Area_List *Plist, void Area_List_print(Area_List* Plist);
int x_ofs, int y_ofs);
uint Area_List_get_num(Area_List* Plist);
// src and dst area lists must have the same number of total areas. Area_List* Area_List_dup_new(Area_List* Plist,
void Area_List_dup(Area_List *Psrc_list, int x_ofs, int y_ofs);
Area_List *Pdst_list,
int x_ofs, int y_ofs);
void Area_List_copy(Area_List *Psrc_list, uint Area_List_get_num(Area_List* Plist);
Area_List *Pdst_list,
int x_ofs, int y_ofs);
void Area_List_clear(Area_List *Plist); // src and dst area lists must have the same number of total areas.
void Area_List_dup(Area_List* Psrc_list,
Area_List* Pdst_list,
int x_ofs, int y_ofs);
void Area_List_set(Area_List *Plist, void Area_List_copy(Area_List* Psrc_list,
Area_List* Pdst_list,
int x_ofs, int y_ofs);
void Area_List_clear(Area_List* Plist);
void Area_List_set(Area_List* Plist,
int x1, int y1, int x2, int y2);
// logical: x and (not y)
void Area_List_remove(Area_List* Plist,
int x1, int y1, int x2, int y2); int x1, int y1, int x2, int y2);
// logical: x and (not y) // logical: x or y
void Area_List_remove(Area_List *Plist, void Area_List_insert(Area_List* Plist,
int x1, int y1, int x2, int y2); int x1, int y1, int x2, int y2,
bool combine);
// logical: x or y // logical: x and y
void Area_List_insert(Area_List *Plist, void Area_List_intersect_area(Area_List* Plist,
int x1, int y1, int x2, int y2, int x1, int y1, int x2, int y2);
bool combine);
// logical: x and y // logical: x and y
void Area_List_intersect_area(Area_List *Plist, void Area_List_intersect_Area_List(Area_List* Pouter_list,
int x1, int y1, int x2, int y2); Area_List* Pinner_list,
Area_List* Pdst_list);
// logical: x and y Area_List_Ptr Area_List_create_optimal(Area_List_Ptr Plist);
void Area_List_intersect_Area_List(Area_List *Pouter_list,
Area_List *Pinner_list,
Area_List *Pdst_list);
Area_List_Ptr Area_List_create_optimal(Area_List_Ptr Plist); } // namespace crnlib
} // namespace crnlib
crnlib/crn_assert.cpp
+32 -38
View File
@@ -8,62 +8,56 @@
static bool g_fail_exceptions; static bool g_fail_exceptions;
static bool g_exit_on_failure = true; static bool g_exit_on_failure = true;
void crnlib_enable_fail_exceptions(bool enabled) void crnlib_enable_fail_exceptions(bool enabled) {
{ g_fail_exceptions = enabled;
g_fail_exceptions = enabled;
} }
void crnlib_assert(const char* pExp, const char* pFile, unsigned line) void crnlib_assert(const char* pExp, const char* pFile, unsigned line) {
{ char buf[512];
char buf[512];
sprintf_s(buf, sizeof(buf), "%s(%u): Assertion failed: \"%s\"\n", pFile, line, pExp); sprintf_s(buf, sizeof(buf), "%s(%u): Assertion failed: \"%s\"\n", pFile, line, pExp);
crnlib_output_debug_string(buf); crnlib_output_debug_string(buf);
fputs(buf, stderr); fputs(buf, stderr);
if (crnlib_is_debugger_present()) if (crnlib_is_debugger_present())
crnlib_debug_break(); crnlib_debug_break();
} }
void crnlib_fail(const char* pExp, const char* pFile, unsigned line) void crnlib_fail(const char* pExp, const char* pFile, unsigned line) {
{ char buf[512];
char buf[512];
sprintf_s(buf, sizeof(buf), "%s(%u): Failure: \"%s\"\n", pFile, line, pExp); sprintf_s(buf, sizeof(buf), "%s(%u): Failure: \"%s\"\n", pFile, line, pExp);
crnlib_output_debug_string(buf); crnlib_output_debug_string(buf);
fputs(buf, stderr); fputs(buf, stderr);
if (crnlib_is_debugger_present()) if (crnlib_is_debugger_present())
crnlib_debug_break(); crnlib_debug_break();
#if CRNLIB_USE_WIN32_API #if CRNLIB_USE_WIN32_API
if (g_fail_exceptions) if (g_fail_exceptions)
RaiseException(CRNLIB_FAIL_EXCEPTION_CODE, 0, 0, NULL); RaiseException(CRNLIB_FAIL_EXCEPTION_CODE, 0, 0, NULL);
else else
#endif #endif
if (g_exit_on_failure) if (g_exit_on_failure)
exit(EXIT_FAILURE); exit(EXIT_FAILURE);
} }
void trace(const char* pFmt, va_list args) void trace(const char* pFmt, va_list args) {
{ if (crnlib_is_debugger_present()) {
if (crnlib_is_debugger_present()) char buf[512];
{ vsprintf_s(buf, sizeof(buf), pFmt, args);
char buf[512];
vsprintf_s(buf, sizeof(buf), pFmt, args);
crnlib_output_debug_string(buf); crnlib_output_debug_string(buf);
} }
}; };
void trace(const char* pFmt, ...) void trace(const char* pFmt, ...) {
{ va_list args;
va_list args; va_start(args, pFmt);
va_start(args, pFmt); trace(pFmt, args);
trace(pFmt, args); va_end(args);
va_end(args);
}; };
crnlib/crn_assert.h
+32 -26
View File
@@ -9,16 +9,19 @@ void crnlib_assert(const char* pExp, const char* pFile, unsigned line);
void crnlib_fail(const char* pExp, const char* pFile, unsigned line); void crnlib_fail(const char* pExp, const char* pFile, unsigned line);
#ifdef NDEBUG #ifdef NDEBUG
#define CRNLIB_ASSERT(x) ((void)0) #define CRNLIB_ASSERT(x) ((void)0)
#undef CRNLIB_ASSERTS_ENABLED #undef CRNLIB_ASSERTS_ENABLED
#else #else
#define CRNLIB_ASSERT(_exp) (void)( (!!(_exp)) || (crnlib_assert(#_exp, __FILE__, __LINE__), 0) ) #define CRNLIB_ASSERT(_exp) (void)((!!(_exp)) || (crnlib_assert(#_exp, __FILE__, __LINE__), 0))
#define CRNLIB_ASSERTS_ENABLED #define CRNLIB_ASSERTS_ENABLED
#endif #endif
#define CRNLIB_VERIFY(_exp) (void)( (!!(_exp)) || (crnlib_assert(#_exp, __FILE__, __LINE__), 0) ) #define CRNLIB_VERIFY(_exp) (void)((!!(_exp)) || (crnlib_assert(#_exp, __FILE__, __LINE__), 0))
#define CRNLIB_FAIL(msg) do { crnlib_fail(#msg, __FILE__, __LINE__); } while(0) #define CRNLIB_FAIL(msg) \
do { \
crnlib_fail(#msg, __FILE__, __LINE__); \
} while (0)
#define CRNLIB_ASSERT_OPEN_RANGE(x, l, h) CRNLIB_ASSERT((x >= l) && (x < h)) #define CRNLIB_ASSERT_OPEN_RANGE(x, l, h) CRNLIB_ASSERT((x >= l) && (x < h))
#define CRNLIB_ASSERT_CLOSED_RANGE(x, l, h) CRNLIB_ASSERT((x >= l) && (x <= h)) #define CRNLIB_ASSERT_CLOSED_RANGE(x, l, h) CRNLIB_ASSERT((x >= l) && (x <= h))
@@ -27,35 +30,38 @@ void trace(const char* pFmt, va_list args);
void trace(const char* pFmt, ...); void trace(const char* pFmt, ...);
// Borrowed from boost libraries. // Borrowed from boost libraries.
template <bool x> struct crnlib_assume_failure; template <bool x>
template <> struct crnlib_assume_failure<true> { enum { blah = 1 }; }; struct crnlib_assume_failure;
template<int x> struct crnlib_assume_try { }; template <>
struct crnlib_assume_failure<true> {
enum { blah = 1 };
};
template <int x>
struct crnlib_assume_try {};
#define CRNLIB_JOINER_FINAL(a, b) a##b #define CRNLIB_JOINER_FINAL(a, b) a##b
#define CRNLIB_JOINER(a, b) CRNLIB_JOINER_FINAL(a, b) #define CRNLIB_JOINER(a, b) CRNLIB_JOINER_FINAL(a, b)
#define CRNLIB_JOIN(a, b) CRNLIB_JOINER(a, b) #define CRNLIB_JOIN(a, b) CRNLIB_JOINER(a, b)
#define CRNLIB_ASSUME(p) typedef crnlib_assume_try < sizeof(crnlib_assume_failure< (bool)(p) > ) > CRNLIB_JOIN(crnlib_assume_typedef, __COUNTER__) #define CRNLIB_ASSUME(p) typedef crnlib_assume_try<sizeof(crnlib_assume_failure<(bool)(p)>)> CRNLIB_JOIN(crnlib_assume_typedef, __COUNTER__)
#ifdef NDEBUG #ifdef NDEBUG
template<typename T> inline T crnlib_assert_range(T i, T m) template <typename T>
{ inline T crnlib_assert_range(T i, T) {
m; return i;
return i;
} }
template<typename T> inline T crnlib_assert_range_incl(T i, T m) template <typename T>
{ inline T crnlib_assert_range_incl(T i, T) {
m; return i;
return i;
} }
#else #else
template<typename T> inline T crnlib_assert_range(T i, T m) template <typename T>
{ inline T crnlib_assert_range(T i, T m) {
CRNLIB_ASSERT((i >= 0) && (i < m)); CRNLIB_ASSERT((i >= 0) && (i < m));
return i; return i;
} }
template<typename T> inline T crnlib_assert_range_incl(T i, T m) template <typename T>
{ inline T crnlib_assert_range_incl(T i, T m) {
CRNLIB_ASSERT((i >= 0) && (i <= m)); CRNLIB_ASSERT((i >= 0) && (i <= m));
return i; return i;
} }
#endif #endif
crnlib/crn_atomics.h
+133 -157
View File
@@ -11,198 +11,174 @@
#endif #endif
#if defined(__GNUC__) && CRNLIB_PLATFORM_PC #if defined(__GNUC__) && CRNLIB_PLATFORM_PC
extern __inline__ __attribute__((__always_inline__,__gnu_inline__)) void crnlib_yield_processor() extern __inline__ __attribute__((__always_inline__, __gnu_inline__)) void crnlib_yield_processor() {
{ __asm__ __volatile__("pause");
__asm__ __volatile__("pause");
} }
#else #else
CRNLIB_FORCE_INLINE void crnlib_yield_processor() CRNLIB_FORCE_INLINE void crnlib_yield_processor() {
{
#if CRNLIB_USE_MSVC_INTRINSICS #if CRNLIB_USE_MSVC_INTRINSICS
#if CRNLIB_PLATFORM_PC_X64 #if CRNLIB_PLATFORM_PC_X64
_mm_pause(); _mm_pause();
#else
YieldProcessor();
#endif
#else #else
// No implementation YieldProcessor();
#endif
#else
// No implementation
#endif #endif
} }
#endif #endif
#if CRNLIB_USE_WIN32_ATOMIC_FUNCTIONS #if CRNLIB_USE_WIN32_ATOMIC_FUNCTIONS
extern "C" __int64 _InterlockedCompareExchange64(__int64 volatile * Destination, __int64 Exchange, __int64 Comperand); extern "C" __int64 _InterlockedCompareExchange64(__int64 volatile* Destination, __int64 Exchange, __int64 Comperand);
#if defined(_MSC_VER) #if defined(_MSC_VER)
#pragma intrinsic(_InterlockedCompareExchange64) #pragma intrinsic(_InterlockedCompareExchange64)
#endif #endif
#endif // CRNLIB_USE_WIN32_ATOMIC_FUNCTIONS #endif // CRNLIB_USE_WIN32_ATOMIC_FUNCTIONS
namespace crnlib namespace crnlib {
{
#if CRNLIB_USE_WIN32_ATOMIC_FUNCTIONS #if CRNLIB_USE_WIN32_ATOMIC_FUNCTIONS
typedef LONG atomic32_t; typedef LONG atomic32_t;
typedef LONGLONG atomic64_t; typedef LONGLONG atomic64_t;
// Returns the original value. // Returns the original value.
inline atomic32_t atomic_compare_exchange32(atomic32_t volatile *pDest, atomic32_t exchange, atomic32_t comparand) inline atomic32_t atomic_compare_exchange32(atomic32_t volatile* pDest, atomic32_t exchange, atomic32_t comparand) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return InterlockedCompareExchange(pDest, exchange, comparand);
return InterlockedCompareExchange(pDest, exchange, comparand); }
}
// Returns the original value. // Returns the original value.
inline atomic64_t atomic_compare_exchange64(atomic64_t volatile *pDest, atomic64_t exchange, atomic64_t comparand) inline atomic64_t atomic_compare_exchange64(atomic64_t volatile* pDest, atomic64_t exchange, atomic64_t comparand) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 7) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 7) == 0); return _InterlockedCompareExchange64(pDest, exchange, comparand);
return _InterlockedCompareExchange64(pDest, exchange, comparand); }
}
// Returns the resulting incremented value. // Returns the resulting incremented value.
inline atomic32_t atomic_increment32(atomic32_t volatile *pDest) inline atomic32_t atomic_increment32(atomic32_t volatile* pDest) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return InterlockedIncrement(pDest);
return InterlockedIncrement(pDest); }
}
// Returns the resulting decremented value. // Returns the resulting decremented value.
inline atomic32_t atomic_decrement32(atomic32_t volatile *pDest) inline atomic32_t atomic_decrement32(atomic32_t volatile* pDest) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return InterlockedDecrement(pDest);
return InterlockedDecrement(pDest); }
}
// Returns the original value. // Returns the original value.
inline atomic32_t atomic_exchange32(atomic32_t volatile *pDest, atomic32_t val) inline atomic32_t atomic_exchange32(atomic32_t volatile* pDest, atomic32_t val) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return InterlockedExchange(pDest, val);
return InterlockedExchange(pDest, val); }
}
// Returns the resulting value. // Returns the resulting value.
inline atomic32_t atomic_add32(atomic32_t volatile *pDest, atomic32_t val) inline atomic32_t atomic_add32(atomic32_t volatile* pDest, atomic32_t val) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return InterlockedExchangeAdd(pDest, val) + val;
return InterlockedExchangeAdd(pDest, val) + val; }
}
// Returns the original value. // Returns the original value.
inline atomic32_t atomic_exchange_add32(atomic32_t volatile *pDest, atomic32_t val) inline atomic32_t atomic_exchange_add32(atomic32_t volatile* pDest, atomic32_t val) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return InterlockedExchangeAdd(pDest, val);
return InterlockedExchangeAdd(pDest, val); }
}
#elif CRNLIB_USE_GCC_ATOMIC_BUILTINS #elif CRNLIB_USE_GCC_ATOMIC_BUILTINS
typedef long atomic32_t; typedef long atomic32_t;
typedef long long atomic64_t; typedef long long atomic64_t;
// Returns the original value. // Returns the original value.
inline atomic32_t atomic_compare_exchange32(atomic32_t volatile *pDest, atomic32_t exchange, atomic32_t comparand) inline atomic32_t atomic_compare_exchange32(atomic32_t volatile* pDest, atomic32_t exchange, atomic32_t comparand) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return __sync_val_compare_and_swap(pDest, comparand, exchange);
return __sync_val_compare_and_swap(pDest, comparand, exchange); }
}
// Returns the original value. // Returns the original value.
inline atomic64_t atomic_compare_exchange64(atomic64_t volatile *pDest, atomic64_t exchange, atomic64_t comparand) inline atomic64_t atomic_compare_exchange64(atomic64_t volatile* pDest, atomic64_t exchange, atomic64_t comparand) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 7) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 7) == 0); return __sync_val_compare_and_swap(pDest, comparand, exchange);
return __sync_val_compare_and_swap(pDest, comparand, exchange); }
}
// Returns the resulting incremented value. // Returns the resulting incremented value.
inline atomic32_t atomic_increment32(atomic32_t volatile *pDest) inline atomic32_t atomic_increment32(atomic32_t volatile* pDest) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return __sync_add_and_fetch(pDest, 1);
return __sync_add_and_fetch(pDest, 1); }
}
// Returns the resulting decremented value. // Returns the resulting decremented value.
inline atomic32_t atomic_decrement32(atomic32_t volatile *pDest) inline atomic32_t atomic_decrement32(atomic32_t volatile* pDest) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return __sync_sub_and_fetch(pDest, 1);
return __sync_sub_and_fetch(pDest, 1); }
}
// Returns the original value. // Returns the original value.
inline atomic32_t atomic_exchange32(atomic32_t volatile *pDest, atomic32_t val) inline atomic32_t atomic_exchange32(atomic32_t volatile* pDest, atomic32_t val) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return __sync_lock_test_and_set(pDest, val);
return __sync_lock_test_and_set(pDest, val); }
}
// Returns the resulting value. // Returns the resulting value.
inline atomic32_t atomic_add32(atomic32_t volatile *pDest, atomic32_t val) inline atomic32_t atomic_add32(atomic32_t volatile* pDest, atomic32_t val) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return __sync_add_and_fetch(pDest, val);
return __sync_add_and_fetch(pDest, val); }
}
// Returns the original value. // Returns the original value.
inline atomic32_t atomic_exchange_add32(atomic32_t volatile *pDest, atomic32_t val) inline atomic32_t atomic_exchange_add32(atomic32_t volatile* pDest, atomic32_t val) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return __sync_fetch_and_add(pDest, val);
return __sync_fetch_and_add(pDest, val); }
}
#else #else
#define CRNLIB_NO_ATOMICS 1 #define CRNLIB_NO_ATOMICS 1
// Atomic ops not supported - but try to do something reasonable. Assumes no threading at all. // Atomic ops not supported - but try to do something reasonable. Assumes no threading at all.
typedef long atomic32_t; typedef long atomic32_t;
typedef long long atomic64_t; typedef long long atomic64_t;
inline atomic32_t atomic_compare_exchange32(atomic32_t volatile *pDest, atomic32_t exchange, atomic32_t comparand) inline atomic32_t atomic_compare_exchange32(atomic32_t volatile* pDest, atomic32_t exchange, atomic32_t comparand) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); atomic32_t cur = *pDest;
atomic32_t cur = *pDest; if (cur == comparand)
if (cur == comparand) *pDest = exchange;
*pDest = exchange; return cur;
return cur; }
}
inline atomic64_t atomic_compare_exchange64(atomic64_t volatile *pDest, atomic64_t exchange, atomic64_t comparand) inline atomic64_t atomic_compare_exchange64(atomic64_t volatile* pDest, atomic64_t exchange, atomic64_t comparand) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 7) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 7) == 0); atomic64_t cur = *pDest;
atomic64_t cur = *pDest; if (cur == comparand)
if (cur == comparand) *pDest = exchange;
*pDest = exchange; return cur;
return cur; }
}
inline atomic32_t atomic_increment32(atomic32_t volatile *pDest) inline atomic32_t atomic_increment32(atomic32_t volatile* pDest) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return (*pDest += 1);
return (*pDest += 1); }
}
inline atomic32_t atomic_decrement32(atomic32_t volatile *pDest) inline atomic32_t atomic_decrement32(atomic32_t volatile* pDest) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return (*pDest -= 1);
return (*pDest -= 1); }
}
inline atomic32_t atomic_exchange32(atomic32_t volatile *pDest, atomic32_t val) inline atomic32_t atomic_exchange32(atomic32_t volatile* pDest, atomic32_t val) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); atomic32_t cur = *pDest;
atomic32_t cur = *pDest; *pDest = val;
*pDest = val; return cur;
return cur; }
}
inline atomic32_t atomic_add32(atomic32_t volatile *pDest, atomic32_t val) inline atomic32_t atomic_add32(atomic32_t volatile* pDest, atomic32_t val) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); return (*pDest += val);
return (*pDest += val); }
}
inline atomic32_t atomic_exchange_add32(atomic32_t volatile *pDest, atomic32_t val) inline atomic32_t atomic_exchange_add32(atomic32_t volatile* pDest, atomic32_t val) {
{ CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(pDest) & 3) == 0); atomic32_t cur = *pDest;
atomic32_t cur = *pDest; *pDest += val;
*pDest += val; return cur;
return cur; }
}
#endif #endif
} // namespace crnlib } // namespace crnlib
#endif // CRN_ATOMICS_H #endif // CRN_ATOMICS_H
crnlib/crn_buffer_stream.h
+125 -143
View File
@@ -3,194 +3,176 @@
#pragma once #pragma once
#include "crn_data_stream.h" #include "crn_data_stream.h"
namespace crnlib namespace crnlib {
{ class buffer_stream : public data_stream {
class buffer_stream : public data_stream public:
{ buffer_stream()
public: : data_stream(),
buffer_stream() : m_pBuf(NULL),
data_stream(), m_size(0),
m_pBuf(NULL), m_ofs(0) {
m_size(0), }
m_ofs(0)
{
}
buffer_stream(void* p, uint size) : buffer_stream(void* p, uint size)
data_stream(), : data_stream(),
m_pBuf(NULL), m_pBuf(NULL),
m_size(0), m_size(0),
m_ofs(0) m_ofs(0) {
{ open(p, size);
open(p, size); }
}
buffer_stream(const void* p, uint size) : buffer_stream(const void* p, uint size)
data_stream(), : data_stream(),
m_pBuf(NULL), m_pBuf(NULL),
m_size(0), m_size(0),
m_ofs(0) m_ofs(0) {
{ open(p, size);
open(p, size); }
}
virtual ~buffer_stream() virtual ~buffer_stream() {
{ }
}
bool open(const void* p, uint size) bool open(const void* p, uint size) {
{ CRNLIB_ASSERT(p);
CRNLIB_ASSERT(p);
close(); close();
if ((!p) || (!size)) if ((!p) || (!size))
return false; return false;
m_opened = true; m_opened = true;
m_pBuf = (uint8*)(p); m_pBuf = (uint8*)(p);
m_size = size; m_size = size;
m_ofs = 0; m_ofs = 0;
m_attribs = cDataStreamSeekable | cDataStreamReadable; m_attribs = cDataStreamSeekable | cDataStreamReadable;
return true; return true;
} }
bool open(void* p, uint size) bool open(void* p, uint size) {
{ CRNLIB_ASSERT(p);
CRNLIB_ASSERT(p);
close(); close();
if ((!p) || (!size)) if ((!p) || (!size))
return false; return false;
m_opened = true; m_opened = true;
m_pBuf = static_cast<uint8*>(p); m_pBuf = static_cast<uint8*>(p);
m_size = size; m_size = size;
m_ofs = 0; m_ofs = 0;
m_attribs = cDataStreamSeekable | cDataStreamWritable | cDataStreamReadable; m_attribs = cDataStreamSeekable | cDataStreamWritable | cDataStreamReadable;
return true; return true;
} }
virtual bool close() virtual bool close() {
{ if (m_opened) {
if (m_opened) m_opened = false;
{ m_pBuf = NULL;
m_opened = false; m_size = 0;
m_pBuf = NULL; m_ofs = 0;
m_size = 0; return true;
m_ofs = 0; }
return true;
}
return false; return false;
} }
const void* get_buf() const { return m_pBuf; } const void* get_buf() const { return m_pBuf; }
void* get_buf() { return m_pBuf; } void* get_buf() { return m_pBuf; }
virtual const void* get_ptr() const { return m_pBuf; } virtual const void* get_ptr() const { return m_pBuf; }
virtual uint read(void* pBuf, uint len) virtual uint read(void* pBuf, uint len) {
{ CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF));
CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF));
if ((!m_opened) || (!is_readable()) || (!len)) if ((!m_opened) || (!is_readable()) || (!len))
return 0; return 0;
CRNLIB_ASSERT(m_ofs <= m_size); CRNLIB_ASSERT(m_ofs <= m_size);
uint bytes_left = m_size - m_ofs; uint bytes_left = m_size - m_ofs;
len = math::minimum<uint>(len, bytes_left); len = math::minimum<uint>(len, bytes_left);
if (len) if (len)
memcpy(pBuf, &m_pBuf[m_ofs], len); memcpy(pBuf, &m_pBuf[m_ofs], len);
m_ofs += len; m_ofs += len;
return len; return len;
} }
virtual uint write(const void* pBuf, uint len) virtual uint write(const void* pBuf, uint len) {
{ CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF));
CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF));
if ((!m_opened) || (!is_writable()) || (!len)) if ((!m_opened) || (!is_writable()) || (!len))
return 0; return 0;
CRNLIB_ASSERT(m_ofs <= m_size); CRNLIB_ASSERT(m_ofs <= m_size);
uint bytes_left = m_size - m_ofs; uint bytes_left = m_size - m_ofs;
len = math::minimum<uint>(len, bytes_left); len = math::minimum<uint>(len, bytes_left);
if (len) if (len)
memcpy(&m_pBuf[m_ofs], pBuf, len); memcpy(&m_pBuf[m_ofs], pBuf, len);
m_ofs += len; m_ofs += len;
return len; return len;
} }
virtual bool flush() virtual bool flush() {
{ if (!m_opened)
if (!m_opened) return false;
return false;
return true; return true;
} }
virtual uint64 get_size() virtual uint64 get_size() {
{ if (!m_opened)
if (!m_opened) return 0;
return 0;
return m_size; return m_size;
} }
virtual uint64 get_remaining() virtual uint64 get_remaining() {
{ if (!m_opened)
if (!m_opened) return 0;
return 0;
CRNLIB_ASSERT(m_ofs <= m_size); CRNLIB_ASSERT(m_ofs <= m_size);
return m_size - m_ofs; return m_size - m_ofs;
} }
virtual uint64 get_ofs() virtual uint64 get_ofs() {
{ if (!m_opened)
if (!m_opened) return 0;
return 0;
return m_ofs; return m_ofs;
} }
virtual bool seek(int64 ofs, bool relative) virtual bool seek(int64 ofs, bool relative) {
{ if ((!m_opened) || (!is_seekable()))
if ((!m_opened) || (!is_seekable())) return false;
return false;
int64 new_ofs = relative ? (m_ofs + ofs) : ofs; int64 new_ofs = relative ? (m_ofs + ofs) : ofs;
if (new_ofs < 0) if (new_ofs < 0)
return false; return false;
else if (new_ofs > m_size) else if (new_ofs > m_size)
return false; return false;
m_ofs = static_cast<uint>(new_ofs); m_ofs = static_cast<uint>(new_ofs);
post_seek(); post_seek();
return true; return true;
} }
private: private:
uint8* m_pBuf; uint8* m_pBuf;
uint m_size; uint m_size;
uint m_ofs; uint m_ofs;
}; };
} // namespace crnlib
} // namespace crnlib
crnlib/crn_cfile_stream.h
+201 -227
View File
@@ -3,239 +3,213 @@
#pragma once #pragma once
#include "crn_data_stream.h" #include "crn_data_stream.h"
namespace crnlib namespace crnlib {
{ class cfile_stream : public data_stream {
class cfile_stream : public data_stream public:
{ cfile_stream()
public: : data_stream(), m_pFile(NULL), m_size(0), m_ofs(0), m_has_ownership(false) {
cfile_stream() : data_stream(), m_pFile(NULL), m_size(0), m_ofs(0), m_has_ownership(false) }
{
cfile_stream(FILE* pFile, const char* pFilename, uint attribs, bool has_ownership)
: data_stream(), m_pFile(NULL), m_size(0), m_ofs(0), m_has_ownership(false) {
open(pFile, pFilename, attribs, has_ownership);
}
cfile_stream(const char* pFilename, uint attribs = cDataStreamReadable | cDataStreamSeekable, bool open_existing = false)
: data_stream(), m_pFile(NULL), m_size(0), m_ofs(0), m_has_ownership(false) {
open(pFilename, attribs, open_existing);
}
virtual ~cfile_stream() {
close();
}
virtual bool close() {
clear_error();
if (m_opened) {
bool status = true;
if (m_has_ownership) {
if (EOF == fclose(m_pFile))
status = false;
} }
cfile_stream(FILE* pFile, const char* pFilename, uint attribs, bool has_ownership) : m_pFile = NULL;
data_stream(), m_pFile(NULL), m_size(0), m_ofs(0), m_has_ownership(false) m_opened = false;
{ m_size = 0;
open(pFile, pFilename, attribs, has_ownership); m_ofs = 0;
m_has_ownership = false;
return status;
}
return false;
}
bool open(FILE* pFile, const char* pFilename, uint attribs, bool has_ownership) {
CRNLIB_ASSERT(pFile);
CRNLIB_ASSERT(pFilename);
close();
set_name(pFilename);
m_pFile = pFile;
m_has_ownership = has_ownership;
m_attribs = static_cast<uint16>(attribs);
m_ofs = crn_ftell(m_pFile);
crn_fseek(m_pFile, 0, SEEK_END);
m_size = crn_ftell(m_pFile);
crn_fseek(m_pFile, m_ofs, SEEK_SET);
m_opened = true;
return true;
}
bool open(const char* pFilename, uint attribs = cDataStreamReadable | cDataStreamSeekable, bool open_existing = false) {
CRNLIB_ASSERT(pFilename);
close();
m_attribs = static_cast<uint16>(attribs);
const char* pMode;
if ((is_readable()) && (is_writable()))
pMode = open_existing ? "r+b" : "w+b";
else if (is_writable())
pMode = open_existing ? "ab" : "wb";
else if (is_readable())
pMode = "rb";
else {
set_error();
return false;
}
FILE* pFile = NULL;
crn_fopen(&pFile, pFilename, pMode);
m_has_ownership = true;
if (!pFile) {
set_error();
return false;
}
// TODO: Change stream class to support UCS2 filenames.
return open(pFile, pFilename, attribs, true);
}
FILE* get_file() const { return m_pFile; }
virtual uint read(void* pBuf, uint len) {
CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF));
if (!m_opened || (!is_readable()) || (!len))
return 0;
len = static_cast<uint>(math::minimum<uint64>(len, get_remaining()));
if (fread(pBuf, 1, len, m_pFile) != len) {
set_error();
return 0;
}
m_ofs += len;
return len;
}
virtual uint write(const void* pBuf, uint len) {
CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF));
if (!m_opened || (!is_writable()) || (!len))
return 0;
if (fwrite(pBuf, 1, len, m_pFile) != len) {
set_error();
return 0;
}
m_ofs += len;
m_size = math::maximum(m_size, m_ofs);
return len;
}
virtual bool flush() {
if ((!m_opened) || (!is_writable()))
return false;
if (EOF == fflush(m_pFile)) {
set_error();
return false;
}
return true;
}
virtual uint64 get_size() {
if (!m_opened)
return 0;
return m_size;
}
virtual uint64 get_remaining() {
if (!m_opened)
return 0;
CRNLIB_ASSERT(m_ofs <= m_size);
return m_size - m_ofs;
}
virtual uint64 get_ofs() {
if (!m_opened)
return 0;
return m_ofs;
}
virtual bool seek(int64 ofs, bool relative) {
if ((!m_opened) || (!is_seekable()))
return false;
int64 new_ofs = relative ? (m_ofs + ofs) : ofs;
if (new_ofs < 0)
return false;
else if (static_cast<uint64>(new_ofs) > m_size)
return false;
if (static_cast<uint64>(new_ofs) != m_ofs) {
if (crn_fseek(m_pFile, new_ofs, SEEK_SET) != 0) {
set_error();
return false;
} }
cfile_stream(const char* pFilename, uint attribs = cDataStreamReadable | cDataStreamSeekable, bool open_existing = false) : m_ofs = new_ofs;
data_stream(), m_pFile(NULL), m_size(0), m_ofs(0), m_has_ownership(false) }
{
open(pFilename, attribs, open_existing);
}
virtual ~cfile_stream() return true;
{ }
close();
}
virtual bool close() static bool read_file_into_array(const char* pFilename, vector<uint8>& buf) {
{ cfile_stream in_stream(pFilename);
clear_error(); if (!in_stream.is_opened())
return false;
return in_stream.read_array(buf);
}
if (m_opened) static bool write_array_to_file(const char* pFilename, const vector<uint8>& buf) {
{ cfile_stream out_stream(pFilename, cDataStreamWritable | cDataStreamSeekable);
bool status = true; if (!out_stream.is_opened())
if (m_has_ownership) return false;
{ return out_stream.write_array(buf);
if (EOF == fclose(m_pFile)) }
status = false;
}
m_pFile = NULL; private:
m_opened = false; FILE* m_pFile;
m_size = 0; uint64 m_size, m_ofs;
m_ofs = 0; bool m_has_ownership;
m_has_ownership = false; };
return status; } // namespace crnlib
}
return false;
}
bool open(FILE* pFile, const char* pFilename, uint attribs, bool has_ownership)
{
CRNLIB_ASSERT(pFile);
CRNLIB_ASSERT(pFilename);
close();
set_name(pFilename);
m_pFile = pFile;
m_has_ownership = has_ownership;
m_attribs = static_cast<uint16>(attribs);
m_ofs = crn_ftell(m_pFile);
crn_fseek(m_pFile, 0, SEEK_END);
m_size = crn_ftell(m_pFile);
crn_fseek(m_pFile, m_ofs, SEEK_SET);
m_opened = true;
return true;
}
bool open(const char* pFilename, uint attribs = cDataStreamReadable | cDataStreamSeekable, bool open_existing = false)
{
CRNLIB_ASSERT(pFilename);
close();
m_attribs = static_cast<uint16>(attribs);
const char* pMode;
if ((is_readable()) && (is_writable()))
pMode = open_existing ? "r+b" : "w+b";
else if (is_writable())
pMode = open_existing ? "ab" : "wb";
else if (is_readable())
pMode = "rb";
else
{
set_error();
return false;
}
FILE* pFile = NULL;
crn_fopen(&pFile, pFilename, pMode);
m_has_ownership = true;
if (!pFile)
{
set_error();
return false;
}
// TODO: Change stream class to support UCS2 filenames.
return open(pFile, pFilename, attribs, true);
}
FILE* get_file() const { return m_pFile; }
virtual uint read(void* pBuf, uint len)
{
CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF));
if (!m_opened || (!is_readable()) || (!len))
return 0;
len = static_cast<uint>(math::minimum<uint64>(len, get_remaining()));
if (fread(pBuf, 1, len, m_pFile) != len)
{
set_error();
return 0;
}
m_ofs += len;
return len;
}
virtual uint write(const void* pBuf, uint len)
{
CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF));
if (!m_opened || (!is_writable()) || (!len))
return 0;
if (fwrite(pBuf, 1, len, m_pFile) != len)
{
set_error();
return 0;
}
m_ofs += len;
m_size = math::maximum(m_size, m_ofs);
return len;
}
virtual bool flush()
{
if ((!m_opened) || (!is_writable()))
return false;
if (EOF == fflush(m_pFile))
{
set_error();
return false;
}
return true;
}
virtual uint64 get_size()
{
if (!m_opened)
return 0;
return m_size;
}
virtual uint64 get_remaining()
{
if (!m_opened)
return 0;
CRNLIB_ASSERT(m_ofs <= m_size);
return m_size - m_ofs;
}
virtual uint64 get_ofs()
{
if (!m_opened)
return 0;
return m_ofs;
}
virtual bool seek(int64 ofs, bool relative)
{
if ((!m_opened) || (!is_seekable()))
return false;
int64 new_ofs = relative ? (m_ofs + ofs) : ofs;
if (new_ofs < 0)
return false;
else if (static_cast<uint64>(new_ofs) > m_size)
return false;
if (static_cast<uint64>(new_ofs) != m_ofs)
{
if (crn_fseek(m_pFile, new_ofs, SEEK_SET) != 0)
{
set_error();
return false;
}
m_ofs = new_ofs;
}
return true;
}
static bool read_file_into_array(const char* pFilename, vector<uint8>& buf)
{
cfile_stream in_stream(pFilename);
if (!in_stream.is_opened())
return false;
return in_stream.read_array(buf);
}
static bool write_array_to_file(const char* pFilename, const vector<uint8>& buf)
{
cfile_stream out_stream(pFilename, cDataStreamWritable|cDataStreamSeekable);
if (!out_stream.is_opened())
return false;
return out_stream.write_array(buf);
}
private:
FILE* m_pFile;
uint64 m_size, m_ofs;
bool m_has_ownership;
};
} // namespace crnlib
crnlib/crn_checksum.cpp
+47 -52
View File
@@ -1,63 +1,58 @@
// File: crn_checksum.cpp // File: crn_checksum.cpp
#include "crn_core.h" #include "crn_core.h"
namespace crnlib namespace crnlib {
{ // From the public domain stb.h header.
// From the public domain stb.h header. uint adler32(const void* pBuf, size_t buflen, uint adler32) {
uint adler32(const void* pBuf, size_t buflen, uint adler32) const uint8* buffer = static_cast<const uint8*>(pBuf);
{
const uint8* buffer = static_cast<const uint8*>(pBuf);
const unsigned long ADLER_MOD = 65521;
unsigned long s1 = adler32 & 0xffff, s2 = adler32 >> 16;
size_t blocklen;
unsigned long i;
blocklen = buflen % 5552; const unsigned long ADLER_MOD = 65521;
while (buflen) { unsigned long s1 = adler32 & 0xffff, s2 = adler32 >> 16;
for (i=0; i + 7 < blocklen; i += 8) { size_t blocklen;
s1 += buffer[0], s2 += s1; unsigned long i;
s1 += buffer[1], s2 += s1;
s1 += buffer[2], s2 += s1;
s1 += buffer[3], s2 += s1;
s1 += buffer[4], s2 += s1;
s1 += buffer[5], s2 += s1;
s1 += buffer[6], s2 += s1;
s1 += buffer[7], s2 += s1;
buffer += 8; blocklen = buflen % 5552;
} while (buflen) {
for (i = 0; i + 7 < blocklen; i += 8) {
s1 += buffer[0], s2 += s1;
s1 += buffer[1], s2 += s1;
s1 += buffer[2], s2 += s1;
s1 += buffer[3], s2 += s1;
s1 += buffer[4], s2 += s1;
s1 += buffer[5], s2 += s1;
s1 += buffer[6], s2 += s1;
s1 += buffer[7], s2 += s1;
for (; i < blocklen; ++i) buffer += 8;
s1 += *buffer++, s2 += s1; }
s1 %= ADLER_MOD, s2 %= ADLER_MOD; for (; i < blocklen; ++i)
buflen -= blocklen; s1 += *buffer++, s2 += s1;
blocklen = 5552;
}
return (s2 << 16) + s1;
}
uint16 crc16(const void* pBuf, size_t len, uint16 crc)
{
crc = ~crc;
const uint8* p = reinterpret_cast<const uint8*>(pBuf); s1 %= ADLER_MOD, s2 %= ADLER_MOD;
while (len) buflen -= blocklen;
{ blocklen = 5552;
const uint16 q = *p++ ^ (crc >> 8); }
crc <<= 8U; return (s2 << 16) + s1;
uint16 r = (q >> 4) ^ q; }
crc ^= r;
r <<= 5U;
crc ^= r;
r <<= 7U;
crc ^= r;
len--;
}
return static_cast<uint16>(~crc); uint16 crc16(const void* pBuf, size_t len, uint16 crc) {
} crc = ~crc;
} // namespace crnlib const uint8* p = reinterpret_cast<const uint8*>(pBuf);
while (len) {
const uint16 q = *p++ ^ (crc >> 8);
crc <<= 8U;
uint16 r = (q >> 4) ^ q;
crc ^= r;
r <<= 5U;
crc ^= r;
r <<= 7U;
crc ^= r;
len--;
}
return static_cast<uint16>(~crc);
}
} // namespace crnlib
crnlib/crn_checksum.h
+8 -9
View File
@@ -1,13 +1,12 @@
// File: crn_checksum.h // File: crn_checksum.h
#pragma once #pragma once
namespace crnlib namespace crnlib {
{ const uint cInitAdler32 = 1U;
const uint cInitAdler32 = 1U; uint adler32(const void* pBuf, size_t buflen, uint adler32 = cInitAdler32);
uint adler32(const void* pBuf, size_t buflen, uint adler32 = cInitAdler32);
// crc16() intended for small buffers - doesn't use an acceleration table.
// crc16() intended for small buffers - doesn't use an acceleration table. const uint cInitCRC16 = 0;
const uint cInitCRC16 = 0; uint16 crc16(const void* pBuf, size_t len, uint16 crc = cInitCRC16);
uint16 crc16(const void* pBuf, size_t len, uint16 crc = cInitCRC16);
} // namespace crnlib } // namespace crnlib
crnlib/crn_clusterizer.h
+529 -593
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_color.h
+882 -976
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_colorized_console.cpp
+84 -94
View File
@@ -6,114 +6,104 @@
#include "crn_winhdr.h" #include "crn_winhdr.h"
#endif #endif
namespace crnlib namespace crnlib {
{ void colorized_console::init() {
void colorized_console::init() console::init();
{ console::add_console_output_func(console_output_func, NULL);
console::init(); }
console::add_console_output_func(console_output_func, NULL);
}
void colorized_console::deinit() void colorized_console::deinit() {
{ console::remove_console_output_func(console_output_func);
console::remove_console_output_func(console_output_func); console::deinit();
console::deinit(); }
}
void colorized_console::tick() void colorized_console::tick() {
{ }
}
#ifdef CRNLIB_USE_WIN32_API #ifdef CRNLIB_USE_WIN32_API
bool colorized_console::console_output_func(eConsoleMessageType type, const char* pMsg, void* pData) bool colorized_console::console_output_func(eConsoleMessageType type, const char* pMsg, void*) {
{ if (console::get_output_disabled())
pData; return true;
if (console::get_output_disabled()) HANDLE cons = GetStdHandle(STD_OUTPUT_HANDLE);
return true;
HANDLE cons = GetStdHandle(STD_OUTPUT_HANDLE); DWORD attr = FOREGROUND_RED | FOREGROUND_GREEN | FOREGROUND_BLUE;
switch (type) {
case cDebugConsoleMessage:
attr = FOREGROUND_BLUE | FOREGROUND_INTENSITY;
break;
case cMessageConsoleMessage:
attr = FOREGROUND_GREEN | FOREGROUND_BLUE | FOREGROUND_INTENSITY;
break;
case cWarningConsoleMessage:
attr = FOREGROUND_GREEN | FOREGROUND_RED | FOREGROUND_INTENSITY;
break;
case cErrorConsoleMessage:
attr = FOREGROUND_RED | FOREGROUND_INTENSITY;
break;
default:
break;
}
DWORD attr = FOREGROUND_RED | FOREGROUND_GREEN | FOREGROUND_BLUE; if (INVALID_HANDLE_VALUE != cons)
switch (type) SetConsoleTextAttribute(cons, (WORD)attr);
{
case cDebugConsoleMessage: attr = FOREGROUND_BLUE | FOREGROUND_INTENSITY; break;
case cMessageConsoleMessage: attr = FOREGROUND_GREEN | FOREGROUND_BLUE | FOREGROUND_INTENSITY; break;
case cWarningConsoleMessage: attr = FOREGROUND_GREEN | FOREGROUND_RED | FOREGROUND_INTENSITY; break;
case cErrorConsoleMessage: attr = FOREGROUND_RED | FOREGROUND_INTENSITY; break;
default: break;
}
if (INVALID_HANDLE_VALUE != cons) if ((console::get_prefixes()) && (console::get_at_beginning_of_line())) {
SetConsoleTextAttribute(cons, (WORD)attr); switch (type) {
case cDebugConsoleMessage:
printf("Debug: %s", pMsg);
break;
case cWarningConsoleMessage:
printf("Warning: %s", pMsg);
break;
case cErrorConsoleMessage:
printf("Error: %s", pMsg);
break;
default:
printf("%s", pMsg);
break;
}
} else {
printf("%s", pMsg);
}
if ((console::get_prefixes()) && (console::get_at_beginning_of_line())) if (console::get_crlf())
{ printf("\n");
switch (type)
{
case cDebugConsoleMessage:
printf("Debug: %s", pMsg);
break;
case cWarningConsoleMessage:
printf("Warning: %s", pMsg);
break;
case cErrorConsoleMessage:
printf("Error: %s", pMsg);
break;
default:
printf("%s", pMsg);
break;
}
}
else
{
printf("%s", pMsg);
}
if (console::get_crlf()) if (INVALID_HANDLE_VALUE != cons)
printf("\n"); SetConsoleTextAttribute(cons, FOREGROUND_RED | FOREGROUND_GREEN | FOREGROUND_BLUE);
if (INVALID_HANDLE_VALUE != cons) return true;
SetConsoleTextAttribute(cons, FOREGROUND_RED | FOREGROUND_GREEN | FOREGROUND_BLUE); }
return true;
}
#else #else
bool colorized_console::console_output_func(eConsoleMessageType type, const char* pMsg, void* pData) bool colorized_console::console_output_func(eConsoleMessageType type, const char* pMsg, void*) {
{ if (console::get_output_disabled())
pData; return true;
if (console::get_output_disabled())
return true;
if ((console::get_prefixes()) && (console::get_at_beginning_of_line())) if ((console::get_prefixes()) && (console::get_at_beginning_of_line())) {
{ switch (type) {
switch (type) case cDebugConsoleMessage:
{ printf("Debug: %s", pMsg);
case cDebugConsoleMessage: break;
printf("Debug: %s", pMsg); case cWarningConsoleMessage:
break; printf("Warning: %s", pMsg);
case cWarningConsoleMessage: break;
printf("Warning: %s", pMsg); case cErrorConsoleMessage:
break; printf("Error: %s", pMsg);
case cErrorConsoleMessage: break;
printf("Error: %s", pMsg); default:
break; printf("%s", pMsg);
default: break;
printf("%s", pMsg); }
break; } else {
} printf("%s", pMsg);
} }
else
{
printf("%s", pMsg);
}
if (console::get_crlf()) if (console::get_crlf())
printf("\n"); printf("\n");
return true; return true;
} }
#endif #endif
} // namespace crnlib } // namespace crnlib
crnlib/crn_colorized_console.h
+10 -12
View File
@@ -3,17 +3,15 @@
#pragma once #pragma once
#include "crn_console.h" #include "crn_console.h"
namespace crnlib namespace crnlib {
{ class colorized_console {
class colorized_console public:
{ static void init();
public: static void deinit();
static void init(); static void tick();
static void deinit();
static void tick();
private: private:
static bool console_output_func(eConsoleMessageType type, const char* pMsg, void* pData); static bool console_output_func(eConsoleMessageType type, const char* pMsg, void* pData);
}; };
} // namespace crnlib } // namespace crnlib
crnlib/crn_command_line_params.cpp
+320 -382
View File
@@ -6,467 +6,405 @@
#include "crn_cfile_stream.h" #include "crn_cfile_stream.h"
#ifdef WIN32 #ifdef WIN32
#define CRNLIB_CMD_LINE_ALLOW_SLASH_PARAMS 1 #define CRNLIB_CMD_LINE_ALLOW_SLASH_PARAMS 1
#endif #endif
#if CRNLIB_USE_WIN32_API #if CRNLIB_USE_WIN32_API
#include "crn_winhdr.h" #include "crn_winhdr.h"
#endif #endif
namespace crnlib namespace crnlib {
{ void get_command_line_as_single_string(dynamic_string& cmd_line, int argc, char* argv[]) {
void get_command_line_as_single_string(dynamic_string& cmd_line, int argc, char *argv[]) argc, argv;
{
argc, argv;
#if CRNLIB_USE_WIN32_API #if CRNLIB_USE_WIN32_API
cmd_line.set(GetCommandLineA()); cmd_line.set(GetCommandLineA());
#else #else
cmd_line.clear(); cmd_line.clear();
for (int i = 0; i < argc; i++) for (int i = 0; i < argc; i++) {
{ dynamic_string tmp(argv[i]);
dynamic_string tmp(argv[i]); if ((tmp.front() != '"') && (tmp.front() != '-') && (tmp.front() != '@'))
if ((tmp.front() != '"') && (tmp.front() != '-') && (tmp.front() != '@')) tmp = "\"" + tmp + "\"";
tmp = "\"" + tmp + "\""; if (cmd_line.get_len())
if (cmd_line.get_len()) cmd_line += " ";
cmd_line += " "; cmd_line += tmp;
cmd_line += tmp; }
}
#endif #endif
} }
command_line_params::command_line_params() command_line_params::command_line_params() {
{ }
}
void command_line_params::clear() void command_line_params::clear() {
{ m_params.clear();
m_params.clear();
m_param_map.clear(); m_param_map.clear();
} }
bool command_line_params::split_params(const char* p, dynamic_string_array& params) bool command_line_params::split_params(const char* p, dynamic_string_array& params) {
{ bool within_param = false;
bool within_param = false; bool within_quote = false;
bool within_quote = false;
uint ofs = 0; uint ofs = 0;
dynamic_string str; dynamic_string str;
while (p[ofs]) while (p[ofs]) {
{ const char c = p[ofs];
const char c = p[ofs];
if (within_param) if (within_param) {
{ if (within_quote) {
if (within_quote) if (c == '"')
{ within_quote = false;
if (c == '"')
within_quote = false;
str.append_char(c); str.append_char(c);
} } else if ((c == ' ') || (c == '\t')) {
else if ((c == ' ') || (c == '\t')) if (!str.is_empty()) {
{ params.push_back(str);
if (!str.is_empty()) str.clear();
{ }
params.push_back(str); within_param = false;
str.clear(); } else {
} if (c == '"')
within_param = false; within_quote = true;
}
else
{
if (c == '"')
within_quote = true;
str.append_char(c); str.append_char(c);
}
}
else if ((c != ' ') && (c != '\t'))
{
within_param = true;
if (c == '"')
within_quote = true;
str.append_char(c);
}
ofs++;
} }
} else if ((c != ' ') && (c != '\t')) {
within_param = true;
if (within_quote) if (c == '"')
{ within_quote = true;
console::error("Unmatched quote in command line \"%s\"", p);
return false;
}
if (!str.is_empty()) str.append_char(c);
params.push_back(str); }
return true; ofs++;
} }
bool command_line_params::load_string_file(const char* pFilename, dynamic_string_array& strings) if (within_quote) {
{ console::error("Unmatched quote in command line \"%s\"", p);
cfile_stream in_stream; return false;
if (!in_stream.open(pFilename, cDataStreamReadable | cDataStreamSeekable)) }
{
console::error("Unable to open file \"%s\" for reading!", pFilename);
return false;
}
dynamic_string ansi_str; if (!str.is_empty())
params.push_back(str);
for ( ; ; ) return true;
{ }
if (!in_stream.read_line(ansi_str))
break;
ansi_str.trim(); bool command_line_params::load_string_file(const char* pFilename, dynamic_string_array& strings) {
if (ansi_str.is_empty()) cfile_stream in_stream;
continue; if (!in_stream.open(pFilename, cDataStreamReadable | cDataStreamSeekable)) {
console::error("Unable to open file \"%s\" for reading!", pFilename);
return false;
}
strings.push_back(dynamic_string(ansi_str.get_ptr())); dynamic_string ansi_str;
}
return true; for (;;) {
} if (!in_stream.read_line(ansi_str))
break;
bool command_line_params::parse(const dynamic_string_array& params, uint n, const param_desc* pParam_desc) ansi_str.trim();
{ if (ansi_str.is_empty())
CRNLIB_ASSERT(n && pParam_desc); continue;
m_params = params; strings.push_back(dynamic_string(ansi_str.get_ptr()));
}
uint arg_index = 0; return true;
while (arg_index < params.size()) }
{
const uint cur_arg_index = arg_index;
const dynamic_string& src_param = params[arg_index++];
if (src_param.is_empty()) bool command_line_params::parse(const dynamic_string_array& params, uint n, const param_desc* pParam_desc) {
continue; CRNLIB_ASSERT(n && pParam_desc);
m_params = params;
uint arg_index = 0;
while (arg_index < params.size()) {
const uint cur_arg_index = arg_index;
const dynamic_string& src_param = params[arg_index++];
if (src_param.is_empty())
continue;
#if CRNLIB_CMD_LINE_ALLOW_SLASH_PARAMS #if CRNLIB_CMD_LINE_ALLOW_SLASH_PARAMS
if ((src_param[0] == '/') || (src_param[0] == '-')) if ((src_param[0] == '/') || (src_param[0] == '-'))
#else #else
if (src_param[0] == '-') if (src_param[0] == '-')
#endif #endif
{ {
if (src_param.get_len() < 2) if (src_param.get_len() < 2) {
{ console::error("Invalid command line parameter: \"%s\"", src_param.get_ptr());
console::error("Invalid command line parameter: \"%s\"", src_param.get_ptr()); return false;
return false;
}
dynamic_string key_str(src_param);
key_str.right(1);
int modifier = 0;
char c = key_str[key_str.get_len() - 1];
if (c == '+')
modifier = 1;
else if (c == '-')
modifier = -1;
if (modifier)
key_str.left(key_str.get_len() - 1);
uint param_index;
for (param_index = 0; param_index < n; param_index++)
if (key_str == pParam_desc[param_index].m_pName)
break;
if (param_index == n)
{
console::error("Unrecognized command line parameter: \"%s\"", src_param.get_ptr());
return false;
}
const param_desc& desc = pParam_desc[param_index];
const uint cMaxValues = 16;
dynamic_string val_str[cMaxValues];
uint num_val_strs = 0;
if (desc.m_num_values)
{
CRNLIB_ASSERT(desc.m_num_values <= cMaxValues);
if ((arg_index + desc.m_num_values) > params.size())
{
console::error("Expected %u value(s) after command line parameter: \"%s\"", desc.m_num_values, src_param.get_ptr());
return false;
}
for (uint v = 0; v < desc.m_num_values; v++)
val_str[num_val_strs++] = params[arg_index++];
}
dynamic_string_array strings;
if ((desc.m_support_listing_file) && (val_str[0].get_len() >= 2) && (val_str[0][0] == '@'))
{
dynamic_string filename(val_str[0]);
filename.right(1);
filename.unquote();
if (!load_string_file(filename.get_ptr(), strings))
{
console::error("Failed loading listing file \"%s\"!", filename.get_ptr());
return false;
}
}
else
{
for (uint v = 0; v < num_val_strs; v++)
{
val_str[v].unquote();
strings.push_back(val_str[v]);
}
}
param_value pv;
pv.m_values.swap(strings);
pv.m_index = cur_arg_index;
pv.m_modifier = (int8)modifier;
m_param_map.insert(std::make_pair(key_str, pv));
}
else
{
param_value pv;
pv.m_values.push_back(src_param);
pv.m_values.back().unquote();
pv.m_index = cur_arg_index;
m_param_map.insert(std::make_pair(g_empty_dynamic_string, pv));
}
} }
return true; dynamic_string key_str(src_param);
}
bool command_line_params::parse(const char* pCmd_line, uint n, const param_desc* pParam_desc, bool skip_first_param) key_str.right(1);
{
CRNLIB_ASSERT(n && pParam_desc);
dynamic_string_array p; int modifier = 0;
if (!split_params(pCmd_line, p)) char c = key_str[key_str.get_len() - 1];
return 0; if (c == '+')
modifier = 1;
else if (c == '-')
modifier = -1;
if (p.empty()) if (modifier)
return 0; key_str.left(key_str.get_len() - 1);
if (skip_first_param) uint param_index;
p.erase(0U); for (param_index = 0; param_index < n; param_index++)
if (key_str == pParam_desc[param_index].m_pName)
break;
return parse(p, n, pParam_desc); if (param_index == n) {
} console::error("Unrecognized command line parameter: \"%s\"", src_param.get_ptr());
return false;
}
bool command_line_params::is_param(uint index) const const param_desc& desc = pParam_desc[param_index];
{
CRNLIB_ASSERT(index < m_params.size());
if (index >= m_params.size())
return false;
const dynamic_string& w = m_params[index]; const uint cMaxValues = 16;
if (w.is_empty()) dynamic_string val_str[cMaxValues];
return false; uint num_val_strs = 0;
if (desc.m_num_values) {
CRNLIB_ASSERT(desc.m_num_values <= cMaxValues);
if ((arg_index + desc.m_num_values) > params.size()) {
console::error("Expected %u value(s) after command line parameter: \"%s\"", desc.m_num_values, src_param.get_ptr());
return false;
}
for (uint v = 0; v < desc.m_num_values; v++)
val_str[num_val_strs++] = params[arg_index++];
}
dynamic_string_array strings;
if ((desc.m_support_listing_file) && (val_str[0].get_len() >= 2) && (val_str[0][0] == '@')) {
dynamic_string filename(val_str[0]);
filename.right(1);
filename.unquote();
if (!load_string_file(filename.get_ptr(), strings)) {
console::error("Failed loading listing file \"%s\"!", filename.get_ptr());
return false;
}
} else {
for (uint v = 0; v < num_val_strs; v++) {
val_str[v].unquote();
strings.push_back(val_str[v]);
}
}
param_value pv;
pv.m_values.swap(strings);
pv.m_index = cur_arg_index;
pv.m_modifier = (int8)modifier;
m_param_map.insert(std::make_pair(key_str, pv));
} else {
param_value pv;
pv.m_values.push_back(src_param);
pv.m_values.back().unquote();
pv.m_index = cur_arg_index;
m_param_map.insert(std::make_pair(g_empty_dynamic_string, pv));
}
}
return true;
}
bool command_line_params::parse(const char* pCmd_line, uint n, const param_desc* pParam_desc, bool skip_first_param) {
CRNLIB_ASSERT(n && pParam_desc);
dynamic_string_array p;
if (!split_params(pCmd_line, p))
return 0;
if (p.empty())
return 0;
if (skip_first_param)
p.erase(0U);
return parse(p, n, pParam_desc);
}
bool command_line_params::is_param(uint index) const {
CRNLIB_ASSERT(index < m_params.size());
if (index >= m_params.size())
return false;
const dynamic_string& w = m_params[index];
if (w.is_empty())
return false;
#if CRNLIB_CMD_LINE_ALLOW_SLASH_PARAMS #if CRNLIB_CMD_LINE_ALLOW_SLASH_PARAMS
return (w.get_len() >= 2) && ((w[0] == '-') || (w[0] == '/')); return (w.get_len() >= 2) && ((w[0] == '-') || (w[0] == '/'));
#else #else
return (w.get_len() >= 2) && (w[0] == '-'); return (w.get_len() >= 2) && (w[0] == '-');
#endif #endif
} }
uint command_line_params::find(uint num_keys, const char** ppKeys, crnlib::vector<param_map_const_iterator>* pIterators, crnlib::vector<uint>* pUnmatched_indices) const uint command_line_params::find(uint num_keys, const char** ppKeys, crnlib::vector<param_map_const_iterator>* pIterators, crnlib::vector<uint>* pUnmatched_indices) const {
{ CRNLIB_ASSERT(ppKeys);
CRNLIB_ASSERT(ppKeys);
if (pUnmatched_indices) if (pUnmatched_indices) {
{ pUnmatched_indices->resize(m_params.size());
pUnmatched_indices->resize(m_params.size()); for (uint i = 0; i < m_params.size(); i++)
for (uint i = 0; i < m_params.size(); i++) (*pUnmatched_indices)[i] = i;
(*pUnmatched_indices)[i] = i; }
uint n = 0;
for (uint i = 0; i < num_keys; i++) {
const char* pKey = ppKeys[i];
param_map_const_iterator begin, end;
find(pKey, begin, end);
while (begin != end) {
if (pIterators)
pIterators->push_back(begin);
if (pUnmatched_indices) {
int k = pUnmatched_indices->find(begin->second.m_index);
if (k >= 0)
pUnmatched_indices->erase_unordered(k);
} }
uint n = 0; n++;
for (uint i = 0; i < num_keys; i++) begin++;
{ }
const char* pKey = ppKeys[i]; }
param_map_const_iterator begin, end; return n;
find(pKey, begin, end); }
while (begin != end) void command_line_params::find(const char* pKey, param_map_const_iterator& begin, param_map_const_iterator& end) const {
{ dynamic_string key(pKey);
if (pIterators) begin = m_param_map.lower_bound(key);
pIterators->push_back(begin); end = m_param_map.upper_bound(key);
}
if (pUnmatched_indices) uint command_line_params::get_count(const char* pKey) const {
{ param_map_const_iterator begin, end;
int k = pUnmatched_indices->find(begin->second.m_index); find(pKey, begin, end);
if (k >= 0)
pUnmatched_indices->erase_unordered(k);
}
n++; uint n = 0;
begin++;
}
}
return n; while (begin != end) {
} n++;
begin++;
}
void command_line_params::find(const char* pKey, param_map_const_iterator& begin, param_map_const_iterator& end) const return n;
{ }
dynamic_string key(pKey);
begin = m_param_map.lower_bound(key);
end = m_param_map.upper_bound(key);
}
uint command_line_params::get_count(const char* pKey) const command_line_params::param_map_const_iterator command_line_params::get_param(const char* pKey, uint index) const {
{ param_map_const_iterator begin, end;
param_map_const_iterator begin, end; find(pKey, begin, end);
find(pKey, begin, end);
uint n = 0; if (begin == end)
return m_param_map.end();
while (begin != end) uint n = 0;
{
n++;
begin++;
}
return n; while ((begin != end) && (n != index)) {
} n++;
begin++;
}
command_line_params::param_map_const_iterator command_line_params::get_param(const char* pKey, uint index) const if (begin == end)
{ return m_param_map.end();
param_map_const_iterator begin, end;
find(pKey, begin, end);
if (begin == end) return begin;
return m_param_map.end(); }
uint n = 0; bool command_line_params::has_value(const char* pKey, uint index) const {
return get_num_values(pKey, index) != 0;
}
while ((begin != end) && (n != index)) uint command_line_params::get_num_values(const char* pKey, uint index) const {
{ param_map_const_iterator it = get_param(pKey, index);
n++;
begin++;
}
if (begin == end) if (it == end())
return m_param_map.end(); return 0;
return begin; return it->second.m_values.size();
} }
bool command_line_params::has_value(const char* pKey, uint index) const bool command_line_params::get_value_as_bool(const char* pKey, uint index, bool def) const {
{ param_map_const_iterator it = get_param(pKey, index);
return get_num_values(pKey, index) != 0; if (it == end())
} return def;
uint command_line_params::get_num_values(const char* pKey, uint index) const if (it->second.m_modifier)
{ return it->second.m_modifier > 0;
param_map_const_iterator it = get_param(pKey, index); else
return true;
}
if (it == end()) int command_line_params::get_value_as_int(const char* pKey, uint index, int def, int l, int h, uint value_index) const {
return 0; param_map_const_iterator it = get_param(pKey, index);
if ((it == end()) || (value_index >= it->second.m_values.size()))
return def;
return it->second.m_values.size(); int val;
} const char* p = it->second.m_values[value_index].get_ptr();
if (!string_to_int(p, val)) {
crnlib::console::warning("Invalid value specified for parameter \"%s\", using default value of %i", pKey, def);
return def;
}
bool command_line_params::get_value_as_bool(const char* pKey, uint index, bool def) const if (val < l) {
{ crnlib::console::warning("Value %i for parameter \"%s\" is out of range, clamping to %i", val, pKey, l);
param_map_const_iterator it = get_param(pKey, index); val = l;
if (it == end()) } else if (val > h) {
return def; crnlib::console::warning("Value %i for parameter \"%s\" is out of range, clamping to %i", val, pKey, h);
val = h;
}
if (it->second.m_modifier) return val;
return it->second.m_modifier > 0; }
else
return true;
}
int command_line_params::get_value_as_int(const char* pKey, uint index, int def, int l, int h, uint value_index) const float command_line_params::get_value_as_float(const char* pKey, uint index, float def, float l, float h, uint value_index) const {
{ param_map_const_iterator it = get_param(pKey, index);
param_map_const_iterator it = get_param(pKey, index); if ((it == end()) || (value_index >= it->second.m_values.size()))
if ((it == end()) || (value_index >= it->second.m_values.size())) return def;
return def;
int val; float val;
const char* p = it->second.m_values[value_index].get_ptr(); const char* p = it->second.m_values[value_index].get_ptr();
if (!string_to_int(p, val)) if (!string_to_float(p, val)) {
{ crnlib::console::warning("Invalid value specified for float parameter \"%s\", using default value of %f", pKey, def);
crnlib::console::warning("Invalid value specified for parameter \"%s\", using default value of %i", pKey, def); return def;
return def; }
}
if (val < l) if (val < l) {
{ crnlib::console::warning("Value %f for parameter \"%s\" is out of range, clamping to %f", val, pKey, l);
crnlib::console::warning("Value %i for parameter \"%s\" is out of range, clamping to %i", val, pKey, l); val = l;
val = l; } else if (val > h) {
} crnlib::console::warning("Value %f for parameter \"%s\" is out of range, clamping to %f", val, pKey, h);
else if (val > h) val = h;
{ }
crnlib::console::warning("Value %i for parameter \"%s\" is out of range, clamping to %i", val, pKey, h);
val = h;
}
return val; return val;
} }
float command_line_params::get_value_as_float(const char* pKey, uint index, float def, float l, float h, uint value_index) const bool command_line_params::get_value_as_string(const char* pKey, uint index, dynamic_string& value, uint value_index) const {
{ param_map_const_iterator it = get_param(pKey, index);
param_map_const_iterator it = get_param(pKey, index); if ((it == end()) || (value_index >= it->second.m_values.size())) {
if ((it == end()) || (value_index >= it->second.m_values.size())) value.empty();
return def; return false;
}
float val; value = it->second.m_values[value_index];
const char* p = it->second.m_values[value_index].get_ptr(); return true;
if (!string_to_float(p, val)) }
{
crnlib::console::warning("Invalid value specified for float parameter \"%s\", using default value of %f", pKey, def);
return def;
}
if (val < l) const dynamic_string& command_line_params::get_value_as_string_or_empty(const char* pKey, uint index, uint value_index) const {
{ param_map_const_iterator it = get_param(pKey, index);
crnlib::console::warning("Value %f for parameter \"%s\" is out of range, clamping to %f", val, pKey, l); if ((it == end()) || (value_index >= it->second.m_values.size()))
val = l; return g_empty_dynamic_string;
}
else if (val > h)
{
crnlib::console::warning("Value %f for parameter \"%s\" is out of range, clamping to %f", val, pKey, h);
val = h;
}
return val; return it->second.m_values[value_index];
} }
bool command_line_params::get_value_as_string(const char* pKey, uint index, dynamic_string& value, uint value_index) const
{
param_map_const_iterator it = get_param(pKey, index);
if ((it == end()) || (value_index >= it->second.m_values.size()))
{
value.empty();
return false;
}
value = it->second.m_values[value_index];
return true;
}
const dynamic_string& command_line_params::get_value_as_string_or_empty(const char* pKey, uint index, uint value_index) const
{
param_map_const_iterator it = get_param(pKey, index);
if ((it == end()) || (value_index >= it->second.m_values.size()))
return g_empty_dynamic_string;
return it->second.m_values[value_index];
}
} // namespace crnlib
} // namespace crnlib
crnlib/crn_command_line_params.h
+51 -54
View File
@@ -4,83 +4,80 @@
#include "crn_value.h" #include "crn_value.h"
#include <map> #include <map>
namespace crnlib namespace crnlib {
{ // Returns the command line passed to the app as a string.
// Returns the command line passed to the app as a string. // On systems where this isn't trivial, this function combines together the separate arguments, quoting and adding spaces as needed.
// On systems where this isn't trivial, this function combines together the separate arguments, quoting and adding spaces as needed. void get_command_line_as_single_string(dynamic_string& cmd_line, int argc, char* argv[]);
void get_command_line_as_single_string(dynamic_string& cmd_line, int argc, char *argv[]);
class command_line_params class command_line_params {
{ public:
public: struct param_value {
struct param_value inline param_value()
{ : m_index(0), m_modifier(0) {}
inline param_value() : m_index(0), m_modifier(0) { }
dynamic_string_array m_values; dynamic_string_array m_values;
uint m_index; uint m_index;
int8 m_modifier; int8 m_modifier;
}; };
typedef std::multimap<dynamic_string, param_value> param_map; typedef std::multimap<dynamic_string, param_value> param_map;
typedef param_map::const_iterator param_map_const_iterator; typedef param_map::const_iterator param_map_const_iterator;
typedef param_map::iterator param_map_iterator; typedef param_map::iterator param_map_iterator;
command_line_params(); command_line_params();
void clear(); void clear();
static bool split_params(const char* p, dynamic_string_array& params); static bool split_params(const char* p, dynamic_string_array& params);
struct param_desc struct param_desc {
{ const char* m_pName;
const char* m_pName; uint m_num_values;
uint m_num_values; bool m_support_listing_file;
bool m_support_listing_file; };
};
bool parse(const dynamic_string_array& params, uint n, const param_desc* pParam_desc); bool parse(const dynamic_string_array& params, uint n, const param_desc* pParam_desc);
bool parse(const char* pCmd_line, uint n, const param_desc* pParam_desc, bool skip_first_param = true); bool parse(const char* pCmd_line, uint n, const param_desc* pParam_desc, bool skip_first_param = true);
const dynamic_string_array& get_array() const { return m_params; } const dynamic_string_array& get_array() const { return m_params; }
bool is_param(uint index) const; bool is_param(uint index) const;
const param_map& get_map() const { return m_param_map; } const param_map& get_map() const { return m_param_map; }
uint get_num_params() const { return static_cast<uint>(m_param_map.size()); } uint get_num_params() const { return static_cast<uint>(m_param_map.size()); }
param_map_const_iterator begin() const { return m_param_map.begin(); } param_map_const_iterator begin() const { return m_param_map.begin(); }
param_map_const_iterator end() const { return m_param_map.end(); } param_map_const_iterator end() const { return m_param_map.end(); }
uint find(uint num_keys, const char** ppKeys, crnlib::vector<param_map_const_iterator>* pIterators, crnlib::vector<uint>* pUnmatched_indices) const; uint find(uint num_keys, const char** ppKeys, crnlib::vector<param_map_const_iterator>* pIterators, crnlib::vector<uint>* pUnmatched_indices) const;
void find(const char* pKey, param_map_const_iterator& begin, param_map_const_iterator& end) const; void find(const char* pKey, param_map_const_iterator& begin, param_map_const_iterator& end) const;
uint get_count(const char* pKey) const; uint get_count(const char* pKey) const;
// Returns end() if param cannot be found, or index is out of range. // Returns end() if param cannot be found, or index is out of range.
param_map_const_iterator get_param(const char* pKey, uint index) const; param_map_const_iterator get_param(const char* pKey, uint index) const;
bool has_key(const char* pKey) const { return get_param(pKey, 0) != end(); } bool has_key(const char* pKey) const { return get_param(pKey, 0) != end(); }
bool has_value(const char* pKey, uint index) const; bool has_value(const char* pKey, uint index) const;
uint get_num_values(const char* pKey, uint index) const; uint get_num_values(const char* pKey, uint index) const;
bool get_value_as_bool(const char* pKey, uint index = 0, bool def = false) const; bool get_value_as_bool(const char* pKey, uint index = 0, bool def = false) const;
int get_value_as_int(const char* pKey, uint index, int def, int l = INT_MIN, int h = INT_MAX, uint value_index = 0) const; int get_value_as_int(const char* pKey, uint index, int def, int l = INT_MIN, int h = INT_MAX, uint value_index = 0) const;
float get_value_as_float(const char* pKey, uint index, float def = 0.0f, float l = -math::cNearlyInfinite, float h = math::cNearlyInfinite, uint value_index = 0) const; float get_value_as_float(const char* pKey, uint index, float def = 0.0f, float l = -math::cNearlyInfinite, float h = math::cNearlyInfinite, uint value_index = 0) const;
bool get_value_as_string(const char* pKey, uint index, dynamic_string& value, uint value_index = 0) const; bool get_value_as_string(const char* pKey, uint index, dynamic_string& value, uint value_index = 0) const;
const dynamic_string& get_value_as_string_or_empty(const char* pKey, uint index = 0, uint value_index = 0) const; const dynamic_string& get_value_as_string_or_empty(const char* pKey, uint index = 0, uint value_index = 0) const;
private: private:
dynamic_string_array m_params; dynamic_string_array m_params;
param_map m_param_map; param_map m_param_map;
static bool load_string_file(const char* pFilename, dynamic_string_array& strings); static bool load_string_file(const char* pFilename, dynamic_string_array& strings);
}; };
} // namespace crnlib } // namespace crnlib
crnlib/crn_comp.cpp
+1252 -2106
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_comp.h
+89 -145
View File
@@ -2,9 +2,7 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
#define CRND_HEADER_FILE_ONLY #include "../inc/crn_defs.h"
#include "../inc/crn_decomp.h"
#undef CRND_HEADER_FILE_ONLY
#include "../inc/crnlib.h" #include "../inc/crnlib.h"
#include "crn_symbol_codec.h" #include "crn_symbol_codec.h"
@@ -13,169 +11,115 @@
#include "crn_image_utils.h" #include "crn_image_utils.h"
#include "crn_texture_comp.h" #include "crn_texture_comp.h"
namespace crnlib namespace crnlib {
{ class crn_comp : public itexture_comp {
class crn_comp : public itexture_comp CRNLIB_NO_COPY_OR_ASSIGNMENT_OP(crn_comp);
{
CRNLIB_NO_COPY_OR_ASSIGNMENT_OP(crn_comp);
public: public:
crn_comp(); crn_comp();
virtual ~crn_comp(); virtual ~crn_comp();
virtual const char *get_ext() const { return "CRN"; } virtual const char* get_ext() const { return "CRN"; }
virtual bool compress_init(const crn_comp_params& params); virtual bool compress_init(const crn_comp_params&) { return true; };
virtual bool compress_pass(const crn_comp_params& params, float *pEffective_bitrate); virtual bool compress_pass(const crn_comp_params& params, float* pEffective_bitrate);
virtual void compress_deinit(); virtual void compress_deinit();
virtual const crnlib::vector<uint8>& get_comp_data() const { return m_comp_data; } virtual const crnlib::vector<uint8>& get_comp_data() const { return m_comp_data; }
virtual crnlib::vector<uint8>& get_comp_data() { return m_comp_data; } virtual crnlib::vector<uint8>& get_comp_data() { return m_comp_data; }
uint get_comp_data_size() const { return m_comp_data.size(); } uint get_comp_data_size() const { return m_comp_data.size(); }
const uint8* get_comp_data_ptr() const { return m_comp_data.size() ? &m_comp_data[0] : NULL; } const uint8* get_comp_data_ptr() const { return m_comp_data.size() ? &m_comp_data[0] : NULL; }
private: private:
task_pool m_task_pool; task_pool m_task_pool;
const crn_comp_params* m_pParams; const crn_comp_params* m_pParams;
image_u8 m_images[cCRNMaxFaces][cCRNMaxLevels]; image_u8 m_images[cCRNMaxFaces][cCRNMaxLevels];
struct level_tag enum comp {
{ cColor,
uint m_width, m_height; cAlpha0,
uint m_chunk_width, m_chunk_height; cAlpha1,
uint m_group_index; cNumComps
uint m_num_chunks; };
uint m_first_chunk;
uint m_group_first_chunk;
} m_levels[cCRNMaxLevels];
struct mip_group bool m_has_comp[cNumComps];
{ bool m_has_etc_color_blocks;
mip_group() : m_first_chunk(0), m_num_chunks(0) { } bool m_has_subblocks;
uint m_first_chunk; struct level_details {
uint m_num_chunks; uint first_block;
}; uint num_blocks;
crnlib::vector<mip_group> m_mip_groups; uint block_width;
};
crnlib::vector<level_details> m_levels;
enum comp uint m_total_blocks;
{ crnlib::vector<uint32> m_color_endpoints;
cColor, crnlib::vector<uint32> m_alpha_endpoints;
cAlpha0, crnlib::vector<uint32> m_color_selectors;
cAlpha1, crnlib::vector<uint64> m_alpha_selectors;
cNumComps crnlib::vector<dxt_hc::endpoint_indices_details> m_endpoint_indices;
}; crnlib::vector<dxt_hc::selector_indices_details> m_selector_indices;
bool m_has_comp[cNumComps]; crnd::crn_header m_crn_header;
crnlib::vector<uint8> m_comp_data;
struct chunk_detail dxt_hc m_hvq;
{
chunk_detail() { utils::zero_object(*this); }
uint m_first_endpoint_index; symbol_histogram m_reference_hist;
uint m_first_selector_index; static_huffman_data_model m_reference_dm;
};
typedef crnlib::vector<chunk_detail> chunk_detail_vec;
chunk_detail_vec m_chunk_details;
crnlib::vector<uint> m_endpoint_indices[cNumComps]; crnlib::vector<uint16> m_endpoint_remaping[2];
crnlib::vector<uint> m_selector_indices[cNumComps]; symbol_histogram m_endpoint_index_hist[2];
static_huffman_data_model m_endpoint_index_dm[2];
uint m_total_chunks; crnlib::vector<uint16> m_selector_remaping[2];
dxt_hc::pixel_chunk_vec m_chunks; symbol_histogram m_selector_index_hist[2];
static_huffman_data_model m_selector_index_dm[2];
crnd::crn_header m_crn_header; crnlib::vector<uint8> m_packed_blocks[cCRNMaxLevels];
crnlib::vector<uint8> m_comp_data; crnlib::vector<uint8> m_packed_data_models;
crnlib::vector<uint8> m_packed_color_endpoints;
crnlib::vector<uint8> m_packed_color_selectors;
crnlib::vector<uint8> m_packed_alpha_endpoints;
crnlib::vector<uint8> m_packed_alpha_selectors;
dxt_hc m_hvq; bool pack_color_endpoints(crnlib::vector<uint8>& packed_data, const crnlib::vector<uint16>& remapping);
bool pack_color_endpoints_etc(crnlib::vector<uint8>& packed_data, const crnlib::vector<uint16>& remapping);
bool pack_color_selectors(crnlib::vector<uint8>& packed_data, const crnlib::vector<uint16>& remapping);
bool pack_alpha_endpoints(crnlib::vector<uint8>& packed_data, const crnlib::vector<uint16>& remapping);
bool pack_alpha_selectors(crnlib::vector<uint8>& packed_data, const crnlib::vector<uint16>& remapping);
bool pack_blocks(
uint group,
bool clear_histograms,
symbol_codec* pCodec,
const crnlib::vector<uint16>* pColor_endpoint_remap,
const crnlib::vector<uint16>* pColor_selector_remap,
const crnlib::vector<uint16>* pAlpha_endpoint_remap,
const crnlib::vector<uint16>* pAlpha_selector_remap
);
symbol_histogram m_chunk_encoding_hist; bool alias_images();
static_huffman_data_model m_chunk_encoding_dm; void clear();
bool quantize_images();
symbol_histogram m_endpoint_index_hist[2]; void optimize_color_endpoints_task(uint64 data, void* pData_ptr);
static_huffman_data_model m_endpoint_index_dm[2]; // color, alpha void optimize_color_selectors();
void optimize_color();
symbol_histogram m_selector_index_hist[2]; void optimize_alpha_endpoints_task(uint64 data, void* pData_ptr);
static_huffman_data_model m_selector_index_dm[2]; // color, alpha void optimize_alpha_selectors();
void optimize_alpha();
crnlib::vector<uint8> m_packed_chunks[cCRNMaxLevels]; bool pack_data_models();
crnlib::vector<uint8> m_packed_data_models; static void append_vec(crnlib::vector<uint8>& a, const void* p, uint size);
crnlib::vector<uint8> m_packed_color_endpoints; static void append_vec(crnlib::vector<uint8>& a, const crnlib::vector<uint8>& b);
crnlib::vector<uint8> m_packed_color_selectors; bool create_comp_data();
crnlib::vector<uint8> m_packed_alpha_endpoints;
crnlib::vector<uint8> m_packed_alpha_selectors;
void clear(); bool update_progress(uint phase_index, uint subphase_index, uint subphase_total);
bool compress_internal();
};
void append_chunks(const image_u8& img, uint num_chunks_x, uint num_chunks_y, dxt_hc::pixel_chunk_vec& chunks, float weight); } // namespace crnlib
static float color_endpoint_similarity_func(uint index_a, uint index_b, void* pContext);
static float alpha_endpoint_similarity_func(uint index_a, uint index_b, void* pContext);
void sort_color_endpoint_codebook(crnlib::vector<uint>& remapping, const crnlib::vector<uint>& endpoints);
void sort_alpha_endpoint_codebook(crnlib::vector<uint>& remapping, const crnlib::vector<uint>& endpoints);
bool pack_color_endpoints(crnlib::vector<uint8>& data, const crnlib::vector<uint>& remapping, const crnlib::vector<uint>& endpoint_indices, uint trial_index);
bool pack_alpha_endpoints(crnlib::vector<uint8>& data, const crnlib::vector<uint>& remapping, const crnlib::vector<uint>& endpoint_indices, uint trial_index);
static float color_selector_similarity_func(uint index_a, uint index_b, void* pContext);
static float alpha_selector_similarity_func(uint index_a, uint index_b, void* pContext);
void sort_selector_codebook(crnlib::vector<uint>& remapping, const crnlib::vector<dxt_hc::selectors>& selectors, const uint8* pTo_linear);
bool pack_selectors(
crnlib::vector<uint8>& packed_data,
const crnlib::vector<uint>& selector_indices,
const crnlib::vector<dxt_hc::selectors>& selectors,
const crnlib::vector<uint>& remapping,
uint max_selector_value,
const uint8* pTo_linear,
uint trial_index);
bool alias_images();
void create_chunks();
bool quantize_chunks();
void create_chunk_indices();
bool pack_chunks(
uint first_chunk, uint num_chunks,
bool clear_histograms,
symbol_codec* pCodec,
const crnlib::vector<uint>* pColor_endpoint_remap,
const crnlib::vector<uint>* pColor_selector_remap,
const crnlib::vector<uint>* pAlpha_endpoint_remap,
const crnlib::vector<uint>* pAlpha_selector_remap);
bool pack_chunks_simulation(
uint first_chunk, uint num_chunks,
uint& total_bits,
const crnlib::vector<uint>* pColor_endpoint_remap,
const crnlib::vector<uint>* pColor_selector_remap,
const crnlib::vector<uint>* pAlpha_endpoint_remap,
const crnlib::vector<uint>* pAlpha_selector_remap);
void optimize_color_endpoint_codebook_task(uint64 data, void* pData_ptr);
bool optimize_color_endpoint_codebook(crnlib::vector<uint>& remapping);
void optimize_color_selector_codebook_task(uint64 data, void* pData_ptr);
bool optimize_color_selector_codebook(crnlib::vector<uint>& remapping);
void optimize_alpha_endpoint_codebook_task(uint64 data, void* pData_ptr);
bool optimize_alpha_endpoint_codebook(crnlib::vector<uint>& remapping);
void optimize_alpha_selector_codebook_task(uint64 data, void* pData_ptr);
bool optimize_alpha_selector_codebook(crnlib::vector<uint>& remapping);
bool create_comp_data();
bool pack_data_models();
bool update_progress(uint phase_index, uint subphase_index, uint subphase_total);
bool compress_internal();
static void append_vec(crnlib::vector<uint8>& a, const void* p, uint size);
static void append_vec(crnlib::vector<uint8>& a, const crnlib::vector<uint8>& b);
};
} // namespace crnlib
crnlib/crn_console.cpp
+153 -175
View File
@@ -5,218 +5,196 @@
#include "crn_data_stream.h" #include "crn_data_stream.h"
#include "crn_threading.h" #include "crn_threading.h"
namespace crnlib namespace crnlib {
{ eConsoleMessageType console::m_default_category = cInfoConsoleMessage;
eConsoleMessageType console::m_default_category = cInfoConsoleMessage; crnlib::vector<console::console_func> console::m_output_funcs;
crnlib::vector<console::console_func> console::m_output_funcs; bool console::m_crlf = true;
bool console::m_crlf = true; bool console::m_prefixes = true;
bool console::m_prefixes = true; bool console::m_output_disabled;
bool console::m_output_disabled; data_stream* console::m_pLog_stream;
data_stream* console::m_pLog_stream; mutex* console::m_pMutex;
mutex* console::m_pMutex; uint console::m_num_messages[cCMTTotal];
uint console::m_num_messages[cCMTTotal]; bool console::m_at_beginning_of_line = true;
bool console::m_at_beginning_of_line = true;
const uint cConsoleBufSize = 4096; const uint cConsoleBufSize = 4096;
void console::init() void console::init() {
{ if (!m_pMutex) {
if (!m_pMutex) m_pMutex = crnlib_new<mutex>();
{ }
m_pMutex = crnlib_new<mutex>(); }
}
}
void console::deinit() void console::deinit() {
{ if (m_pMutex) {
if (m_pMutex) crnlib_delete(m_pMutex);
{ m_pMutex = NULL;
crnlib_delete(m_pMutex); }
m_pMutex = NULL; }
}
}
void console::disable_crlf() void console::disable_crlf() {
{ init();
init();
m_crlf = false; m_crlf = false;
} }
void console::enable_crlf() void console::enable_crlf() {
{ init();
init();
m_crlf = true; m_crlf = true;
} }
void console::vprintf(eConsoleMessageType type, const char* p, va_list args) void console::vprintf(eConsoleMessageType type, const char* p, va_list args) {
{ init();
init();
scoped_mutex lock(*m_pMutex); scoped_mutex lock(*m_pMutex);
m_num_messages[type]++; m_num_messages[type]++;
char buf[cConsoleBufSize]; char buf[cConsoleBufSize];
vsprintf_s(buf, cConsoleBufSize, p, args); vsprintf_s(buf, cConsoleBufSize, p, args);
bool handled = false; bool handled = false;
if (m_output_funcs.size()) if (m_output_funcs.size()) {
{ for (uint i = 0; i < m_output_funcs.size(); i++)
for (uint i = 0; i < m_output_funcs.size(); i++) if (m_output_funcs[i].m_func(type, buf, m_output_funcs[i].m_pData))
if (m_output_funcs[i].m_func(type, buf, m_output_funcs[i].m_pData)) handled = true;
handled = true; }
}
const char* pPrefix = NULL; const char* pPrefix = NULL;
if ((m_prefixes) && (m_at_beginning_of_line)) if ((m_prefixes) && (m_at_beginning_of_line)) {
{ switch (type) {
switch (type) case cDebugConsoleMessage:
{ pPrefix = "Debug: ";
case cDebugConsoleMessage: pPrefix = "Debug: "; break; break;
case cWarningConsoleMessage: pPrefix = "Warning: "; break; case cWarningConsoleMessage:
case cErrorConsoleMessage: pPrefix = "Error: "; break; pPrefix = "Warning: ";
default: break; break;
} case cErrorConsoleMessage:
} pPrefix = "Error: ";
break;
default:
break;
}
}
if ((!m_output_disabled) && (!handled)) if ((!m_output_disabled) && (!handled)) {
{ if (pPrefix)
if (pPrefix) ::printf("%s", pPrefix);
::printf("%s", pPrefix); ::printf(m_crlf ? "%s\n" : "%s", buf);
::printf(m_crlf ? "%s\n" : "%s", buf); }
}
uint n = strlen(buf); uint n = strlen(buf);
m_at_beginning_of_line = (m_crlf) || ((n) && (buf[n - 1] == '\n')); m_at_beginning_of_line = (m_crlf) || ((n) && (buf[n - 1] == '\n'));
if ((type != cProgressConsoleMessage) && (m_pLog_stream)) if ((type != cProgressConsoleMessage) && (m_pLog_stream)) {
{ // Yes this is bad.
// Yes this is bad. dynamic_string tmp_buf(buf);
dynamic_string tmp_buf(buf);
tmp_buf.translate_lf_to_crlf(); tmp_buf.translate_lf_to_crlf();
m_pLog_stream->printf(m_crlf ? "%s\r\n" : "%s", tmp_buf.get_ptr()); m_pLog_stream->printf(m_crlf ? "%s\r\n" : "%s", tmp_buf.get_ptr());
m_pLog_stream->flush(); m_pLog_stream->flush();
} }
} }
void console::printf(eConsoleMessageType type, const char* p, ...) void console::printf(eConsoleMessageType type, const char* p, ...) {
{ va_list args;
va_list args; va_start(args, p);
va_start(args, p); vprintf(type, p, args);
vprintf(type, p, args); va_end(args);
va_end(args); }
}
void console::printf(const char* p, ...) void console::printf(const char* p, ...) {
{ va_list args;
va_list args; va_start(args, p);
va_start(args, p); vprintf(m_default_category, p, args);
vprintf(m_default_category, p, args); va_end(args);
va_end(args); }
}
void console::set_default_category(eConsoleMessageType category) void console::set_default_category(eConsoleMessageType category) {
{ init();
init();
m_default_category = category; m_default_category = category;
} }
eConsoleMessageType console::get_default_category() eConsoleMessageType console::get_default_category() {
{ init();
init();
return m_default_category; return m_default_category;
} }
void console::add_console_output_func(console_output_func pFunc, void* pData) void console::add_console_output_func(console_output_func pFunc, void* pData) {
{ init();
init();
scoped_mutex lock(*m_pMutex); scoped_mutex lock(*m_pMutex);
m_output_funcs.push_back(console_func(pFunc, pData)); m_output_funcs.push_back(console_func(pFunc, pData));
} }
void console::remove_console_output_func(console_output_func pFunc) void console::remove_console_output_func(console_output_func pFunc) {
{ init();
init();
scoped_mutex lock(*m_pMutex); scoped_mutex lock(*m_pMutex);
for (int i = m_output_funcs.size() - 1; i >= 0; i--) for (int i = m_output_funcs.size() - 1; i >= 0; i--) {
{ if (m_output_funcs[i].m_func == pFunc) {
if (m_output_funcs[i].m_func == pFunc) m_output_funcs.erase(m_output_funcs.begin() + i);
{ }
m_output_funcs.erase(m_output_funcs.begin() + i); }
}
}
if (!m_output_funcs.size()) if (!m_output_funcs.size()) {
{ m_output_funcs.clear();
m_output_funcs.clear(); }
} }
}
void console::progress(const char* p, ...) void console::progress(const char* p, ...) {
{ va_list args;
va_list args; va_start(args, p);
va_start(args, p); vprintf(cProgressConsoleMessage, p, args);
vprintf(cProgressConsoleMessage, p, args); va_end(args);
va_end(args); }
}
void console::info(const char* p, ...) void console::info(const char* p, ...) {
{ va_list args;
va_list args; va_start(args, p);
va_start(args, p); vprintf(cInfoConsoleMessage, p, args);
vprintf(cInfoConsoleMessage, p, args); va_end(args);
va_end(args); }
}
void console::message(const char* p, ...) void console::message(const char* p, ...) {
{ va_list args;
va_list args; va_start(args, p);
va_start(args, p); vprintf(cMessageConsoleMessage, p, args);
vprintf(cMessageConsoleMessage, p, args); va_end(args);
va_end(args); }
}
void console::cons(const char* p, ...) void console::cons(const char* p, ...) {
{ va_list args;
va_list args; va_start(args, p);
va_start(args, p); vprintf(cConsoleConsoleMessage, p, args);
vprintf(cConsoleConsoleMessage, p, args); va_end(args);
va_end(args); }
}
void console::debug(const char* p, ...) void console::debug(const char* p, ...) {
{ va_list args;
va_list args; va_start(args, p);
va_start(args, p); vprintf(cDebugConsoleMessage, p, args);
vprintf(cDebugConsoleMessage, p, args); va_end(args);
va_end(args); }
}
void console::warning(const char* p, ...) void console::warning(const char* p, ...) {
{ va_list args;
va_list args; va_start(args, p);
va_start(args, p); vprintf(cWarningConsoleMessage, p, args);
vprintf(cWarningConsoleMessage, p, args); va_end(args);
va_end(args); }
}
void console::error(const char* p, ...) void console::error(const char* p, ...) {
{ va_list args;
va_list args; va_start(args, p);
va_start(args, p); vprintf(cErrorConsoleMessage, p, args);
vprintf(cErrorConsoleMessage, p, args); va_end(args);
va_end(args); }
}
} // namespace crnlib } // namespace crnlib
crnlib/crn_console.h
+85 -92
View File
@@ -7,122 +7,115 @@
#include <tchar.h> #include <tchar.h>
#include <conio.h> #include <conio.h>
#endif #endif
namespace crnlib namespace crnlib {
{ class dynamic_string;
class dynamic_string; class data_stream;
class data_stream; class mutex;
class mutex;
enum eConsoleMessageType enum eConsoleMessageType {
{ cDebugConsoleMessage, // debugging messages
cDebugConsoleMessage, // debugging messages cProgressConsoleMessage, // progress messages
cProgressConsoleMessage, // progress messages cInfoConsoleMessage, // ordinary messages
cInfoConsoleMessage, // ordinary messages cConsoleConsoleMessage, // user console output
cConsoleConsoleMessage, // user console output cMessageConsoleMessage, // high importance messages
cMessageConsoleMessage, // high importance messages cWarningConsoleMessage, // warnings
cWarningConsoleMessage, // warnings cErrorConsoleMessage, // errors
cErrorConsoleMessage, // errors
cCMTTotal cCMTTotal
}; };
typedef bool (*console_output_func)(eConsoleMessageType type, const char* pMsg, void* pData); typedef bool (*console_output_func)(eConsoleMessageType type, const char* pMsg, void* pData);
class console class console {
{ public:
public: static void init();
static void init(); static void deinit();
static void deinit();
static bool is_initialized() { return m_pMutex != NULL; } static bool is_initialized() { return m_pMutex != NULL; }
static void set_default_category(eConsoleMessageType category); static void set_default_category(eConsoleMessageType category);
static eConsoleMessageType get_default_category(); static eConsoleMessageType get_default_category();
static void add_console_output_func(console_output_func pFunc, void* pData); static void add_console_output_func(console_output_func pFunc, void* pData);
static void remove_console_output_func(console_output_func pFunc); static void remove_console_output_func(console_output_func pFunc);
static void printf(const char* p, ...); static void printf(const char* p, ...);
static void vprintf(eConsoleMessageType type, const char* p, va_list args); static void vprintf(eConsoleMessageType type, const char* p, va_list args);
static void printf(eConsoleMessageType type, const char* p, ...); static void printf(eConsoleMessageType type, const char* p, ...);
static void cons(const char* p, ...); static void cons(const char* p, ...);
static void debug(const char* p, ...); static void debug(const char* p, ...);
static void progress(const char* p, ...); static void progress(const char* p, ...);
static void info(const char* p, ...); static void info(const char* p, ...);
static void message(const char* p, ...); static void message(const char* p, ...);
static void warning(const char* p, ...); static void warning(const char* p, ...);
static void error(const char* p, ...); static void error(const char* p, ...);
// FIXME: All console state is currently global! // FIXME: All console state is currently global!
static void disable_prefixes(); static void disable_prefixes();
static void enable_prefixes(); static void enable_prefixes();
static bool get_prefixes() { return m_prefixes; } static bool get_prefixes() { return m_prefixes; }
static bool get_at_beginning_of_line() { return m_at_beginning_of_line; } static bool get_at_beginning_of_line() { return m_at_beginning_of_line; }
static void disable_crlf(); static void disable_crlf();
static void enable_crlf(); static void enable_crlf();
static bool get_crlf() { return m_crlf; } static bool get_crlf() { return m_crlf; }
static void disable_output() { m_output_disabled = true; } static void disable_output() { m_output_disabled = true; }
static void enable_output() { m_output_disabled = false; } static void enable_output() { m_output_disabled = false; }
static bool get_output_disabled() { return m_output_disabled; } static bool get_output_disabled() { return m_output_disabled; }
static void set_log_stream(data_stream* pStream) { m_pLog_stream = pStream; } static void set_log_stream(data_stream* pStream) { m_pLog_stream = pStream; }
static data_stream* get_log_stream() { return m_pLog_stream; } static data_stream* get_log_stream() { return m_pLog_stream; }
static uint get_num_messages(eConsoleMessageType type) { return m_num_messages[type]; } static uint get_num_messages(eConsoleMessageType type) { return m_num_messages[type]; }
private: private:
static eConsoleMessageType m_default_category; static eConsoleMessageType m_default_category;
struct console_func struct console_func {
{ console_func(console_output_func func = NULL, void* pData = NULL)
console_func(console_output_func func = NULL, void* pData = NULL) : m_func(func), m_pData(pData) { } : m_func(func), m_pData(pData) {}
console_output_func m_func; console_output_func m_func;
void* m_pData; void* m_pData;
}; };
static crnlib::vector<console_func> m_output_funcs; static crnlib::vector<console_func> m_output_funcs;
static bool m_crlf, m_prefixes, m_output_disabled; static bool m_crlf, m_prefixes, m_output_disabled;
static data_stream* m_pLog_stream; static data_stream* m_pLog_stream;
static mutex* m_pMutex; static mutex* m_pMutex;
static uint m_num_messages[cCMTTotal]; static uint m_num_messages[cCMTTotal];
static bool m_at_beginning_of_line; static bool m_at_beginning_of_line;
}; };
#if defined(WIN32) #if defined(WIN32)
inline int crn_getch() inline int crn_getch() {
{ return _getch();
return _getch(); }
}
#elif defined(__GNUC__) #elif defined(__GNUC__)
#include <termios.h> #include <termios.h>
#include <unistd.h> #include <unistd.h>
inline int crn_getch() inline int crn_getch() {
{ struct termios oldt, newt;
struct termios oldt, newt; int ch;
int ch; tcgetattr(STDIN_FILENO, &oldt);
tcgetattr(STDIN_FILENO, &oldt); newt = oldt;
newt = oldt; newt.c_lflag &= ~(ICANON | ECHO);
newt.c_lflag &= ~(ICANON | ECHO); tcsetattr(STDIN_FILENO, TCSANOW, &newt);
tcsetattr(STDIN_FILENO, TCSANOW, &newt); ch = getchar();
ch = getchar(); tcsetattr(STDIN_FILENO, TCSANOW, &oldt);
tcsetattr(STDIN_FILENO, TCSANOW, &oldt); return ch;
return ch; }
} #else
#else inline int crn_getch() {
inline int crn_getch() printf("crn_getch: Unimplemented");
{ return 0;
printf("crn_getch: Unimplemented"); }
return 0;
}
#endif #endif
} // namespace crnlib } // namespace crnlib
crnlib/crn_core.cpp
+4 -5
View File
@@ -6,9 +6,8 @@
#include "crn_winhdr.h" #include "crn_winhdr.h"
#endif #endif
namespace crnlib namespace crnlib {
{ const char* g_copyright_str = "Copyright (c) 2010-2016 Richard Geldreich, Jr. and Binomial LLC";
const char *g_copyright_str = "Copyright (c) 2010-2012 Rich Geldreich and Tenacious Software LLC"; const char* g_sig_str = "C8cfRlaorj0wLtnMSxrBJxTC85rho2L9hUZKHcBL";
const char *g_sig_str = "C8cfRlaorj0wLtnMSxrBJxTC85rho2L9hUZKHcBL";
} // namespace crnlib } // namespace crnlib
crnlib/crn_core.h
+116 -116
View File
@@ -3,124 +3,124 @@
#pragma once #pragma once
#if defined(WIN32) && defined(_MSC_VER) #if defined(WIN32) && defined(_MSC_VER)
#pragma warning (disable: 4201) // nonstandard extension used : nameless struct/union #pragma warning(disable : 4201) // nonstandard extension used : nameless struct/union
#pragma warning (disable: 4127) // conditional expression is constant #pragma warning(disable : 4127) // conditional expression is constant
#pragma warning (disable: 4793) // function compiled as native #pragma warning(disable : 4793) // function compiled as native
#pragma warning (disable: 4324) // structure was padded due to __declspec(align()) #pragma warning(disable : 4324) // structure was padded due to __declspec(align())
#endif #endif
#if defined(WIN32) && !defined(CRNLIB_ANSI_CPLUSPLUS) #if defined(WIN32) && !defined(CRNLIB_ANSI_CPLUSPLUS)
// MSVC or MinGW, x86 or x64, Win32 API's for threading and Win32 Interlocked API's or GCC built-ins for atomic ops. // MSVC or MinGW, x86 or x64, Win32 API's for threading and Win32 Interlocked API's or GCC built-ins for atomic ops.
#ifdef NDEBUG #ifdef NDEBUG
// Ensure checked iterators are disabled. Note: Be sure anything else that links against this lib also #define's this stuff, or remove this crap! // Ensure checked iterators are disabled. Note: Be sure anything else that links against this lib also #define's this stuff, or remove this crap!
#define _SECURE_SCL 0 #define _SECURE_SCL 0
#define _HAS_ITERATOR_DEBUGGING 0 #define _HAS_ITERATOR_DEBUGGING 0
#endif #endif
#ifndef _DLL #ifndef _DLL
// If we're using the DLL form of the run-time libs, we're also going to be enabling exceptions because we'll be building CLR apps. // If we're using the DLL form of the run-time libs, we're also going to be enabling exceptions because we'll be building CLR apps.
// Otherwise, we disable exceptions for a small speed boost. // Otherwise, we disable exceptions for a small speed boost.
#define _HAS_EXCEPTIONS 0 #define _HAS_EXCEPTIONS 0
#endif #endif
#define NOMINMAX #define NOMINMAX
#define CRNLIB_USE_WIN32_API 1 #define CRNLIB_USE_WIN32_API 1
#if defined(__MINGW32__) || defined(__MINGW64__) #if defined(__MINGW32__) || defined(__MINGW64__)
#define CRNLIB_USE_GCC_ATOMIC_BUILTINS 1 #define CRNLIB_USE_GCC_ATOMIC_BUILTINS 1
#else
#define CRNLIB_USE_WIN32_ATOMIC_FUNCTIONS 1
#endif
#define CRNLIB_PLATFORM_PC 1
#if defined(_WIN64) || defined(__MINGW64__) || defined(_LP64) || defined(__LP64__)
#define CRNLIB_PLATFORM_PC_X64 1
#define CRNLIB_64BIT_POINTERS 1
#define CRNLIB_CPU_HAS_64BIT_REGISTERS 1
#define CRNLIB_LITTLE_ENDIAN_CPU 1
#else
#define CRNLIB_PLATFORM_PC_X86 1
#define CRNLIB_64BIT_POINTERS 0
#define CRNLIB_CPU_HAS_64BIT_REGISTERS 0
#define CRNLIB_LITTLE_ENDIAN_CPU 1
#endif
#define CRNLIB_USE_UNALIGNED_INT_LOADS 1
#define CRNLIB_RESTRICT __restrict
#define CRNLIB_FORCE_INLINE __forceinline
#if defined(_MSC_VER) || defined(__MINGW32__) || defined(__MINGW64__)
#define CRNLIB_USE_MSVC_INTRINSICS 1
#endif
#define CRNLIB_INT64_FORMAT_SPECIFIER "%I64i"
#define CRNLIB_UINT64_FORMAT_SPECIFIER "%I64u"
#define CRNLIB_STDCALL __stdcall
#define CRNLIB_MEMORY_IMPORT_BARRIER
#define CRNLIB_MEMORY_EXPORT_BARRIER
#elif defined(__GNUC__) && !defined(CRNLIB_ANSI_CPLUSPLUS)
// GCC x86 or x64, pthreads for threading and GCC built-ins for atomic ops.
#define CRNLIB_PLATFORM_PC 1
#if defined(_WIN64) || defined(__MINGW64__) || defined(_LP64) || defined(__LP64__)
#define CRNLIB_PLATFORM_PC_X64 1
#define CRNLIB_64BIT_POINTERS 1
#define CRNLIB_CPU_HAS_64BIT_REGISTERS 1
#else
#define CRNLIB_PLATFORM_PC_X86 1
#define CRNLIB_64BIT_POINTERS 0
#define CRNLIB_CPU_HAS_64BIT_REGISTERS 0
#endif
#define CRNLIB_USE_UNALIGNED_INT_LOADS 1
#define CRNLIB_LITTLE_ENDIAN_CPU 1
#define CRNLIB_USE_PTHREADS_API 1
#define CRNLIB_USE_GCC_ATOMIC_BUILTINS 1
#define CRNLIB_RESTRICT
#define CRNLIB_FORCE_INLINE inline __attribute__((__always_inline__,__gnu_inline__))
#define CRNLIB_INT64_FORMAT_SPECIFIER "%lli"
#define CRNLIB_UINT64_FORMAT_SPECIFIER "%llu"
#define CRNLIB_STDCALL
#define CRNLIB_MEMORY_IMPORT_BARRIER
#define CRNLIB_MEMORY_EXPORT_BARRIER
#else #else
// Vanilla ANSI-C/C++ #define CRNLIB_USE_WIN32_ATOMIC_FUNCTIONS 1
// No threading support, unaligned loads are NOT okay. #endif
#if defined(_WIN64) || defined(__MINGW64__) || defined(_LP64) || defined(__LP64__)
#define CRNLIB_64BIT_POINTERS 1
#define CRNLIB_CPU_HAS_64BIT_REGISTERS 1
#else
#define CRNLIB_64BIT_POINTERS 0
#define CRNLIB_CPU_HAS_64BIT_REGISTERS 0
#endif
#define CRNLIB_USE_UNALIGNED_INT_LOADS 0 #define CRNLIB_PLATFORM_PC 1
#if __BIG_ENDIAN__ #if defined(_WIN64) || defined(__MINGW64__) || defined(_LP64) || defined(__LP64__)
#define CRNLIB_BIG_ENDIAN_CPU 1 #define CRNLIB_PLATFORM_PC_X64 1
#else #define CRNLIB_64BIT_POINTERS 1
#define CRNLIB_LITTLE_ENDIAN_CPU 1 #define CRNLIB_CPU_HAS_64BIT_REGISTERS 1
#endif #define CRNLIB_LITTLE_ENDIAN_CPU 1
#else
#define CRNLIB_PLATFORM_PC_X86 1
#define CRNLIB_64BIT_POINTERS 0
#define CRNLIB_CPU_HAS_64BIT_REGISTERS 0
#define CRNLIB_LITTLE_ENDIAN_CPU 1
#endif
#define CRNLIB_USE_GCC_ATOMIC_BUILTINS 0 #define CRNLIB_USE_UNALIGNED_INT_LOADS 1
#define CRNLIB_USE_WIN32_ATOMIC_FUNCTIONS 0 #define CRNLIB_RESTRICT __restrict
#define CRNLIB_FORCE_INLINE __forceinline
#define CRNLIB_RESTRICT #if defined(_MSC_VER) || defined(__MINGW32__) || defined(__MINGW64__)
#define CRNLIB_FORCE_INLINE inline #define CRNLIB_USE_MSVC_INTRINSICS 1
#endif
#define CRNLIB_INT64_FORMAT_SPECIFIER "%I64i" #define CRNLIB_INT64_FORMAT_SPECIFIER "%I64i"
#define CRNLIB_UINT64_FORMAT_SPECIFIER "%I64u" #define CRNLIB_UINT64_FORMAT_SPECIFIER "%I64u"
#define CRNLIB_STDCALL #define CRNLIB_STDCALL __stdcall
#define CRNLIB_MEMORY_IMPORT_BARRIER #define CRNLIB_MEMORY_IMPORT_BARRIER
#define CRNLIB_MEMORY_EXPORT_BARRIER #define CRNLIB_MEMORY_EXPORT_BARRIER
#elif defined(__GNUC__) && !defined(CRNLIB_ANSI_CPLUSPLUS)
// GCC x86 or x64, pthreads for threading and GCC built-ins for atomic ops.
#define CRNLIB_PLATFORM_PC 1
#if defined(_WIN64) || defined(__MINGW64__) || defined(_LP64) || defined(__LP64__)
#define CRNLIB_PLATFORM_PC_X64 1
#define CRNLIB_64BIT_POINTERS 1
#define CRNLIB_CPU_HAS_64BIT_REGISTERS 1
#else
#define CRNLIB_PLATFORM_PC_X86 1
#define CRNLIB_64BIT_POINTERS 0
#define CRNLIB_CPU_HAS_64BIT_REGISTERS 0
#endif
#define CRNLIB_USE_UNALIGNED_INT_LOADS 1
#define CRNLIB_LITTLE_ENDIAN_CPU 1
#define CRNLIB_USE_PTHREADS_API 1
#define CRNLIB_USE_GCC_ATOMIC_BUILTINS 1
#define CRNLIB_RESTRICT
#define CRNLIB_FORCE_INLINE inline __attribute__((__always_inline__, __gnu_inline__))
#define CRNLIB_INT64_FORMAT_SPECIFIER "%lli"
#define CRNLIB_UINT64_FORMAT_SPECIFIER "%llu"
#define CRNLIB_STDCALL
#define CRNLIB_MEMORY_IMPORT_BARRIER
#define CRNLIB_MEMORY_EXPORT_BARRIER
#else
// Vanilla ANSI-C/C++
// No threading support, unaligned loads are NOT okay.
#if defined(_WIN64) || defined(__MINGW64__) || defined(_LP64) || defined(__LP64__)
#define CRNLIB_64BIT_POINTERS 1
#define CRNLIB_CPU_HAS_64BIT_REGISTERS 1
#else
#define CRNLIB_64BIT_POINTERS 0
#define CRNLIB_CPU_HAS_64BIT_REGISTERS 0
#endif
#define CRNLIB_USE_UNALIGNED_INT_LOADS 0
#if __BIG_ENDIAN__
#define CRNLIB_BIG_ENDIAN_CPU 1
#else
#define CRNLIB_LITTLE_ENDIAN_CPU 1
#endif
#define CRNLIB_USE_GCC_ATOMIC_BUILTINS 0
#define CRNLIB_USE_WIN32_ATOMIC_FUNCTIONS 0
#define CRNLIB_RESTRICT
#define CRNLIB_FORCE_INLINE inline
#define CRNLIB_INT64_FORMAT_SPECIFIER "%I64i"
#define CRNLIB_UINT64_FORMAT_SPECIFIER "%I64u"
#define CRNLIB_STDCALL
#define CRNLIB_MEMORY_IMPORT_BARRIER
#define CRNLIB_MEMORY_EXPORT_BARRIER
#endif #endif
#define CRNLIB_SLOW_STRING_LEN_CHECKS 1 #define CRNLIB_SLOW_STRING_LEN_CHECKS 1
@@ -139,11 +139,11 @@
#include <errno.h> #include <errno.h>
#ifdef min #ifdef min
#undef min #undef min
#endif #endif
#ifdef max #ifdef max
#undef max #undef max
#endif #endif
#define CRNLIB_FALSE (0) #define CRNLIB_FALSE (0)
@@ -151,17 +151,17 @@
#define CRNLIB_MAX_PATH (260) #define CRNLIB_MAX_PATH (260)
#ifdef _DEBUG #ifdef _DEBUG
#define CRNLIB_BUILD_DEBUG #define CRNLIB_BUILD_DEBUG
#else #else
#define CRNLIB_BUILD_RELEASE #define CRNLIB_BUILD_RELEASE
#ifndef NDEBUG #ifndef NDEBUG
#define NDEBUG #define NDEBUG
#endif #endif
#ifdef DEBUG #ifdef DEBUG
#error DEBUG cannot be defined in CRNLIB_BUILD_RELEASE #error DEBUG cannot be defined in CRNLIB_BUILD_RELEASE
#endif #endif
#endif #endif
#include "crn_types.h" #include "crn_types.h"
crnlib/crn_data_stream.cpp
+89 -103
View File
@@ -3,126 +3,112 @@
#include "crn_core.h" #include "crn_core.h"
#include "crn_data_stream.h" #include "crn_data_stream.h"
namespace crnlib namespace crnlib {
{ data_stream::data_stream()
data_stream::data_stream() : : m_attribs(0),
m_attribs(0), m_opened(false),
m_opened(false), m_error(false), m_got_cr(false) m_error(false),
{ m_got_cr(false) {
} }
data_stream::data_stream(const char* pName, uint attribs) : data_stream::data_stream(const char* pName, uint attribs)
m_name(pName), : m_name(pName),
m_attribs(static_cast<uint16>(attribs)), m_attribs(static_cast<uint16>(attribs)),
m_opened(false), m_error(false), m_got_cr(false) m_opened(false),
{ m_error(false),
} m_got_cr(false) {
}
uint64 data_stream::skip(uint64 len) uint64 data_stream::skip(uint64 len) {
{ uint64 total_bytes_read = 0;
uint64 total_bytes_read = 0;
const uint cBufSize = 1024; const uint cBufSize = 1024;
uint8 buf[cBufSize]; uint8 buf[cBufSize];
while (len) while (len) {
{ const uint64 bytes_to_read = math::minimum<uint64>(sizeof(buf), len);
const uint64 bytes_to_read = math::minimum<uint64>(sizeof(buf), len); const uint64 bytes_read = read(buf, static_cast<uint>(bytes_to_read));
const uint64 bytes_read = read(buf, static_cast<uint>(bytes_to_read)); total_bytes_read += bytes_read;
total_bytes_read += bytes_read;
if (bytes_read != bytes_to_read) if (bytes_read != bytes_to_read)
break; break;
len -= bytes_read; len -= bytes_read;
} }
return total_bytes_read; return total_bytes_read;
} }
bool data_stream::read_line(dynamic_string& str) bool data_stream::read_line(dynamic_string& str) {
{ str.empty();
str.empty();
for ( ; ; ) for (;;) {
{ const int c = read_byte();
const int c = read_byte();
const bool prev_got_cr = m_got_cr; const bool prev_got_cr = m_got_cr;
m_got_cr = false; m_got_cr = false;
if (c < 0) if (c < 0) {
{
if (!str.is_empty())
break;
return false;
}
else if ((26 == c) || (!c))
continue;
else if (13 == c)
{
m_got_cr = true;
break;
}
else if (10 == c)
{
if (prev_got_cr)
continue;
break;
}
str.append_char(static_cast<char>(c));
}
return true;
}
bool data_stream::printf(const char* p, ...)
{
va_list args;
va_start(args, p);
dynamic_string buf;
buf.format_args(p, args);
va_end(args);
return write(buf.get_ptr(), buf.get_len() * sizeof(char)) == buf.get_len() * sizeof(char);
}
bool data_stream::write_line(const dynamic_string& str)
{
if (!str.is_empty()) if (!str.is_empty())
return write(str.get_ptr(), str.get_len()) == str.get_len(); break;
return true; return false;
} } else if ((26 == c) || (!c))
continue;
else if (13 == c) {
m_got_cr = true;
break;
} else if (10 == c) {
if (prev_got_cr)
continue;
bool data_stream::read_array(vector<uint8>& buf) break;
{ }
if (buf.size() < get_remaining())
{
if (get_remaining() > 1024U*1024U*1024U)
return false;
buf.resize((uint)get_remaining()); str.append_char(static_cast<char>(c));
} }
if (!get_remaining()) return true;
{ }
buf.resize(0);
return true;
}
return read(&buf[0], buf.size()) == buf.size(); bool data_stream::printf(const char* p, ...) {
} va_list args;
bool data_stream::write_array(const vector<uint8>& buf) va_start(args, p);
{ dynamic_string buf;
if (!buf.empty()) buf.format_args(p, args);
return write(&buf[0], buf.size()) == buf.size(); va_end(args);
return true;
}
} // namespace crnlib return write(buf.get_ptr(), buf.get_len() * sizeof(char)) == buf.get_len() * sizeof(char);
}
bool data_stream::write_line(const dynamic_string& str) {
if (!str.is_empty())
return write(str.get_ptr(), str.get_len()) == str.get_len();
return true;
}
bool data_stream::read_array(vector<uint8>& buf) {
if (buf.size() < get_remaining()) {
if (get_remaining() > 1024U * 1024U * 1024U)
return false;
buf.resize((uint)get_remaining());
}
if (!get_remaining()) {
buf.resize(0);
return true;
}
return read(&buf[0], buf.size()) == buf.size();
}
bool data_stream::write_array(const vector<uint8>& buf) {
if (!buf.empty())
return write(&buf[0], buf.size()) == buf.size();
return true;
}
} // namespace crnlib
crnlib/crn_data_stream.h
+69 -60
View File
@@ -2,88 +2,97 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
namespace crnlib namespace crnlib {
{ enum data_stream_attribs {
enum data_stream_attribs cDataStreamReadable = 1,
{ cDataStreamWritable = 2,
cDataStreamReadable = 1, cDataStreamSeekable = 4
cDataStreamWritable = 2, };
cDataStreamSeekable = 4
};
const int64 DATA_STREAM_SIZE_UNKNOWN = cINT64_MAX; const int64 DATA_STREAM_SIZE_UNKNOWN = cINT64_MAX;
const int64 DATA_STREAM_SIZE_INFINITE = cUINT64_MAX; const int64 DATA_STREAM_SIZE_INFINITE = cUINT64_MAX;
class data_stream class data_stream {
{ data_stream(const data_stream&);
data_stream(const data_stream&); data_stream& operator=(const data_stream&);
data_stream& operator= (const data_stream&);
public: public:
data_stream(); data_stream();
data_stream(const char* pName, uint attribs); data_stream(const char* pName, uint attribs);
virtual ~data_stream() { } virtual ~data_stream() {}
virtual data_stream *get_parent() { return NULL; } virtual data_stream* get_parent() { return NULL; }
virtual bool close() { m_opened = false; m_error = false; m_got_cr = false; return true; } virtual bool close() {
m_opened = false;
m_error = false;
m_got_cr = false;
return true;
}
typedef uint16 attribs_t; typedef uint16 attribs_t;
inline attribs_t get_attribs() const { return m_attribs; } inline attribs_t get_attribs() const { return m_attribs; }
inline bool is_opened() const { return m_opened; } inline bool is_opened() const { return m_opened; }
inline bool is_readable() const { return utils::is_bit_set(m_attribs, cDataStreamReadable); } inline bool is_readable() const { return utils::is_bit_set(m_attribs, cDataStreamReadable); }
inline bool is_writable() const { return utils::is_bit_set(m_attribs, cDataStreamWritable); } inline bool is_writable() const { return utils::is_bit_set(m_attribs, cDataStreamWritable); }
inline bool is_seekable() const { return utils::is_bit_set(m_attribs, cDataStreamSeekable); } inline bool is_seekable() const { return utils::is_bit_set(m_attribs, cDataStreamSeekable); }
inline bool get_error() const { return m_error; } inline bool get_error() const { return m_error; }
inline const dynamic_string& get_name() const { return m_name; } inline const dynamic_string& get_name() const { return m_name; }
inline void set_name(const char* pName) { m_name.set(pName); } inline void set_name(const char* pName) { m_name.set(pName); }
virtual uint read(void* pBuf, uint len) = 0; virtual uint read(void* pBuf, uint len) = 0;
virtual uint64 skip(uint64 len); virtual uint64 skip(uint64 len);
virtual uint write(const void* pBuf, uint len) = 0; virtual uint write(const void* pBuf, uint len) = 0;
virtual bool flush() = 0; virtual bool flush() = 0;
virtual bool is_size_known() const { return true; } virtual bool is_size_known() const { return true; }
// Returns DATA_STREAM_SIZE_UNKNOWN if size hasn't been determined yet, or DATA_STREAM_SIZE_INFINITE for infinite streams. // Returns DATA_STREAM_SIZE_UNKNOWN if size hasn't been determined yet, or DATA_STREAM_SIZE_INFINITE for infinite streams.
virtual uint64 get_size() = 0; virtual uint64 get_size() = 0;
virtual uint64 get_remaining() = 0; virtual uint64 get_remaining() = 0;
virtual uint64 get_ofs() = 0; virtual uint64 get_ofs() = 0;
virtual bool seek(int64 ofs, bool relative) = 0; virtual bool seek(int64 ofs, bool relative) = 0;
virtual const void* get_ptr() const { return NULL; } virtual const void* get_ptr() const { return NULL; }
inline int read_byte() { uint8 c; if (read(&c, 1) != 1) return -1; return c; } inline int read_byte() {
inline bool write_byte(uint8 c) { return write(&c, 1) == 1; } uint8 c;
if (read(&c, 1) != 1)
return -1;
return c;
}
inline bool write_byte(uint8 c) { return write(&c, 1) == 1; }
bool read_line(dynamic_string& str); bool read_line(dynamic_string& str);
bool printf(const char* p, ...); bool printf(const char* p, ...);
bool write_line(const dynamic_string& str); bool write_line(const dynamic_string& str);
bool write_bom() { uint16 bom = 0xFEFF; return write(&bom, sizeof(bom)) == sizeof(bom); } bool write_bom() {
uint16 bom = 0xFEFF;
return write(&bom, sizeof(bom)) == sizeof(bom);
}
bool read_array(vector<uint8>& buf); bool read_array(vector<uint8>& buf);
bool write_array(const vector<uint8>& buf); bool write_array(const vector<uint8>& buf);
protected: protected:
dynamic_string m_name; dynamic_string m_name;
attribs_t m_attribs; attribs_t m_attribs;
bool m_opened : 1; bool m_opened : 1;
bool m_error : 1; bool m_error : 1;
bool m_got_cr : 1; bool m_got_cr : 1;
inline void set_error() { m_error = true; } inline void set_error() { m_error = true; }
inline void clear_error() { m_error = false; } inline void clear_error() { m_error = false; }
inline void post_seek() { m_got_cr = false; } inline void post_seek() { m_got_cr = false; }
}; };
} // namespace crnlib
} // namespace crnlib
crnlib/crn_data_stream_serializer.h
+467 -440
View File
@@ -3,466 +3,493 @@
#pragma once #pragma once
#include "crn_data_stream.h" #include "crn_data_stream.h"
namespace crnlib namespace crnlib {
{ // Defaults to little endian mode.
// Defaults to little endian mode. class data_stream_serializer {
class data_stream_serializer public:
{ data_stream_serializer()
public: : m_pStream(NULL), m_little_endian(true) {}
data_stream_serializer() : m_pStream(NULL), m_little_endian(true) { } data_stream_serializer(data_stream* pStream)
data_stream_serializer(data_stream* pStream) : m_pStream(pStream), m_little_endian(true) { } : m_pStream(pStream), m_little_endian(true) {}
data_stream_serializer(data_stream& stream) : m_pStream(&stream), m_little_endian(true) { } data_stream_serializer(data_stream& stream)
data_stream_serializer(const data_stream_serializer& other) : m_pStream(other.m_pStream), m_little_endian(other.m_little_endian) { } : m_pStream(&stream), m_little_endian(true) {}
data_stream_serializer(const data_stream_serializer& other)
data_stream_serializer& operator= (const data_stream_serializer& rhs) { m_pStream = rhs.m_pStream; m_little_endian = rhs.m_little_endian; return *this; } : m_pStream(other.m_pStream), m_little_endian(other.m_little_endian) {}
data_stream* get_stream() const { return m_pStream; } data_stream_serializer& operator=(const data_stream_serializer& rhs) {
void set_stream(data_stream* pStream) { m_pStream = pStream; } m_pStream = rhs.m_pStream;
m_little_endian = rhs.m_little_endian;
return *this;
}
const dynamic_string& get_name() const { return m_pStream ? m_pStream->get_name() : g_empty_dynamic_string; } data_stream* get_stream() const { return m_pStream; }
void set_stream(data_stream* pStream) { m_pStream = pStream; }
bool get_error() { return m_pStream ? m_pStream->get_error() : false; } const dynamic_string& get_name() const { return m_pStream ? m_pStream->get_name() : g_empty_dynamic_string; }
bool get_little_endian() const { return m_little_endian; }
void set_little_endian(bool little_endian) { m_little_endian = little_endian; }
bool write(const void* pBuf, uint len)
{
return m_pStream->write(pBuf, len) == len;
}
bool read(void* pBuf, uint len)
{
return m_pStream->read(pBuf, len) == len;
}
// size = size of each element, count = number of elements, returns actual count of elements written
uint write(const void* pBuf, uint size, uint count)
{
uint actual_size = size * count;
if (!actual_size)
return 0;
uint n = m_pStream->write(pBuf, actual_size);
if (n == actual_size)
return count;
return n / size;
}
// size = size of each element, count = number of elements, returns actual count of elements read bool get_error() { return m_pStream ? m_pStream->get_error() : false; }
uint read(void* pBuf, uint size, uint count)
{
uint actual_size = size * count;
if (!actual_size)
return 0;
uint n = m_pStream->read(pBuf, actual_size);
if (n == actual_size)
return count;
return n / size;
}
bool write_chars(const char* pBuf, uint len)
{
return write(pBuf, len);
}
bool read_chars(char* pBuf, uint len)
{
return read(pBuf, len);
}
bool skip(uint len)
{
return m_pStream->skip(len) == len;
}
template<typename T>
bool write_object(const T& obj)
{
if (m_little_endian == c_crnlib_little_endian_platform)
return write(&obj, sizeof(obj));
else
{
uint8 buf[sizeof(T)];
uint buf_size = sizeof(T);
void* pBuf = buf;
utils::write_obj(obj, pBuf, buf_size, m_little_endian);
return write(buf, sizeof(T));
}
}
template<typename T>
bool read_object(T& obj)
{
if (m_little_endian == c_crnlib_little_endian_platform)
return read(&obj, sizeof(obj));
else
{
uint8 buf[sizeof(T)];
if (!read(buf, sizeof(T)))
return false;
uint buf_size = sizeof(T);
const void* pBuf = buf;
utils::read_obj(obj, pBuf, buf_size, m_little_endian);
return true;
}
}
template<typename T>
bool write_value(T value)
{
return write_object(value);
}
template<typename T>
T read_value(const T& on_error_value = T())
{
T result;
if (!read_object(result))
result = on_error_value;
return result;
}
template<typename T>
bool write_enum(T e)
{
int val = static_cast<int>(e);
return write_object(val);
}
template<typename T> bool get_little_endian() const { return m_little_endian; }
T read_enum() void set_little_endian(bool little_endian) { m_little_endian = little_endian; }
{
return static_cast<T>(read_value<int>()); bool write(const void* pBuf, uint len) {
return m_pStream->write(pBuf, len) == len;
}
bool read(void* pBuf, uint len) {
return m_pStream->read(pBuf, len) == len;
}
// size = size of each element, count = number of elements, returns actual count of elements written
uint write(const void* pBuf, uint size, uint count) {
uint actual_size = size * count;
if (!actual_size)
return 0;
uint n = m_pStream->write(pBuf, actual_size);
if (n == actual_size)
return count;
return n / size;
}
// size = size of each element, count = number of elements, returns actual count of elements read
uint read(void* pBuf, uint size, uint count) {
uint actual_size = size * count;
if (!actual_size)
return 0;
uint n = m_pStream->read(pBuf, actual_size);
if (n == actual_size)
return count;
return n / size;
}
bool write_chars(const char* pBuf, uint len) {
return write(pBuf, len);
}
bool read_chars(char* pBuf, uint len) {
return read(pBuf, len);
}
bool skip(uint len) {
return m_pStream->skip(len) == len;
}
template <typename T>
bool write_object(const T& obj) {
if (m_little_endian == c_crnlib_little_endian_platform)
return write(&obj, sizeof(obj));
else {
uint8 buf[sizeof(T)];
uint buf_size = sizeof(T);
void* pBuf = buf;
utils::write_obj(obj, pBuf, buf_size, m_little_endian);
return write(buf, sizeof(T));
}
}
template <typename T>
bool read_object(T& obj) {
if (m_little_endian == c_crnlib_little_endian_platform)
return read(&obj, sizeof(obj));
else {
uint8 buf[sizeof(T)];
if (!read(buf, sizeof(T)))
return false;
uint buf_size = sizeof(T);
const void* pBuf = buf;
utils::read_obj(obj, pBuf, buf_size, m_little_endian);
return true;
}
}
template <typename T>
bool write_value(T value) {
return write_object(value);
}
template <typename T>
T read_value(const T& on_error_value = T()) {
T result;
if (!read_object(result))
result = on_error_value;
return result;
}
template <typename T>
bool write_enum(T e) {
int val = static_cast<int>(e);
return write_object(val);
}
template <typename T>
T read_enum() {
return static_cast<T>(read_value<int>());
}
// Writes uint using a simple variable length code (VLC).
bool write_uint_vlc(uint val) {
do {
uint8 c = static_cast<uint8>(val) & 0x7F;
if (val <= 0x7F)
c |= 0x80;
if (!write_value(c))
return false;
val >>= 7;
} while (val);
return true;
}
// Reads uint using a simple variable length code (VLC).
bool read_uint_vlc(uint& val) {
val = 0;
uint shift = 0;
for (;;) {
if (shift >= 32)
return false;
uint8 c;
if (!read_object(c))
return false;
val |= ((c & 0x7F) << shift);
shift += 7;
if (c & 0x80)
break;
}
return true;
}
bool write_c_str(const char* p) {
uint len = static_cast<uint>(strlen(p));
if (!write_uint_vlc(len))
return false;
return write_chars(p, len);
}
bool read_c_str(char* pBuf, uint buf_size) {
uint len;
if (!read_uint_vlc(len))
return false;
if ((len + 1) > buf_size)
return false;
pBuf[len] = '\0';
return read_chars(pBuf, len);
}
bool write_string(const dynamic_string& str) {
if (!write_uint_vlc(str.get_len()))
return false;
return write_chars(str.get_ptr(), str.get_len());
}
bool read_string(dynamic_string& str) {
uint len;
if (!read_uint_vlc(len))
return false;
if (!str.set_len(len))
return false;
if (len) {
if (!read_chars(str.get_ptr_raw(), len))
return false;
if (memchr(str.get_ptr(), 0, len) != NULL) {
str.truncate(0);
return false;
} }
}
// Writes uint using a simple variable length code (VLC).
bool write_uint_vlc(uint val) return true;
{ }
do
{ template <typename T>
uint8 c = static_cast<uint8>(val) & 0x7F; bool write_vector(const T& vec) {
if (val <= 0x7F) if (!write_uint_vlc(vec.size()))
c |= 0x80; return false;
if (!write_value(c)) for (uint i = 0; i < vec.size(); i++) {
return false; *this << vec[i];
if (get_error())
val >>= 7; return false;
} while (val); }
return true; return true;
} };
// Reads uint using a simple variable length code (VLC). template <typename T>
bool read_uint_vlc(uint& val) bool read_vector(T& vec, uint num_expected = UINT_MAX) {
{ uint size;
val = 0; if (!read_uint_vlc(size))
uint shift = 0; return false;
for ( ; ; ) if ((size * sizeof(T::value_type)) >= 2U * 1024U * 1024U * 1024U)
{ return false;
if (shift >= 32)
return false; if ((num_expected != UINT_MAX) && (size != num_expected))
return false;
uint8 c;
if (!read_object(c)) vec.resize(size);
return false; for (uint i = 0; i < vec.size(); i++) {
*this >> vec[i];
val |= ((c & 0x7F) << shift);
shift += 7; if (get_error())
return false;
if (c & 0x80) }
break;
} return true;
}
return true;
} bool read_entire_file(crnlib::vector<uint8>& buf) {
return m_pStream->read_array(buf);
bool write_c_str(const char* p) }
{
uint len = static_cast<uint>(strlen(p)); bool write_entire_file(const crnlib::vector<uint8>& buf) {
if (!write_uint_vlc(len)) return m_pStream->write_array(buf);
}
// Got this idea from the Molly Rocket forums.
// fmt may contain the characters "1", "2", or "4".
bool writef(char* fmt, ...) {
va_list v;
va_start(v, fmt);
while (*fmt) {
switch (*fmt++) {
case '1': {
const uint8 x = static_cast<uint8>(va_arg(v, uint));
if (!write_value(x))
return false; return false;
}
return write_chars(p, len); case '2': {
const uint16 x = static_cast<uint16>(va_arg(v, uint));
if (!write_value(x))
return false;
}
case '4': {
const uint32 x = static_cast<uint32>(va_arg(v, uint));
if (!write_value(x))
return false;
}
case ' ':
case ',': {
break;
}
default: {
CRNLIB_ASSERT(0);
return false;
}
} }
}
bool read_c_str(char* pBuf, uint buf_size)
{ va_end(v);
uint len; return true;
if (!read_uint_vlc(len)) }
// Got this idea from the Molly Rocket forums.
// fmt may contain the characters "1", "2", or "4".
bool readf(char* fmt, ...) {
va_list v;
va_start(v, fmt);
while (*fmt) {
switch (*fmt++) {
case '1': {
uint8* x = va_arg(v, uint8*);
CRNLIB_ASSERT(x);
if (!read_object(*x))
return false; return false;
if ((len + 1) > buf_size) }
case '2': {
uint16* x = va_arg(v, uint16*);
CRNLIB_ASSERT(x);
if (!read_object(*x))
return false; return false;
}
pBuf[len] = '\0'; case '4': {
uint32* x = va_arg(v, uint32*);
return read_chars(pBuf, len); CRNLIB_ASSERT(x);
if (!read_object(*x))
return false;
}
case ' ':
case ',': {
break;
}
default: {
CRNLIB_ASSERT(0);
return false;
}
} }
}
bool write_string(const dynamic_string& str)
{
if (!write_uint_vlc(str.get_len()))
return false;
return write_chars(str.get_ptr(), str.get_len()); va_end(v);
} return true;
}
bool read_string(dynamic_string& str)
{
uint len;
if (!read_uint_vlc(len))
return false;
if (!str.set_len(len))
return false;
if (len)
{
if (!read_chars(str.get_ptr_raw(), len))
return false;
if (memchr(str.get_ptr(), 0, len) != NULL)
{
str.truncate(0);
return false;
}
}
return true;
}
template<typename T>
bool write_vector(const T& vec)
{
if (!write_uint_vlc(vec.size()))
return false;
for (uint i = 0; i < vec.size(); i++)
{
*this << vec[i];
if (get_error())
return false;
}
return true;
};
template<typename T>
bool read_vector(T& vec, uint num_expected = UINT_MAX)
{
uint size;
if (!read_uint_vlc(size))
return false;
if ((size * sizeof(T::value_type)) >= 2U*1024U*1024U*1024U)
return false;
if ((num_expected != UINT_MAX) && (size != num_expected))
return false;
vec.resize(size);
for (uint i = 0; i < vec.size(); i++)
{
*this >> vec[i];
if (get_error())
return false;
}
return true;
}
bool read_entire_file(crnlib::vector<uint8>& buf) private:
{ data_stream* m_pStream;
return m_pStream->read_array(buf);
}
bool write_entire_file(const crnlib::vector<uint8>& buf) bool m_little_endian;
{ };
return m_pStream->write_array(buf);
}
// Got this idea from the Molly Rocket forums.
// fmt may contain the characters "1", "2", or "4".
bool writef(char *fmt, ...)
{
va_list v;
va_start(v, fmt);
while (*fmt)
{
switch (*fmt++)
{
case '1':
{
const uint8 x = static_cast<uint8>(va_arg(v, uint));
if (!write_value(x))
return false;
}
case '2':
{
const uint16 x = static_cast<uint16>(va_arg(v, uint));
if (!write_value(x))
return false;
}
case '4':
{
const uint32 x = static_cast<uint32>(va_arg(v, uint));
if (!write_value(x))
return false;
}
case ' ':
case ',':
{
break;
}
default:
{
CRNLIB_ASSERT(0);
return false;
}
}
}
va_end(v);
return true;
}
// Got this idea from the Molly Rocket forums.
// fmt may contain the characters "1", "2", or "4".
bool readf(char *fmt, ...)
{
va_list v;
va_start(v, fmt);
while (*fmt) // Write operators
{ inline data_stream_serializer& operator<<(data_stream_serializer& serializer, bool val) {
switch (*fmt++) serializer.write_value(val);
{ return serializer;
case '1': }
{ inline data_stream_serializer& operator<<(data_stream_serializer& serializer, int8 val) {
uint8* x = va_arg(v, uint8*); serializer.write_value(val);
CRNLIB_ASSERT(x); return serializer;
if (!read_object(*x)) }
return false; inline data_stream_serializer& operator<<(data_stream_serializer& serializer, uint8 val) {
} serializer.write_value(val);
case '2': return serializer;
{ }
uint16* x = va_arg(v, uint16*); inline data_stream_serializer& operator<<(data_stream_serializer& serializer, int16 val) {
CRNLIB_ASSERT(x); serializer.write_value(val);
if (!read_object(*x)) return serializer;
return false; }
} inline data_stream_serializer& operator<<(data_stream_serializer& serializer, uint16 val) {
case '4': serializer.write_value(val);
{ return serializer;
uint32* x = va_arg(v, uint32*); }
CRNLIB_ASSERT(x); inline data_stream_serializer& operator<<(data_stream_serializer& serializer, int32 val) {
if (!read_object(*x)) serializer.write_value(val);
return false; return serializer;
} }
case ' ': inline data_stream_serializer& operator<<(data_stream_serializer& serializer, uint32 val) {
case ',': serializer.write_uint_vlc(val);
{ return serializer;
break; }
} inline data_stream_serializer& operator<<(data_stream_serializer& serializer, int64 val) {
default: serializer.write_value(val);
{ return serializer;
CRNLIB_ASSERT(0); }
return false; inline data_stream_serializer& operator<<(data_stream_serializer& serializer, uint64 val) {
} serializer.write_value(val);
} return serializer;
} }
inline data_stream_serializer& operator<<(data_stream_serializer& serializer, long val) {
serializer.write_value(val);
return serializer;
}
inline data_stream_serializer& operator<<(data_stream_serializer& serializer, unsigned long val) {
serializer.write_value(val);
return serializer;
}
inline data_stream_serializer& operator<<(data_stream_serializer& serializer, float val) {
serializer.write_value(val);
return serializer;
}
inline data_stream_serializer& operator<<(data_stream_serializer& serializer, double val) {
serializer.write_value(val);
return serializer;
}
inline data_stream_serializer& operator<<(data_stream_serializer& serializer, const char* p) {
serializer.write_c_str(p);
return serializer;
}
va_end(v); inline data_stream_serializer& operator<<(data_stream_serializer& serializer, const dynamic_string& str) {
return true; serializer.write_string(str);
} return serializer;
}
private:
data_stream* m_pStream;
bool m_little_endian;
};
// Write operators
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, bool val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, int8 val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, uint8 val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, int16 val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, uint16 val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, int32 val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, uint32 val) { serializer.write_uint_vlc(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, int64 val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, uint64 val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, long val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, unsigned long val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, float val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, double val) { serializer.write_value(val); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, const char* p) { serializer.write_c_str(p); return serializer; }
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, const dynamic_string& str)
{
serializer.write_string(str);
return serializer;
}
template<typename T>
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, const crnlib::vector<T>& vec)
{
serializer.write_vector(vec);
return serializer;
}
template<typename T>
inline data_stream_serializer& operator<< (data_stream_serializer& serializer, const T* p)
{
serializer.write_object(*p);
return serializer;
}
// Read operators
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, bool& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, int8& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, uint8& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, int16& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, uint16& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, int32& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, uint32& val) { serializer.read_uint_vlc(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, int64& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, uint64& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, long& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, unsigned long& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, float& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, double& val) { serializer.read_object(val); return serializer; }
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, dynamic_string& str)
{
serializer.read_string(str);
return serializer;
}
template<typename T>
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, crnlib::vector<T>& vec)
{
serializer.read_vector(vec);
return serializer;
}
template<typename T>
inline data_stream_serializer& operator>> (data_stream_serializer& serializer, T* p)
{
serializer.read_object(*p);
return serializer;
}
} // namespace crnlib
template <typename T>
inline data_stream_serializer& operator<<(data_stream_serializer& serializer, const crnlib::vector<T>& vec) {
serializer.write_vector(vec);
return serializer;
}
template <typename T>
inline data_stream_serializer& operator<<(data_stream_serializer& serializer, const T* p) {
serializer.write_object(*p);
return serializer;
}
// Read operators
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, bool& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, int8& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, uint8& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, int16& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, uint16& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, int32& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, uint32& val) {
serializer.read_uint_vlc(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, int64& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, uint64& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, long& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, unsigned long& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, float& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, double& val) {
serializer.read_object(val);
return serializer;
}
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, dynamic_string& str) {
serializer.read_string(str);
return serializer;
}
template <typename T>
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, crnlib::vector<T>& vec) {
serializer.read_vector(vec);
return serializer;
}
template <typename T>
inline data_stream_serializer& operator>>(data_stream_serializer& serializer, T* p) {
serializer.read_object(*p);
return serializer;
}
} // namespace crnlib
crnlib/crn_dds_comp.cpp
+196 -228
View File
@@ -5,255 +5,223 @@
#include "crn_dynamic_stream.h" #include "crn_dynamic_stream.h"
#include "crn_lzma_codec.h" #include "crn_lzma_codec.h"
namespace crnlib namespace crnlib {
{ dds_comp::dds_comp()
dds_comp::dds_comp() : : m_pParams(NULL),
m_pParams(NULL),
m_pixel_fmt(PIXEL_FMT_INVALID), m_pixel_fmt(PIXEL_FMT_INVALID),
m_pQDXT_state(NULL) m_pQDXT_state(NULL) {
{ }
}
dds_comp::~dds_comp() dds_comp::~dds_comp() {
{ crnlib_delete(m_pQDXT_state);
crnlib_delete(m_pQDXT_state); }
}
void dds_comp::clear() void dds_comp::clear() {
{ m_src_tex.clear();
m_src_tex.clear(); m_packed_tex.clear();
m_packed_tex.clear(); m_comp_data.clear();
m_comp_data.clear(); m_pParams = NULL;
m_pParams = NULL; m_pixel_fmt = PIXEL_FMT_INVALID;
m_pixel_fmt = PIXEL_FMT_INVALID; m_task_pool.deinit();
m_task_pool.deinit(); if (m_pQDXT_state) {
if (m_pQDXT_state) crnlib_delete(m_pQDXT_state);
{ m_pQDXT_state = NULL;
crnlib_delete(m_pQDXT_state); }
m_pQDXT_state = NULL; }
bool dds_comp::create_dds_tex(mipmapped_texture& dds_tex) {
image_u8 images[cCRNMaxFaces][cCRNMaxLevels];
bool has_alpha = false;
for (uint face_index = 0; face_index < m_pParams->m_faces; face_index++) {
for (uint level_index = 0; level_index < m_pParams->m_levels; level_index++) {
const uint width = math::maximum(1U, m_pParams->m_width >> level_index);
const uint height = math::maximum(1U, m_pParams->m_height >> level_index);
if (!m_pParams->m_pImages[face_index][level_index])
return false;
images[face_index][level_index].alias((color_quad_u8*)m_pParams->m_pImages[face_index][level_index], width, height);
if (!has_alpha)
has_alpha = image_utils::has_alpha(images[face_index][level_index]);
}
}
for (uint face_index = 0; face_index < m_pParams->m_faces; face_index++)
for (uint level_index = 0; level_index < m_pParams->m_levels; level_index++)
images[face_index][level_index].set_component_valid(3, has_alpha);
image_utils::conversion_type conv_type = image_utils::get_image_conversion_type_from_crn_format((crn_format)m_pParams->m_format);
if (conv_type != image_utils::cConversion_Invalid) {
for (uint face_index = 0; face_index < m_pParams->m_faces; face_index++) {
for (uint level_index = 0; level_index < m_pParams->m_levels; level_index++) {
image_u8 cooked_image(images[face_index][level_index]);
image_utils::convert_image(cooked_image, conv_type);
images[face_index][level_index].swap(cooked_image);
} }
} }
}
bool dds_comp::create_dds_tex(mipmapped_texture &dds_tex)
{
image_u8 images[cCRNMaxFaces][cCRNMaxLevels];
bool has_alpha = false; face_vec faces(m_pParams->m_faces);
for (uint face_index = 0; face_index < m_pParams->m_faces; face_index++)
{
for (uint level_index = 0; level_index < m_pParams->m_levels; level_index++)
{
const uint width = math::maximum(1U, m_pParams->m_width >> level_index);
const uint height = math::maximum(1U, m_pParams->m_height >> level_index);
if (!m_pParams->m_pImages[face_index][level_index]) for (uint face_index = 0; face_index < m_pParams->m_faces; face_index++) {
return false; for (uint level_index = 0; level_index < m_pParams->m_levels; level_index++) {
mip_level* pMip = crnlib_new<mip_level>();
images[face_index][level_index].alias((color_quad_u8*)m_pParams->m_pImages[face_index][level_index], width, height); image_u8* pImage = crnlib_new<image_u8>();
if (!has_alpha) pImage->swap(images[face_index][level_index]);
has_alpha = image_utils::has_alpha(images[face_index][level_index]); pMip->assign(pImage);
}
}
for (uint face_index = 0; face_index < m_pParams->m_faces; face_index++)
for (uint level_index = 0; level_index < m_pParams->m_levels; level_index++)
images[face_index][level_index].set_component_valid(3, has_alpha);
image_utils::conversion_type conv_type = image_utils::get_image_conversion_type_from_crn_format((crn_format)m_pParams->m_format); faces[face_index].push_back(pMip);
if (conv_type != image_utils::cConversion_Invalid) }
{ }
for (uint face_index = 0; face_index < m_pParams->m_faces; face_index++)
{
for (uint level_index = 0; level_index < m_pParams->m_levels; level_index++)
{
image_u8 cooked_image(images[face_index][level_index]);
image_utils::convert_image(cooked_image, conv_type); dds_tex.assign(faces);
images[face_index][level_index].swap(cooked_image);
}
}
}
face_vec faces(m_pParams->m_faces);
for (uint face_index = 0; face_index < m_pParams->m_faces; face_index++)
{
for (uint level_index = 0; level_index < m_pParams->m_levels; level_index++)
{
mip_level *pMip = crnlib_new<mip_level>();
image_u8 *pImage = crnlib_new<image_u8>();
pImage->swap(images[face_index][level_index]);
pMip->assign(pImage);
faces[face_index].push_back(pMip);
}
}
dds_tex.assign(faces);
#ifdef CRNLIB_BUILD_DEBUG #ifdef CRNLIB_BUILD_DEBUG
CRNLIB_ASSERT(dds_tex.check()); CRNLIB_ASSERT(dds_tex.check());
#endif #endif
return true; return true;
} }
static bool progress_callback_func(uint percentage_complete, void* pUser_data_ptr) static bool progress_callback_func(uint percentage_complete, void* pUser_data_ptr) {
{ const crn_comp_params& params = *(const crn_comp_params*)pUser_data_ptr;
const crn_comp_params& params = *(const crn_comp_params*)pUser_data_ptr; return params.m_pProgress_func(0, 1, percentage_complete, 100, params.m_pProgress_func_data) != 0;
return params.m_pProgress_func(0, 1, percentage_complete, 100, params.m_pProgress_func_data) != 0; }
}
static bool progress_callback_func_phase_0(uint percentage_complete, void* pUser_data_ptr) static bool progress_callback_func_phase_0(uint percentage_complete, void* pUser_data_ptr) {
{ const crn_comp_params& params = *(const crn_comp_params*)pUser_data_ptr;
const crn_comp_params& params = *(const crn_comp_params*)pUser_data_ptr; return params.m_pProgress_func(0, 2, percentage_complete, 100, params.m_pProgress_func_data) != 0;
return params.m_pProgress_func(0, 2, percentage_complete, 100, params.m_pProgress_func_data) != 0; }
}
static bool progress_callback_func_phase_1(uint percentage_complete, void* pUser_data_ptr) static bool progress_callback_func_phase_1(uint percentage_complete, void* pUser_data_ptr) {
{ const crn_comp_params& params = *(const crn_comp_params*)pUser_data_ptr;
const crn_comp_params& params = *(const crn_comp_params*)pUser_data_ptr; return params.m_pProgress_func(1, 2, percentage_complete, 100, params.m_pProgress_func_data) != 0;
return params.m_pProgress_func(1, 2, percentage_complete, 100, params.m_pProgress_func_data) != 0; }
}
bool dds_comp::convert_to_dxt(const crn_comp_params& params) bool dds_comp::convert_to_dxt(const crn_comp_params& params) {
{ if ((params.m_quality_level == cCRNMaxQualityLevel) || (params.m_format == cCRNFmtDXT3)) {
if ((params.m_quality_level == cCRNMaxQualityLevel) || (params.m_format == cCRNFmtDXT3)) m_packed_tex = m_src_tex;
{ if (!m_packed_tex.convert(m_pixel_fmt, false, m_pack_params))
m_packed_tex = m_src_tex; return false;
if (!m_packed_tex.convert(m_pixel_fmt, false, m_pack_params)) } else {
return false; const bool hierarchical = (params.m_flags & cCRNCompFlagHierarchical) != 0;
}
else
{
const bool hierarchical = (params.m_flags & cCRNCompFlagHierarchical) != 0;
m_q1_params.m_quality_level = params.m_quality_level;
m_q1_params.m_hierarchical = hierarchical;
m_q5_params.m_quality_level = params.m_quality_level; m_q1_params.m_quality_level = params.m_quality_level;
m_q5_params.m_hierarchical = hierarchical; m_q1_params.m_hierarchical = hierarchical;
if (!m_pQDXT_state) m_q5_params.m_quality_level = params.m_quality_level;
{ m_q5_params.m_hierarchical = hierarchical;
m_pQDXT_state = crnlib_new<mipmapped_texture::qdxt_state>(m_task_pool);
if (params.m_pProgress_func)
{
m_q1_params.m_pProgress_func = progress_callback_func_phase_0;
m_q1_params.m_pProgress_data = (void*)&params;
m_q5_params.m_pProgress_func = progress_callback_func_phase_0;
m_q5_params.m_pProgress_data = (void*)&params;
}
if (!m_src_tex.qdxt_pack_init(*m_pQDXT_state, m_packed_tex, m_q1_params, m_q5_params, m_pixel_fmt, false)) if (!m_pQDXT_state) {
return false; m_pQDXT_state = crnlib_new<mipmapped_texture::qdxt_state>(m_task_pool);
if (params.m_pProgress_func)
{
m_q1_params.m_pProgress_func = progress_callback_func_phase_1;
m_q5_params.m_pProgress_func = progress_callback_func_phase_1;
}
}
else
{
if (params.m_pProgress_func)
{
m_q1_params.m_pProgress_func = progress_callback_func;
m_q1_params.m_pProgress_data = (void*)&params;
m_q5_params.m_pProgress_func = progress_callback_func;
m_q5_params.m_pProgress_data = (void*)&params;
}
}
if (!m_src_tex.qdxt_pack(*m_pQDXT_state, m_packed_tex, m_q1_params, m_q5_params)) if (params.m_pProgress_func) {
return false; m_q1_params.m_pProgress_func = progress_callback_func_phase_0;
} m_q1_params.m_pProgress_data = (void*)&params;
m_q5_params.m_pProgress_func = progress_callback_func_phase_0;
return true; m_q5_params.m_pProgress_data = (void*)&params;
}
bool dds_comp::compress_init(const crn_comp_params& params)
{
clear();
m_pParams = &params;
if ((math::minimum(m_pParams->m_width, m_pParams->m_height) < 1) || (math::maximum(m_pParams->m_width, m_pParams->m_height) > cCRNMaxLevelResolution))
return false;
if (math::minimum(m_pParams->m_faces, m_pParams->m_levels) < 1)
return false;
if (!create_dds_tex(m_src_tex))
return false;
m_pack_params.init(*m_pParams);
if (params.m_pProgress_func)
{
m_pack_params.m_pProgress_callback = progress_callback_func;
m_pack_params.m_pProgress_callback_user_data_ptr = (void*)&params;
}
m_pixel_fmt = pixel_format_helpers::convert_crn_format_to_pixel_format(static_cast<crn_format>(m_pParams->m_format));
if (m_pixel_fmt == PIXEL_FMT_INVALID)
return false;
if ((m_pixel_fmt == PIXEL_FMT_DXT1) && (m_src_tex.has_alpha()) && (m_pack_params.m_use_both_block_types) && (m_pParams->m_flags & cCRNCompFlagDXT1AForTransparency))
m_pixel_fmt = PIXEL_FMT_DXT1A;
if (!m_task_pool.init(m_pParams->m_num_helper_threads))
return false;
m_pack_params.m_pTask_pool = &m_task_pool;
const bool hierarchical = (params.m_flags & cCRNCompFlagHierarchical) != 0;
m_q1_params.init(m_pack_params, params.m_quality_level, hierarchical);
m_q5_params.init(m_pack_params, params.m_quality_level, hierarchical);
return true;
}
bool dds_comp::compress_pass(const crn_comp_params& params, float *pEffective_bitrate)
{
if (pEffective_bitrate) *pEffective_bitrate = 0.0f;
if (!m_pParams)
return false;
if (!convert_to_dxt(params))
return false;
dynamic_stream out_stream;
out_stream.reserve(512*1024);
data_stream_serializer serializer(out_stream);
if (!m_packed_tex.write_dds(serializer))
return false;
out_stream.reserve(0);
m_comp_data.swap(out_stream.get_buf());
if (pEffective_bitrate)
{
lzma_codec lossless_codec;
crnlib::vector<uint8> cmp_tex_bytes;
if (lossless_codec.pack(m_comp_data.get_ptr(), m_comp_data.size(), cmp_tex_bytes))
{
uint comp_size = cmp_tex_bytes.size();
if (comp_size)
{
*pEffective_bitrate = (comp_size * 8.0f) / m_src_tex.get_total_pixels_in_all_faces_and_mips();
}
}
} }
return true; if (!m_src_tex.qdxt_pack_init(*m_pQDXT_state, m_packed_tex, m_q1_params, m_q5_params, m_pixel_fmt, false))
} return false;
void dds_comp::compress_deinit() if (params.m_pProgress_func) {
{ m_q1_params.m_pProgress_func = progress_callback_func_phase_1;
clear(); m_q5_params.m_pProgress_func = progress_callback_func_phase_1;
} }
} else {
if (params.m_pProgress_func) {
m_q1_params.m_pProgress_func = progress_callback_func;
m_q1_params.m_pProgress_data = (void*)&params;
m_q5_params.m_pProgress_func = progress_callback_func;
m_q5_params.m_pProgress_data = (void*)&params;
}
}
} // namespace crnlib if (!m_src_tex.qdxt_pack(*m_pQDXT_state, m_packed_tex, m_q1_params, m_q5_params))
return false;
}
return true;
}
bool dds_comp::compress_init(const crn_comp_params& params) {
clear();
m_pParams = &params;
if ((math::minimum(m_pParams->m_width, m_pParams->m_height) < 1) || (math::maximum(m_pParams->m_width, m_pParams->m_height) > cCRNMaxLevelResolution))
return false;
if (math::minimum(m_pParams->m_faces, m_pParams->m_levels) < 1)
return false;
if (!create_dds_tex(m_src_tex))
return false;
m_pack_params.init(*m_pParams);
if (params.m_pProgress_func) {
m_pack_params.m_pProgress_callback = progress_callback_func;
m_pack_params.m_pProgress_callback_user_data_ptr = (void*)&params;
}
m_pixel_fmt = pixel_format_helpers::convert_crn_format_to_pixel_format(static_cast<crn_format>(m_pParams->m_format));
if (m_pixel_fmt == PIXEL_FMT_INVALID)
return false;
if ((m_pixel_fmt == PIXEL_FMT_DXT1) && (m_src_tex.has_alpha()) && (m_pack_params.m_use_both_block_types) && (m_pParams->m_flags & cCRNCompFlagDXT1AForTransparency))
m_pixel_fmt = PIXEL_FMT_DXT1A;
if (!m_task_pool.init(m_pParams->m_num_helper_threads))
return false;
m_pack_params.m_pTask_pool = &m_task_pool;
const bool hierarchical = (params.m_flags & cCRNCompFlagHierarchical) != 0;
m_q1_params.init(m_pack_params, params.m_quality_level, hierarchical);
m_q5_params.init(m_pack_params, params.m_quality_level, hierarchical);
return true;
}
bool dds_comp::compress_pass(const crn_comp_params& params, float* pEffective_bitrate) {
if (pEffective_bitrate)
*pEffective_bitrate = 0.0f;
if (!m_pParams)
return false;
if (!convert_to_dxt(params))
return false;
dynamic_stream out_stream;
out_stream.reserve(512 * 1024);
data_stream_serializer serializer(out_stream);
if (!m_packed_tex.write_dds(serializer))
return false;
out_stream.reserve(0);
m_comp_data.swap(out_stream.get_buf());
if (pEffective_bitrate) {
lzma_codec lossless_codec;
crnlib::vector<uint8> cmp_tex_bytes;
if (lossless_codec.pack(m_comp_data.get_ptr(), m_comp_data.size(), cmp_tex_bytes)) {
uint comp_size = cmp_tex_bytes.size();
if (comp_size) {
*pEffective_bitrate = (comp_size * 8.0f) / m_src_tex.get_total_pixels_in_all_faces_and_mips();
}
}
}
return true;
}
void dds_comp::compress_deinit() {
clear();
}
} // namespace crnlib
crnlib/crn_dds_comp.h
+28 -30
View File
@@ -5,44 +5,42 @@
#include "crn_mipmapped_texture.h" #include "crn_mipmapped_texture.h"
#include "crn_texture_comp.h" #include "crn_texture_comp.h"
namespace crnlib namespace crnlib {
{ class dds_comp : public itexture_comp {
class dds_comp : public itexture_comp CRNLIB_NO_COPY_OR_ASSIGNMENT_OP(dds_comp);
{
CRNLIB_NO_COPY_OR_ASSIGNMENT_OP(dds_comp);
public: public:
dds_comp(); dds_comp();
virtual ~dds_comp(); virtual ~dds_comp();
virtual const char *get_ext() const { return "DDS"; } virtual const char* get_ext() const { return "DDS"; }
virtual bool compress_init(const crn_comp_params& params); virtual bool compress_init(const crn_comp_params& params);
virtual bool compress_pass(const crn_comp_params& params, float *pEffective_bitrate); virtual bool compress_pass(const crn_comp_params& params, float* pEffective_bitrate);
virtual void compress_deinit(); virtual void compress_deinit();
virtual const crnlib::vector<uint8>& get_comp_data() const { return m_comp_data; } virtual const crnlib::vector<uint8>& get_comp_data() const { return m_comp_data; }
virtual crnlib::vector<uint8>& get_comp_data() { return m_comp_data; } virtual crnlib::vector<uint8>& get_comp_data() { return m_comp_data; }
private: private:
mipmapped_texture m_src_tex; mipmapped_texture m_src_tex;
mipmapped_texture m_packed_tex; mipmapped_texture m_packed_tex;
crnlib::vector<uint8> m_comp_data; crnlib::vector<uint8> m_comp_data;
const crn_comp_params* m_pParams; const crn_comp_params* m_pParams;
pixel_format m_pixel_fmt; pixel_format m_pixel_fmt;
dxt_image::pack_params m_pack_params; dxt_image::pack_params m_pack_params;
task_pool m_task_pool; task_pool m_task_pool;
qdxt1_params m_q1_params; qdxt1_params m_q1_params;
qdxt5_params m_q5_params; qdxt5_params m_q5_params;
mipmapped_texture::qdxt_state *m_pQDXT_state; mipmapped_texture::qdxt_state* m_pQDXT_state;
void clear(); void clear();
bool create_dds_tex(mipmapped_texture &dds_tex); bool create_dds_tex(mipmapped_texture& dds_tex);
bool convert_to_dxt(const crn_comp_params& params); bool convert_to_dxt(const crn_comp_params& params);
}; };
} // namespace crnlib } // namespace crnlib
crnlib/crn_dxt.cpp
+350 -359
View File
@@ -7,380 +7,371 @@
#include "crn_dxt_fast.h" #include "crn_dxt_fast.h"
#include "crn_intersect.h" #include "crn_intersect.h"
namespace crnlib namespace crnlib {
{ const uint8 g_dxt5_from_linear[cDXT5SelectorValues] = {0U, 2U, 3U, 4U, 5U, 6U, 7U, 1U};
const uint8 g_dxt5_from_linear[cDXT5SelectorValues] = { 0U, 2U, 3U, 4U, 5U, 6U, 7U, 1U }; const uint8 g_dxt5_to_linear[cDXT5SelectorValues] = {0U, 7U, 1U, 2U, 3U, 4U, 5U, 6U};
const uint8 g_dxt5_to_linear[cDXT5SelectorValues] = { 0U, 7U, 1U, 2U, 3U, 4U, 5U, 6U };
const uint8 g_dxt5_alpha6_to_linear[cDXT5SelectorValues] = { 0U, 5U, 1U, 2U, 3U, 4U, 0U, 0U }; const uint8 g_dxt5_alpha6_to_linear[cDXT5SelectorValues] = {0U, 5U, 1U, 2U, 3U, 4U, 0U, 0U};
const uint8 g_dxt1_from_linear[cDXT1SelectorValues] = { 0U, 2U, 3U, 1U }; const uint8 g_dxt1_from_linear[cDXT1SelectorValues] = {0U, 2U, 3U, 1U};
const uint8 g_dxt1_to_linear[cDXT1SelectorValues] = { 0U, 3U, 1U, 2U }; const uint8 g_dxt1_to_linear[cDXT1SelectorValues] = {0U, 3U, 1U, 2U};
const uint8 g_six_alpha_invert_table[cDXT5SelectorValues] = { 1, 0, 5, 4, 3, 2, 6, 7 }; const uint8 g_six_alpha_invert_table[cDXT5SelectorValues] = {1, 0, 5, 4, 3, 2, 6, 7};
const uint8 g_eight_alpha_invert_table[cDXT5SelectorValues] = { 1, 0, 7, 6, 5, 4, 3, 2 }; const uint8 g_eight_alpha_invert_table[cDXT5SelectorValues] = {1, 0, 7, 6, 5, 4, 3, 2};
const char* get_dxt_format_string(dxt_format fmt) const char* get_dxt_format_string(dxt_format fmt) {
{ switch (fmt) {
switch (fmt) case cDXT1:
{ return "DXT1";
case cDXT1: return "DXT1"; case cDXT1A:
case cDXT1A: return "DXT1A"; return "DXT1A";
case cDXT3: return "DXT3"; case cDXT3:
case cDXT5: return "DXT5"; return "DXT3";
case cDXT5A: return "DXT5A"; case cDXT5:
case cDXN_XY: return "DXN_XY"; return "DXT5";
case cDXN_YX: return "DXN_YX"; case cDXT5A:
case cETC1: return "ETC1"; return "DXT5A";
default: break; case cDXN_XY:
} return "DXN_XY";
CRNLIB_ASSERT(false); case cDXN_YX:
return "?"; return "DXN_YX";
} case cETC1:
return "ETC1";
case cETC2:
return "ETC2";
case cETC2A:
return "ETC2A";
case cETC1S:
return "ETC1S";
case cETC2AS:
return "ETC2AS";
default:
break;
}
CRNLIB_ASSERT(false);
return "?";
}
const char* get_dxt_compressor_name(crn_dxt_compressor_type c) const char* get_dxt_compressor_name(crn_dxt_compressor_type c) {
{ switch (c) {
switch (c) case cCRNDXTCompressorCRN:
{ return "CRN";
case cCRNDXTCompressorCRN: return "CRN"; case cCRNDXTCompressorCRNF:
case cCRNDXTCompressorCRNF: return "CRNF"; return "CRNF";
case cCRNDXTCompressorRYG: return "RYG"; case cCRNDXTCompressorRYG:
return "RYG";
#if CRNLIB_SUPPORT_ATI_COMPRESS #if CRNLIB_SUPPORT_ATI_COMPRESS
case cCRNDXTCompressorATI: return "ATI"; case cCRNDXTCompressorATI:
return "ATI";
#endif #endif
default: break; default:
} break;
CRNLIB_ASSERT(false); }
return "?"; CRNLIB_ASSERT(false);
} return "?";
}
uint get_dxt_format_bits_per_pixel(dxt_format fmt)
{
switch (fmt)
{
case cDXT1:
case cDXT1A:
case cDXT5A:
case cETC1:
return 4;
case cDXT3:
case cDXT5:
case cDXN_XY:
case cDXN_YX:
return 8;
default: break;
}
CRNLIB_ASSERT(false);
return 0;
}
bool get_dxt_format_has_alpha(dxt_format fmt)
{
switch (fmt)
{
case cDXT1A:
case cDXT3:
case cDXT5:
case cDXT5A:
return true;
default: break;
}
return false;
}
uint16 dxt1_block::pack_color(const color_quad_u8& color, bool scaled, uint bias)
{
uint r = color.r;
uint g = color.g;
uint b = color.b;
if (scaled)
{
r = (r * 31U + bias) / 255U;
g = (g * 63U + bias) / 255U;
b = (b * 31U + bias) / 255U;
}
r = math::minimum(r, 31U);
g = math::minimum(g, 63U);
b = math::minimum(b, 31U);
return static_cast<uint16>(b | (g << 5U) | (r << 11U));
}
uint16 dxt1_block::pack_color(uint r, uint g, uint b, bool scaled, uint bias)
{
return pack_color(color_quad_u8(r, g, b, 0), scaled, bias);
}
color_quad_u8 dxt1_block::unpack_color(uint16 packed_color, bool scaled, uint alpha)
{
uint b = packed_color & 31U;
uint g = (packed_color >> 5U) & 63U;
uint r = (packed_color >> 11U) & 31U;
if (scaled)
{
b = (b << 3U) | (b >> 2U);
g = (g << 2U) | (g >> 4U);
r = (r << 3U) | (r >> 2U);
}
return color_quad_u8(cNoClamp, r, g, b, math::minimum(alpha, 255U));
}
void dxt1_block::unpack_color(uint& r, uint& g, uint& b, uint16 packed_color, bool scaled)
{
color_quad_u8 c(unpack_color(packed_color, scaled, 0));
r = c.r;
g = c.g;
b = c.b;
}
void dxt1_block::get_block_colors_NV5x(color_quad_u8* pDst, uint16 packed_col0, uint16 packed_col1, bool color4)
{
color_quad_u8 col0(unpack_color(packed_col0, false));
color_quad_u8 col1(unpack_color(packed_col1, false));
pDst[0].r = (3 * col0.r * 22) / 8;
pDst[0].b = (3 * col0.b * 22) / 8;
pDst[0].g = (col0.g << 2) | (col0.g >> 4);
pDst[0].a = 0xFF;
pDst[1].r = (3 * col1.r * 22) / 8;
pDst[1].g = (col1.g << 2) | (col1.g >> 4);
pDst[1].b = (3 * col1.b * 22) / 8;
pDst[1].a = 0xFF;
int gdiff = pDst[1].g - pDst[0].g;
if (color4) //(packed_col0 > packed_col1)
{
pDst[2].r = static_cast<uint8>(((2 * col0.r + col1.r) * 22) / 8);
pDst[2].g = static_cast<uint8>((256 * pDst[0].g + gdiff/4 + 128 + gdiff * 80) / 256);
pDst[2].b = static_cast<uint8>(((2 * col0.b + col1.b) * 22) / 8);
pDst[2].a = 0xFF;
pDst[3].r = static_cast<uint8>(((2 * col1.r + col0.r) * 22) / 8);
pDst[3].g = static_cast<uint8>((256 * pDst[1].g - gdiff/4 + 128 - gdiff * 80) / 256);
pDst[3].b = static_cast<uint8>(((2 * col1.b + col0.b) * 22) / 8);
pDst[3].a = 0xFF;
}
else {
pDst[2].r = static_cast<uint8>(((col0.r + col1.r) * 33) / 8);
pDst[2].g = static_cast<uint8>((256 * pDst[0].g + gdiff/4 + 128 + gdiff * 128) / 256);
pDst[2].b = static_cast<uint8>(((col0.b + col1.b) * 33) / 8);
pDst[2].a = 0xFF;
pDst[3].r = 0x00;
pDst[3].g = 0x00;
pDst[3].b = 0x00;
pDst[3].a = 0x00;
}
}
uint dxt1_block::get_block_colors3(color_quad_u8* pDst, uint16 color0, uint16 color1)
{
color_quad_u8 c0(unpack_color(color0, true));
color_quad_u8 c1(unpack_color(color1, true));
pDst[0] = c0;
pDst[1] = c1;
pDst[2].set_noclamp_rgba( (c0.r + c1.r) >> 1U, (c0.g + c1.g) >> 1U, (c0.b + c1.b) >> 1U, 255U);
pDst[3].set_noclamp_rgba(0, 0, 0, 0);
return 3;
}
uint dxt1_block::get_block_colors4(color_quad_u8* pDst, uint16 color0, uint16 color1)
{
color_quad_u8 c0(unpack_color(color0, true));
color_quad_u8 c1(unpack_color(color1, true));
pDst[0] = c0;
pDst[1] = c1;
// The compiler changes the div3 into a mul by recip+shift.
pDst[2].set_noclamp_rgba( (c0.r * 2 + c1.r) / 3, (c0.g * 2 + c1.g) / 3, (c0.b * 2 + c1.b) / 3, 255U);
pDst[3].set_noclamp_rgba( (c1.r * 2 + c0.r) / 3, (c1.g * 2 + c0.g) / 3, (c1.b * 2 + c0.b) / 3, 255U);
uint get_dxt_format_bits_per_pixel(dxt_format fmt) {
switch (fmt) {
case cDXT1:
case cDXT1A:
case cDXT5A:
case cETC1:
case cETC2:
case cETC1S:
return 4; return 4;
} case cDXT3:
case cDXT5:
uint dxt1_block::get_block_colors3_round(color_quad_u8* pDst, uint16 color0, uint16 color1) case cDXN_XY:
{ case cDXN_YX:
color_quad_u8 c0(unpack_color(color0, true)); case cETC2A:
color_quad_u8 c1(unpack_color(color1, true)); case cETC2AS:
pDst[0] = c0;
pDst[1] = c1;
pDst[2].set_noclamp_rgba( (c0.r + c1.r + 1) >> 1U, (c0.g + c1.g + 1) >> 1U, (c0.b + c1.b + 1) >> 1U, 255U);
pDst[3].set_noclamp_rgba(0, 0, 0, 0);
return 3;
}
uint dxt1_block::get_block_colors4_round(color_quad_u8* pDst, uint16 color0, uint16 color1)
{
color_quad_u8 c0(unpack_color(color0, true));
color_quad_u8 c1(unpack_color(color1, true));
pDst[0] = c0;
pDst[1] = c1;
// 12/14/08 - Supposed to round according to DX docs, but this conflicts with the OpenGL S3TC spec. ?
// The compiler changes the div3 into a mul by recip+shift.
pDst[2].set_noclamp_rgba( (c0.r * 2 + c1.r + 1) / 3, (c0.g * 2 + c1.g + 1) / 3, (c0.b * 2 + c1.b + 1) / 3, 255U);
pDst[3].set_noclamp_rgba( (c1.r * 2 + c0.r + 1) / 3, (c1.g * 2 + c0.g + 1) / 3, (c1.b * 2 + c0.b + 1) / 3, 255U);
return 4;
}
uint dxt1_block::get_block_colors(color_quad_u8* pDst, uint16 color0, uint16 color1)
{
if (color0 > color1)
return get_block_colors4(pDst, color0, color1);
else
return get_block_colors3(pDst, color0, color1);
}
uint dxt1_block::get_block_colors_round(color_quad_u8* pDst, uint16 color0, uint16 color1)
{
if (color0 > color1)
return get_block_colors4_round(pDst, color0, color1);
else
return get_block_colors3_round(pDst, color0, color1);
}
color_quad_u8 dxt1_block::unpack_endpoint(uint32 endpoints, uint index, bool scaled, uint alpha)
{
CRNLIB_ASSERT(index < 2);
return unpack_color( static_cast<uint16>((endpoints >> (index * 16U)) & 0xFFFFU), scaled, alpha );
}
uint dxt1_block::pack_endpoints(uint lo, uint hi)
{
CRNLIB_ASSERT((lo <= 0xFFFFU) && (hi <= 0xFFFFU));
return lo | (hi << 16U);
}
void dxt3_block::set_alpha(uint x, uint y, uint value, bool scaled)
{
CRNLIB_ASSERT((x < cDXTBlockSize) && (y < cDXTBlockSize));
if (scaled)
{
CRNLIB_ASSERT(value <= 0xFF);
value = (value * 15U + 128U) / 255U;
}
else
{
CRNLIB_ASSERT(value <= 0xF);
}
uint ofs = (y << 1U) + (x >> 1U);
uint c = m_alpha[ofs];
c &= ~(0xF << ((x & 1U) << 2U));
c |= (value << ((x & 1U) << 2U));
m_alpha[ofs] = static_cast<uint8>(c);
}
uint dxt3_block::get_alpha(uint x, uint y, bool scaled) const
{
CRNLIB_ASSERT((x < cDXTBlockSize) && (y < cDXTBlockSize));
uint value = m_alpha[(y << 1U) + (x >> 1U)];
if (x & 1)
value >>= 4;
value &= 0xF;
if (scaled)
value = (value << 4U) | value;
return value;
}
uint dxt5_block::get_block_values6(color_quad_u8* pDst, uint l, uint h)
{
pDst[0].a = static_cast<uint8>(l);
pDst[1].a = static_cast<uint8>(h);
pDst[2].a = static_cast<uint8>((l * 4 + h ) / 5);
pDst[3].a = static_cast<uint8>((l * 3 + h * 2) / 5);
pDst[4].a = static_cast<uint8>((l * 2 + h * 3) / 5);
pDst[5].a = static_cast<uint8>((l + h * 4) / 5);
pDst[6].a = 0;
pDst[7].a = 255;
return 6;
}
uint dxt5_block::get_block_values8(color_quad_u8* pDst, uint l, uint h)
{
pDst[0].a = static_cast<uint8>(l);
pDst[1].a = static_cast<uint8>(h);
pDst[2].a = static_cast<uint8>((l * 6 + h ) / 7);
pDst[3].a = static_cast<uint8>((l * 5 + h * 2) / 7);
pDst[4].a = static_cast<uint8>((l * 4 + h * 3) / 7);
pDst[5].a = static_cast<uint8>((l * 3 + h * 4) / 7);
pDst[6].a = static_cast<uint8>((l * 2 + h * 5) / 7);
pDst[7].a = static_cast<uint8>((l + h * 6) / 7);
return 8; return 8;
} default:
break;
}
CRNLIB_ASSERT(false);
return 0;
}
uint dxt5_block::get_block_values(color_quad_u8* pDst, uint l, uint h) bool get_dxt_format_has_alpha(dxt_format fmt) {
{ switch (fmt) {
if (l > h) case cDXT1A:
return get_block_values8(pDst, l, h); case cDXT3:
else case cDXT5:
return get_block_values6(pDst, l, h); case cDXT5A:
} case cETC2A:
case cETC2AS:
return true;
default:
break;
}
return false;
}
uint dxt5_block::get_block_values6(uint* pDst, uint l, uint h) uint16 dxt1_block::pack_color(const color_quad_u8& color, bool scaled, uint bias) {
{ uint r = color.r;
pDst[0] = l; uint g = color.g;
pDst[1] = h; uint b = color.b;
pDst[2] = (l * 4 + h ) / 5;
pDst[3] = (l * 3 + h * 2) / 5;
pDst[4] = (l * 2 + h * 3) / 5;
pDst[5] = (l + h * 4) / 5;
pDst[6] = 0;
pDst[7] = 255;
return 6;
}
uint dxt5_block::get_block_values8(uint* pDst, uint l, uint h) if (scaled) {
{ r = (r * 31U + bias) / 255U;
pDst[0] = l; g = (g * 63U + bias) / 255U;
pDst[1] = h; b = (b * 31U + bias) / 255U;
pDst[2] = (l * 6 + h ) / 7; }
pDst[3] = (l * 5 + h * 2) / 7;
pDst[4] = (l * 4 + h * 3) / 7;
pDst[5] = (l * 3 + h * 4) / 7;
pDst[6] = (l * 2 + h * 5) / 7;
pDst[7] = (l + h * 6) / 7;
return 8;
}
uint dxt5_block::unpack_endpoint(uint packed, uint index) r = math::minimum(r, 31U);
{ g = math::minimum(g, 63U);
CRNLIB_ASSERT(index < 2); b = math::minimum(b, 31U);
return (packed >> (8 * index)) & 0xFF;
}
uint dxt5_block::pack_endpoints(uint lo, uint hi) return static_cast<uint16>(b | (g << 5U) | (r << 11U));
{ }
CRNLIB_ASSERT((lo <= 0xFF) && (hi <= 0xFF));
return lo | (hi << 8U);
}
uint dxt5_block::get_block_values(uint* pDst, uint l, uint h) uint16 dxt1_block::pack_color(uint r, uint g, uint b, bool scaled, uint bias) {
{ return pack_color(color_quad_u8(r, g, b, 0), scaled, bias);
if (l > h) }
return get_block_values8(pDst, l, h);
else
return get_block_values6(pDst, l, h);
}
} // namespace crnlib color_quad_u8 dxt1_block::unpack_color(uint16 packed_color, bool scaled, uint alpha) {
uint b = packed_color & 31U;
uint g = (packed_color >> 5U) & 63U;
uint r = (packed_color >> 11U) & 31U;
if (scaled) {
b = (b << 3U) | (b >> 2U);
g = (g << 2U) | (g >> 4U);
r = (r << 3U) | (r >> 2U);
}
return color_quad_u8(cNoClamp, r, g, b, math::minimum(alpha, 255U));
}
void dxt1_block::unpack_color(uint& r, uint& g, uint& b, uint16 packed_color, bool scaled) {
color_quad_u8 c(unpack_color(packed_color, scaled, 0));
r = c.r;
g = c.g;
b = c.b;
}
void dxt1_block::get_block_colors_NV5x(color_quad_u8* pDst, uint16 packed_col0, uint16 packed_col1, bool color4) {
color_quad_u8 col0(unpack_color(packed_col0, false));
color_quad_u8 col1(unpack_color(packed_col1, false));
pDst[0].r = (3 * col0.r * 22) / 8;
pDst[0].b = (3 * col0.b * 22) / 8;
pDst[0].g = (col0.g << 2) | (col0.g >> 4);
pDst[0].a = 0xFF;
pDst[1].r = (3 * col1.r * 22) / 8;
pDst[1].g = (col1.g << 2) | (col1.g >> 4);
pDst[1].b = (3 * col1.b * 22) / 8;
pDst[1].a = 0xFF;
int gdiff = pDst[1].g - pDst[0].g;
if (color4) //(packed_col0 > packed_col1)
{
pDst[2].r = static_cast<uint8>(((2 * col0.r + col1.r) * 22) / 8);
pDst[2].g = static_cast<uint8>((256 * pDst[0].g + gdiff / 4 + 128 + gdiff * 80) / 256);
pDst[2].b = static_cast<uint8>(((2 * col0.b + col1.b) * 22) / 8);
pDst[2].a = 0xFF;
pDst[3].r = static_cast<uint8>(((2 * col1.r + col0.r) * 22) / 8);
pDst[3].g = static_cast<uint8>((256 * pDst[1].g - gdiff / 4 + 128 - gdiff * 80) / 256);
pDst[3].b = static_cast<uint8>(((2 * col1.b + col0.b) * 22) / 8);
pDst[3].a = 0xFF;
} else {
pDst[2].r = static_cast<uint8>(((col0.r + col1.r) * 33) / 8);
pDst[2].g = static_cast<uint8>((256 * pDst[0].g + gdiff / 4 + 128 + gdiff * 128) / 256);
pDst[2].b = static_cast<uint8>(((col0.b + col1.b) * 33) / 8);
pDst[2].a = 0xFF;
pDst[3].r = 0x00;
pDst[3].g = 0x00;
pDst[3].b = 0x00;
pDst[3].a = 0x00;
}
}
uint dxt1_block::get_block_colors3(color_quad_u8* pDst, uint16 color0, uint16 color1) {
color_quad_u8 c0(unpack_color(color0, true));
color_quad_u8 c1(unpack_color(color1, true));
pDst[0] = c0;
pDst[1] = c1;
pDst[2].set_noclamp_rgba((c0.r + c1.r) >> 1U, (c0.g + c1.g) >> 1U, (c0.b + c1.b) >> 1U, 255U);
pDst[3].set_noclamp_rgba(0, 0, 0, 0);
return 3;
}
uint dxt1_block::get_block_colors4(color_quad_u8* pDst, uint16 color0, uint16 color1) {
color_quad_u8 c0(unpack_color(color0, true));
color_quad_u8 c1(unpack_color(color1, true));
pDst[0] = c0;
pDst[1] = c1;
// The compiler changes the div3 into a mul by recip+shift.
pDst[2].set_noclamp_rgba((c0.r * 2 + c1.r) / 3, (c0.g * 2 + c1.g) / 3, (c0.b * 2 + c1.b) / 3, 255U);
pDst[3].set_noclamp_rgba((c1.r * 2 + c0.r) / 3, (c1.g * 2 + c0.g) / 3, (c1.b * 2 + c0.b) / 3, 255U);
return 4;
}
uint dxt1_block::get_block_colors3_round(color_quad_u8* pDst, uint16 color0, uint16 color1) {
color_quad_u8 c0(unpack_color(color0, true));
color_quad_u8 c1(unpack_color(color1, true));
pDst[0] = c0;
pDst[1] = c1;
pDst[2].set_noclamp_rgba((c0.r + c1.r + 1) >> 1U, (c0.g + c1.g + 1) >> 1U, (c0.b + c1.b + 1) >> 1U, 255U);
pDst[3].set_noclamp_rgba(0, 0, 0, 0);
return 3;
}
uint dxt1_block::get_block_colors4_round(color_quad_u8* pDst, uint16 color0, uint16 color1) {
color_quad_u8 c0(unpack_color(color0, true));
color_quad_u8 c1(unpack_color(color1, true));
pDst[0] = c0;
pDst[1] = c1;
// 12/14/08 - Supposed to round according to DX docs, but this conflicts with the OpenGL S3TC spec. ?
// The compiler changes the div3 into a mul by recip+shift.
pDst[2].set_noclamp_rgba((c0.r * 2 + c1.r + 1) / 3, (c0.g * 2 + c1.g + 1) / 3, (c0.b * 2 + c1.b + 1) / 3, 255U);
pDst[3].set_noclamp_rgba((c1.r * 2 + c0.r + 1) / 3, (c1.g * 2 + c0.g + 1) / 3, (c1.b * 2 + c0.b + 1) / 3, 255U);
return 4;
}
uint dxt1_block::get_block_colors(color_quad_u8* pDst, uint16 color0, uint16 color1) {
if (color0 > color1)
return get_block_colors4(pDst, color0, color1);
else
return get_block_colors3(pDst, color0, color1);
}
uint dxt1_block::get_block_colors_round(color_quad_u8* pDst, uint16 color0, uint16 color1) {
if (color0 > color1)
return get_block_colors4_round(pDst, color0, color1);
else
return get_block_colors3_round(pDst, color0, color1);
}
color_quad_u8 dxt1_block::unpack_endpoint(uint32 endpoints, uint index, bool scaled, uint alpha) {
CRNLIB_ASSERT(index < 2);
return unpack_color(static_cast<uint16>((endpoints >> (index * 16U)) & 0xFFFFU), scaled, alpha);
}
uint dxt1_block::pack_endpoints(uint lo, uint hi) {
CRNLIB_ASSERT((lo <= 0xFFFFU) && (hi <= 0xFFFFU));
return lo | (hi << 16U);
}
void dxt3_block::set_alpha(uint x, uint y, uint value, bool scaled) {
CRNLIB_ASSERT((x < cDXTBlockSize) && (y < cDXTBlockSize));
if (scaled) {
CRNLIB_ASSERT(value <= 0xFF);
value = (value * 15U + 128U) / 255U;
} else {
CRNLIB_ASSERT(value <= 0xF);
}
uint ofs = (y << 1U) + (x >> 1U);
uint c = m_alpha[ofs];
c &= ~(0xF << ((x & 1U) << 2U));
c |= (value << ((x & 1U) << 2U));
m_alpha[ofs] = static_cast<uint8>(c);
}
uint dxt3_block::get_alpha(uint x, uint y, bool scaled) const {
CRNLIB_ASSERT((x < cDXTBlockSize) && (y < cDXTBlockSize));
uint value = m_alpha[(y << 1U) + (x >> 1U)];
if (x & 1)
value >>= 4;
value &= 0xF;
if (scaled)
value = (value << 4U) | value;
return value;
}
uint dxt5_block::get_block_values6(color_quad_u8* pDst, uint l, uint h) {
pDst[0].a = static_cast<uint8>(l);
pDst[1].a = static_cast<uint8>(h);
pDst[2].a = static_cast<uint8>((l * 4 + h) / 5);
pDst[3].a = static_cast<uint8>((l * 3 + h * 2) / 5);
pDst[4].a = static_cast<uint8>((l * 2 + h * 3) / 5);
pDst[5].a = static_cast<uint8>((l + h * 4) / 5);
pDst[6].a = 0;
pDst[7].a = 255;
return 6;
}
uint dxt5_block::get_block_values8(color_quad_u8* pDst, uint l, uint h) {
pDst[0].a = static_cast<uint8>(l);
pDst[1].a = static_cast<uint8>(h);
pDst[2].a = static_cast<uint8>((l * 6 + h) / 7);
pDst[3].a = static_cast<uint8>((l * 5 + h * 2) / 7);
pDst[4].a = static_cast<uint8>((l * 4 + h * 3) / 7);
pDst[5].a = static_cast<uint8>((l * 3 + h * 4) / 7);
pDst[6].a = static_cast<uint8>((l * 2 + h * 5) / 7);
pDst[7].a = static_cast<uint8>((l + h * 6) / 7);
return 8;
}
uint dxt5_block::get_block_values(color_quad_u8* pDst, uint l, uint h) {
if (l > h)
return get_block_values8(pDst, l, h);
else
return get_block_values6(pDst, l, h);
}
uint dxt5_block::get_block_values6(uint* pDst, uint l, uint h) {
pDst[0] = l;
pDst[1] = h;
pDst[2] = (l * 4 + h) / 5;
pDst[3] = (l * 3 + h * 2) / 5;
pDst[4] = (l * 2 + h * 3) / 5;
pDst[5] = (l + h * 4) / 5;
pDst[6] = 0;
pDst[7] = 255;
return 6;
}
uint dxt5_block::get_block_values8(uint* pDst, uint l, uint h) {
pDst[0] = l;
pDst[1] = h;
pDst[2] = (l * 6 + h) / 7;
pDst[3] = (l * 5 + h * 2) / 7;
pDst[4] = (l * 4 + h * 3) / 7;
pDst[5] = (l * 3 + h * 4) / 7;
pDst[6] = (l * 2 + h * 5) / 7;
pDst[7] = (l + h * 6) / 7;
return 8;
}
uint dxt5_block::unpack_endpoint(uint packed, uint index) {
CRNLIB_ASSERT(index < 2);
return (packed >> (8 * index)) & 0xFF;
}
uint dxt5_block::pack_endpoints(uint lo, uint hi) {
CRNLIB_ASSERT((lo <= 0xFF) && (hi <= 0xFF));
return lo | (hi << 8U);
}
uint dxt5_block::get_block_values(uint* pDst, uint l, uint h) {
if (l > h)
return get_block_values8(pDst, l, h);
else
return get_block_values6(pDst, l, h);
}
} // namespace crnlib
crnlib/crn_dxt.h
+270 -306
View File
@@ -11,351 +11,315 @@
#define CRNLIB_DXT_ALT_ROUNDING 1 #define CRNLIB_DXT_ALT_ROUNDING 1
namespace crnlib namespace crnlib {
{ enum dxt_constants {
enum dxt_constants cDXT1BytesPerBlock = 8U,
{ cDXT5NBytesPerBlock = 16U,
cDXT1BytesPerBlock = 8U,
cDXT5NBytesPerBlock = 16U,
cDXT5SelectorBits = 3U, cDXT5SelectorBits = 3U,
cDXT5SelectorValues = 1U << cDXT5SelectorBits, cDXT5SelectorValues = 1U << cDXT5SelectorBits,
cDXT5SelectorMask = cDXT5SelectorValues - 1U, cDXT5SelectorMask = cDXT5SelectorValues - 1U,
cDXT1SelectorBits = 2U, cDXT1SelectorBits = 2U,
cDXT1SelectorValues = 1U << cDXT1SelectorBits, cDXT1SelectorValues = 1U << cDXT1SelectorBits,
cDXT1SelectorMask = cDXT1SelectorValues - 1U, cDXT1SelectorMask = cDXT1SelectorValues - 1U,
cDXTBlockShift = 2U, cDXTBlockShift = 2U,
cDXTBlockSize = 1U << cDXTBlockShift cDXTBlockSize = 1U << cDXTBlockShift
}; };
enum dxt_format enum dxt_format {
{ cDXTInvalid = -1,
cDXTInvalid = -1,
// cDXT1/1A must appear first! // cDXT1/1A must appear first!
cDXT1, cDXT1,
cDXT1A, cDXT1A,
cDXT3, cDXT3,
cDXT5, cDXT5,
cDXT5A, cDXT5A,
cDXN_XY, // inverted relative to standard ATI2, 360's DXN cDXN_XY, // inverted relative to standard ATI2, 360's DXN
cDXN_YX, // standard ATI2, cDXN_YX, // standard ATI2,
cETC1 // Ericsson texture compression (color only, 4x4 blocks, 4bpp, 64-bits/block) cETC1,
}; cETC2,
cETC2A,
cETC1S,
cETC2AS,
};
const float cDXT1MaxLinearValue = 3.0f; const float cDXT1MaxLinearValue = 3.0f;
const float cDXT1InvMaxLinearValue = 1.0f/3.0f; const float cDXT1InvMaxLinearValue = 1.0f / 3.0f;
const float cDXT5MaxLinearValue = 7.0f; const float cDXT5MaxLinearValue = 7.0f;
const float cDXT5InvMaxLinearValue = 1.0f/7.0f; const float cDXT5InvMaxLinearValue = 1.0f / 7.0f;
// Converts DXT1 raw color selector index to a linear value. // Converts DXT1 raw color selector index to a linear value.
extern const uint8 g_dxt1_to_linear[cDXT1SelectorValues]; extern const uint8 g_dxt1_to_linear[cDXT1SelectorValues];
// Converts DXT5 raw alpha selector index to a linear value. // Converts DXT5 raw alpha selector index to a linear value.
extern const uint8 g_dxt5_to_linear[cDXT5SelectorValues]; extern const uint8 g_dxt5_to_linear[cDXT5SelectorValues];
// Converts DXT1 linear color selector index to a raw value (inverse of g_dxt1_to_linear). // Converts DXT1 linear color selector index to a raw value (inverse of g_dxt1_to_linear).
extern const uint8 g_dxt1_from_linear[cDXT1SelectorValues]; extern const uint8 g_dxt1_from_linear[cDXT1SelectorValues];
// Converts DXT5 linear alpha selector index to a raw value (inverse of g_dxt5_to_linear). // Converts DXT5 linear alpha selector index to a raw value (inverse of g_dxt5_to_linear).
extern const uint8 g_dxt5_from_linear[cDXT5SelectorValues]; extern const uint8 g_dxt5_from_linear[cDXT5SelectorValues];
extern const uint8 g_dxt5_alpha6_to_linear[cDXT5SelectorValues]; extern const uint8 g_dxt5_alpha6_to_linear[cDXT5SelectorValues];
extern const uint8 g_six_alpha_invert_table[cDXT5SelectorValues]; extern const uint8 g_six_alpha_invert_table[cDXT5SelectorValues];
extern const uint8 g_eight_alpha_invert_table[cDXT5SelectorValues]; extern const uint8 g_eight_alpha_invert_table[cDXT5SelectorValues];
const char* get_dxt_format_string(dxt_format fmt); const char* get_dxt_format_string(dxt_format fmt);
uint get_dxt_format_bits_per_pixel(dxt_format fmt); uint get_dxt_format_bits_per_pixel(dxt_format fmt);
bool get_dxt_format_has_alpha(dxt_format fmt); bool get_dxt_format_has_alpha(dxt_format fmt);
const char* get_dxt_quality_string(crn_dxt_quality q); const char* get_dxt_quality_string(crn_dxt_quality q);
const char* get_dxt_compressor_name(crn_dxt_compressor_type c); const char* get_dxt_compressor_name(crn_dxt_compressor_type c);
struct dxt1_block struct dxt1_block {
{ uint8 m_low_color[2];
uint8 m_low_color[2]; uint8 m_high_color[2];
uint8 m_high_color[2];
enum { cNumSelectorBytes = 4 }; enum { cNumSelectorBytes = 4 };
uint8 m_selectors[cNumSelectorBytes]; uint8 m_selectors[cNumSelectorBytes];
inline void clear() inline void clear() {
{ utils::zero_this(this);
utils::zero_this(this); }
// These methods assume the in-memory rep is in LE byte order.
inline uint get_low_color() const {
return m_low_color[0] | (m_low_color[1] << 8U);
}
inline uint get_high_color() const {
return m_high_color[0] | (m_high_color[1] << 8U);
}
inline void set_low_color(uint16 c) {
m_low_color[0] = static_cast<uint8>(c & 0xFF);
m_low_color[1] = static_cast<uint8>((c >> 8) & 0xFF);
}
inline void set_high_color(uint16 c) {
m_high_color[0] = static_cast<uint8>(c & 0xFF);
m_high_color[1] = static_cast<uint8>((c >> 8) & 0xFF);
}
inline bool is_constant_color_block() const { return get_low_color() == get_high_color(); }
inline bool is_alpha_block() const { return get_low_color() <= get_high_color(); }
inline bool is_non_alpha_block() const { return !is_alpha_block(); }
inline uint get_selector(uint x, uint y) const {
CRNLIB_ASSERT((x < 4U) && (y < 4U));
return (m_selectors[y] >> (x * cDXT1SelectorBits)) & cDXT1SelectorMask;
}
inline void set_selector(uint x, uint y, uint val) {
CRNLIB_ASSERT((x < 4U) && (y < 4U) && (val < 4U));
m_selectors[y] &= (~(cDXT1SelectorMask << (x * cDXT1SelectorBits)));
m_selectors[y] |= (val << (x * cDXT1SelectorBits));
}
inline void flip_x(uint w = 4, uint h = 4) {
for (uint x = 0; x < (w / 2); x++) {
for (uint y = 0; y < h; y++) {
const uint c = get_selector(x, y);
set_selector(x, y, get_selector((w - 1) - x, y));
set_selector((w - 1) - x, y, c);
} }
}
}
// These methods assume the in-memory rep is in LE byte order. inline void flip_y(uint w = 4, uint h = 4) {
inline uint get_low_color() const for (uint y = 0; y < (h / 2); y++) {
{ for (uint x = 0; x < w; x++) {
return m_low_color[0] | (m_low_color[1] << 8U); const uint c = get_selector(x, y);
set_selector(x, y, get_selector(x, (h - 1) - y));
set_selector(x, (h - 1) - y, c);
} }
}
}
inline uint get_high_color() const static uint16 pack_color(const color_quad_u8& color, bool scaled, uint bias = 127U);
{ static uint16 pack_color(uint r, uint g, uint b, bool scaled, uint bias = 127U);
return m_high_color[0] | (m_high_color[1] << 8U);
static color_quad_u8 unpack_color(uint16 packed_color, bool scaled, uint alpha = 255U);
static void unpack_color(uint& r, uint& g, uint& b, uint16 packed_color, bool scaled);
static uint get_block_colors3(color_quad_u8* pDst, uint16 color0, uint16 color1);
static uint get_block_colors3_round(color_quad_u8* pDst, uint16 color0, uint16 color1);
static uint get_block_colors4(color_quad_u8* pDst, uint16 color0, uint16 color1);
static uint get_block_colors4_round(color_quad_u8* pDst, uint16 color0, uint16 color1);
// pDst must point to an array at least cDXT1SelectorValues long.
static uint get_block_colors(color_quad_u8* pDst, uint16 color0, uint16 color1);
static uint get_block_colors_round(color_quad_u8* pDst, uint16 color0, uint16 color1);
static color_quad_u8 unpack_endpoint(uint32 endpoints, uint index, bool scaled, uint alpha = 255U);
static uint pack_endpoints(uint lo, uint hi);
static void get_block_colors_NV5x(color_quad_u8* pDst, uint16 packed_col0, uint16 packed_col1, bool color4);
};
CRNLIB_DEFINE_BITWISE_COPYABLE(dxt1_block);
struct dxt3_block {
enum { cNumAlphaBytes = 8 };
uint8 m_alpha[cNumAlphaBytes];
void set_alpha(uint x, uint y, uint value, bool scaled);
uint get_alpha(uint x, uint y, bool scaled) const;
inline void flip_x(uint w = 4, uint h = 4) {
for (uint x = 0; x < (w / 2); x++) {
for (uint y = 0; y < h; y++) {
const uint c = get_alpha(x, y, false);
set_alpha(x, y, get_alpha((w - 1) - x, y, false), false);
set_alpha((w - 1) - x, y, c, false);
} }
}
}
inline void set_low_color(uint16 c) inline void flip_y(uint w = 4, uint h = 4) {
{ for (uint y = 0; y < (h / 2); y++) {
m_low_color[0] = static_cast<uint8>(c & 0xFF); for (uint x = 0; x < w; x++) {
m_low_color[1] = static_cast<uint8>((c >> 8) & 0xFF); const uint c = get_alpha(x, y, false);
set_alpha(x, y, get_alpha(x, (h - 1) - y, false), false);
set_alpha(x, (h - 1) - y, c, false);
} }
}
}
};
inline void set_high_color(uint16 c) CRNLIB_DEFINE_BITWISE_COPYABLE(dxt3_block);
{
m_high_color[0] = static_cast<uint8>(c & 0xFF); struct dxt5_block {
m_high_color[1] = static_cast<uint8>((c >> 8) & 0xFF); uint8 m_endpoints[2];
enum { cNumSelectorBytes = 6 };
uint8 m_selectors[cNumSelectorBytes];
inline void clear() {
utils::zero_this(this);
}
inline uint get_low_alpha() const {
return m_endpoints[0];
}
inline uint get_high_alpha() const {
return m_endpoints[1];
}
inline void set_low_alpha(uint i) {
CRNLIB_ASSERT(i <= cUINT8_MAX);
m_endpoints[0] = static_cast<uint8>(i);
}
inline void set_high_alpha(uint i) {
CRNLIB_ASSERT(i <= cUINT8_MAX);
m_endpoints[1] = static_cast<uint8>(i);
}
inline bool is_alpha6_block() const { return get_low_alpha() <= get_high_alpha(); }
uint get_endpoints_as_word() const { return m_endpoints[0] | (m_endpoints[1] << 8); }
uint get_selectors_as_word(uint index) {
CRNLIB_ASSERT(index < 3);
return m_selectors[index * 2] | (m_selectors[index * 2 + 1] << 8);
}
inline uint get_selector(uint x, uint y) const {
CRNLIB_ASSERT((x < 4U) && (y < 4U));
uint selector_index = (y * 4) + x;
uint bit_index = selector_index * cDXT5SelectorBits;
uint byte_index = bit_index >> 3;
uint bit_ofs = bit_index & 7;
uint v = m_selectors[byte_index];
if (byte_index < (cNumSelectorBytes - 1))
v |= (m_selectors[byte_index + 1] << 8);
return (v >> bit_ofs) & 7;
}
inline void set_selector(uint x, uint y, uint val) {
CRNLIB_ASSERT((x < 4U) && (y < 4U) && (val < 8U));
uint selector_index = (y * 4) + x;
uint bit_index = selector_index * cDXT5SelectorBits;
uint byte_index = bit_index >> 3;
uint bit_ofs = bit_index & 7;
uint v = m_selectors[byte_index];
if (byte_index < (cNumSelectorBytes - 1))
v |= (m_selectors[byte_index + 1] << 8);
v &= (~(7 << bit_ofs));
v |= (val << bit_ofs);
m_selectors[byte_index] = static_cast<uint8>(v);
if (byte_index < (cNumSelectorBytes - 1))
m_selectors[byte_index + 1] = static_cast<uint8>(v >> 8);
}
inline void flip_x(uint w = 4, uint h = 4) {
for (uint x = 0; x < (w / 2); x++) {
for (uint y = 0; y < h; y++) {
const uint c = get_selector(x, y);
set_selector(x, y, get_selector((w - 1) - x, y));
set_selector((w - 1) - x, y, c);
} }
}
}
inline bool is_constant_color_block() const { return get_low_color() == get_high_color(); } inline void flip_y(uint w = 4, uint h = 4) {
inline bool is_alpha_block() const { return get_low_color() <= get_high_color(); } for (uint y = 0; y < (h / 2); y++) {
inline bool is_non_alpha_block() const { return !is_alpha_block(); } for (uint x = 0; x < w; x++) {
const uint c = get_selector(x, y);
inline uint get_selector(uint x, uint y) const set_selector(x, y, get_selector(x, (h - 1) - y));
{ set_selector(x, (h - 1) - y, c);
CRNLIB_ASSERT((x < 4U) && (y < 4U));
return (m_selectors[y] >> (x * cDXT1SelectorBits)) & cDXT1SelectorMask;
} }
}
}
inline void set_selector(uint x, uint y, uint val) enum { cMaxSelectorValues = 8 };
{
CRNLIB_ASSERT((x < 4U) && (y < 4U) && (val < 4U));
m_selectors[y] &= (~(cDXT1SelectorMask << (x * cDXT1SelectorBits))); // Results written to alpha channel.
m_selectors[y] |= (val << (x * cDXT1SelectorBits)); static uint get_block_values6(color_quad_u8* pDst, uint l, uint h);
} static uint get_block_values8(color_quad_u8* pDst, uint l, uint h);
static uint get_block_values(color_quad_u8* pDst, uint l, uint h);
inline void flip_x(uint w = 4, uint h = 4) static uint get_block_values6(uint* pDst, uint l, uint h);
{ static uint get_block_values8(uint* pDst, uint l, uint h);
for (uint x = 0; x < (w / 2); x++) // pDst must point to an array at least cDXT5SelectorValues long.
{ static uint get_block_values(uint* pDst, uint l, uint h);
for (uint y = 0; y < h; y++)
{
const uint c = get_selector(x, y);
set_selector(x, y, get_selector((w - 1) - x, y));
set_selector((w - 1) - x, y, c);
}
}
}
inline void flip_y(uint w = 4, uint h = 4) static uint unpack_endpoint(uint packed, uint index);
{ static uint pack_endpoints(uint lo, uint hi);
for (uint y = 0; y < (h / 2); y++) };
{
for (uint x = 0; x < w; x++)
{
const uint c = get_selector(x, y);
set_selector(x, y, get_selector(x, (h - 1) - y));
set_selector(x, (h - 1) - y, c);
}
}
}
static uint16 pack_color(const color_quad_u8& color, bool scaled, uint bias = 127U);
static uint16 pack_color(uint r, uint g, uint b, bool scaled, uint bias = 127U);
static color_quad_u8 unpack_color(uint16 packed_color, bool scaled, uint alpha = 255U); CRNLIB_DEFINE_BITWISE_COPYABLE(dxt5_block);
static void unpack_color(uint& r, uint& g, uint& b, uint16 packed_color, bool scaled);
static uint get_block_colors3(color_quad_u8* pDst, uint16 color0, uint16 color1); struct dxt_pixel_block {
static uint get_block_colors3_round(color_quad_u8* pDst, uint16 color0, uint16 color1); color_quad_u8 m_pixels[cDXTBlockSize][cDXTBlockSize]; // [y][x]
static uint get_block_colors4(color_quad_u8* pDst, uint16 color0, uint16 color1);
static uint get_block_colors4_round(color_quad_u8* pDst, uint16 color0, uint16 color1);
// pDst must point to an array at least cDXT1SelectorValues long.
static uint get_block_colors(color_quad_u8* pDst, uint16 color0, uint16 color1);
static uint get_block_colors_round(color_quad_u8* pDst, uint16 color0, uint16 color1);
static color_quad_u8 unpack_endpoint(uint32 endpoints, uint index, bool scaled, uint alpha = 255U);
static uint pack_endpoints(uint lo, uint hi);
static void get_block_colors_NV5x(color_quad_u8* pDst, uint16 packed_col0, uint16 packed_col1, bool color4);
};
CRNLIB_DEFINE_BITWISE_COPYABLE(dxt1_block);
struct dxt3_block
{
enum { cNumAlphaBytes = 8 };
uint8 m_alpha[cNumAlphaBytes];
void set_alpha(uint x, uint y, uint value, bool scaled);
uint get_alpha(uint x, uint y, bool scaled) const;
inline void flip_x(uint w = 4, uint h = 4)
{
for (uint x = 0; x < (w / 2); x++)
{
for (uint y = 0; y < h; y++)
{
const uint c = get_alpha(x, y, false);
set_alpha(x, y, get_alpha((w - 1) - x, y, false), false);
set_alpha((w - 1) - x, y, c, false);
}
}
}
inline void flip_y(uint w = 4, uint h = 4)
{
for (uint y = 0; y < (h / 2); y++)
{
for (uint x = 0; x < w; x++)
{
const uint c = get_alpha(x, y, false);
set_alpha(x, y, get_alpha(x, (h - 1) - y, false), false);
set_alpha(x, (h - 1) - y, c, false);
}
}
}
};
CRNLIB_DEFINE_BITWISE_COPYABLE(dxt3_block);
struct dxt5_block
{
uint8 m_endpoints[2];
enum { cNumSelectorBytes = 6 };
uint8 m_selectors[cNumSelectorBytes];
inline void clear()
{
utils::zero_this(this);
}
inline uint get_low_alpha() const
{
return m_endpoints[0];
}
inline uint get_high_alpha() const
{
return m_endpoints[1];
}
inline void set_low_alpha(uint i)
{
CRNLIB_ASSERT(i <= cUINT8_MAX);
m_endpoints[0] = static_cast<uint8>(i);
}
inline void set_high_alpha(uint i)
{
CRNLIB_ASSERT(i <= cUINT8_MAX);
m_endpoints[1] = static_cast<uint8>(i);
}
inline bool is_alpha6_block() const { return get_low_alpha() <= get_high_alpha(); }
uint get_endpoints_as_word() const { return m_endpoints[0] | (m_endpoints[1] << 8); }
uint get_selectors_as_word(uint index) { CRNLIB_ASSERT(index < 3); return m_selectors[index * 2] | (m_selectors[index * 2 + 1] << 8); }
inline uint get_selector(uint x, uint y) const
{
CRNLIB_ASSERT((x < 4U) && (y < 4U));
uint selector_index = (y * 4) + x;
uint bit_index = selector_index * cDXT5SelectorBits;
uint byte_index = bit_index >> 3;
uint bit_ofs = bit_index & 7;
uint v = m_selectors[byte_index];
if (byte_index < (cNumSelectorBytes - 1))
v |= (m_selectors[byte_index + 1] << 8);
return (v >> bit_ofs) & 7;
}
inline void set_selector(uint x, uint y, uint val)
{
CRNLIB_ASSERT((x < 4U) && (y < 4U) && (val < 8U));
uint selector_index = (y * 4) + x;
uint bit_index = selector_index * cDXT5SelectorBits;
uint byte_index = bit_index >> 3;
uint bit_ofs = bit_index & 7;
uint v = m_selectors[byte_index];
if (byte_index < (cNumSelectorBytes - 1))
v |= (m_selectors[byte_index + 1] << 8);
v &= (~(7 << bit_ofs));
v |= (val << bit_ofs);
m_selectors[byte_index] = static_cast<uint8>(v);
if (byte_index < (cNumSelectorBytes - 1))
m_selectors[byte_index + 1] = static_cast<uint8>(v >> 8);
}
inline void flip_x(uint w = 4, uint h = 4)
{
for (uint x = 0; x < (w / 2); x++)
{
for (uint y = 0; y < h; y++)
{
const uint c = get_selector(x, y);
set_selector(x, y, get_selector((w - 1) - x, y));
set_selector((w - 1) - x, y, c);
}
}
}
inline void flip_y(uint w = 4, uint h = 4)
{
for (uint y = 0; y < (h / 2); y++)
{
for (uint x = 0; x < w; x++)
{
const uint c = get_selector(x, y);
set_selector(x, y, get_selector(x, (h - 1) - y));
set_selector(x, (h - 1) - y, c);
}
}
}
enum { cMaxSelectorValues = 8 };
// Results written to alpha channel.
static uint get_block_values6(color_quad_u8* pDst, uint l, uint h);
static uint get_block_values8(color_quad_u8* pDst, uint l, uint h);
static uint get_block_values(color_quad_u8* pDst, uint l, uint h);
static uint get_block_values6(uint* pDst, uint l, uint h);
static uint get_block_values8(uint* pDst, uint l, uint h);
// pDst must point to an array at least cDXT5SelectorValues long.
static uint get_block_values(uint* pDst, uint l, uint h);
static uint unpack_endpoint(uint packed, uint index);
static uint pack_endpoints(uint lo, uint hi);
};
CRNLIB_DEFINE_BITWISE_COPYABLE(dxt5_block);
struct dxt_pixel_block
{
color_quad_u8 m_pixels[cDXTBlockSize][cDXTBlockSize]; // [y][x]
inline void clear()
{
utils::zero_object(*this);
}
};
CRNLIB_DEFINE_BITWISE_COPYABLE(dxt_pixel_block);
} // namespace crnlib
inline void clear() {
utils::zero_object(*this);
}
};
CRNLIB_DEFINE_BITWISE_COPYABLE(dxt_pixel_block);
} // namespace crnlib
crnlib/crn_dxt1.cpp
+1713 -2066
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_dxt1.h
+264 -340
View File
@@ -3,350 +3,274 @@
#pragma once #pragma once
#include "crn_dxt.h" #include "crn_dxt.h"
namespace crnlib namespace crnlib {
{ struct dxt1_solution_coordinates {
struct dxt1_solution_coordinates inline dxt1_solution_coordinates()
{ : m_low_color(0), m_high_color(0) {}
inline dxt1_solution_coordinates() : m_low_color(0), m_high_color(0){ }
inline dxt1_solution_coordinates(uint16 l, uint16 h) : m_low_color(l), m_high_color(h) { } inline dxt1_solution_coordinates(uint16 l, uint16 h)
: m_low_color(l), m_high_color(h) {}
inline dxt1_solution_coordinates(const color_quad_u8& l, const color_quad_u8& h, bool scaled = true) : inline dxt1_solution_coordinates(const color_quad_u8& l, const color_quad_u8& h, bool scaled = true)
m_low_color(dxt1_block::pack_color(l, scaled)), : m_low_color(dxt1_block::pack_color(l, scaled)),
m_high_color(dxt1_block::pack_color(h, scaled)) m_high_color(dxt1_block::pack_color(h, scaled)) {
{ }
}
inline dxt1_solution_coordinates(vec3F nl, vec3F nh) inline dxt1_solution_coordinates(vec3F nl, vec3F nh) {
{
#if CRNLIB_DXT_ALT_ROUNDING #if CRNLIB_DXT_ALT_ROUNDING
// Umm, wtf? // Umm, wtf?
nl.clamp(0.0f, .999f); nl.clamp(0.0f, .999f);
nh.clamp(0.0f, .999f); nh.clamp(0.0f, .999f);
color_quad_u8 l( (int)floor(nl[0] * 32.0f), (int)floor(nl[1] * 64.0f), (int)floor(nl[2] * 32.0f), 255); color_quad_u8 l((int)floor(nl[0] * 32.0f), (int)floor(nl[1] * 64.0f), (int)floor(nl[2] * 32.0f), 255);
color_quad_u8 h( (int)floor(nh[0] * 32.0f), (int)floor(nh[1] * 64.0f), (int)floor(nh[2] * 32.0f), 255); color_quad_u8 h((int)floor(nh[0] * 32.0f), (int)floor(nh[1] * 64.0f), (int)floor(nh[2] * 32.0f), 255);
#else #else
// Fixes the bins // Fixes the bins
color_quad_u8 l( (int)floor(.5f + nl[0] * 31.0f), (int)floor(.5f + nl[1] * 63.0f), (int)floor(.5f + nl[2] * 31.0f), 255); color_quad_u8 l((int)floor(.5f + nl[0] * 31.0f), (int)floor(.5f + nl[1] * 63.0f), (int)floor(.5f + nl[2] * 31.0f), 255);
color_quad_u8 h( (int)floor(.5f + nh[0] * 31.0f), (int)floor(.5f + nh[1] * 63.0f), (int)floor(.5f + nh[2] * 31.0f), 255); color_quad_u8 h((int)floor(.5f + nh[0] * 31.0f), (int)floor(.5f + nh[1] * 63.0f), (int)floor(.5f + nh[2] * 31.0f), 255);
#endif #endif
m_low_color = dxt1_block::pack_color(l, false); m_low_color = dxt1_block::pack_color(l, false);
m_high_color = dxt1_block::pack_color(h, false); m_high_color = dxt1_block::pack_color(h, false);
} }
uint16 m_low_color; uint16 m_low_color;
uint16 m_high_color; uint16 m_high_color;
inline void clear() inline void clear() {
{ m_low_color = 0;
m_low_color = 0; m_high_color = 0;
m_high_color = 0; }
}
inline dxt1_solution_coordinates& canonicalize() {
inline dxt1_solution_coordinates& canonicalize() if (m_low_color < m_high_color)
{ utils::swap(m_low_color, m_high_color);
if (m_low_color < m_high_color) return *this;
utils::swap(m_low_color, m_high_color); }
return *this;
} inline operator size_t() const { return fast_hash(this, sizeof(*this)); }
inline operator size_t() const { return fast_hash(this, sizeof(*this)); } inline bool operator==(const dxt1_solution_coordinates& other) const {
uint16 l0 = math::minimum(m_low_color, m_high_color);
inline bool operator== (const dxt1_solution_coordinates& other) const uint16 h0 = math::maximum(m_low_color, m_high_color);
{
uint16 l0 = math::minimum(m_low_color, m_high_color); uint16 l1 = math::minimum(other.m_low_color, other.m_high_color);
uint16 h0 = math::maximum(m_low_color, m_high_color); uint16 h1 = math::maximum(other.m_low_color, other.m_high_color);
uint16 l1 = math::minimum(other.m_low_color, other.m_high_color); return (l0 == l1) && (h0 == h1);
uint16 h1 = math::maximum(other.m_low_color, other.m_high_color); }
return (l0 == l1) && (h0 == h1); inline bool operator!=(const dxt1_solution_coordinates& other) const {
} return !(*this == other);
}
inline bool operator!= (const dxt1_solution_coordinates& other) const
{ inline bool operator<(const dxt1_solution_coordinates& other) const {
return !(*this == other); uint16 l0 = math::minimum(m_low_color, m_high_color);
} uint16 h0 = math::maximum(m_low_color, m_high_color);
inline bool operator< (const dxt1_solution_coordinates& other) const uint16 l1 = math::minimum(other.m_low_color, other.m_high_color);
{ uint16 h1 = math::maximum(other.m_low_color, other.m_high_color);
uint16 l0 = math::minimum(m_low_color, m_high_color);
uint16 h0 = math::maximum(m_low_color, m_high_color); if (l0 < l1)
return true;
uint16 l1 = math::minimum(other.m_low_color, other.m_high_color); else if (l0 == l1) {
uint16 h1 = math::maximum(other.m_low_color, other.m_high_color); if (h0 < h1)
return true;
if (l0 < l1) }
return true;
else if (l0 == l1) return false;
{ }
if (h0 < h1) };
return true;
} typedef crnlib::vector<dxt1_solution_coordinates> dxt1_solution_coordinates_vec;
return false; CRNLIB_DEFINE_BITWISE_COPYABLE(dxt1_solution_coordinates);
}
}; struct unique_color {
inline unique_color() {}
typedef crnlib::vector<dxt1_solution_coordinates> dxt1_solution_coordinates_vec; inline unique_color(const color_quad_u8& color, uint weight)
: m_color(color), m_weight(weight) {}
CRNLIB_DEFINE_BITWISE_COPYABLE(dxt1_solution_coordinates);
color_quad_u8 m_color;
struct unique_color uint m_weight;
{
inline unique_color() { } inline bool operator<(const unique_color& c) const {
inline unique_color(const color_quad_u8& color, uint weight) : m_color(color), m_weight(weight) { } return *reinterpret_cast<const uint32*>(&m_color) < *reinterpret_cast<const uint32*>(&c.m_color);
}
color_quad_u8 m_color;
uint m_weight; inline bool operator==(const unique_color& c) const {
return *reinterpret_cast<const uint32*>(&m_color) == *reinterpret_cast<const uint32*>(&c.m_color);
inline bool operator< (const unique_color& c) const }
{ };
return *reinterpret_cast<const uint32*>(&m_color) < *reinterpret_cast<const uint32*>(&c.m_color);
} CRNLIB_DEFINE_BITWISE_COPYABLE(unique_color);
inline bool operator== (const unique_color& c) const class dxt1_endpoint_optimizer {
{ public:
return *reinterpret_cast<const uint32*>(&m_color) == *reinterpret_cast<const uint32*>(&c.m_color); dxt1_endpoint_optimizer();
}
}; struct params {
params()
CRNLIB_DEFINE_BITWISE_COPYABLE(unique_color); : m_block_index(0),
m_pPixels(NULL),
class dxt1_endpoint_optimizer m_num_pixels(0),
{ m_dxt1a_alpha_threshold(128U),
public: m_quality(cCRNDXTQualityUber),
dxt1_endpoint_optimizer(); m_pixels_have_alpha(false),
m_use_alpha_blocks(true),
struct params m_perceptual(true),
{ m_grayscale_sampling(false),
params() : m_endpoint_caching(true),
m_block_index(0), m_use_transparent_indices_for_black(false),
m_pPixels(NULL), m_force_alpha_blocks(false) {
m_num_pixels(0), }
m_dxt1a_alpha_threshold(128U),
m_quality(cCRNDXTQualityUber), uint m_block_index;
m_pixels_have_alpha(false),
m_use_alpha_blocks(true), const color_quad_u8* m_pPixels;
m_perceptual(true), uint m_num_pixels;
m_grayscale_sampling(false), uint m_dxt1a_alpha_threshold;
m_endpoint_caching(true),
m_use_transparent_indices_for_black(false), crn_dxt_quality m_quality;
m_force_alpha_blocks(false)
{ bool m_pixels_have_alpha;
m_color_weights[0] = 1; bool m_use_alpha_blocks;
m_color_weights[1] = 1; bool m_perceptual;
m_color_weights[2] = 1; bool m_grayscale_sampling;
} bool m_endpoint_caching;
bool m_use_transparent_indices_for_black;
uint m_block_index; bool m_force_alpha_blocks;
};
const color_quad_u8* m_pPixels;
uint m_num_pixels; struct results {
uint m_dxt1a_alpha_threshold; inline results()
: m_pSelectors(NULL) {}
crn_dxt_quality m_quality;
uint64 m_error;
bool m_pixels_have_alpha;
bool m_use_alpha_blocks; uint16 m_low_color;
bool m_perceptual; uint16 m_high_color;
bool m_grayscale_sampling;
bool m_endpoint_caching; uint8* m_pSelectors;
bool m_use_transparent_indices_for_black; bool m_alpha_block;
bool m_force_alpha_blocks; bool m_reordered;
int m_color_weights[3]; bool m_alternate_rounding;
}; bool m_enforce_selector;
uint8 m_enforced_selector;
struct results };
{
inline results() : m_pSelectors(NULL) { } bool compute(const params& p, results& r);
uint64 m_error; private:
const params* m_pParams;
uint16 m_low_color; results* m_pResults;
uint16 m_high_color;
bool m_perceptual;
uint8* m_pSelectors; bool m_evaluate_hc;
bool m_alpha_block;
}; typedef crnlib::vector<unique_color> unique_color_vec;
struct solution //typedef crnlib::hash_map<uint32, uint32, bit_hasher<uint32> > unique_color_hash_map;
{ typedef crnlib::hash_map<uint32, uint32> unique_color_hash_map;
solution() { } unique_color_hash_map m_unique_color_hash_map;
solution(const solution& other) unique_color_vec m_unique_colors; // excludes transparent colors!
{ unique_color_vec m_evaluated_colors;
m_results = other.m_results; unique_color_vec m_temp_unique_colors;
m_selectors = other.m_selectors;
m_results.m_pSelectors = m_selectors.begin(); struct {
} uint64 low, high;
} m_rDist[32], m_gDist[64], m_bDist[32];
solution& operator= (const solution& rhs)
{ uint m_total_unique_color_weight;
if (this == &rhs)
return *this; bool m_has_transparent_pixels;
m_results = rhs.m_results; vec3F_array m_norm_unique_colors;
m_selectors = rhs.m_selectors; vec3F m_mean_norm_color;
m_results.m_pSelectors = m_selectors.begin();
vec3F_array m_norm_unique_colors_weighted;
return *this; vec3F m_mean_norm_color_weighted;
}
vec3F m_principle_axis;
results m_results;
crnlib::vector<uint8> m_selectors; crnlib::vector<uint16> m_unique_packed_colors;
crnlib::vector<uint8> m_trial_selectors;
inline bool operator< (const solution& other) const
{ crnlib::vector<vec3F> m_low_coords;
return m_results.m_error < other.m_results.m_error; crnlib::vector<vec3F> m_high_coords;
}
static inline bool coords_equal(const solution& lhs, const solution& rhs) enum { cMaxPrevResults = 4 };
{ dxt1_solution_coordinates m_prev_results[cMaxPrevResults];
return (lhs.m_results.m_low_color == rhs.m_results.m_low_color) && (lhs.m_results.m_high_color == rhs.m_results.m_high_color); uint m_num_prev_results;
}
}; crnlib::vector<vec3I> m_lo_cells;
typedef crnlib::vector<solution> solution_vec; crnlib::vector<vec3I> m_hi_cells;
bool compute(const params& p, results& r, solution_vec* pSolutions = NULL); struct potential_solution {
potential_solution()
private: : m_coords(), m_error(cUINT64_MAX), m_alpha_block(false) {
const params* m_pParams; }
results* m_pResults;
solution_vec* m_pSolutions; dxt1_solution_coordinates m_coords;
crnlib::vector<uint8> m_selectors;
bool m_perceptual; uint64 m_error;
bool m_has_color_weighting; bool m_alpha_block;
bool m_alternate_rounding;
typedef crnlib::vector<unique_color> unique_color_vec; bool m_enforce_selector;
uint8 m_enforced_selector;
//typedef crnlib::hash_map<uint32, uint32, bit_hasher<uint32> > unique_color_hash_map;
typedef crnlib::hash_map<uint32, uint32> unique_color_hash_map; void clear() {
unique_color_hash_map m_unique_color_hash_map; m_coords.clear();
m_selectors.resize(0);
unique_color_vec m_unique_colors; // excludes transparent colors! m_error = cUINT64_MAX;
unique_color_vec m_temp_unique_colors; m_alpha_block = false;
}
uint m_total_unique_color_weight;
bool are_selectors_all_equal() const {
bool m_has_transparent_pixels; if (m_selectors.empty())
return false;
vec3F_array m_norm_unique_colors; const uint s = m_selectors[0];
vec3F m_mean_norm_color; for (uint i = 1; i < m_selectors.size(); i++)
if (m_selectors[i] != s)
vec3F_array m_norm_unique_colors_weighted; return false;
vec3F m_mean_norm_color_weighted; return true;
}
vec3F m_principle_axis; };
bool m_all_pixels_grayscale; potential_solution m_trial_solution;
potential_solution m_best_solution;
crnlib::vector<uint16> m_unique_packed_colors;
crnlib::vector<uint8> m_trial_selectors; typedef crnlib::hash_map<uint, empty_type> solution_hash_map;
solution_hash_map m_solutions_tried;
crnlib::vector<vec3F> m_low_coords;
crnlib::vector<vec3F> m_high_coords; bool refine_solution(int refinement_level = 0);
enum { cMaxPrevResults = 4 }; bool evaluate_solution(const dxt1_solution_coordinates& coords, bool alternate_rounding = false);
dxt1_solution_coordinates m_prev_results[cMaxPrevResults]; bool evaluate_solution_uber(const dxt1_solution_coordinates& coords, bool alternate_rounding);
uint m_num_prev_results; bool evaluate_solution_fast(const dxt1_solution_coordinates& coords, bool alternate_rounding);
bool evaluate_solution_hc_perceptual(const dxt1_solution_coordinates& coords, bool alternate_rounding);
crnlib::vector<vec3I> m_lo_cells; bool evaluate_solution_hc_uniform(const dxt1_solution_coordinates& coords, bool alternate_rounding);
crnlib::vector<vec3I> m_hi_cells; void compute_selectors();
void compute_selectors_hc();
uint m_total_evals;
void find_unique_colors();
struct potential_solution void handle_multicolor_block();
{ void compute_pca(vec3F& axis, const vec3F_array& norm_colors, const vec3F& def);
potential_solution() : m_coords(), m_error(cUINT64_MAX), m_alpha_block(false), m_valid(false) void compute_vectors(const vec3F& perceptual_weights);
{ void return_solution();
} void try_combinatorial_encoding();
void compute_endpoint_component_errors(uint comp_index, uint64 (&error)[4][256], uint64 (&best_remaining_error)[4]);
dxt1_solution_coordinates m_coords; void optimize_endpoint_comps();
crnlib::vector<uint8> m_selectors; void optimize_endpoints(vec3F& low_color, vec3F& high_color);
uint64 m_error; bool try_alpha_as_black_optimization();
bool m_alpha_block; bool try_average_block_as_solid();
bool m_valid; bool try_median4(const vec3F& low_color, const vec3F& high_color);
void clear() void compute_internal(const params& p, results& r);
{
m_coords.clear(); unique_color lerp_color(const color_quad_u8& a, const color_quad_u8& b, float f, int rounding = 1);
m_selectors.resize(0);
m_error = cUINT64_MAX; inline uint color_distance(bool perceptual, const color_quad_u8& e1, const color_quad_u8& e2, bool alpha);
m_alpha_block = false; };
m_valid = false;
} } // namespace crnlib
bool are_selectors_all_equal() const
{
if (m_selectors.empty())
return false;
const uint s = m_selectors[0];
for (uint i = 1; i < m_selectors.size(); i++)
if (m_selectors[i] != s)
return false;
return true;
}
};
potential_solution m_trial_solution;
potential_solution m_best_solution;
typedef crnlib::hash_map<uint, empty_type> solution_hash_map;
solution_hash_map m_solutions_tried;
bool refine_solution(int refinement_level = 0);
bool evaluate_solution(
const dxt1_solution_coordinates& coords,
bool early_out,
potential_solution* pBest_solution,
bool alternate_rounding = false);
bool evaluate_solution_uber(
potential_solution& solution,
const dxt1_solution_coordinates& coords,
bool early_out,
potential_solution* pBest_solution,
bool alternate_rounding = false);
bool evaluate_solution_fast(
potential_solution& solution,
const dxt1_solution_coordinates& coords,
bool early_out,
potential_solution* pBest_solution,
bool alternate_rounding = false);
void clear();
void find_unique_colors();
bool handle_all_transparent_block();
bool handle_solid_block();
bool handle_multicolor_block();
bool handle_grayscale_block();
void compute_pca(vec3F& axis, const vec3F_array& norm_colors, const vec3F& def);
void compute_vectors(const vec3F& perceptual_weights);
void return_solution(results& results, const potential_solution& solution);
void try_combinatorial_encoding();
void optimize_endpoint_comps();
bool optimize_endpoints(vec3F& low_color, vec3F& high_color);
bool try_alpha_as_black_optimization();
bool try_average_block_as_solid();
bool try_median4(const vec3F& low_color, const vec3F& high_color);
bool compute_internal(const params& p, results& r, solution_vec* pSolutions);
unique_color lerp_color(const color_quad_u8& a, const color_quad_u8& b, float f, int rounding = 1);
inline uint color_distance(bool perceptual, const color_quad_u8& e1, const color_quad_u8& e2, bool alpha);
static inline vec3F unpack_to_vec3F_raw(uint16 packed_color);
static inline vec3F unpack_to_vec3F(uint16 packed_color);
};
inline void swap(dxt1_endpoint_optimizer::solution& a, dxt1_endpoint_optimizer::solution& b)
{
std::swap(a.m_results, b.m_results);
a.m_selectors.swap(b.m_selectors);
}
} // namespace crnlib
crnlib/crn_dxt5a.cpp
+164 -184
View File
@@ -6,204 +6,184 @@
#include "crn_dxt_fast.h" #include "crn_dxt_fast.h"
#include "crn_intersect.h" #include "crn_intersect.h"
namespace crnlib namespace crnlib {
{ dxt5_endpoint_optimizer::dxt5_endpoint_optimizer()
dxt5_endpoint_optimizer::dxt5_endpoint_optimizer() : : m_pParams(NULL),
m_pParams(NULL), m_pResults(NULL) {
m_pResults(NULL) m_unique_values.reserve(16);
{ m_unique_value_weights.reserve(16);
m_unique_values.reserve(16); }
m_unique_value_weights.reserve(16);
}
bool dxt5_endpoint_optimizer::compute(const params& p, results& r) bool dxt5_endpoint_optimizer::compute(const params& p, results& r) {
{ m_pParams = &p;
m_pParams = &p; m_pResults = &r;
m_pResults = &r;
if ((!p.m_num_pixels) || (!p.m_pPixels)) if ((!p.m_num_pixels) || (!p.m_pPixels))
return false; return false;
m_unique_values.resize(0); m_unique_values.resize(0);
m_unique_value_weights.resize(0); m_unique_value_weights.resize(0);
for (uint i = 0; i < 256; i++) for (uint i = 0; i < 256; i++)
m_unique_value_map[i] = -1; m_unique_value_map[i] = -1;
for (uint i = 0; i < p.m_num_pixels; i++) for (uint i = 0; i < p.m_num_pixels; i++) {
{ uint alpha = p.m_pPixels[i][p.m_comp_index];
uint alpha = p.m_pPixels[i][p.m_comp_index];
int index = m_unique_value_map[alpha]; int index = m_unique_value_map[alpha];
if (index == -1) if (index == -1) {
{ index = m_unique_values.size();
index = m_unique_values.size();
m_unique_value_map[alpha] = index; m_unique_value_map[alpha] = index;
m_unique_values.push_back(static_cast<uint8>(alpha)); m_unique_values.push_back(static_cast<uint8>(alpha));
m_unique_value_weights.push_back(0); m_unique_value_weights.push_back(0);
} }
m_unique_value_weights[index]++; m_unique_value_weights[index]++;
} }
if (m_unique_values.size() == 1) if (m_unique_values.size() == 1) {
{ r.m_block_type = 0;
r.m_block_type = 0; r.m_reordered = false;
r.m_error = 0; r.m_error = 0;
r.m_first_endpoint = m_unique_values[0]; r.m_first_endpoint = m_unique_values[0];
r.m_second_endpoint = m_unique_values[0]; r.m_second_endpoint = m_unique_values[0];
memset(r.m_pSelectors, 0, p.m_num_pixels); memset(r.m_pSelectors, 0, p.m_num_pixels);
return true; return true;
}
m_trial_selectors.resize(m_unique_values.size());
m_best_selectors.resize(m_unique_values.size());
r.m_error = cUINT64_MAX;
for (uint i = 0; i < m_unique_values.size() - 1; i++) {
const uint low_endpoint = m_unique_values[i];
for (uint j = i + 1; j < m_unique_values.size(); j++) {
const uint high_endpoint = m_unique_values[j];
evaluate_solution(low_endpoint, high_endpoint);
}
}
if ((m_pParams->m_quality >= cCRNDXTQualityBetter) && (m_pResults->m_error)) {
m_flags.resize(256 * 256);
m_flags.clear_all_bits();
const int cProbeAmount = (m_pParams->m_quality == cCRNDXTQualityUber) ? 16 : 8;
for (int l_delta = -cProbeAmount; l_delta <= cProbeAmount; l_delta++) {
const int l = m_pResults->m_first_endpoint + l_delta;
if (l < 0)
continue;
else if (l > 255)
break;
const uint bit_index = l * 256;
for (int h_delta = -cProbeAmount; h_delta <= cProbeAmount; h_delta++) {
const int h = m_pResults->m_second_endpoint + h_delta;
if (h < 0)
continue;
else if (h > 255)
break;
//if (m_flags.get_bit(bit_index + h))
// continue;
if ((m_flags.get_bit(bit_index + h)) || (m_flags.get_bit(h * 256 + l)))
continue;
m_flags.set_bit(bit_index + h);
evaluate_solution(static_cast<uint>(l), static_cast<uint>(h));
}
}
}
m_pResults->m_reordered = false;
if (m_pResults->m_first_endpoint == m_pResults->m_second_endpoint) {
for (uint i = 0; i < m_best_selectors.size(); i++)
m_best_selectors[i] = 0;
} else if (m_pResults->m_block_type) {
//if (l > h)
// eight alpha
// else
// six alpha
if (m_pResults->m_first_endpoint > m_pResults->m_second_endpoint) {
utils::swap(m_pResults->m_first_endpoint, m_pResults->m_second_endpoint);
m_pResults->m_reordered = true;
for (uint i = 0; i < m_best_selectors.size(); i++)
m_best_selectors[i] = g_six_alpha_invert_table[m_best_selectors[i]];
}
} else if (!(m_pResults->m_first_endpoint > m_pResults->m_second_endpoint)) {
utils::swap(m_pResults->m_first_endpoint, m_pResults->m_second_endpoint);
m_pResults->m_reordered = true;
for (uint i = 0; i < m_best_selectors.size(); i++)
m_best_selectors[i] = g_eight_alpha_invert_table[m_best_selectors[i]];
}
for (uint i = 0; i < m_pParams->m_num_pixels; i++) {
uint alpha = m_pParams->m_pPixels[i][m_pParams->m_comp_index];
int index = m_unique_value_map[alpha];
m_pResults->m_pSelectors[i] = m_best_selectors[index];
}
return true;
}
void dxt5_endpoint_optimizer::evaluate_solution(uint low_endpoint, uint high_endpoint) {
for (uint block_type = 0; block_type < (m_pParams->m_use_both_block_types ? 2U : 1U); block_type++) {
uint selector_values[8];
if (!block_type)
dxt5_block::get_block_values8(selector_values, low_endpoint, high_endpoint);
else
dxt5_block::get_block_values6(selector_values, low_endpoint, high_endpoint);
uint64 trial_error = 0;
for (uint i = 0; i < m_unique_values.size(); i++) {
const uint val = m_unique_values[i];
const uint weight = m_unique_value_weights[i];
uint best_selector_error = UINT_MAX;
uint best_selector = 0;
for (uint j = 0; j < 8; j++) {
int selector_error = val - selector_values[j];
selector_error = selector_error * selector_error * (int)weight;
if (static_cast<uint>(selector_error) < best_selector_error) {
best_selector_error = selector_error;
best_selector = j;
if (!best_selector_error)
break;
}
} }
m_trial_selectors.resize(m_unique_values.size()); m_trial_selectors[i] = static_cast<uint8>(best_selector);
m_best_selectors.resize(m_unique_values.size()); trial_error += best_selector_error;
r.m_error = cUINT64_MAX; if (trial_error > m_pResults->m_error)
break;
}
for (uint i = 0; i < m_unique_values.size() - 1; i++) if (trial_error < m_pResults->m_error) {
{ m_pResults->m_error = trial_error;
const uint low_endpoint = m_unique_values[i]; m_pResults->m_first_endpoint = static_cast<uint8>(low_endpoint);
m_pResults->m_second_endpoint = static_cast<uint8>(high_endpoint);
m_pResults->m_block_type = static_cast<uint8>(block_type);
m_best_selectors.swap(m_trial_selectors);
for (uint j = i + 1; j < m_unique_values.size(); j++) if (!trial_error)
{ break;
const uint high_endpoint = m_unique_values[j]; }
}
}
evaluate_solution(low_endpoint, high_endpoint); } // namespace crnlib
}
}
if ((m_pParams->m_quality >= cCRNDXTQualityBetter) && (m_pResults->m_error))
{
m_flags.resize(256 * 256);
m_flags.clear_all_bits();
const int cProbeAmount = (m_pParams->m_quality == cCRNDXTQualityUber) ? 16 : 8;
for (int l_delta = -cProbeAmount; l_delta <= cProbeAmount; l_delta++)
{
const int l = m_pResults->m_first_endpoint + l_delta;
if (l < 0)
continue;
else if (l > 255)
break;
const uint bit_index = l * 256;
for (int h_delta = -cProbeAmount; h_delta <= cProbeAmount; h_delta++)
{
const int h = m_pResults->m_second_endpoint + h_delta;
if (h < 0)
continue;
else if (h > 255)
break;
//if (m_flags.get_bit(bit_index + h))
// continue;
if ((m_flags.get_bit(bit_index + h)) || (m_flags.get_bit(h * 256 + l)))
continue;
m_flags.set_bit(bit_index + h);
evaluate_solution(static_cast<uint>(l), static_cast<uint>(h));
}
}
}
if (m_pResults->m_first_endpoint == m_pResults->m_second_endpoint)
{
for (uint i = 0; i < m_best_selectors.size(); i++)
m_best_selectors[i] = 0;
}
else if (m_pResults->m_block_type)
{
//if (l > h)
// eight alpha
// else
// six alpha
if (m_pResults->m_first_endpoint > m_pResults->m_second_endpoint)
{
utils::swap(m_pResults->m_first_endpoint, m_pResults->m_second_endpoint);
for (uint i = 0; i < m_best_selectors.size(); i++)
m_best_selectors[i] = g_six_alpha_invert_table[m_best_selectors[i]];
}
}
else if (!(m_pResults->m_first_endpoint > m_pResults->m_second_endpoint))
{
utils::swap(m_pResults->m_first_endpoint, m_pResults->m_second_endpoint);
for (uint i = 0; i < m_best_selectors.size(); i++)
m_best_selectors[i] = g_eight_alpha_invert_table[m_best_selectors[i]];
}
for (uint i = 0; i < m_pParams->m_num_pixels; i++)
{
uint alpha = m_pParams->m_pPixels[i][m_pParams->m_comp_index];
int index = m_unique_value_map[alpha];
m_pResults->m_pSelectors[i] = m_best_selectors[index];
}
return true;
}
void dxt5_endpoint_optimizer::evaluate_solution(uint low_endpoint, uint high_endpoint)
{
for (uint block_type = 0; block_type < (m_pParams->m_use_both_block_types ? 2U : 1U); block_type++)
{
uint selector_values[8];
if (!block_type)
dxt5_block::get_block_values8(selector_values, low_endpoint, high_endpoint);
else
dxt5_block::get_block_values6(selector_values, low_endpoint, high_endpoint);
uint64 trial_error = 0;
for (uint i = 0; i < m_unique_values.size(); i++)
{
const uint val = m_unique_values[i];
const uint weight = m_unique_value_weights[i];
uint best_selector_error = UINT_MAX;
uint best_selector = 0;
for (uint j = 0; j < 8; j++)
{
int selector_error = val - selector_values[j];
selector_error = selector_error * selector_error * (int)weight;
if (static_cast<uint>(selector_error) < best_selector_error)
{
best_selector_error = selector_error;
best_selector = j;
if (!best_selector_error)
break;
}
}
m_trial_selectors[i] = static_cast<uint8>(best_selector);
trial_error += best_selector_error;
if (trial_error > m_pResults->m_error)
break;
}
if (trial_error < m_pResults->m_error)
{
m_pResults->m_error = trial_error;
m_pResults->m_first_endpoint = static_cast<uint8>(low_endpoint);
m_pResults->m_second_endpoint = static_cast<uint8>(high_endpoint);
m_pResults->m_block_type = static_cast<uint8>(block_type);
m_best_selectors.swap(m_trial_selectors);
if (!trial_error)
break;
}
}
}
} // namespace crnlib
crnlib/crn_dxt5a.h
+41 -45
View File
@@ -3,64 +3,60 @@
#pragma once #pragma once
#include "crn_dxt.h" #include "crn_dxt.h"
namespace crnlib namespace crnlib {
{ class dxt5_endpoint_optimizer {
class dxt5_endpoint_optimizer public:
{ dxt5_endpoint_optimizer();
public:
dxt5_endpoint_optimizer();
struct params struct params {
{ params()
params() : : m_block_index(0),
m_block_index(0), m_pPixels(NULL),
m_pPixels(NULL), m_num_pixels(0),
m_num_pixels(0), m_comp_index(3),
m_comp_index(3), m_quality(cCRNDXTQualityUber),
m_quality(cCRNDXTQualityUber), m_use_both_block_types(true) {
m_use_both_block_types(true) }
{
}
uint m_block_index; uint m_block_index;
const color_quad_u8* m_pPixels; const color_quad_u8* m_pPixels;
uint m_num_pixels; uint m_num_pixels;
uint m_comp_index; uint m_comp_index;
crn_dxt_quality m_quality; crn_dxt_quality m_quality;
bool m_use_both_block_types; bool m_use_both_block_types;
}; };
struct results struct results {
{ uint8* m_pSelectors;
uint8* m_pSelectors;
uint64 m_error; uint64 m_error;
uint8 m_first_endpoint; uint8 m_first_endpoint;
uint8 m_second_endpoint; uint8 m_second_endpoint;
uint8 m_block_type; // 1 if 6-alpha, otherwise 8-alpha uint8 m_block_type; // 1 if 6-alpha, otherwise 8-alpha
}; bool m_reordered;
};
bool compute(const params& p, results& r); bool compute(const params& p, results& r);
private: private:
const params* m_pParams; const params* m_pParams;
results* m_pResults; results* m_pResults;
crnlib::vector<uint8> m_unique_values; crnlib::vector<uint8> m_unique_values;
crnlib::vector<uint> m_unique_value_weights; crnlib::vector<uint> m_unique_value_weights;
crnlib::vector<uint8> m_trial_selectors; crnlib::vector<uint8> m_trial_selectors;
crnlib::vector<uint8> m_best_selectors; crnlib::vector<uint8> m_best_selectors;
int m_unique_value_map[256]; int m_unique_value_map[256];
sparse_bit_array m_flags; sparse_bit_array m_flags;
void evaluate_solution(uint low_endpoint, uint high_endpoint); void evaluate_solution(uint low_endpoint, uint high_endpoint);
}; };
} // namespace crnlib } // namespace crnlib
crnlib/crn_dxt_endpoint_refiner.cpp
+184 -337
View File
@@ -4,359 +4,206 @@
#include "crn_dxt_endpoint_refiner.h" #include "crn_dxt_endpoint_refiner.h"
#include "crn_dxt1.h" #include "crn_dxt1.h"
namespace crnlib namespace crnlib {
{ dxt_endpoint_refiner::dxt_endpoint_refiner()
dxt_endpoint_refiner::dxt_endpoint_refiner() : : m_pParams(NULL),
m_pParams(NULL), m_pResults(NULL) {
m_pResults(NULL) }
{
}
bool dxt_endpoint_refiner::refine(const params& p, results& r) bool dxt_endpoint_refiner::refine(const params& p, results& r) {
{ if (!p.m_num_pixels)
if (!p.m_num_pixels) return false;
return false;
m_pParams = &p; m_pParams = &p;
m_pResults = &r; m_pResults = &r;
r.m_error = cUINT64_MAX; r.m_error = cUINT64_MAX;
r.m_low_color = 0; r.m_low_color = 0;
r.m_high_color = 0; r.m_high_color = 0;
double alpha2_sum = 0.0f; double alpha2_sum = 0.0f;
double beta2_sum = 0.0f; double beta2_sum = 0.0f;
double alphabeta_sum = 0.0f; double alphabeta_sum = 0.0f;
vec<3, double> alphax_sum( 0.0f ); vec<3, double> alphax_sum(0.0f);
vec<3, double> betax_sum( 0.0f ); vec<3, double> betax_sum(0.0f);
vec<3, double> first_color( 0.0f ); vec<3, double> first_color(0.0f);
// This linear solver is from Squish. // This linear solver is from Squish.
for( uint i = 0; i < p.m_num_pixels; ++i ) for (uint i = 0; i < p.m_num_pixels; ++i) {
{ uint8 c = p.m_pSelectors[i];
uint8 c = p.m_pSelectors[i];
double k; double k;
if (p.m_dxt1_selectors) if (p.m_dxt1_selectors)
k = g_dxt1_to_linear[c] * 1.0f/3.0f; k = g_dxt1_to_linear[c] * 1.0f / 3.0f;
else else
k = g_dxt5_to_linear[c] * 1.0f/7.0f; k = g_dxt5_to_linear[c] * 1.0f / 7.0f;
double alpha = 1.0f - k; double alpha = 1.0f - k;
double beta = k; double beta = k;
vec<3, double> x; vec<3, double> x;
if (p.m_dxt1_selectors) if (p.m_dxt1_selectors)
x.set( p.m_pPixels[i][0] * 1.0f/255.0f, p.m_pPixels[i][1] * 1.0f/255.0f, p.m_pPixels[i][2] * 1.0f/255.0f ); x.set(p.m_pPixels[i][0] * 1.0f / 255.0f, p.m_pPixels[i][1] * 1.0f / 255.0f, p.m_pPixels[i][2] * 1.0f / 255.0f);
else else
x.set( p.m_pPixels[i][p.m_alpha_comp_index]/255.0f ); x.set(p.m_pPixels[i][p.m_alpha_comp_index] / 255.0f);
if (!i) if (!i)
first_color = x; first_color = x;
alpha2_sum += alpha*alpha; alpha2_sum += alpha * alpha;
beta2_sum += beta*beta; beta2_sum += beta * beta;
alphabeta_sum += alpha*beta; alphabeta_sum += alpha * beta;
alphax_sum += alpha*x; alphax_sum += alpha * x;
betax_sum += beta*x; betax_sum += beta * x;
}
// zero where non-determinate
vec<3, double> a, b;
if (beta2_sum == 0.0f) {
a = alphax_sum / alpha2_sum;
b.clear();
} else if (alpha2_sum == 0.0f) {
a.clear();
b = betax_sum / beta2_sum;
} else {
double factor = alpha2_sum * beta2_sum - alphabeta_sum * alphabeta_sum;
if (factor != 0.0f) {
a = (alphax_sum * beta2_sum - betax_sum * alphabeta_sum) / factor;
b = (betax_sum * alpha2_sum - alphax_sum * alphabeta_sum) / factor;
} else {
a = first_color;
b = first_color;
}
}
vec3F l(0.0f), h(0.0f);
l = a;
h = b;
l.clamp(0.0f, 1.0f);
h.clamp(0.0f, 1.0f);
if (p.m_dxt1_selectors)
optimize_dxt1(l, h);
else
optimize_dxt5(l, h);
return r.m_error < p.m_error_to_beat;
}
void dxt_endpoint_refiner::optimize_dxt5(vec3F low_color, vec3F high_color) {
uint8 L0 = math::clamp<int>(low_color[0] * 256.0f, 0, 255);
uint8 H0 = math::clamp<int>(high_color[0] * 256.0f, 0, 255);
uint64 hist[8] = {}, D2[8] = {}, DD[8] = {};
for (uint c = m_pParams->m_alpha_comp_index, i = 0; i < m_pParams->m_num_pixels; i++) {
uint8 a = m_pParams->m_pPixels[i][c];
uint8 s = m_pParams->m_pSelectors[i];
hist[s]++;
D2[s] += a * 2;
DD[s] += a * a;
}
uint16 solutions[529];
uint solutions_count = 0;
solutions[solutions_count++] = L0 == H0 ? H0 ? H0 - 1 << 8 | L0 : 1 : L0 > H0 ? H0 << 8 | L0 : L0 << 8 | H0;
uint8 minL = L0 <= 11 ? 0 : L0 - 11, maxL = L0 >= 244 ? 255 : L0 + 11;
uint8 minH = H0 <= 11 ? 0 : H0 - 11, maxH = H0 >= 244 ? 255 : H0 + 11;
for (uint16 L = minL; L <= maxL; L++) {
for (uint16 H = minH; H <= maxH; H++) {
if ((maxH < L || L <= H || H < minL) && (L != L0 || H != H0) && (L != H0 || H != L0))
solutions[solutions_count++] = L == H ? H ? H - 1 << 8 | L : 1 : L > H ? H << 8 | L : L << 8 | H;
}
}
for (uint i = 0; i < solutions_count; i++) {
uint8 L = solutions[i] & 0xFF;
uint8 H = solutions[i] >> 8;
uint values[8];
dxt5_block::get_block_values8(values, L, H);
uint64 error = 0;
for (uint64 s = 0; s < 8; s++)
error += hist[s] * values[s] * values[s] - D2[s] * values[s] + DD[s];
if (error < m_pResults->m_error) {
m_pResults->m_low_color = L;
m_pResults->m_high_color = H;
m_pResults->m_error = error;
if (!m_pResults->m_error)
return;
}
}
}
void dxt_endpoint_refiner::optimize_dxt1(vec3F low_color, vec3F high_color) {
uint16 L0 = math::clamp<int>(low_color[0] * 32.0f, 0, 31) << 11 | math::clamp<int>(low_color[1] * 64.0f, 0, 63) << 5 | math::clamp<int>(low_color[2] * 32.0f, 0, 31);
uint16 H0 = math::clamp<int>(high_color[0] * 32.0f, 0, 31) << 11 | math::clamp<int>(high_color[1] * 64.0f, 0, 63) << 5 | math::clamp<int>(high_color[2] * 32.0f, 0, 31);
uint64 hist[4] = {}, D2[4][3] = {}, DD[4][3] = {};
for (uint i = 0; i < m_pParams->m_num_pixels; i++) {
const color_quad_u8& pixel = m_pParams->m_pPixels[i];
uint8 s = m_pParams->m_pSelectors[i];
hist[s]++;
for (uint c = 0; c < 3; c++) {
D2[s][c] += pixel[c] * 2;
DD[s][c] += pixel[c] * pixel[c];
}
}
crnlib::vector<uint> solutions(54);
bool preserveL = hist[0] + hist[2] > hist[1] + hist[3];
bool improved = true;
for (uint iterations = 8; improved && iterations; iterations--) {
improved = false;
uint solutions_count = 0;
for (uint16 b0 = L0 & 31, g0 = L0 >> 5 & 63, r0 = L0 >> 11 & 31, b = b0 ? b0 - 1 : b0; b <= b0 + 1 && b <= 31; b++) {
for (uint16 g = g0 ? g0 - 1 : g0; g <= g0 + 1 && g <= 63; g++) {
for (uint16 r = r0 ? r0 - 1 : r0; r <= r0 + 1 && r <= 31; r++) {
uint16 L = r << 11 | g << 5 | b;
if (L != L0)
solutions[solutions_count++] = L > H0 ? L | H0 << 16 : H0 | L << 16;
}
} }
}
// zero where non-determinate for (uint16 b0 = H0 & 31, g0 = H0 >> 5 & 63, r0 = H0 >> 11 & 31, b = b0 ? b0 - 1 : b0; b <= b0 + 1 && b <= 31; b++) {
vec<3, double> a, b; for (uint16 g = g0 ? g0 - 1 : g0; g <= g0 + 1 && g <= 63; g++) {
if( beta2_sum == 0.0f ) for (uint16 r = r0 ? r0 - 1 : r0; r <= r0 + 1 && r <= 31; r++) {
{ uint16 H = r << 11 | g << 5 | b;
a = alphax_sum / alpha2_sum; if (H != H0)
b.clear(); solutions[solutions_count++] = H > L0 ? H | L0 << 16 : L0 | H << 16;
}
} }
else if( alpha2_sum == 0.0f ) }
{ std::sort(solutions.begin(), solutions.begin() + solutions_count);
a.clear(); for (uint i = 0; i < solutions_count; i++) {
b = betax_sum / beta2_sum; if (i && solutions[i] == solutions[i - 1])
continue;
uint16 L = solutions[i] & 0xFFFF;
uint16 H = solutions[i] >> 16;
if (L == H) {
L += !preserveL ? ~L & 0x1F ? 0x1 : ~L & 0xF800 ? 0x800 : ~L & 0x7E0 ? 0x20 : 0 : !L ? 0x1 : 0;
H -= preserveL ? H & 0x1F ? 0x1 : H & 0xF800 ? 0x800 : H & 0x7E0 ? 0x20 : 0 : H == 0xFFFF ? 0x1 : 0;
} }
else color_quad_u8 block_colors[4];
{ dxt1_block::get_block_colors4(block_colors, L, H);
double factor = alpha2_sum*beta2_sum - alphabeta_sum*alphabeta_sum; uint64 error = 0;
if (factor != 0.0f) for (uint64 s = 0, d[3]; s < 4; s++) {
{ for (uint c = 0; c < 3; c++)
a = ( alphax_sum*beta2_sum - betax_sum*alphabeta_sum ) / factor; d[c] = hist[s] * block_colors[s][c] * block_colors[s][c] - D2[s][c] * block_colors[s][c] + DD[s][c];
b = ( betax_sum*alpha2_sum - alphax_sum*alphabeta_sum ) / factor; error += m_pParams->m_perceptual ? d[0] * 8 + d[1] * 25 + d[2] : d[0] + d[1] + d[2];
}
else
{
a = first_color;
b = first_color;
}
} }
if (error < m_pResults->m_error) {
vec3F l(0.0f), h(0.0f); m_pResults->m_low_color = L0 = L;
l = a; m_pResults->m_high_color = H0 = H;
h = b; m_pResults->m_error = error;
if (!m_pResults->m_error)
l.clamp(0.0f, 1.0f); return;
h.clamp(0.0f, 1.0f); improved = true;
if (p.m_dxt1_selectors)
optimize_dxt1(l, h);
else
optimize_dxt5(l, h);
//if (r.m_low_color < r.m_high_color)
// utils::swap(r.m_low_color, r.m_high_color);
return r.m_error < p.m_error_to_beat;
}
void dxt_endpoint_refiner::optimize_dxt5(vec3F low_color, vec3F high_color)
{
float nl = low_color[0];
float nh = high_color[0];
#if CRNLIB_DXT_ALT_ROUNDING
nl = math::clamp(nl, 0.0f, .999f);
nh = math::clamp(nh, 0.0f, .999f);
uint il = (int)floor(nl * 256.0f);
uint ih = (int)floor(nh * 256.0f);
#else
uint il = (int)floor(.5f + math::clamp(nl, 0.0f, 1.0f) * 255.0f);
uint ih = (int)floor(.5f + math::clamp(nh, 0.0f, 1.0f) * 255.0f);
#endif
crnlib::vector<uint> trial_solutions;
trial_solutions.reserve(256);
trial_solutions.push_back(il | (ih << 8));
sparse_bit_array flags;
flags.resize(256 * 256);
flags.set_bit((il * 256) + ih);
const int cProbeAmount = 11;
for (int l_delta = -cProbeAmount; l_delta <= cProbeAmount; l_delta++)
{
const int l = il + l_delta;
if (l < 0)
continue;
else if (l > 255)
break;
const uint bit_index = l * 256;
for (int h_delta = -cProbeAmount; h_delta <= cProbeAmount; h_delta++)
{
const int h = ih + h_delta;
if (h < 0)
continue;
else if (h > 255)
break;
if ((flags.get_bit(bit_index + h)) || (flags.get_bit(h * 256 + l)))
continue;
flags.set_bit(bit_index + h);
trial_solutions.push_back(l | (h << 8));
}
} }
}
}
}
for (uint trial = 0; trial < trial_solutions.size(); trial++) } // namespace crnlib
{
uint l = trial_solutions[trial] & 0xFF;
uint h = trial_solutions[trial] >> 8;
if (l == h)
{
if (h)
h--;
else
l++;
}
else if (l < h)
{
utils::swap(l, h);
}
CRNLIB_ASSERT(l > h);
uint values[cDXT5SelectorValues];
dxt5_block::get_block_values8(values, l, h);
uint total_error = 0;
for (uint j = 0; j < m_pParams->m_num_pixels; j++)
{
int p = m_pParams->m_pPixels[j][m_pParams->m_alpha_comp_index];
int c = values[m_pParams->m_pSelectors[j]];
int error = p - c;
error *= error;
total_error += error;
if (total_error > m_pResults->m_error)
break;
}
if (total_error < m_pResults->m_error)
{
m_pResults->m_error = total_error;
m_pResults->m_low_color = static_cast<uint16>(l);
m_pResults->m_high_color = static_cast<uint16>(h);
if (m_pResults->m_error == 0)
return;
}
}
}
void dxt_endpoint_refiner::optimize_dxt1(vec3F low_color, vec3F high_color)
{
uint selector_hist[4];
utils::zero_object(selector_hist);
for (uint i = 0; i < m_pParams->m_num_pixels; i++)
selector_hist[m_pParams->m_pSelectors[i]]++;
dxt1_solution_coordinates c(low_color, high_color);
for (uint pass = 0; pass < 8; pass++)
{
const uint64 initial_error = m_pResults->m_error;
dxt1_solution_coordinates_vec coords_to_try;
coords_to_try.resize(0);
color_quad_u8 lc(dxt1_block::unpack_color(c.m_low_color, false));
color_quad_u8 hc(dxt1_block::unpack_color(c.m_high_color, false));
for (int i = 0; i < 27; i++)
{
if (13 == i) continue;
const int ir = (i % 3) - 1;
const int ig = ((i / 3) % 3) - 1;
const int ib = ((i / 9) % 3) - 1;
int r = lc.r + ir;
int g = lc.g + ig;
int b = lc.b + ib;
if ((r < 0) || (r > 31)|| (g < 0) || (g > 63) || (b < 0) || (b > 31)) continue;
coords_to_try.push_back(
dxt1_solution_coordinates(dxt1_block::pack_color(r, g, b, false), c.m_high_color)
);
}
for (int i = 0; i < 27; i++)
{
if (13 == i) continue;
const int ir = (i % 3) - 1;
const int ig = ((i / 3) % 3) - 1;
const int ib = ((i / 9) % 3) - 1;
int r = hc.r + ir;
int g = hc.g + ig;
int b = hc.b + ib;
if ((r < 0) || (r > 31)|| (g < 0) || (g > 63) || (b < 0) || (b > 31)) continue;
coords_to_try.push_back(dxt1_solution_coordinates(c.m_low_color, dxt1_block::pack_color(r, g, b, false)));
}
std::sort(coords_to_try.begin(), coords_to_try.end());
dxt1_solution_coordinates_vec::const_iterator p_last = std::unique(coords_to_try.begin(), coords_to_try.end());
uint num_coords_to_try = (uint)(p_last - coords_to_try.begin());
for (uint i = 0; i < num_coords_to_try; i++)
{
color_quad_u8 block_colors[4];
uint16 l = coords_to_try[i].m_low_color;
uint16 h = coords_to_try[i].m_high_color;
if (l < h)
utils::swap(l, h);
else if (l == h)
{
color_quad_u8 lc(dxt1_block::unpack_color(l, false));
color_quad_u8 hc(dxt1_block::unpack_color(h, false));
bool retry = false;
if ((selector_hist[0] + selector_hist[2]) > (selector_hist[1] + selector_hist[3]))
{
// l affects the output more than h, so muck with h
if (hc[2] != 0)
hc[2]--;
else if (hc[0] != 0)
hc[0]--;
else if (hc[1] != 0)
hc[1]--;
else
retry = true;
}
else
{
// h affects the output more than l, so muck with l
if (lc[2] != 31)
lc[2]++;
else if (lc[0] != 31)
lc[0]++;
else if (lc[1] != 63)
lc[1]++;
else
retry = true;
}
if (retry)
{
if (l == 0)
l++;
else
h--;
}
else
{
l = dxt1_block::pack_color(lc, false);
h = dxt1_block::pack_color(hc, false);
}
CRNLIB_ASSERT(l > h);
}
dxt1_block::get_block_colors4(block_colors, l, h);
uint total_error = 0;
for (uint j = 0; j < m_pParams->m_num_pixels; j++)
{
const color_quad_u8& c = block_colors[m_pParams->m_pSelectors[j]];
total_error += color::color_distance(m_pParams->m_perceptual, c, m_pParams->m_pPixels[j], false);
if (total_error > m_pResults->m_error)
break;
}
if (total_error < m_pResults->m_error)
{
m_pResults->m_error = total_error;
m_pResults->m_low_color = l;
m_pResults->m_high_color = h;
CRNLIB_ASSERT(l > h);
if (m_pResults->m_error == 0)
return;
}
}
if (m_pResults->m_error == initial_error)
break;
c.m_low_color = m_pResults->m_low_color;
c.m_high_color = m_pResults->m_high_color;
}
}
} // namespace crnlib
crnlib/crn_dxt_endpoint_refiner.h
+40 -45
View File
@@ -3,60 +3,55 @@
#pragma once #pragma once
#include "crn_dxt.h" #include "crn_dxt.h"
namespace crnlib namespace crnlib {
{ // TODO: Experimental/Not fully implemented
// TODO: Experimental/Not fully implemented class dxt_endpoint_refiner {
class dxt_endpoint_refiner public:
{ dxt_endpoint_refiner();
public:
dxt_endpoint_refiner();
struct params struct params {
{ params()
params() : : m_block_index(0),
m_block_index(0), m_pPixels(NULL),
m_pPixels(NULL), m_num_pixels(0),
m_num_pixels(0), m_pSelectors(NULL),
m_pSelectors(NULL), m_alpha_comp_index(0),
m_alpha_comp_index(0), m_error_to_beat(cUINT64_MAX),
m_error_to_beat(cUINT64_MAX), m_dxt1_selectors(true),
m_dxt1_selectors(true), m_perceptual(true),
m_perceptual(true), m_highest_quality(true) {
m_highest_quality(true) }
{
}
uint m_block_index; uint m_block_index;
const color_quad_u8* m_pPixels; const color_quad_u8* m_pPixels;
uint m_num_pixels; uint m_num_pixels;
const uint8* m_pSelectors; const uint8* m_pSelectors;
uint m_alpha_comp_index; uint m_alpha_comp_index;
uint64 m_error_to_beat; uint64 m_error_to_beat;
bool m_dxt1_selectors; bool m_dxt1_selectors;
bool m_perceptual; bool m_perceptual;
bool m_highest_quality; bool m_highest_quality;
}; };
struct results struct results {
{ uint16 m_low_color;
uint16 m_low_color; uint16 m_high_color;
uint16 m_high_color; uint64 m_error;
uint64 m_error; };
};
bool refine(const params& p, results& r); bool refine(const params& p, results& r);
private: private:
const params* m_pParams; const params* m_pParams;
results* m_pResults; results* m_pResults;
void optimize_dxt1(vec3F low_color, vec3F high_color); void optimize_dxt1(vec3F low_color, vec3F high_color);
void optimize_dxt5(vec3F low_color, vec3F high_color); void optimize_dxt5(vec3F low_color, vec3F high_color);
}; };
} // namespace crnlib } // namespace crnlib
crnlib/crn_dxt_fast.cpp
+773 -853
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_dxt_fast.h
+12 -14
View File
@@ -4,18 +4,16 @@
#include "crn_color.h" #include "crn_color.h"
#include "crn_dxt.h" #include "crn_dxt.h"
namespace crnlib namespace crnlib {
{ namespace dxt_fast {
namespace dxt_fast void compress_color_block(uint n, const color_quad_u8* block, uint& low16, uint& high16, uint8* pSelectors, bool refine = false);
{ void compress_color_block(dxt1_block* pDXT1_block, const color_quad_u8* pBlock, bool refine = false);
void compress_color_block(uint n, const color_quad_u8* block, uint& low16, uint& high16, uint8* pSelectors, bool refine = false);
void compress_color_block(dxt1_block* pDXT1_block, const color_quad_u8* pBlock, bool refine = false);
void compress_alpha_block(uint n, const color_quad_u8* block, uint& low8, uint& high8, uint8* pSelectors, uint comp_index);
void compress_alpha_block(dxt5_block* pDXT5_block, const color_quad_u8* pBlock, uint comp_index);
void find_representative_colors(uint n, const color_quad_u8* pBlock, color_quad_u8& lo, color_quad_u8& hi);
} // namespace dxt_fast
} // namespace crnlib void compress_alpha_block(uint n, const color_quad_u8* block, uint& low8, uint& high8, uint8* pSelectors, uint comp_index);
void compress_alpha_block(dxt5_block* pDXT5_block, const color_quad_u8* pBlock, uint comp_index);
void find_representative_colors(uint n, const color_quad_u8* pBlock, color_quad_u8& lo, color_quad_u8& hi);
} // namespace dxt_fast
} // namespace crnlib
crnlib/crn_dxt_hc.cpp
+1231 -2459
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_dxt_hc.h
+192 -418
View File
@@ -14,426 +14,200 @@
#define CRN_NO_FUNCTION_DEFINITIONS #define CRN_NO_FUNCTION_DEFINITIONS
#include "../inc/crnlib.h" #include "../inc/crnlib.h"
namespace crnlib namespace crnlib {
{ const uint cTotalCompressionPhases = 25;
const uint cTotalCompressionPhases = 25;
class dxt_hc class dxt_hc {
{ public:
public: dxt_hc();
dxt_hc(); ~dxt_hc();
~dxt_hc();
struct pixel_chunk struct endpoint_indices_details {
{ union {
pixel_chunk() { clear(); } struct {
uint16 color;
dxt_pixel_block m_blocks[cChunkBlockHeight][cChunkBlockWidth]; uint16 alpha0;
uint16 alpha1;
const color_quad_u8& operator() (uint cx, uint cy) const
{
CRNLIB_ASSERT((cx < cChunkPixelWidth) && (cy < cChunkPixelHeight));
return m_blocks[cy >> cBlockPixelHeightShift][cx >> cBlockPixelWidthShift].m_pixels
[cy & (cBlockPixelHeight - 1)][cx & (cBlockPixelWidth - 1)];
}
color_quad_u8& operator() (uint cx, uint cy)
{
CRNLIB_ASSERT((cx < cChunkPixelWidth) && (cy < cChunkPixelHeight));
return m_blocks[cy >> cBlockPixelHeightShift][cx >> cBlockPixelWidthShift].m_pixels
[cy & (cBlockPixelHeight - 1)][cx & (cBlockPixelWidth - 1)];
}
inline void clear()
{
utils::zero_object(*this);
m_weight = 1.0f;
}
float m_weight;
}; };
uint16 component[3];
};
uint8 reference;
endpoint_indices_details() { utils::zero_object(*this); }
};
typedef crnlib::vector<pixel_chunk> pixel_chunk_vec; struct selector_indices_details {
union {
struct params struct {
{ uint16 color;
params() : uint16 alpha0;
m_color_endpoint_codebook_size(3072), uint16 alpha1;
m_color_selector_codebook_size(3072),
m_alpha_endpoint_codebook_size(3072),
m_alpha_selector_codebook_size(3072),
m_adaptive_tile_color_psnr_derating(2.0f), // was 3.4f
m_adaptive_tile_alpha_psnr_derating(2.0f),
m_adaptive_tile_color_alpha_weighting_ratio(3.0f),
m_num_levels(0),
m_format(cDXT1),
m_hierarchical(true),
m_perceptual(true),
m_debugging(false),
m_pProgress_func(NULL),
m_pProgress_func_data(NULL)
{
m_alpha_component_indices[0] = 3;
m_alpha_component_indices[1] = 0;
for (uint i = 0; i < cCRNMaxLevels; i++)
{
m_levels[i].m_first_chunk = 0;
m_levels[i].m_num_chunks = 0;
}
}
// Valid range for codebook sizes: [32,8192] (non-power of two values are okay)
uint m_color_endpoint_codebook_size;
uint m_color_selector_codebook_size;
uint m_alpha_endpoint_codebook_size;
uint m_alpha_selector_codebook_size;
// Higher values cause fewer 8x4, 4x8, and 4x4 blocks to be utilized less often (lower quality/smaller files).
// Lower values cause the encoder to use large tiles less often (better quality/larger files).
// Valid range: [0.0,100.0].
// A value of 0 will cause the encoder to only use tiles larger than 4x4 if doing so would incur to quality loss.
float m_adaptive_tile_color_psnr_derating;
float m_adaptive_tile_alpha_psnr_derating;
float m_adaptive_tile_color_alpha_weighting_ratio;
uint m_alpha_component_indices[2];
struct miplevel_desc
{
uint m_first_chunk;
uint m_num_chunks;
};
// The mip level data is optional!
miplevel_desc m_levels[cCRNMaxLevels];
uint m_num_levels;
dxt_format m_format;
// If m_hierarchical is false, only 4x4 blocks will be used by the encoder (leading to higher quality/larger files).
bool m_hierarchical;
// If m_perceptual is true, perceptual color metrics will be used by the encoder.
bool m_perceptual;
bool m_debugging;
crn_progress_callback_func m_pProgress_func;
void* m_pProgress_func_data;
}; };
uint16 component[3];
void clear(); };
selector_indices_details() { utils::zero_object(*this); }
// Main compression function };
bool compress(const params& p, uint num_chunks, const pixel_chunk* pChunks, task_pool& task_pool);
struct tile_details {
// Output accessors crnlib::vector<color_quad_u8> pixels;
inline uint get_num_chunks() const { return m_num_chunks; } float weight;
vec<6, float> color_endpoint;
struct chunk_encoding vec<2, float> alpha_endpoints[2];
{ uint16 cluster_indices[3];
chunk_encoding() { utils::zero_object(*this); }; };
crnlib::vector<tile_details> m_tiles;
// Index into g_chunk_encodings. uint m_num_tiles;
uint8 m_encoding_index; float m_color_derating[cCRNMaxLevels][8];
float m_alpha_derating[8];
// Number of tiles, endpoint indices. float m_uint8_to_float[256];
uint8 m_num_tiles;
color_quad_u8 (*m_blocks)[16];
// Color, alpha0, alpha1 uint m_num_blocks;
enum { cColorIndex = 0, cAlpha0Index = 1, cAlpha1Index = 2 }; crnlib::vector<float> m_block_weights;
uint16 m_endpoint_indices[3][cChunkMaxTiles]; crnlib::vector<uint8> m_block_encodings;
uint16 m_selector_indices[3][cChunkBlockHeight][cChunkBlockWidth]; // [block_y][block_x] crnlib::vector<uint64> m_block_selectors[3];
}; crnlib::vector<uint32> m_color_selectors;
crnlib::vector<uint64> m_alpha_selectors;
typedef crnlib::vector<chunk_encoding> chunk_encoding_vec; crnlib::vector<bool> m_color_selectors_used;
crnlib::vector<bool> m_alpha_selectors_used;
inline const chunk_encoding& get_chunk_encoding(uint chunk_index) const { return m_chunk_encoding[chunk_index]; } crnlib::vector<uint> m_tile_indices;
inline const chunk_encoding_vec& get_chunk_encoding_vec() const { return m_chunk_encoding; } crnlib::vector<endpoint_indices_details> m_endpoint_indices;
crnlib::vector<selector_indices_details> m_selector_indices;
struct selectors
{ struct params {
selectors() { utils::zero_object(*this); } params()
: m_num_blocks(0),
uint8 m_selectors[cBlockPixelHeight][cBlockPixelWidth]; m_num_levels(0),
m_num_faces(0),
uint8 get_by_index(uint i) const { CRNLIB_ASSERT(i < (cBlockPixelWidth * cBlockPixelHeight)); const uint8* p = (const uint8*)m_selectors; return *(p + i); } m_format(cDXT1),
void set_by_index(uint i, uint v) { CRNLIB_ASSERT(i < (cBlockPixelWidth * cBlockPixelHeight)); uint8* p = (uint8*)m_selectors; *(p + i) = static_cast<uint8>(v); } m_perceptual(true),
}; m_hierarchical(true),
typedef crnlib::vector<selectors> selectors_vec; m_color_endpoint_codebook_size(3072),
m_color_selector_codebook_size(3072),
// Color endpoints m_alpha_endpoint_codebook_size(3072),
inline uint get_color_endpoint_codebook_size() const { return m_color_endpoints.size(); } m_alpha_selector_codebook_size(3072),
inline uint get_color_endpoint(uint codebook_index) const { return m_color_endpoints[codebook_index]; } m_adaptive_tile_color_psnr_derating(2.0f),
const crnlib::vector<uint>& get_color_endpoint_vec() const { return m_color_endpoints; } m_adaptive_tile_alpha_psnr_derating(2.0f),
m_adaptive_tile_color_alpha_weighting_ratio(3.0f),
// Color selectors m_debugging(false),
uint get_color_selector_codebook_size() const { return m_color_selectors.size(); } m_pProgress_func(0),
const selectors& get_color_selectors(uint codebook_index) const { return m_color_selectors[codebook_index]; } m_pProgress_func_data(0) {
const crnlib::vector<selectors>& get_color_selectors_vec() const { return m_color_selectors; } m_alpha_component_indices[0] = 3;
m_alpha_component_indices[1] = 0;
// Alpha endpoints for (uint i = 0; i < cCRNMaxLevels; i++) {
inline uint get_alpha_endpoint_codebook_size() const { return m_alpha_endpoints.size(); } m_levels[i].m_first_block = 0;
inline uint get_alpha_endpoint(uint codebook_index) const { return m_alpha_endpoints[codebook_index]; } m_levels[i].m_num_blocks = 0;
const crnlib::vector<uint>& get_alpha_endpoint_vec() const { return m_alpha_endpoints; } m_levels[i].m_block_width = 0;
}
// Alpha selectors }
uint get_alpha_selector_codebook_size() const { return m_alpha_selectors.size(); }
const selectors& get_alpha_selectors(uint codebook_index) const { return m_alpha_selectors[codebook_index]; } uint m_num_blocks;
const crnlib::vector<selectors>& get_alpha_selectors_vec() const { return m_alpha_selectors; } uint m_num_levels;
uint m_num_faces;
// Debug images
const pixel_chunk_vec& get_compressed_chunk_pixels() const { return m_dbg_chunk_pixels; } struct {
const pixel_chunk_vec& get_compressed_chunk_pixels_tile_vis() const { return m_dbg_chunk_pixels_tile_vis; } uint m_first_block;
const pixel_chunk_vec& get_compressed_chunk_pixels_color_quantized() const { return m_dbg_chunk_pixels_color_quantized; } uint m_num_blocks;
const pixel_chunk_vec& get_compressed_chunk_pixels_alpha_quantized() const { return m_dbg_chunk_pixels_alpha_quantized; } uint m_block_width;
const pixel_chunk_vec& get_compressed_chunk_pixels_final() const { return m_dbg_chunk_pixels_final; } float m_weight;
} m_levels[cCRNMaxLevels];
const pixel_chunk_vec& get_compressed_chunk_pixels_orig_color_selectors() const { return m_dbg_chunk_pixels_orig_color_selectors; }
const pixel_chunk_vec& get_compressed_chunk_pixels_quantized_color_selectors() const { return m_dbg_chunk_pixels_quantized_color_selectors; } dxt_format m_format;
const pixel_chunk_vec& get_compressed_chunk_pixels_final_color_selectors() const { return m_dbg_chunk_pixels_final_color_selectors; } bool m_perceptual;
bool m_hierarchical;
const pixel_chunk_vec& get_compressed_chunk_pixels_orig_alpha_selectors() const { return m_dbg_chunk_pixels_orig_alpha_selectors; }
const pixel_chunk_vec& get_compressed_chunk_pixels_quantized_alpha_selectors() const { return m_dbg_chunk_pixels_quantized_alpha_selectors; } uint m_color_endpoint_codebook_size;
const pixel_chunk_vec& get_compressed_chunk_pixels_final_alpha_selectors() const { return m_dbg_chunk_pixels_final_alpha_selectors; } uint m_color_selector_codebook_size;
uint m_alpha_endpoint_codebook_size;
static void create_debug_image_from_chunks(uint num_chunks_x, uint num_chunks_y, const pixel_chunk_vec& chunks, const chunk_encoding_vec *pChunk_encodings, image_u8& img, bool serpentine_scan, int comp_index = -1); uint m_alpha_selector_codebook_size;
private: float m_adaptive_tile_color_psnr_derating;
params m_params; float m_adaptive_tile_alpha_psnr_derating;
float m_adaptive_tile_color_alpha_weighting_ratio;
uint m_num_chunks; uint m_alpha_component_indices[2];
const pixel_chunk* m_pChunks;
task_pool* m_pTask_pool;
chunk_encoding_vec m_chunk_encoding; bool m_debugging;
crn_progress_callback_func m_pProgress_func;
uint m_num_alpha_blocks; // 0, 1, or 2 void* m_pProgress_func_data;
bool m_has_color_blocks; };
bool m_has_alpha0_blocks;
bool m_has_alpha1_blocks; void clear();
bool compress(
struct compressed_tile color_quad_u8 (*blocks)[16],
{ crnlib::vector<endpoint_indices_details>& endpoint_indices,
uint m_endpoint_cluster_index; crnlib::vector<selector_indices_details>& selector_indices,
uint m_first_endpoint; crnlib::vector<uint32>& color_endpoints,
uint m_second_endpoint; crnlib::vector<uint32>& alpha_endpoints,
crnlib::vector<uint32>& color_selectors,
uint8 m_selectors[cChunkPixelWidth * cChunkPixelHeight]; crnlib::vector<uint64>& alpha_selectors,
const params& p
void set_selector(uint x, uint y, uint s) );
{
CRNLIB_ASSERT((x < m_pixel_width) && (y < m_pixel_height)); private:
m_selectors[x + y * m_pixel_width] = static_cast<uint8>(s); params m_params;
}
uint m_num_alpha_blocks;
uint get_selector(uint x, uint y) const bool m_has_color_blocks;
{ bool m_has_etc_color_blocks;
CRNLIB_ASSERT((x < m_pixel_width) && (y < m_pixel_height)); bool m_has_subblocks;
return m_selectors[x + y * m_pixel_width];
} enum {
cColor = 0,
uint8 m_pixel_width; cAlpha0 = 1,
uint8 m_pixel_height; cAlpha1 = 2,
cNumComps = 3
uint8 m_layout_index; };
bool m_alpha_encoding; struct color_cluster {
}; color_cluster() : first_endpoint(0), second_endpoint(0) {}
crnlib::vector<uint> blocks[3];
struct compressed_chunk crnlib::vector<color_quad_u8> pixels;
{ uint first_endpoint;
compressed_chunk() { utils::zero_object(*this); } uint second_endpoint;
color_quad_u8 color_values[4];
uint8 m_encoding_index; };
crnlib::vector<color_cluster> m_color_clusters;
uint8 m_num_tiles;
struct alpha_cluster {
compressed_tile m_tiles[cChunkMaxTiles]; alpha_cluster() : first_endpoint(0), second_endpoint(0) {}
compressed_tile m_quantized_tiles[cChunkMaxTiles]; crnlib::vector<uint> blocks[3];
crnlib::vector<color_quad_u8> pixels;
uint16 m_endpoint_cluster_index[cChunkMaxTiles]; uint first_endpoint;
uint16 m_selector_cluster_index[cChunkBlockHeight][cChunkBlockWidth]; uint second_endpoint;
}; uint alpha_values[8];
bool refined_alpha;
typedef crnlib::vector<compressed_chunk> compressed_chunk_vec; uint refined_alpha_values[8];
enum };
{ crnlib::vector<alpha_cluster> m_alpha_clusters;
cColorChunks = 0,
cAlpha0Chunks = 1, crn_thread_id_t m_main_thread_id;
cAlpha1Chunks = 2, bool m_canceled;
task_pool* m_pTask_pool;
cNumCompressedChunkVecs = 3
}; int m_prev_phase_index;
compressed_chunk_vec m_compressed_chunks[cNumCompressedChunkVecs]; int m_prev_percentage_complete;
volatile atomic32_t m_encoding_hist[cNumChunkEncodings]; vec<6, float> palettize_color(color_quad_u8* pixels, uint pixels_count);
vec<2, float> palettize_alpha(color_quad_u8* pixels, uint pixels_count, uint comp_index);
atomic32_t m_total_tiles; void determine_tiles_task(uint64 data, void* pData_ptr);
void determine_tiles_task_etc(uint64 data, void* pData_ptr);
void compress_dxt1_block(
dxt1_endpoint_optimizer::results& results, void determine_color_endpoint_codebook_task(uint64 data, void* pData_ptr);
uint chunk_index, const image_u8& chunk, uint x_ofs, uint y_ofs, uint width, uint height, void determine_color_endpoint_codebook_task_etc(uint64 data, void* pData_ptr);
uint8* pSelectors); void determine_color_endpoint_clusters_task(uint64 data, void* pData_ptr);
void determine_color_endpoints();
void compress_dxt5_block(
dxt5_endpoint_optimizer::results& results, void determine_alpha_endpoint_codebook_task(uint64 data, void* pData_ptr);
uint chunk_index, const image_u8& chunk, uint x_ofs, uint y_ofs, uint width, uint height, uint component_index, void determine_alpha_endpoint_clusters_task(uint64 data, void* pData_ptr);
uint8* pAlpha_selectors); void determine_alpha_endpoints();
void determine_compressed_chunks_task(uint64 data, void* pData_ptr); void create_color_selector_codebook_task(uint64 data, void* pData_ptr);
bool determine_compressed_chunks(); void create_color_selector_codebook();
struct tile_cluster void create_alpha_selector_codebook_task(uint64 data, void* pData_ptr);
{ void create_alpha_selector_codebook();
tile_cluster() : m_first_endpoint(0), m_second_endpoint(0), m_error(0), m_alpha_encoding(false) { }
bool update_progress(uint phase_index, uint subphase_index, uint subphase_total);
// first = chunk, second = tile };
// if an alpha tile, second's upper 16 bits contains the alpha index (0 or 1)
crnlib::vector< std::pair<uint, uint> > m_tiles; } // namespace crnlib
uint m_first_endpoint;
uint m_second_endpoint;
uint64 m_error;
bool m_alpha_encoding;
};
typedef crnlib::vector<tile_cluster> tile_cluster_vec;
tile_cluster_vec m_color_clusters;
tile_cluster_vec m_alpha_clusters;
selectors_vec m_color_selectors;
selectors_vec m_alpha_selectors;
// For each selector, this array indicates every chunk/tile/tile block that use this color selector.
struct block_id
{
block_id() { utils::zero_object(*this); }
block_id(uint chunk_index, uint alpha_index, uint tile_index, uint block_x, uint block_y) :
m_chunk_index(chunk_index), m_alpha_index((uint8)alpha_index), m_tile_index((uint8)tile_index), m_block_x((uint8)block_x), m_block_y((uint8)block_y) { }
uint m_chunk_index;
uint8 m_alpha_index;
uint8 m_tile_index;
uint8 m_block_x;
uint8 m_block_y;
};
typedef crnlib::vector< crnlib::vector< block_id > > chunk_blocks_using_selectors_vec;
chunk_blocks_using_selectors_vec m_chunk_blocks_using_color_selectors;
chunk_blocks_using_selectors_vec m_chunk_blocks_using_alpha_selectors; // second's upper 16 bits contain alpha index!
crnlib::vector<uint> m_color_endpoints; // not valid until end, only for user access
crnlib::vector<uint> m_alpha_endpoints; // not valid until end, only for user access
// Debugging
pixel_chunk_vec m_dbg_chunk_pixels;
pixel_chunk_vec m_dbg_chunk_pixels_tile_vis;
pixel_chunk_vec m_dbg_chunk_pixels_color_quantized;
pixel_chunk_vec m_dbg_chunk_pixels_alpha_quantized;
pixel_chunk_vec m_dbg_chunk_pixels_orig_color_selectors;
pixel_chunk_vec m_dbg_chunk_pixels_quantized_color_selectors;
pixel_chunk_vec m_dbg_chunk_pixels_final_color_selectors;
pixel_chunk_vec m_dbg_chunk_pixels_orig_alpha_selectors;
pixel_chunk_vec m_dbg_chunk_pixels_quantized_alpha_selectors;
pixel_chunk_vec m_dbg_chunk_pixels_final_alpha_selectors;
pixel_chunk_vec m_dbg_chunk_pixels_final;
crn_thread_id_t m_main_thread_id;
bool m_canceled;
task_pool* m_pTask_pool;
int m_prev_phase_index;
int m_prev_percentage_complete;
typedef vec<6, float> vec6F;
typedef vec<16, float> vec16F;
typedef tree_clusterizer<vec2F> vec2F_tree_vq;
typedef tree_clusterizer<vec6F> vec6F_tree_vq;
typedef tree_clusterizer<vec16F> vec16F_tree_vq;
struct assign_color_endpoint_clusters_state
{
CRNLIB_NO_COPY_OR_ASSIGNMENT_OP(assign_color_endpoint_clusters_state);
assign_color_endpoint_clusters_state(vec6F_tree_vq& vq, crnlib::vector< crnlib::vector<vec6F> >& training_vecs) :
m_vq(vq), m_training_vecs(training_vecs) { }
vec6F_tree_vq& m_vq;
crnlib::vector< crnlib::vector<vec6F> >& m_training_vecs;
};
struct create_selector_codebook_state
{
CRNLIB_NO_COPY_OR_ASSIGNMENT_OP(create_selector_codebook_state);
create_selector_codebook_state(dxt_hc& hc, bool alpha_blocks, uint comp_index_start, uint comp_index_end, vec16F_tree_vq& selector_vq, chunk_blocks_using_selectors_vec& chunk_blocks_using_selectors, selectors_vec& selectors_cb) :
m_hc(hc),
m_alpha_blocks(alpha_blocks),
m_comp_index_start(comp_index_start),
m_comp_index_end(comp_index_end),
m_selector_vq(selector_vq),
m_chunk_blocks_using_selectors(chunk_blocks_using_selectors),
m_selectors_cb(selectors_cb)
{
}
dxt_hc& m_hc;
bool m_alpha_blocks;
uint m_comp_index_start;
uint m_comp_index_end;
vec16F_tree_vq& m_selector_vq;
chunk_blocks_using_selectors_vec& m_chunk_blocks_using_selectors;
selectors_vec& m_selectors_cb;
mutable spinlock m_chunk_blocks_using_selectors_lock;
};
void assign_color_endpoint_clusters_task(uint64 data, void* pData_ptr);
bool determine_color_endpoint_clusters();
struct determine_alpha_endpoint_clusters_state
{
vec2F_tree_vq m_vq;
crnlib::vector< crnlib::vector<vec2F> > m_training_vecs[2];
};
void determine_alpha_endpoint_clusters_task(uint64 data, void* pData_ptr);
bool determine_alpha_endpoint_clusters();
void determine_color_endpoint_codebook_task(uint64 data, void* pData_ptr);
bool determine_color_endpoint_codebook();
void determine_alpha_endpoint_codebook_task(uint64 data, void* pData_ptr);
bool determine_alpha_endpoint_codebook();
void create_quantized_debug_images();
void create_selector_codebook_task(uint64 data, void* pData_ptr);
bool create_selector_codebook(bool alpha_blocks);
bool refine_quantized_color_endpoints();
bool refine_quantized_color_selectors();
bool refine_quantized_alpha_endpoints();
bool refine_quantized_alpha_selectors();
void create_final_debug_image();
bool create_chunk_encodings();
bool update_progress(uint phase_index, uint subphase_index, uint subphase_total);
bool compress_internal(const params& p, uint num_chunks, const pixel_chunk* pChunks);
};
CRNLIB_DEFINE_BITWISE_COPYABLE(dxt_hc::pixel_chunk);
CRNLIB_DEFINE_BITWISE_COPYABLE(dxt_hc::chunk_encoding);
CRNLIB_DEFINE_BITWISE_COPYABLE(dxt_hc::selectors);
} // namespace crnlib
crnlib/crn_dxt_hc_common.cpp
+27 -33
View File
@@ -3,45 +3,39 @@
#include "crn_core.h" #include "crn_core.h"
#include "crn_dxt_hc_common.h" #include "crn_dxt_hc_common.h"
namespace crnlib namespace crnlib {
{ chunk_encoding_desc g_chunk_encodings[cNumChunkEncodings] =
chunk_encoding_desc g_chunk_encodings[cNumChunkEncodings] = {
{ {1, {{0, 0, 8, 8, 0}}},
{ 1, { { 0, 0, 8, 8, 0 } } },
{ 2, { { 0, 0, 8, 4, 1 }, { 0, 4, 8, 4, 2 } } }, {2, {{0, 0, 8, 4, 1}, {0, 4, 8, 4, 2}}},
{ 2, { { 0, 0, 4, 8, 3 }, { 4, 0, 4, 8, 4 } } }, {2, {{0, 0, 4, 8, 3}, {4, 0, 4, 8, 4}}},
{ 3, { { 0, 0, 8, 4, 1 }, { 0, 4, 4, 4, 7 }, { 4, 4, 4, 4, 8 } } }, {3, {{0, 0, 8, 4, 1}, {0, 4, 4, 4, 7}, {4, 4, 4, 4, 8}}},
{ 3, { { 0, 4, 8, 4, 2 }, { 0, 0, 4, 4, 5 }, { 4, 0, 4, 4, 6 } } }, {3, {{0, 4, 8, 4, 2}, {0, 0, 4, 4, 5}, {4, 0, 4, 4, 6}}},
{ 3, { { 0, 0, 4, 8, 3 }, { 4, 0, 4, 4, 6 }, { 4, 4, 4, 4, 8 } } }, {3, {{0, 0, 4, 8, 3}, {4, 0, 4, 4, 6}, {4, 4, 4, 4, 8}}},
{ 3, { { 4, 0, 4, 8, 4 }, { 0, 0, 4, 4, 5 }, { 0, 4, 4, 4, 7 } } }, {3, {{4, 0, 4, 8, 4}, {0, 0, 4, 4, 5}, {0, 4, 4, 4, 7}}},
{ 4, { { 0, 0, 4, 4, 5 }, { 4, 0, 4, 4, 6 }, { 0, 4, 4, 4, 7 }, { 4, 4, 4, 4, 8 } } } {4, {{0, 0, 4, 4, 5}, {4, 0, 4, 4, 6}, {0, 4, 4, 4, 7}, {4, 4, 4, 4, 8}}}};
};
chunk_tile_desc g_chunk_tile_layouts[cNumChunkTileLayouts] = chunk_tile_desc g_chunk_tile_layouts[cNumChunkTileLayouts] =
{ {
// 2x2 // 2x2
{ 0, 0, 8, 8, 0 }, {0, 0, 8, 8, 0},
// 2x1 // 2x1
{ 0, 0, 8, 4, 1 }, {0, 0, 8, 4, 1},
{ 0, 4, 8, 4, 2 }, {0, 4, 8, 4, 2},
// 1x2
{ 0, 0, 4, 8, 3 },
{ 4, 0, 4, 8, 4 },
// 1x1
{ 0, 0, 4, 4, 5 },
{ 4, 0, 4, 4, 6 },
{ 0, 4, 4, 4, 7 },
{ 4, 4, 4, 4, 8 }
};
} // namespace crnlib
// 1x2
{0, 0, 4, 8, 3},
{4, 0, 4, 8, 4},
// 1x1
{0, 0, 4, 4, 5},
{4, 0, 4, 4, 6},
{0, 4, 4, 4, 7},
{4, 4, 4, 4, 8}};
} // namespace crnlib
crnlib/crn_dxt_hc_common.h
+29 -32
View File
@@ -2,42 +2,39 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
namespace crnlib namespace crnlib {
{ struct chunk_tile_desc {
struct chunk_tile_desc // These values are in pixels, and always a multiple of cBlockPixelWidth/cBlockPixelHeight.
{ uint m_x_ofs;
// These values are in pixels, and always a multiple of cBlockPixelWidth/cBlockPixelHeight. uint m_y_ofs;
uint m_x_ofs; uint m_width;
uint m_y_ofs; uint m_height;
uint m_width; uint m_layout_index;
uint m_height; };
uint m_layout_index;
};
struct chunk_encoding_desc struct chunk_encoding_desc {
{ uint m_num_tiles;
uint m_num_tiles; chunk_tile_desc m_tiles[4];
chunk_tile_desc m_tiles[4]; };
};
const uint cChunkPixelWidth = 8; const uint cChunkPixelWidth = 8;
const uint cChunkPixelHeight = 8; const uint cChunkPixelHeight = 8;
const uint cChunkBlockWidth = 2; const uint cChunkBlockWidth = 2;
const uint cChunkBlockHeight = 2; const uint cChunkBlockHeight = 2;
const uint cChunkMaxTiles = 4; const uint cChunkMaxTiles = 4;
const uint cBlockPixelWidthShift = 2; const uint cBlockPixelWidthShift = 2;
const uint cBlockPixelHeightShift = 2; const uint cBlockPixelHeightShift = 2;
const uint cBlockPixelWidth = 4; const uint cBlockPixelWidth = 4;
const uint cBlockPixelHeight = 4; const uint cBlockPixelHeight = 4;
const uint cNumChunkEncodings = 8; const uint cNumChunkEncodings = 8;
extern chunk_encoding_desc g_chunk_encodings[cNumChunkEncodings]; extern chunk_encoding_desc g_chunk_encodings[cNumChunkEncodings];
const uint cNumChunkTileLayouts = 9; const uint cNumChunkTileLayouts = 9;
const uint cFirst4x4ChunkTileLayout = 5; const uint cFirst4x4ChunkTileLayout = 5;
extern chunk_tile_desc g_chunk_tile_layouts[cNumChunkTileLayouts]; extern chunk_tile_desc g_chunk_tile_layouts[cNumChunkTileLayouts];
} // namespace crnlib } // namespace crnlib
crnlib/crn_dxt_image.cpp
+1516 -1564
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_dxt_image.h
+228 -220
View File
@@ -11,238 +11,246 @@
#define CRNLIB_SUPPORT_ATI_COMPRESS 0 #define CRNLIB_SUPPORT_ATI_COMPRESS 0
namespace crnlib namespace crnlib {
{ class task_pool;
class task_pool;
class dxt_image class dxt_image {
{ public:
public: dxt_image();
dxt_image(); dxt_image(const dxt_image& other);
dxt_image(const dxt_image& other); dxt_image& operator=(const dxt_image& rhs);
dxt_image& operator= (const dxt_image& rhs);
void clear();
inline bool is_valid() const { return m_blocks_x > 0; } void clear();
uint get_width() const { return m_width; }
uint get_height() const { return m_height; }
uint get_blocks_x() const { return m_blocks_x; }
uint get_blocks_y() const { return m_blocks_y; }
uint get_total_blocks() const { return m_blocks_x * m_blocks_y; }
uint get_elements_per_block() const { return m_num_elements_per_block; }
uint get_bytes_per_block() const { return m_bytes_per_block; }
dxt_format get_format() const { return m_format; }
bool has_color() const { return (m_format == cDXT1) || (m_format == cDXT1A) || (m_format == cDXT3) || (m_format == cDXT5) || (m_format == cETC1); }
// Will be pretty slow if the image is DXT1, as this method scans for alpha blocks/selectors.
bool has_alpha() const;
enum element_type
{
cUnused = 0,
cColorDXT1, // DXT1 color block
cAlphaDXT3, // DXT3 alpha block (only)
cAlphaDXT5, // DXT5 alpha block (only)
cColorETC1, // ETC1 color block inline bool is_valid() const { return m_blocks_x > 0; }
};
element_type get_element_type(uint element_index) const { CRNLIB_ASSERT(element_index < m_num_elements_per_block); return m_element_type[element_index]; }
//Returns -1 for RGB, or [0,3]
int8 get_element_component_index(uint element_index) const { CRNLIB_ASSERT(element_index < m_num_elements_per_block); return m_element_component_index[element_index]; }
struct element
{
uint8 m_bytes[8];
uint get_le_word(uint index) const { CRNLIB_ASSERT(index < 4); return m_bytes[index*2] | (m_bytes[index * 2 + 1] << 8); }
uint get_be_word(uint index) const { CRNLIB_ASSERT(index < 4); return m_bytes[index*2 + 1] | (m_bytes[index * 2] << 8); }
void set_le_word(uint index, uint val) { CRNLIB_ASSERT((index < 4) && (val <= cUINT16_MAX)); m_bytes[index*2] = static_cast<uint8>(val & 0xFF); m_bytes[index * 2 + 1] = static_cast<uint8>((val >> 8) & 0xFF); }
void set_be_word(uint index, uint val) { CRNLIB_ASSERT((index < 4) && (val <= cUINT16_MAX)); m_bytes[index*2+1] = static_cast<uint8>(val & 0xFF); m_bytes[index * 2] = static_cast<uint8>((val >> 8) & 0xFF); }
void clear()
{
memset(this, 0, sizeof(*this));
}
};
typedef crnlib::vector<element> element_vec;
bool init(dxt_format fmt, uint width, uint height, bool clear_elements);
bool init(dxt_format fmt, uint width, uint height, uint num_elements, element* pElements, bool create_copy);
struct pack_params
{
pack_params()
{
clear();
}
void clear() uint get_width() const { return m_width; }
{ uint get_height() const { return m_height; }
m_quality = cCRNDXTQualityUber;
m_perceptual = true;
m_dithering = false;
m_grayscale_sampling = false;
m_use_both_block_types = true;
m_endpoint_caching = true;
m_compressor = cCRNDXTCompressorCRN;
m_pProgress_callback = NULL;
m_pProgress_callback_user_data_ptr = NULL;
m_dxt1a_alpha_threshold = 128;
m_num_helper_threads = 0;
m_progress_start = 0;
m_progress_range = 100;
m_use_transparent_indices_for_black = false;
m_pTask_pool = NULL;
m_color_weights[0] = 1;
m_color_weights[1] = 1;
m_color_weights[2] = 1;
}
void init(const crn_comp_params &params) uint get_blocks_x() const { return m_blocks_x; }
{ uint get_blocks_y() const { return m_blocks_y; }
m_perceptual = (params.m_flags & cCRNCompFlagPerceptual) != 0; uint get_total_blocks() const { return m_blocks_x * m_blocks_y; }
m_num_helper_threads = params.m_num_helper_threads;
m_use_both_block_types = (params.m_flags & cCRNCompFlagUseBothBlockTypes) != 0;
m_use_transparent_indices_for_black = (params.m_flags & cCRNCompFlagUseTransparentIndicesForBlack) != 0;
m_dxt1a_alpha_threshold = params.m_dxt1a_alpha_threshold;
m_quality = params.m_dxt_quality;
m_endpoint_caching = (params.m_flags & cCRNCompFlagDisableEndpointCaching) == 0;
m_grayscale_sampling = (params.m_flags & cCRNCompFlagGrayscaleSampling) != 0;
m_compressor = params.m_dxt_compressor_type;
}
uint m_dxt1a_alpha_threshold;
uint m_num_helper_threads;
crn_dxt_quality m_quality;
crn_dxt_compressor_type m_compressor;
bool m_perceptual;
bool m_dithering;
bool m_grayscale_sampling;
bool m_use_both_block_types;
bool m_endpoint_caching;
bool m_use_transparent_indices_for_black;
typedef bool (*progress_callback_func)(uint percentage_complete, void* pUser_data_ptr);
progress_callback_func m_pProgress_callback;
void* m_pProgress_callback_user_data_ptr;
uint m_progress_start;
uint m_progress_range;
task_pool *m_pTask_pool; uint get_elements_per_block() const { return m_num_elements_per_block; }
uint get_bytes_per_block() const { return m_bytes_per_block; }
int m_color_weights[3]; dxt_format get_format() const { return m_format; }
};
bool init(dxt_format fmt, const image_u8& img, const pack_params& p = dxt_image::pack_params());
bool unpack(image_u8& img) const;
void endian_swap();
uint get_total_elements() const { return m_elements.size(); }
const element_vec& get_element_vec() const { return m_elements; }
element_vec& get_element_vec() { return m_elements; }
const element& get_element(uint block_x, uint block_y, uint element_index) const;
element& get_element(uint block_x, uint block_y, uint element_index);
const element* get_element_ptr() const { return m_pElements; }
element* get_element_ptr() { return m_pElements; }
uint get_size_in_bytes() const { return m_elements.size() * sizeof(element); }
uint get_row_pitch_in_bytes() const { return m_blocks_x * m_bytes_per_block; }
color_quad_u8 get_pixel(uint x, uint y) const;
uint get_pixel_alpha(uint x, uint y, uint element_index) const;
void set_pixel(uint x, uint y, const color_quad_u8& c, bool perceptual = true);
// get_block_pixels() only sets those components stored in the image!
bool get_block_pixels(uint block_x, uint block_y, color_quad_u8* pPixels) const;
struct set_block_pixels_context bool has_color() const { return (m_format == cDXT1) || (m_format == cDXT1A) || (m_format == cDXT3) || (m_format == cDXT5) || (m_format == cETC1) || (m_format == cETC2) || (m_format == cETC2A) || (m_format == cETC1S) || (m_format == cETC2AS); }
{
dxt1_endpoint_optimizer m_dxt1_optimizer; // Will be pretty slow if the image is DXT1, as this method scans for alpha blocks/selectors.
dxt5_endpoint_optimizer m_dxt5_optimizer; bool has_alpha() const;
pack_etc1_block_context m_etc1_optimizer;
enum element_type {
cUnused = 0,
cColorDXT1, // DXT1 color block
cAlphaDXT3, // DXT3 alpha block (only)
cAlphaDXT5, // DXT5 alpha block (only)
cColorETC1, // ETC1 color block
cColorETC2, // ETC2 color block
cAlphaETC2, // ETC2 alpha block (only)
};
element_type get_element_type(uint element_index) const {
CRNLIB_ASSERT(element_index < m_num_elements_per_block);
return m_element_type[element_index];
}
//Returns -1 for RGB, or [0,3]
int8 get_element_component_index(uint element_index) const {
CRNLIB_ASSERT(element_index < m_num_elements_per_block);
return m_element_component_index[element_index];
}
struct element {
uint8 m_bytes[8];
uint get_le_word(uint index) const {
CRNLIB_ASSERT(index < 4);
return m_bytes[index * 2] | (m_bytes[index * 2 + 1] << 8);
}
uint get_be_word(uint index) const {
CRNLIB_ASSERT(index < 4);
return m_bytes[index * 2 + 1] | (m_bytes[index * 2] << 8);
}
void set_le_word(uint index, uint val) {
CRNLIB_ASSERT((index < 4) && (val <= cUINT16_MAX));
m_bytes[index * 2] = static_cast<uint8>(val & 0xFF);
m_bytes[index * 2 + 1] = static_cast<uint8>((val >> 8) & 0xFF);
}
void set_be_word(uint index, uint val) {
CRNLIB_ASSERT((index < 4) && (val <= cUINT16_MAX));
m_bytes[index * 2 + 1] = static_cast<uint8>(val & 0xFF);
m_bytes[index * 2] = static_cast<uint8>((val >> 8) & 0xFF);
}
void clear() {
memset(this, 0, sizeof(*this));
}
};
typedef crnlib::vector<element> element_vec;
bool init(dxt_format fmt, uint width, uint height, bool clear_elements);
bool init(dxt_format fmt, uint width, uint height, uint num_elements, element* pElements, bool create_copy);
struct pack_params {
pack_params() {
clear();
}
void clear() {
m_quality = cCRNDXTQualityUber;
m_perceptual = true;
m_dithering = false;
m_grayscale_sampling = false;
m_use_both_block_types = true;
m_endpoint_caching = true;
m_compressor = cCRNDXTCompressorCRN;
m_pProgress_callback = NULL;
m_pProgress_callback_user_data_ptr = NULL;
m_dxt1a_alpha_threshold = 128;
m_num_helper_threads = 0;
m_progress_start = 0;
m_progress_range = 100;
m_use_transparent_indices_for_black = false;
m_pTask_pool = NULL;
}
void init(const crn_comp_params& params) {
m_perceptual = (params.m_flags & cCRNCompFlagPerceptual) != 0;
m_num_helper_threads = params.m_num_helper_threads;
m_use_both_block_types = (params.m_flags & cCRNCompFlagUseBothBlockTypes) != 0;
m_use_transparent_indices_for_black = (params.m_flags & cCRNCompFlagUseTransparentIndicesForBlack) != 0;
m_dxt1a_alpha_threshold = params.m_dxt1a_alpha_threshold;
m_quality = params.m_dxt_quality;
m_endpoint_caching = (params.m_flags & cCRNCompFlagDisableEndpointCaching) == 0;
m_grayscale_sampling = (params.m_flags & cCRNCompFlagGrayscaleSampling) != 0;
m_compressor = params.m_dxt_compressor_type;
}
uint m_dxt1a_alpha_threshold;
uint m_num_helper_threads;
crn_dxt_quality m_quality;
crn_dxt_compressor_type m_compressor;
bool m_perceptual;
bool m_dithering;
bool m_grayscale_sampling;
bool m_use_both_block_types;
bool m_endpoint_caching;
bool m_use_transparent_indices_for_black;
typedef bool (*progress_callback_func)(uint percentage_complete, void* pUser_data_ptr);
progress_callback_func m_pProgress_callback;
void* m_pProgress_callback_user_data_ptr;
uint m_progress_start;
uint m_progress_range;
task_pool* m_pTask_pool;
};
bool init(dxt_format fmt, const image_u8& img, const pack_params& p = dxt_image::pack_params());
bool unpack(image_u8& img) const;
void endian_swap();
uint get_total_elements() const { return m_elements.size(); }
const element_vec& get_element_vec() const { return m_elements; }
element_vec& get_element_vec() { return m_elements; }
const element& get_element(uint block_x, uint block_y, uint element_index) const;
element& get_element(uint block_x, uint block_y, uint element_index);
const element* get_element_ptr() const { return m_pElements; }
element* get_element_ptr() { return m_pElements; }
uint get_size_in_bytes() const { return m_elements.size() * sizeof(element); }
uint get_row_pitch_in_bytes() const { return m_blocks_x * m_bytes_per_block; }
color_quad_u8 get_pixel(uint x, uint y) const;
uint get_pixel_alpha(uint x, uint y, uint element_index) const;
void set_pixel(uint x, uint y, const color_quad_u8& c, bool perceptual = true);
// get_block_pixels() only sets those components stored in the image!
bool get_block_pixels(uint block_x, uint block_y, color_quad_u8* pPixels) const;
struct set_block_pixels_context {
dxt1_endpoint_optimizer m_dxt1_optimizer;
dxt5_endpoint_optimizer m_dxt5_optimizer;
pack_etc1_block_context m_etc1_optimizer;
#if CRNLIB_SUPPORT_ETC_A1 #if CRNLIB_SUPPORT_ETC_A1
etc_a1::pack_etc1_block_context m_etc1_a1_optimizer; etc_a1::pack_etc1_block_context m_etc1_a1_optimizer;
#endif #endif
}; };
void set_block_pixels(uint block_x, uint block_y, const color_quad_u8* pPixels, const pack_params& p, set_block_pixels_context& context);
void set_block_pixels(uint block_x, uint block_y, const color_quad_u8* pPixels, const pack_params& p);
void get_block_endpoints(uint block_x, uint block_y, uint element_index, uint& packed_low_endpoint, uint& packed_high_endpoint) const;
// Returns a value representing the component(s) that where actually set, where -1 = RGB.
// This method does not always set every component!
int get_block_endpoints(uint block_x, uint block_y, uint element_index, color_quad_u8& low_endpoint, color_quad_u8& high_endpoint, bool scaled = true) const;
// pColors should point to a 16 entry array, to handle DXT3.
// Returns the number of block colors: 3, 4, 6, 8, or 16.
uint get_block_colors(uint block_x, uint block_y, uint element_index, color_quad_u8* pColors, uint subblock_index = 0);
uint get_subblock_index(uint x, uint y, uint element_index) const;
uint get_total_subblocks(uint element_index) const;
uint get_selector(uint x, uint y, uint element_index) const;
void change_dxt1_to_dxt1a();
bool can_flip(uint axis_index); void set_block_pixels(uint block_x, uint block_y, const color_quad_u8* pPixels, const pack_params& p, set_block_pixels_context& context);
void set_block_pixels(uint block_x, uint block_y, const color_quad_u8* pPixels, const pack_params& p);
// Returns true if the texture can actually be flipped. void get_block_endpoints(uint block_x, uint block_y, uint element_index, uint& packed_low_endpoint, uint& packed_high_endpoint) const;
bool flip_x();
bool flip_y();
private:
element_vec m_elements;
element* m_pElements;
uint m_width;
uint m_height;
uint m_blocks_x;
uint m_blocks_y;
uint m_total_blocks;
uint m_total_elements;
uint m_num_elements_per_block; // 1 or 2
uint m_bytes_per_block; // 8 or 16
int8 m_element_component_index[2];
element_type m_element_type[2];
dxt_format m_format; // DXT1, 1A, 3, 5, N/3DC, or 5A
bool init_internal(dxt_format fmt, uint width, uint height);
void init_task(uint64 data, void* pData_ptr);
#if CRNLIB_SUPPORT_ATI_COMPRESS // Returns a value representing the component(s) that where actually set, where -1 = RGB.
bool init_ati_compress(dxt_format fmt, const image_u8& img, const pack_params& p); // This method does not always set every component!
#endif int get_block_endpoints(uint block_x, uint block_y, uint element_index, color_quad_u8& low_endpoint, color_quad_u8& high_endpoint, bool scaled = true) const;
void flip_col(uint x); // pColors should point to a 16 entry array, to handle DXT3.
void flip_row(uint y); // Returns the number of block colors: 3, 4, 6, 8, or 16.
}; uint get_block_colors(uint block_x, uint block_y, uint element_index, color_quad_u8* pColors, uint subblock_index = 0);
} // namespace crnlib uint get_subblock_index(uint x, uint y, uint element_index) const;
uint get_total_subblocks(uint element_index) const;
uint get_selector(uint x, uint y, uint element_index) const;
void change_dxt1_to_dxt1a();
bool can_flip(uint axis_index);
// Returns true if the texture can actually be flipped.
bool flip_x();
bool flip_y();
private:
element_vec m_elements;
element* m_pElements;
uint m_width;
uint m_height;
uint m_blocks_x;
uint m_blocks_y;
uint m_total_blocks;
uint m_total_elements;
uint m_num_elements_per_block; // 1 or 2
uint m_bytes_per_block; // 8 or 16
int8 m_element_component_index[2];
element_type m_element_type[2];
dxt_format m_format; // DXT1, 1A, 3, 5, N/3DC, or 5A
bool init_internal(dxt_format fmt, uint width, uint height);
void init_task(uint64 data, void* pData_ptr);
#if CRNLIB_SUPPORT_ATI_COMPRESS
bool init_ati_compress(dxt_format fmt, const image_u8& img, const pack_params& p);
#endif
void flip_col(uint x);
void flip_row(uint y);
};
} // namespace crnlib
crnlib/crn_dynamic_stream.h
+139 -163
View File
@@ -3,204 +3,180 @@
#pragma once #pragma once
#include "crn_data_stream.h" #include "crn_data_stream.h"
namespace crnlib namespace crnlib {
{ class dynamic_stream : public data_stream {
class dynamic_stream : public data_stream public:
{ dynamic_stream(uint initial_size, const char* pName = "dynamic_stream", uint attribs = cDataStreamSeekable | cDataStreamWritable | cDataStreamReadable)
public: : data_stream(pName, attribs),
dynamic_stream(uint initial_size, const char* pName = "dynamic_stream", uint attribs = cDataStreamSeekable | cDataStreamWritable | cDataStreamReadable) : m_ofs(0) {
data_stream(pName, attribs), open(initial_size, pName, attribs);
m_ofs(0) }
{
open(initial_size, pName, attribs); dynamic_stream(const void* pBuf, uint size, const char* pName = "dynamic_stream", uint attribs = cDataStreamSeekable | cDataStreamWritable | cDataStreamReadable)
: data_stream(pName, attribs),
m_ofs(0) {
open(pBuf, size, pName, attribs);
}
dynamic_stream()
: data_stream(),
m_ofs(0) {
open();
}
virtual ~dynamic_stream() {
}
bool open(uint initial_size = 0, const char* pName = "dynamic_stream", uint attribs = cDataStreamSeekable | cDataStreamWritable | cDataStreamReadable) {
close();
m_opened = true;
m_buf.clear();
m_buf.resize(initial_size);
m_ofs = 0;
m_name.set(pName ? pName : "dynamic_stream");
m_attribs = static_cast<attribs_t>(attribs);
return true;
}
bool reopen(const char* pName, uint attribs) {
if (!m_opened) {
return open(0, pName, attribs);
}
m_name.set(pName ? pName : "dynamic_stream");
m_attribs = static_cast<attribs_t>(attribs);
return true;
}
bool open(const void* pBuf, uint size, const char* pName = "dynamic_stream", uint attribs = cDataStreamSeekable | cDataStreamWritable | cDataStreamReadable) {
if (!m_opened) {
m_opened = true;
m_buf.resize(size);
if (size) {
CRNLIB_ASSERT(pBuf);
memcpy(&m_buf[0], pBuf, size);
} }
m_ofs = 0;
m_name.set(pName ? pName : "dynamic_stream");
m_attribs = static_cast<attribs_t>(attribs);
return true;
}
dynamic_stream(const void* pBuf, uint size, const char* pName = "dynamic_stream", uint attribs = cDataStreamSeekable | cDataStreamWritable | cDataStreamReadable) : return false;
data_stream(pName, attribs), }
m_ofs(0)
{
open(pBuf, size, pName, attribs);
}
dynamic_stream() : virtual bool close() {
data_stream(), if (m_opened) {
m_ofs(0) m_opened = false;
{ m_buf.clear();
open(); m_ofs = 0;
} return true;
}
virtual ~dynamic_stream() return false;
{ }
}
bool open(uint initial_size = 0, const char* pName = "dynamic_stream", uint attribs = cDataStreamSeekable | cDataStreamWritable | cDataStreamReadable) const crnlib::vector<uint8>& get_buf() const { return m_buf; }
{ crnlib::vector<uint8>& get_buf() { return m_buf; }
close();
m_opened = true; void reserve(uint size) {
m_buf.clear(); if (m_opened) {
m_buf.resize(initial_size); m_buf.reserve(size);
m_ofs = 0; }
m_name.set(pName ? pName : "dynamic_stream"); }
m_attribs = static_cast<attribs_t>(attribs);
return true;
}
bool reopen(const char* pName, uint attribs) virtual const void* get_ptr() const { return m_buf.empty() ? NULL : &m_buf[0]; }
{
if (!m_opened)
{
return open(0, pName, attribs);
}
m_name.set(pName ? pName : "dynamic_stream"); virtual uint read(void* pBuf, uint len) {
m_attribs = static_cast<attribs_t>(attribs); CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF));
return true;
}
bool open(const void* pBuf, uint size, const char* pName = "dynamic_stream", uint attribs = cDataStreamSeekable | cDataStreamWritable | cDataStreamReadable) if ((!m_opened) || (!is_readable()) || (!len))
{ return 0;
if (!m_opened)
{
m_opened = true;
m_buf.resize(size);
if (size)
{
CRNLIB_ASSERT(pBuf);
memcpy(&m_buf[0], pBuf, size);
}
m_ofs = 0;
m_name.set(pName ? pName : "dynamic_stream");
m_attribs = static_cast<attribs_t>(attribs);
return true;
}
return false; CRNLIB_ASSERT(m_ofs <= m_buf.size());
}
virtual bool close() uint bytes_left = m_buf.size() - m_ofs;
{
if (m_opened)
{
m_opened = false;
m_buf.clear();
m_ofs = 0;
return true;
}
return false; len = math::minimum<uint>(len, bytes_left);
}
const crnlib::vector<uint8>& get_buf() const { return m_buf; } if (len)
crnlib::vector<uint8>& get_buf() { return m_buf; } memcpy(pBuf, &m_buf[m_ofs], len);
void reserve(uint size) m_ofs += len;
{
if (m_opened)
{
m_buf.reserve(size);
}
}
virtual const void* get_ptr() const { return m_buf.empty() ? NULL : &m_buf[0]; } return len;
}
virtual uint read(void* pBuf, uint len) virtual uint write(const void* pBuf, uint len) {
{ CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF));
CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF));
if ((!m_opened) || (!is_readable()) || (!len)) if ((!m_opened) || (!is_writable()) || (!len))
return 0; return 0;
CRNLIB_ASSERT(m_ofs <= m_buf.size()); CRNLIB_ASSERT(m_ofs <= m_buf.size());
uint bytes_left = m_buf.size() - m_ofs; uint new_ofs = m_ofs + len;
if (new_ofs > m_buf.size())
m_buf.resize(new_ofs);
len = math::minimum<uint>(len, bytes_left); memcpy(&m_buf[m_ofs], pBuf, len);
m_ofs = new_ofs;
if (len) return len;
memcpy(pBuf, &m_buf[m_ofs], len); }
m_ofs += len; virtual bool flush() {
if (!m_opened)
return false;
return len; return true;
} }
virtual uint write(const void* pBuf, uint len) virtual uint64 get_size() {
{ if (!m_opened)
CRNLIB_ASSERT(pBuf && (len <= 0x7FFFFFFF)); return 0;
if ((!m_opened) || (!is_writable()) || (!len)) return m_buf.size();
return 0; }
CRNLIB_ASSERT(m_ofs <= m_buf.size()); virtual uint64 get_remaining() {
if (!m_opened)
return 0;
uint new_ofs = m_ofs + len; CRNLIB_ASSERT(m_ofs <= m_buf.size());
if (new_ofs > m_buf.size())
m_buf.resize(new_ofs);
memcpy(&m_buf[m_ofs], pBuf, len); return m_buf.size() - m_ofs;
m_ofs = new_ofs; }
return len; virtual uint64 get_ofs() {
} if (!m_opened)
return 0;
virtual bool flush() return m_ofs;
{ }
if (!m_opened)
return false;
return true; virtual bool seek(int64 ofs, bool relative) {
} if ((!m_opened) || (!is_seekable()))
return false;
virtual uint64 get_size() int64 new_ofs = relative ? (m_ofs + ofs) : ofs;
{
if (!m_opened)
return 0;
return m_buf.size(); if (new_ofs < 0)
} return false;
else if (new_ofs > m_buf.size())
return false;
virtual uint64 get_remaining() m_ofs = static_cast<uint>(new_ofs);
{
if (!m_opened)
return 0;
CRNLIB_ASSERT(m_ofs <= m_buf.size()); post_seek();
return m_buf.size() - m_ofs; return true;
} }
virtual uint64 get_ofs() private:
{ crnlib::vector<uint8> m_buf;
if (!m_opened) uint m_ofs;
return 0; };
return m_ofs;
}
virtual bool seek(int64 ofs, bool relative)
{
if ((!m_opened) || (!is_seekable()))
return false;
int64 new_ofs = relative ? (m_ofs + ofs) : ofs;
if (new_ofs < 0)
return false;
else if (new_ofs > m_buf.size())
return false;
m_ofs = static_cast<uint>(new_ofs);
post_seek();
return true;
}
private:
crnlib::vector<uint8> m_buf;
uint m_ofs;
};
} // namespace crnlib
} // namespace crnlib
crnlib/crn_dynamic_string.cpp
+494 -579
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_dynamic_string.h
+131 -119
View File
@@ -2,172 +2,184 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
namespace crnlib namespace crnlib {
{ enum { cMaxDynamicStringLen = cUINT16_MAX - 1 };
enum { cMaxDynamicStringLen = cUINT16_MAX - 1 }; class dynamic_string {
class dynamic_string public:
{ inline dynamic_string()
public: : m_buf_size(0), m_len(0), m_pStr(NULL) {}
inline dynamic_string() : m_buf_size(0), m_len(0), m_pStr(NULL) { } dynamic_string(eVarArg dummy, const char* p, ...);
dynamic_string(eVarArg dummy, const char* p, ...); dynamic_string(const char* p);
dynamic_string(const char* p); dynamic_string(const char* p, uint len);
dynamic_string(const char* p, uint len); dynamic_string(const dynamic_string& other);
dynamic_string(const dynamic_string& other);
inline ~dynamic_string() { if (m_pStr) crnlib_delete_array(m_pStr); } inline ~dynamic_string() {
if (m_pStr)
crnlib_delete_array(m_pStr);
}
// Truncates the string to 0 chars and frees the buffer. // Truncates the string to 0 chars and frees the buffer.
void clear(); void clear();
void optimize(); void optimize();
// Truncates the string to 0 chars, but does not free the buffer. // Truncates the string to 0 chars, but does not free the buffer.
void empty(); void empty();
inline const char *assume_ownership() { const char *p = m_pStr; m_pStr = NULL; m_len = 0; m_buf_size = 0; return p; } inline const char* assume_ownership() {
const char* p = m_pStr;
m_pStr = NULL;
m_len = 0;
m_buf_size = 0;
return p;
}
inline uint get_len() const { return m_len; } inline uint get_len() const { return m_len; }
inline bool is_empty() const { return !m_len; } inline bool is_empty() const { return !m_len; }
inline const char* get_ptr() const { return m_pStr ? m_pStr : ""; } inline const char* get_ptr() const { return m_pStr ? m_pStr : ""; }
inline const char* c_str() const { return get_ptr(); } inline const char* c_str() const { return get_ptr(); }
inline const char* get_ptr_raw() const { return m_pStr; } inline const char* get_ptr_raw() const { return m_pStr; }
inline char* get_ptr_raw() { return m_pStr; } inline char* get_ptr_raw() { return m_pStr; }
inline char front() const { return m_len ? m_pStr[0] : '\0'; } inline char front() const { return m_len ? m_pStr[0] : '\0'; }
inline char back() const { return m_len ? m_pStr[m_len - 1] : '\0'; } inline char back() const { return m_len ? m_pStr[m_len - 1] : '\0'; }
inline char operator[] (uint i) const { CRNLIB_ASSERT(i <= m_len); return get_ptr()[i]; } inline char operator[](uint i) const {
CRNLIB_ASSERT(i <= m_len);
return get_ptr()[i];
}
inline operator size_t() const { return fast_hash(get_ptr(), m_len) ^ fast_hash(&m_len, sizeof(m_len)); } inline operator size_t() const { return fast_hash(get_ptr(), m_len) ^ fast_hash(&m_len, sizeof(m_len)); }
int compare(const char* p, bool case_sensitive = false) const; int compare(const char* p, bool case_sensitive = false) const;
int compare(const dynamic_string& rhs, bool case_sensitive = false) const; int compare(const dynamic_string& rhs, bool case_sensitive = false) const;
inline bool operator== (const dynamic_string& rhs) const { return compare(rhs) == 0; } inline bool operator==(const dynamic_string& rhs) const { return compare(rhs) == 0; }
inline bool operator== (const char* p) const { return compare(p) == 0; } inline bool operator==(const char* p) const { return compare(p) == 0; }
inline bool operator!= (const dynamic_string& rhs) const { return compare(rhs) != 0; } inline bool operator!=(const dynamic_string& rhs) const { return compare(rhs) != 0; }
inline bool operator!= (const char* p) const { return compare(p) != 0; } inline bool operator!=(const char* p) const { return compare(p) != 0; }
inline bool operator< (const dynamic_string& rhs) const { return compare(rhs) < 0; } inline bool operator<(const dynamic_string& rhs) const { return compare(rhs) < 0; }
inline bool operator< (const char* p) const { return compare(p) < 0; } inline bool operator<(const char* p) const { return compare(p) < 0; }
inline bool operator> (const dynamic_string& rhs) const { return compare(rhs) > 0; } inline bool operator>(const dynamic_string& rhs) const { return compare(rhs) > 0; }
inline bool operator> (const char* p) const { return compare(p) > 0; } inline bool operator>(const char* p) const { return compare(p) > 0; }
inline bool operator<= (const dynamic_string& rhs) const { return compare(rhs) <= 0; } inline bool operator<=(const dynamic_string& rhs) const { return compare(rhs) <= 0; }
inline bool operator<= (const char* p) const { return compare(p) <= 0; } inline bool operator<=(const char* p) const { return compare(p) <= 0; }
inline bool operator>= (const dynamic_string& rhs) const { return compare(rhs) >= 0; } inline bool operator>=(const dynamic_string& rhs) const { return compare(rhs) >= 0; }
inline bool operator>= (const char* p) const { return compare(p) >= 0; } inline bool operator>=(const char* p) const { return compare(p) >= 0; }
friend inline bool operator== (const char* p, const dynamic_string& rhs) { return rhs.compare(p) == 0; } friend inline bool operator==(const char* p, const dynamic_string& rhs) { return rhs.compare(p) == 0; }
dynamic_string& set(const char* p, uint max_len = UINT_MAX); dynamic_string& set(const char* p, uint max_len = UINT_MAX);
dynamic_string& set(const dynamic_string& other, uint max_len = UINT_MAX); dynamic_string& set(const dynamic_string& other, uint max_len = UINT_MAX);
bool set_len(uint new_len, char fill_char = ' '); bool set_len(uint new_len, char fill_char = ' ');
// Set from non-zero terminated buffer. // Set from non-zero terminated buffer.
dynamic_string& set_from_buf(const void* pBuf, uint buf_size); dynamic_string& set_from_buf(const void* pBuf, uint buf_size);
dynamic_string& operator= (const dynamic_string& rhs) { return set(rhs); } dynamic_string& operator=(const dynamic_string& rhs) { return set(rhs); }
dynamic_string& operator= (const char* p) { return set(p); } dynamic_string& operator=(const char* p) { return set(p); }
dynamic_string& set_char(uint index, char c); dynamic_string& set_char(uint index, char c);
dynamic_string& append_char(char c); dynamic_string& append_char(char c);
dynamic_string& append_char(int c) { CRNLIB_ASSERT((c >= 0) && (c <= 255)); return append_char(static_cast<char>(c)); } dynamic_string& append_char(int c) {
dynamic_string& truncate(uint new_len); CRNLIB_ASSERT((c >= 0) && (c <= 255));
dynamic_string& tolower(); return append_char(static_cast<char>(c));
dynamic_string& toupper(); }
dynamic_string& truncate(uint new_len);
dynamic_string& tolower();
dynamic_string& toupper();
dynamic_string& append(const char* p); dynamic_string& append(const char* p);
dynamic_string& append(const dynamic_string& other); dynamic_string& append(const dynamic_string& other);
dynamic_string& operator += (const char* p) { return append(p); } dynamic_string& operator+=(const char* p) { return append(p); }
dynamic_string& operator += (const dynamic_string& other) { return append(other); } dynamic_string& operator+=(const dynamic_string& other) { return append(other); }
friend dynamic_string operator+ (const char* p, const dynamic_string& a); friend dynamic_string operator+(const char* p, const dynamic_string& a);
friend dynamic_string operator+ (const dynamic_string& a, const char* p); friend dynamic_string operator+(const dynamic_string& a, const char* p);
friend dynamic_string operator+ (const dynamic_string& a, const dynamic_string& b); friend dynamic_string operator+(const dynamic_string& a, const dynamic_string& b);
dynamic_string& format_args(const char* p, va_list args); dynamic_string& format_args(const char* p, va_list args);
dynamic_string& format(const char* p, ...); dynamic_string& format(const char* p, ...);
dynamic_string& crop(uint start, uint len); dynamic_string& crop(uint start, uint len);
dynamic_string& substring(uint start, uint end); dynamic_string& substring(uint start, uint end);
dynamic_string& left(uint len); dynamic_string& left(uint len);
dynamic_string& mid(uint start, uint len); dynamic_string& mid(uint start, uint len);
dynamic_string& right(uint start); dynamic_string& right(uint start);
dynamic_string& tail(uint num); dynamic_string& tail(uint num);
dynamic_string& unquote(); dynamic_string& unquote();
uint count_char(char c) const; uint count_char(char c) const;
int find_left(const char* p, bool case_sensitive = false) const; int find_left(const char* p, bool case_sensitive = false) const;
int find_left(char c) const; int find_left(char c) const;
int find_right(char c) const; int find_right(char c) const;
int find_right(const char* p, bool case_sensitive = false) const; int find_right(const char* p, bool case_sensitive = false) const;
bool contains(const char* p, bool case_sensitive = false) const; bool contains(const char* p, bool case_sensitive = false) const;
dynamic_string& trim(); dynamic_string& trim();
dynamic_string& trim_crlf(); dynamic_string& trim_crlf();
dynamic_string& remap(int from_char, int to_char); dynamic_string& remap(int from_char, int to_char);
void swap(dynamic_string& other); void swap(dynamic_string& other);
// Returns -1 on failure, or the number of bytes written. // Returns -1 on failure, or the number of bytes written.
int serialize(void* pBuf, uint buf_size, bool little_endian) const; int serialize(void* pBuf, uint buf_size, bool little_endian) const;
// Returns -1 on failure, or the number of bytes read. // Returns -1 on failure, or the number of bytes read.
int deserialize(const void* pBuf, uint buf_size, bool little_endian); int deserialize(const void* pBuf, uint buf_size, bool little_endian);
void translate_lf_to_crlf(); void translate_lf_to_crlf();
static inline char *create_raw_buffer(uint& buf_size_in_chars); static inline char* create_raw_buffer(uint& buf_size_in_chars);
static inline void free_raw_buffer(char *p) { crnlib_delete_array(p); } static inline void free_raw_buffer(char* p) { crnlib_delete_array(p); }
dynamic_string& set_from_raw_buf_and_assume_ownership(char *pBuf, uint buf_size_in_chars, uint len_in_chars); dynamic_string& set_from_raw_buf_and_assume_ownership(char* pBuf, uint buf_size_in_chars, uint len_in_chars);
private:
uint16 m_buf_size; private:
uint16 m_len; uint16 m_buf_size;
char* m_pStr; uint16 m_len;
char* m_pStr;
#ifdef CRNLIB_BUILD_DEBUG #ifdef CRNLIB_BUILD_DEBUG
void check() const; void check() const;
#else #else
inline void check() const { } inline void check() const {}
#endif #endif
bool expand_buf(uint new_buf_size, bool preserve_contents); bool expand_buf(uint new_buf_size, bool preserve_contents);
const char* get_ptr_priv() const { return m_pStr ? m_pStr : ""; } const char* get_ptr_priv() const { return m_pStr ? m_pStr : ""; }
char* get_ptr_priv() { return (char*)(m_pStr ? m_pStr : ""); } char* get_ptr_priv() { return (char*)(m_pStr ? m_pStr : ""); }
bool ensure_buf(uint len, bool preserve_contents = true); bool ensure_buf(uint len, bool preserve_contents = true);
}; };
typedef crnlib::vector<dynamic_string> dynamic_string_array; typedef crnlib::vector<dynamic_string> dynamic_string_array;
extern dynamic_string g_empty_dynamic_string; extern dynamic_string g_empty_dynamic_string;
CRNLIB_DEFINE_BITWISE_MOVABLE(dynamic_string); CRNLIB_DEFINE_BITWISE_MOVABLE(dynamic_string);
inline void swap (dynamic_string& a, dynamic_string& b) inline void swap(dynamic_string& a, dynamic_string& b) {
{ a.swap(b);
a.swap(b); }
}
inline char *dynamic_string::create_raw_buffer(uint& buf_size_in_chars) inline char* dynamic_string::create_raw_buffer(uint& buf_size_in_chars) {
{ if (buf_size_in_chars > cUINT16_MAX) {
if (buf_size_in_chars > cUINT16_MAX) CRNLIB_ASSERT(0);
{ return NULL;
CRNLIB_ASSERT(0); }
return NULL; buf_size_in_chars = math::minimum<uint>(cUINT16_MAX, math::next_pow2(buf_size_in_chars));
} return crnlib_new_array<char>(buf_size_in_chars);
buf_size_in_chars = math::minimum<uint>(cUINT16_MAX, math::next_pow2(buf_size_in_chars)); }
return crnlib_new_array<char>(buf_size_in_chars); } // namespace crnlib
}
} // namespace crnlib
crnlib/crn_etc.cpp
+1568 -1423
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_etc.h
+537 -603
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_file_utils.cpp
+405 -457
View File
@@ -18,561 +18,509 @@
#include <libgen.h> #include <libgen.h>
#endif #endif
namespace crnlib namespace crnlib {
{
#if CRNLIB_USE_WIN32_API #if CRNLIB_USE_WIN32_API
bool file_utils::is_read_only(const char* pFilename) bool file_utils::is_read_only(const char* pFilename) {
{ uint32 dst_file_attribs = GetFileAttributesA(pFilename);
uint32 dst_file_attribs = GetFileAttributesA(pFilename); if (dst_file_attribs == INVALID_FILE_ATTRIBUTES)
if (dst_file_attribs == INVALID_FILE_ATTRIBUTES) return false;
return false; if (dst_file_attribs & FILE_ATTRIBUTE_READONLY)
if (dst_file_attribs & FILE_ATTRIBUTE_READONLY) return true;
return true; return false;
return false; }
}
bool file_utils::disable_read_only(const char* pFilename)
{
uint32 dst_file_attribs = GetFileAttributesA(pFilename);
if (dst_file_attribs == INVALID_FILE_ATTRIBUTES)
return false;
if (dst_file_attribs & FILE_ATTRIBUTE_READONLY)
{
dst_file_attribs &= ~FILE_ATTRIBUTE_READONLY;
if (SetFileAttributesA(pFilename, dst_file_attribs))
return true;
}
return false;
}
bool file_utils::is_older_than(const char* pSrcFilename, const char* pDstFilename)
{
WIN32_FILE_ATTRIBUTE_DATA src_file_attribs;
const BOOL src_file_exists = GetFileAttributesExA(pSrcFilename, GetFileExInfoStandard, &src_file_attribs);
WIN32_FILE_ATTRIBUTE_DATA dst_file_attribs;
const BOOL dest_file_exists = GetFileAttributesExA(pDstFilename, GetFileExInfoStandard, &dst_file_attribs);
if ((dest_file_exists) && (src_file_exists))
{
LONG timeComp = CompareFileTime(&src_file_attribs.ftLastWriteTime, &dst_file_attribs.ftLastWriteTime);
if (timeComp < 0)
return true;
}
return false;
}
bool file_utils::does_file_exist(const char* pFilename)
{
const DWORD fullAttributes = GetFileAttributesA(pFilename);
if (fullAttributes == INVALID_FILE_ATTRIBUTES)
return false;
if (fullAttributes & FILE_ATTRIBUTE_DIRECTORY)
return false;
bool file_utils::disable_read_only(const char* pFilename) {
uint32 dst_file_attribs = GetFileAttributesA(pFilename);
if (dst_file_attribs == INVALID_FILE_ATTRIBUTES)
return false;
if (dst_file_attribs & FILE_ATTRIBUTE_READONLY) {
dst_file_attribs &= ~FILE_ATTRIBUTE_READONLY;
if (SetFileAttributesA(pFilename, dst_file_attribs))
return true; return true;
} }
return false;
}
bool file_utils::does_dir_exist(const char* pDir) bool file_utils::is_older_than(const char* pSrcFilename, const char* pDstFilename) {
{ WIN32_FILE_ATTRIBUTE_DATA src_file_attribs;
//-- Get the file attributes. const BOOL src_file_exists = GetFileAttributesExA(pSrcFilename, GetFileExInfoStandard, &src_file_attribs);
DWORD fullAttributes = GetFileAttributesA(pDir);
if (fullAttributes == INVALID_FILE_ATTRIBUTES) WIN32_FILE_ATTRIBUTE_DATA dst_file_attribs;
return false; const BOOL dest_file_exists = GetFileAttributesExA(pDstFilename, GetFileExInfoStandard, &dst_file_attribs);
if (fullAttributes & FILE_ATTRIBUTE_DIRECTORY)
return true;
return false;
}
bool file_utils::get_file_size(const char* pFilename, uint64& file_size)
{
file_size = 0;
WIN32_FILE_ATTRIBUTE_DATA attr;
if (0 == GetFileAttributesExA(pFilename, GetFileExInfoStandard, &attr))
return false;
if (attr.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)
return false;
file_size = static_cast<uint64>(attr.nFileSizeLow) | (static_cast<uint64>(attr.nFileSizeHigh) << 32U);
if ((dest_file_exists) && (src_file_exists)) {
LONG timeComp = CompareFileTime(&src_file_attribs.ftLastWriteTime, &dst_file_attribs.ftLastWriteTime);
if (timeComp < 0)
return true; return true;
} }
#elif defined( __GNUC__ ) return false;
bool file_utils::is_read_only(const char* pFilename) }
{
pFilename;
// TODO
return false;
}
bool file_utils::disable_read_only(const char* pFilename) bool file_utils::does_file_exist(const char* pFilename) {
{ const DWORD fullAttributes = GetFileAttributesA(pFilename);
pFilename;
// TODO
return false;
}
bool file_utils::is_older_than(const char *pSrcFilename, const char* pDstFilename) if (fullAttributes == INVALID_FILE_ATTRIBUTES)
{ return false;
pSrcFilename, pDstFilename;
// TODO
return false;
}
bool file_utils::does_file_exist(const char* pFilename) if (fullAttributes & FILE_ATTRIBUTE_DIRECTORY)
{ return false;
struct stat stat_buf;
int result = stat(pFilename, &stat_buf);
if (result)
return false;
if (S_ISREG(stat_buf.st_mode))
return true;
return false;
}
bool file_utils::does_dir_exist(const char* pDir) return true;
{ }
struct stat stat_buf;
int result = stat(pDir, &stat_buf);
if (result)
return false;
if (S_ISDIR(stat_buf.st_mode) || S_ISLNK(stat_buf.st_mode))
return true;
return false;
}
bool file_utils::get_file_size(const char* pFilename, uint64& file_size) bool file_utils::does_dir_exist(const char* pDir) {
{ //-- Get the file attributes.
file_size = 0; DWORD fullAttributes = GetFileAttributesA(pDir);
struct stat stat_buf;
int result = stat(pFilename, &stat_buf); if (fullAttributes == INVALID_FILE_ATTRIBUTES)
if (result) return false;
return false;
if (!S_ISREG(stat_buf.st_mode)) if (fullAttributes & FILE_ATTRIBUTE_DIRECTORY)
return false; return true;
file_size = stat_buf.st_size;
return true; return false;
} }
bool file_utils::get_file_size(const char* pFilename, uint64& file_size) {
file_size = 0;
WIN32_FILE_ATTRIBUTE_DATA attr;
if (0 == GetFileAttributesExA(pFilename, GetFileExInfoStandard, &attr))
return false;
if (attr.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)
return false;
file_size = static_cast<uint64>(attr.nFileSizeLow) | (static_cast<uint64>(attr.nFileSizeHigh) << 32U);
return true;
}
#elif defined(__GNUC__)
bool file_utils::is_read_only(const char* pFilename) {
pFilename;
// TODO
return false;
}
bool file_utils::disable_read_only(const char* pFilename) {
pFilename;
// TODO
return false;
}
bool file_utils::is_older_than(const char* pSrcFilename, const char* pDstFilename) {
pSrcFilename, pDstFilename;
// TODO
return false;
}
bool file_utils::does_file_exist(const char* pFilename) {
struct stat stat_buf;
int result = stat(pFilename, &stat_buf);
if (result)
return false;
if (S_ISREG(stat_buf.st_mode))
return true;
return false;
}
bool file_utils::does_dir_exist(const char* pDir) {
struct stat stat_buf;
int result = stat(pDir, &stat_buf);
if (result)
return false;
if (S_ISDIR(stat_buf.st_mode) || S_ISLNK(stat_buf.st_mode))
return true;
return false;
}
bool file_utils::get_file_size(const char* pFilename, uint64& file_size) {
file_size = 0;
struct stat stat_buf;
int result = stat(pFilename, &stat_buf);
if (result)
return false;
if (!S_ISREG(stat_buf.st_mode))
return false;
file_size = stat_buf.st_size;
return true;
}
#else #else
bool file_utils::is_read_only(const char* pFilename) bool file_utils::is_read_only(const char* pFilename) {
{ return false;
return false; }
}
bool file_utils::disable_read_only(const char* pFilename) bool file_utils::disable_read_only(const char* pFilename) {
{ pFilename;
pFilename; // TODO
// TODO return false;
return false; }
}
bool file_utils::is_older_than(const char *pSrcFilename, const char* pDstFilename) bool file_utils::is_older_than(const char* pSrcFilename, const char* pDstFilename) {
{ return false;
return false; }
}
bool file_utils::does_file_exist(const char* pFilename) bool file_utils::does_file_exist(const char* pFilename) {
{ FILE* pFile;
FILE* pFile; crn_fopen(&pFile, pFilename, "rb");
crn_fopen(&pFile, pFilename, "rb"); if (!pFile)
if (!pFile) return false;
return false; fclose(pFile);
fclose(pFile); return true;
return true; }
}
bool file_utils::does_dir_exist(const char* pDir) bool file_utils::does_dir_exist(const char* pDir) {
{ return false;
return false; }
}
bool file_utils::get_file_size(const char* pFilename, uint64& file_size) bool file_utils::get_file_size(const char* pFilename, uint64& file_size) {
{ FILE* pFile;
FILE* pFile; crn_fopen(&pFile, pFilename, "rb");
crn_fopen(&pFile, pFilename, "rb"); if (!pFile)
if (!pFile) return false;
return false; crn_fseek(pFile, 0, SEEK_END);
crn_fseek(pFile, 0, SEEK_END); file_size = crn_ftell(pFile);
file_size = crn_ftell(pFile); fclose(pFile);
fclose(pFile); return true;
return true; }
}
#endif #endif
bool file_utils::get_file_size(const char* pFilename, uint32& file_size) bool file_utils::get_file_size(const char* pFilename, uint32& file_size) {
{ uint64 file_size64;
uint64 file_size64; if (!get_file_size(pFilename, file_size64)) {
if (!get_file_size(pFilename, file_size64)) file_size = 0;
{ return false;
file_size = 0; }
return false;
}
if (file_size64 > cUINT32_MAX) if (file_size64 > cUINT32_MAX)
file_size64 = cUINT32_MAX; file_size64 = cUINT32_MAX;
file_size = static_cast<uint32>(file_size64); file_size = static_cast<uint32>(file_size64);
return true; return true;
} }
bool file_utils::is_path_separator(char c) bool file_utils::is_path_separator(char c) {
{
#ifdef WIN32 #ifdef WIN32
return (c == '/') || (c == '\\'); return (c == '/') || (c == '\\');
#else #else
return (c == '/'); return (c == '/');
#endif #endif
} }
bool file_utils::is_path_or_drive_separator(char c) bool file_utils::is_path_or_drive_separator(char c) {
{
#ifdef WIN32 #ifdef WIN32
return (c == '/') || (c == '\\') || (c == ':'); return (c == '/') || (c == '\\') || (c == ':');
#else #else
return (c == '/'); return (c == '/');
#endif #endif
} }
bool file_utils::is_drive_separator(char c) bool file_utils::is_drive_separator(char c) {
{
#ifdef WIN32 #ifdef WIN32
return (c == ':'); return (c == ':');
#else #else
c; c;
return false; return false;
#endif #endif
} }
bool file_utils::split_path(const char* p, dynamic_string* pDrive, dynamic_string* pDir, dynamic_string* pFilename, dynamic_string* pExt) bool file_utils::split_path(const char* p, dynamic_string* pDrive, dynamic_string* pDir, dynamic_string* pFilename, dynamic_string* pExt) {
{ CRNLIB_ASSERT(p);
CRNLIB_ASSERT(p);
#ifdef WIN32 #ifdef WIN32
char drive_buf[_MAX_DRIVE]; char drive_buf[_MAX_DRIVE];
char dir_buf[_MAX_DIR]; char dir_buf[_MAX_DIR];
char fname_buf[_MAX_FNAME]; char fname_buf[_MAX_FNAME];
char ext_buf[_MAX_EXT]; char ext_buf[_MAX_EXT];
#ifdef _MSC_VER #ifdef _MSC_VER
// Compiling with MSVC // Compiling with MSVC
errno_t error = _splitpath_s(p, errno_t error = _splitpath_s(p,
pDrive ? drive_buf : NULL, pDrive ? _MAX_DRIVE : 0, pDrive ? drive_buf : NULL, pDrive ? _MAX_DRIVE : 0,
pDir ? dir_buf : NULL, pDir ? _MAX_DIR : 0, pDir ? dir_buf : NULL, pDir ? _MAX_DIR : 0,
pFilename ? fname_buf : NULL, pFilename ? _MAX_FNAME : 0, pFilename ? fname_buf : NULL, pFilename ? _MAX_FNAME : 0,
pExt ? ext_buf : NULL, pExt ? _MAX_EXT : 0); pExt ? ext_buf : NULL, pExt ? _MAX_EXT : 0);
if (error != 0) if (error != 0)
return false; return false;
#else #else
// Compiling with MinGW // Compiling with MinGW
_splitpath(p, _splitpath(p,
pDrive ? drive_buf : NULL, pDrive ? drive_buf : NULL,
pDir ? dir_buf : NULL, pDir ? dir_buf : NULL,
pFilename ? fname_buf : NULL, pFilename ? fname_buf : NULL,
pExt ? ext_buf : NULL); pExt ? ext_buf : NULL);
#endif #endif
if (pDrive) *pDrive = drive_buf; if (pDrive)
if (pDir) *pDir = dir_buf; *pDrive = drive_buf;
if (pFilename) *pFilename = fname_buf; if (pDir)
if (pExt) *pExt = ext_buf; *pDir = dir_buf;
if (pFilename)
*pFilename = fname_buf;
if (pExt)
*pExt = ext_buf;
#else #else
char dirtmp[1024]; char dirtmp[1024];
char nametmp[1024]; char nametmp[1024];
strcpy_safe(dirtmp, sizeof(dirtmp), p); strcpy_safe(dirtmp, sizeof(dirtmp), p);
strcpy_safe(nametmp, sizeof(nametmp), p); strcpy_safe(nametmp, sizeof(nametmp), p);
if (pDrive) pDrive->clear(); if (pDrive)
pDrive->clear();
const char *pDirName = dirname(dirtmp); const char* pDirName = dirname(dirtmp);
if (!pDirName) if (!pDirName)
return false; return false;
if (pDir) if (pDir) {
{ pDir->set(pDirName);
pDir->set(pDirName); if ((!pDir->is_empty()) && (pDir->back() != '/'))
if ((!pDir->is_empty()) && (pDir->back() != '/')) pDir->append_char('/');
pDir->append_char('/'); }
}
const char *pBaseName = basename(nametmp); const char* pBaseName = basename(nametmp);
if (!pBaseName) if (!pBaseName)
return false; return false;
if (pFilename) if (pFilename) {
{ pFilename->set(pBaseName);
pFilename->set(pBaseName); remove_extension(*pFilename);
remove_extension(*pFilename); }
}
if (pExt) if (pExt) {
{ pExt->set(pBaseName);
pExt->set(pBaseName); get_extension(*pExt);
get_extension(*pExt); *pExt = "." + *pExt;
*pExt = "." + *pExt; }
} #endif // #ifdef WIN32
#endif // #ifdef WIN32
return true; return true;
} }
bool file_utils::split_path(const char* p, dynamic_string& path, dynamic_string& filename) bool file_utils::split_path(const char* p, dynamic_string& path, dynamic_string& filename) {
{ dynamic_string temp_drive, temp_path, temp_ext;
dynamic_string temp_drive, temp_path, temp_ext; if (!split_path(p, &temp_drive, &temp_path, &filename, &temp_ext))
if (!split_path(p, &temp_drive, &temp_path, &filename, &temp_ext)) return false;
return false;
filename += temp_ext; filename += temp_ext;
combine_path(path, temp_drive.get_ptr(), temp_path.get_ptr()); combine_path(path, temp_drive.get_ptr(), temp_path.get_ptr());
return true; return true;
} }
bool file_utils::get_pathname(const char* p, dynamic_string& path) bool file_utils::get_pathname(const char* p, dynamic_string& path) {
{ dynamic_string temp_drive, temp_path;
dynamic_string temp_drive, temp_path; if (!split_path(p, &temp_drive, &temp_path, NULL, NULL))
if (!split_path(p, &temp_drive, &temp_path, NULL, NULL)) return false;
return false;
combine_path(path, temp_drive.get_ptr(), temp_path.get_ptr()); combine_path(path, temp_drive.get_ptr(), temp_path.get_ptr());
return true; return true;
} }
bool file_utils::get_filename(const char* p, dynamic_string& filename) bool file_utils::get_filename(const char* p, dynamic_string& filename) {
{ dynamic_string temp_ext;
dynamic_string temp_ext; if (!split_path(p, NULL, NULL, &filename, &temp_ext))
if (!split_path(p, NULL, NULL, &filename, &temp_ext)) return false;
return false;
filename += temp_ext; filename += temp_ext;
return true; return true;
} }
void file_utils::combine_path(dynamic_string& dst, const char* pA, const char* pB) void file_utils::combine_path(dynamic_string& dst, const char* pA, const char* pB) {
{ dynamic_string temp(pA);
dynamic_string temp(pA); if ((!temp.is_empty()) && (!is_path_separator(pB[0]))) {
if ((!temp.is_empty()) && (!is_path_separator(pB[0]))) char c = temp[temp.get_len() - 1];
{ if (!is_path_separator(c))
char c = temp[temp.get_len() - 1]; temp.append_char(CRNLIB_PATH_SEPERATOR_CHAR);
if (!is_path_separator(c)) }
temp.append_char(CRNLIB_PATH_SEPERATOR_CHAR); temp += pB;
} dst.swap(temp);
temp += pB; }
dst.swap(temp);
}
void file_utils::combine_path(dynamic_string& dst, const char* pA, const char* pB, const char* pC) void file_utils::combine_path(dynamic_string& dst, const char* pA, const char* pB, const char* pC) {
{ combine_path(dst, pA, pB);
combine_path(dst, pA, pB); combine_path(dst, dst.get_ptr(), pC);
combine_path(dst, dst.get_ptr(), pC); }
}
bool file_utils::full_path(dynamic_string& path) bool file_utils::full_path(dynamic_string& path) {
{
#ifdef WIN32 #ifdef WIN32
char buf[1024]; char buf[1024];
char* p = _fullpath(buf, path.get_ptr(), sizeof(buf)); char* p = _fullpath(buf, path.get_ptr(), sizeof(buf));
if (!p) if (!p)
return false; return false;
#else #else
char buf[PATH_MAX]; char buf[PATH_MAX];
char* p; char* p;
dynamic_string pn, fn; dynamic_string pn, fn;
split_path(path.get_ptr(), pn, fn); split_path(path.get_ptr(), pn, fn);
if ((fn == ".") || (fn == "..")) if ((fn == ".") || (fn == "..")) {
{ p = realpath(path.get_ptr(), buf);
p = realpath(path.get_ptr(), buf); if (!p)
if (!p) return false;
return false; path.set(buf);
path.set(buf); } else {
} if (pn.is_empty())
else pn = "./";
{ p = realpath(pn.get_ptr(), buf);
if (pn.is_empty()) if (!p)
pn = "./"; return false;
p = realpath(pn.get_ptr(), buf); combine_path(path, buf, fn.get_ptr());
if (!p) }
return false;
combine_path(path, buf, fn.get_ptr());
}
#endif #endif
return true; return true;
} }
bool file_utils::get_extension(dynamic_string& filename) bool file_utils::get_extension(dynamic_string& filename) {
{ int sep = -1;
int sep = -1;
#ifdef WIN32 #ifdef WIN32
sep = filename.find_right('\\'); sep = filename.find_right('\\');
#endif #endif
if (sep < 0) if (sep < 0)
sep = filename.find_right('/'); sep = filename.find_right('/');
int dot = filename.find_right('.'); int dot = filename.find_right('.');
if (dot < sep) if (dot < sep) {
{ filename.clear();
filename.clear(); return false;
return false; }
}
filename.right(dot + 1); filename.right(dot + 1);
return true; return true;
} }
bool file_utils::remove_extension(dynamic_string& filename) bool file_utils::remove_extension(dynamic_string& filename) {
{ int sep = -1;
int sep = -1;
#ifdef WIN32 #ifdef WIN32
sep = filename.find_right('\\'); sep = filename.find_right('\\');
#endif #endif
if (sep < 0) if (sep < 0)
sep = filename.find_right('/'); sep = filename.find_right('/');
int dot = filename.find_right('.'); int dot = filename.find_right('.');
if (dot < sep) if (dot < sep)
return false; return false;
filename.left(dot); filename.left(dot);
return true; return true;
} }
bool file_utils::create_path(const dynamic_string& fullpath)
{
bool got_unc = false; got_unc;
dynamic_string cur_path;
const int l = fullpath.get_len();
int n = 0;
while (n < l)
{
const char c = fullpath.get_ptr()[n];
const bool sep = is_path_separator(c);
const bool back_sep = is_path_separator(cur_path.back());
const bool is_last_char = (n == (l - 1));
if ( ((sep) && (!back_sep)) || (is_last_char) )
{
if ((is_last_char) && (!sep))
cur_path.append_char(c);
bool valid = !cur_path.is_empty();
bool file_utils::create_path(const dynamic_string& fullpath) {
#ifdef WIN32 #ifdef WIN32
// reject obvious stuff (drives, beginning of UNC paths): bool got_unc = false;
// c:\b\cool
// \\machine\blah
// \cool\blah
if ((cur_path.get_len() == 2) && (cur_path[1] == ':'))
valid = false;
else if ((cur_path.get_len() >= 2) && (cur_path[0] == '\\') && (cur_path[1] == '\\'))
{
if (!got_unc)
valid = false;
got_unc = true;
}
else if (cur_path == "\\")
valid = false;
#endif #endif
if (cur_path == "/") dynamic_string cur_path;
valid = false;
const int l = fullpath.get_len();
int n = 0;
while (n < l) {
const char c = fullpath.get_ptr()[n];
const bool sep = is_path_separator(c);
const bool back_sep = is_path_separator(cur_path.back());
const bool is_last_char = (n == (l - 1));
if (((sep) && (!back_sep)) || (is_last_char)) {
if ((is_last_char) && (!sep))
cur_path.append_char(c);
bool valid = !cur_path.is_empty();
if ((valid) && (cur_path.get_len()))
{
#ifdef WIN32 #ifdef WIN32
_mkdir(cur_path.get_ptr()); // reject obvious stuff (drives, beginning of UNC paths):
// c:\b\cool
// \\machine\blah
// \cool\blah
if ((cur_path.get_len() == 2) && (cur_path[1] == ':'))
valid = false;
else if ((cur_path.get_len() >= 2) && (cur_path[0] == '\\') && (cur_path[1] == '\\')) {
if (!got_unc)
valid = false;
got_unc = true;
} else if (cur_path == "\\")
valid = false;
#endif
if (cur_path == "/")
valid = false;
if ((valid) && (cur_path.get_len())) {
#ifdef WIN32
_mkdir(cur_path.get_ptr());
#else #else
mkdir(cur_path.get_ptr(), S_IRWXU | S_IRWXG | S_IRWXO ); mkdir(cur_path.get_ptr(), S_IRWXU | S_IRWXG | S_IRWXO);
#endif #endif
}
}
cur_path.append_char(c);
n++;
} }
}
return true; cur_path.append_char(c);
}
void file_utils::trim_trailing_seperator(dynamic_string& path) n++;
{ }
if ((path.get_len()) && (is_path_separator(path.back())))
path.truncate(path.get_len() - 1);
}
// See http://www.codeproject.com/KB/string/wildcmp.aspx return true;
int file_utils::wildcmp(const char* pWild, const char* pString) }
{
const char* cp = NULL, *mp = NULL;
while ((*pString) && (*pWild != '*')) void file_utils::trim_trailing_seperator(dynamic_string& path) {
{ if ((path.get_len()) && (is_path_separator(path.back())))
if ((*pWild != *pString) && (*pWild != '?')) path.truncate(path.get_len() - 1);
return 0; }
pWild++;
pString++;
}
// Either *pString=='\0' or *pWild='*' here. // See http://www.codeproject.com/KB/string/wildcmp.aspx
int file_utils::wildcmp(const char* pWild, const char* pString) {
const char *cp = NULL, *mp = NULL;
while (*pString) while ((*pString) && (*pWild != '*')) {
{ if ((*pWild != *pString) && (*pWild != '?'))
if (*pWild == '*') return 0;
{ pWild++;
if (!*++pWild) pString++;
return 1; }
mp = pWild;
cp = pString+1;
}
else if ((*pWild == *pString) || (*pWild == '?'))
{
pWild++;
pString++;
}
else
{
pWild = mp;
pString = cp++;
}
}
while (*pWild == '*') // Either *pString=='\0' or *pWild='*' here.
pWild++;
return !*pWild; while (*pString) {
} if (*pWild == '*') {
if (!*++pWild)
return 1;
mp = pWild;
cp = pString + 1;
} else if ((*pWild == *pString) || (*pWild == '?')) {
pWild++;
pString++;
} else {
pWild = mp;
pString = cp++;
}
}
bool file_utils::write_buf_to_file(const char* pPath, const void* pData, size_t data_size) while (*pWild == '*')
{ pWild++;
FILE *pFile = NULL;
return !*pWild;
}
bool file_utils::write_buf_to_file(const char* pPath, const void* pData, size_t data_size) {
FILE* pFile = NULL;
#ifdef _MSC_VER #ifdef _MSC_VER
// Compiling with MSVC // Compiling with MSVC
if (fopen_s(&pFile, pPath, "wb")) if (fopen_s(&pFile, pPath, "wb"))
return false; return false;
#else #else
pFile = fopen(pPath, "wb"); pFile = fopen(pPath, "wb");
#endif #endif
if (!pFile) if (!pFile)
return false; return false;
bool success = fwrite(pData, 1, data_size, pFile) == data_size; bool success = fwrite(pData, 1, data_size, pFile) == data_size;
fclose(pFile); fclose(pFile);
return success; return success;
} }
} // namespace crnlib } // namespace crnlib
crnlib/crn_file_utils.h
+28 -30
View File
@@ -2,42 +2,40 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
namespace crnlib namespace crnlib {
{ struct file_utils {
struct file_utils // Returns true if pSrcFilename is older than pDstFilename
{ static bool is_read_only(const char* pFilename);
// Returns true if pSrcFilename is older than pDstFilename static bool disable_read_only(const char* pFilename);
static bool is_read_only(const char* pFilename); static bool is_older_than(const char* pSrcFilename, const char* pDstFilename);
static bool disable_read_only(const char* pFilename); static bool does_file_exist(const char* pFilename);
static bool is_older_than(const char *pSrcFilename, const char* pDstFilename); static bool does_dir_exist(const char* pDir);
static bool does_file_exist(const char* pFilename); static bool get_file_size(const char* pFilename, uint64& file_size);
static bool does_dir_exist(const char* pDir); static bool get_file_size(const char* pFilename, uint32& file_size);
static bool get_file_size(const char* pFilename, uint64& file_size);
static bool get_file_size(const char* pFilename, uint32& file_size);
static bool is_path_separator(char c); static bool is_path_separator(char c);
static bool is_path_or_drive_separator(char c); static bool is_path_or_drive_separator(char c);
static bool is_drive_separator(char c); static bool is_drive_separator(char c);
static bool split_path(const char* p, dynamic_string* pDrive, dynamic_string* pDir, dynamic_string* pFilename, dynamic_string* pExt); static bool split_path(const char* p, dynamic_string* pDrive, dynamic_string* pDir, dynamic_string* pFilename, dynamic_string* pExt);
static bool split_path(const char* p, dynamic_string& path, dynamic_string& filename); static bool split_path(const char* p, dynamic_string& path, dynamic_string& filename);
static bool get_pathname(const char* p, dynamic_string& path); static bool get_pathname(const char* p, dynamic_string& path);
static bool get_filename(const char* p, dynamic_string& filename); static bool get_filename(const char* p, dynamic_string& filename);
static void combine_path(dynamic_string& dst, const char* pA, const char* pB); static void combine_path(dynamic_string& dst, const char* pA, const char* pB);
static void combine_path(dynamic_string& dst, const char* pA, const char* pB, const char* pC); static void combine_path(dynamic_string& dst, const char* pA, const char* pB, const char* pC);
static bool full_path(dynamic_string& path); static bool full_path(dynamic_string& path);
static bool get_extension(dynamic_string& filename); static bool get_extension(dynamic_string& filename);
static bool remove_extension(dynamic_string& filename); static bool remove_extension(dynamic_string& filename);
static bool create_path(const dynamic_string& path); static bool create_path(const dynamic_string& path);
static void trim_trailing_seperator(dynamic_string& path); static void trim_trailing_seperator(dynamic_string& path);
static int wildcmp(const char* pWild, const char* pString); static int wildcmp(const char* pWild, const char* pString);
static bool write_buf_to_file(const char* pPath, const void* pData, size_t data_size); static bool write_buf_to_file(const char* pPath, const void* pData, size_t data_size);
}; // struct file_utils }; // struct file_utils
} // namespace crnlib } // namespace crnlib
crnlib/crn_find_files.cpp
+201 -235
View File
@@ -13,275 +13,241 @@
#include <dirent.h> #include <dirent.h>
#endif #endif
namespace crnlib namespace crnlib {
{
#ifdef CRNLIB_USE_WIN32_API #ifdef CRNLIB_USE_WIN32_API
bool find_files::find(const char* pBasepath, const char* pFilespec, uint flags) bool find_files::find(const char* pBasepath, const char* pFilespec, uint flags) {
{ m_last_error = S_OK;
m_last_error = S_OK; m_files.resize(0);
m_files.resize(0);
return find_internal(pBasepath, "", pFilespec, flags, 0); return find_internal(pBasepath, "", pFilespec, flags, 0);
} }
bool find_files::find(const char* pSpec, uint flags) bool find_files::find(const char* pSpec, uint flags) {
{ dynamic_string find_name(pSpec);
dynamic_string find_name(pSpec);
if (!file_utils::full_path(find_name)) if (!file_utils::full_path(find_name))
return false; return false;
dynamic_string find_pathname, find_filename; dynamic_string find_pathname, find_filename;
if (!file_utils::split_path(find_name.get_ptr(), find_pathname, find_filename)) if (!file_utils::split_path(find_name.get_ptr(), find_pathname, find_filename))
return false; return false;
return find(find_pathname.get_ptr(), find_filename.get_ptr(), flags); return find(find_pathname.get_ptr(), find_filename.get_ptr(), flags);
} }
bool find_files::find_internal(const char* pBasepath, const char* pRelpath, const char* pFilespec, uint flags, int level) bool find_files::find_internal(const char* pBasepath, const char* pRelpath, const char* pFilespec, uint flags, int level) {
{ WIN32_FIND_DATAA find_data;
WIN32_FIND_DATAA find_data;
dynamic_string filename; dynamic_string filename;
dynamic_string_array child_paths; dynamic_string_array child_paths;
if (flags & cFlagRecursive) if (flags & cFlagRecursive) {
{ if (strlen(pRelpath))
if (strlen(pRelpath)) file_utils::combine_path(filename, pBasepath, pRelpath, "*");
file_utils::combine_path(filename, pBasepath, pRelpath, "*"); else
else file_utils::combine_path(filename, pBasepath, "*");
file_utils::combine_path(filename, pBasepath, "*");
HANDLE handle = FindFirstFileA(filename.get_ptr(), &find_data); HANDLE handle = FindFirstFileA(filename.get_ptr(), &find_data);
if (handle == INVALID_HANDLE_VALUE) if (handle == INVALID_HANDLE_VALUE) {
{ HRESULT hres = GetLastError();
HRESULT hres = GetLastError(); if ((level == 0) && (hres != NO_ERROR) && (hres != ERROR_FILE_NOT_FOUND)) {
if ((level == 0) && (hres != NO_ERROR) && (hres != ERROR_FILE_NOT_FOUND)) m_last_error = hres;
{ return false;
m_last_error = hres; }
return false; } else {
} do {
} const bool is_dir = (find_data.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) != 0;
else
{
do
{
const bool is_dir = (find_data.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) != 0;
bool skip = !is_dir; bool skip = !is_dir;
if (is_dir) if (is_dir)
skip = (strcmp(find_data.cFileName, ".") == 0) || (strcmp(find_data.cFileName, "..") == 0); skip = (strcmp(find_data.cFileName, ".") == 0) || (strcmp(find_data.cFileName, "..") == 0);
if (find_data.dwFileAttributes & (FILE_ATTRIBUTE_SYSTEM | FILE_ATTRIBUTE_TEMPORARY)) if (find_data.dwFileAttributes & (FILE_ATTRIBUTE_SYSTEM | FILE_ATTRIBUTE_TEMPORARY))
skip = true; skip = true;
if (find_data.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN) if (find_data.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN) {
{ if ((flags & cFlagAllowHidden) == 0)
if ((flags & cFlagAllowHidden) == 0) skip = true;
skip = true; }
}
if (!skip) if (!skip) {
{ dynamic_string child_path(find_data.cFileName);
dynamic_string child_path(find_data.cFileName); if ((!child_path.count_char('?')) && (!child_path.count_char('*')))
if ((!child_path.count_char('?')) && (!child_path.count_char('*'))) child_paths.push_back(child_path);
child_paths.push_back(child_path); }
}
} while (FindNextFileA(handle, &find_data) != 0); } while (FindNextFileA(handle, &find_data) != 0);
HRESULT hres = GetLastError(); HRESULT hres = GetLastError();
FindClose(handle); FindClose(handle);
handle = INVALID_HANDLE_VALUE; handle = INVALID_HANDLE_VALUE;
if (hres != ERROR_NO_MORE_FILES) if (hres != ERROR_NO_MORE_FILES) {
{ m_last_error = hres;
m_last_error = hres; return false;
return false; }
} }
} }
if (strlen(pRelpath))
file_utils::combine_path(filename, pBasepath, pRelpath, pFilespec);
else
file_utils::combine_path(filename, pBasepath, pFilespec);
HANDLE handle = FindFirstFileA(filename.get_ptr(), &find_data);
if (handle == INVALID_HANDLE_VALUE) {
HRESULT hres = GetLastError();
if ((level == 0) && (hres != NO_ERROR) && (hres != ERROR_FILE_NOT_FOUND)) {
m_last_error = hres;
return false;
}
} else {
do {
const bool is_dir = (find_data.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) != 0;
bool skip = false;
if (is_dir)
skip = (strcmp(find_data.cFileName, ".") == 0) || (strcmp(find_data.cFileName, "..") == 0);
if (find_data.dwFileAttributes & (FILE_ATTRIBUTE_SYSTEM | FILE_ATTRIBUTE_TEMPORARY))
skip = true;
if (find_data.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN) {
if ((flags & cFlagAllowHidden) == 0)
skip = true;
} }
if (strlen(pRelpath)) if (!skip) {
file_utils::combine_path(filename, pBasepath, pRelpath, pFilespec); if (((is_dir) && (flags & cFlagAllowDirs)) || ((!is_dir) && (flags & cFlagAllowFiles))) {
else m_files.resize(m_files.size() + 1);
file_utils::combine_path(filename, pBasepath, pFilespec); file_desc& file = m_files.back();
file.m_is_dir = is_dir;
HANDLE handle = FindFirstFileA(filename.get_ptr(), &find_data); file.m_base = pBasepath;
if (handle == INVALID_HANDLE_VALUE) file.m_name = find_data.cFileName;
{ file.m_rel = pRelpath;
HRESULT hres = GetLastError(); if (strlen(pRelpath))
if ((level == 0) && (hres != NO_ERROR) && (hres != ERROR_FILE_NOT_FOUND)) file_utils::combine_path(file.m_fullname, pBasepath, pRelpath, find_data.cFileName);
{ else
m_last_error = hres; file_utils::combine_path(file.m_fullname, pBasepath, find_data.cFileName);
return false; }
}
}
else
{
do
{
const bool is_dir = (find_data.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) != 0;
bool skip = false;
if (is_dir)
skip = (strcmp(find_data.cFileName, ".") == 0) || (strcmp(find_data.cFileName, "..") == 0);
if (find_data.dwFileAttributes & (FILE_ATTRIBUTE_SYSTEM | FILE_ATTRIBUTE_TEMPORARY))
skip = true;
if (find_data.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN)
{
if ((flags & cFlagAllowHidden) == 0)
skip = true;
}
if (!skip)
{
if (((is_dir) && (flags & cFlagAllowDirs)) || ((!is_dir) && (flags & cFlagAllowFiles)))
{
m_files.resize(m_files.size() + 1);
file_desc& file = m_files.back();
file.m_is_dir = is_dir;
file.m_base = pBasepath;
file.m_name = find_data.cFileName;
file.m_rel = pRelpath;
if (strlen(pRelpath))
file_utils::combine_path(file.m_fullname, pBasepath, pRelpath, find_data.cFileName);
else
file_utils::combine_path(file.m_fullname, pBasepath, find_data.cFileName);
}
}
} while (FindNextFileA(handle, &find_data) != 0);
HRESULT hres = GetLastError();
FindClose(handle);
if (hres != ERROR_NO_MORE_FILES)
{
m_last_error = hres;
return false;
}
} }
for (uint i = 0; i < child_paths.size(); i++) } while (FindNextFileA(handle, &find_data) != 0);
{
dynamic_string child_path;
if (strlen(pRelpath))
file_utils::combine_path(child_path, pRelpath, child_paths[i].get_ptr());
else
child_path = child_paths[i];
if (!find_internal(pBasepath, child_path.get_ptr(), pFilespec, flags, level + 1)) HRESULT hres = GetLastError();
return false;
}
return true; FindClose(handle);
}
if (hres != ERROR_NO_MORE_FILES) {
m_last_error = hres;
return false;
}
}
for (uint i = 0; i < child_paths.size(); i++) {
dynamic_string child_path;
if (strlen(pRelpath))
file_utils::combine_path(child_path, pRelpath, child_paths[i].get_ptr());
else
child_path = child_paths[i];
if (!find_internal(pBasepath, child_path.get_ptr(), pFilespec, flags, level + 1))
return false;
}
return true;
}
#elif defined(__GNUC__) #elif defined(__GNUC__)
bool find_files::find(const char* pBasepath, const char* pFilespec, uint flags) bool find_files::find(const char* pBasepath, const char* pFilespec, uint flags) {
{ m_files.resize(0);
m_files.resize(0); return find_internal(pBasepath, "", pFilespec, flags, 0);
return find_internal(pBasepath, "", pFilespec, flags, 0); }
}
bool find_files::find(const char* pSpec, uint flags) bool find_files::find(const char* pSpec, uint flags) {
{ dynamic_string find_name(pSpec);
dynamic_string find_name(pSpec);
if (!file_utils::full_path(find_name)) if (!file_utils::full_path(find_name))
return false; return false;
dynamic_string find_pathname, find_filename; dynamic_string find_pathname, find_filename;
if (!file_utils::split_path(find_name.get_ptr(), find_pathname, find_filename)) if (!file_utils::split_path(find_name.get_ptr(), find_pathname, find_filename))
return false; return false;
return find(find_pathname.get_ptr(), find_filename.get_ptr(), flags); return find(find_pathname.get_ptr(), find_filename.get_ptr(), flags);
} }
bool find_files::find_internal(const char* pBasepath, const char* pRelpath, const char* pFilespec, uint flags, int level) bool find_files::find_internal(const char* pBasepath, const char* pRelpath, const char* pFilespec, uint flags, int level) {
{ dynamic_string pathname;
dynamic_string pathname; if (strlen(pRelpath))
file_utils::combine_path(pathname, pBasepath, pRelpath);
else
pathname = pBasepath;
if (!pathname.is_empty()) {
char c = pathname.back();
if (c != '/')
pathname += "/";
}
DIR* dp = opendir(pathname.get_ptr());
if (!dp)
return level ? true : false;
dynamic_string_array paths;
for (;;) {
struct dirent* ep = readdir(dp);
if (!ep)
break;
if ((strcmp(ep->d_name, ".") == 0) || (strcmp(ep->d_name, "..") == 0))
continue;
const bool is_directory = (ep->d_type & DT_DIR) != 0;
const bool is_file = (ep->d_type & DT_REG) != 0;
dynamic_string filename(ep->d_name);
if (is_directory) {
if (flags & cFlagRecursive) {
paths.push_back(filename);
}
}
if (((is_file) && (flags & cFlagAllowFiles)) || ((is_directory) && (flags & cFlagAllowDirs))) {
if (0 == fnmatch(pFilespec, filename.get_ptr(), 0)) {
m_files.resize(m_files.size() + 1);
file_desc& file = m_files.back();
file.m_is_dir = is_directory;
file.m_base = pBasepath;
file.m_rel = pRelpath;
file.m_name = filename;
file.m_fullname = pathname + filename;
}
}
}
closedir(dp);
dp = NULL;
if (flags & cFlagRecursive) {
for (uint i = 0; i < paths.size(); i++) {
dynamic_string childpath;
if (strlen(pRelpath)) if (strlen(pRelpath))
file_utils::combine_path(pathname, pBasepath, pRelpath); file_utils::combine_path(childpath, pRelpath, paths[i].get_ptr());
else else
pathname = pBasepath; childpath = paths[i];
if (!pathname.is_empty()) if (!find_internal(pBasepath, childpath.get_ptr(), pFilespec, flags, level + 1))
{ return false;
char c = pathname.back(); }
if (c != '/') }
pathname += "/";
}
DIR *dp = opendir(pathname.get_ptr()); return true;
}
if (!dp)
return level ? true : false;
dynamic_string_array paths;
for ( ; ; )
{
struct dirent *ep = readdir(dp);
if (!ep)
break;
if ((strcmp(ep->d_name, ".") == 0) || (strcmp(ep->d_name, "..") == 0))
continue;
const bool is_directory = (ep->d_type & DT_DIR) != 0;
const bool is_file = (ep->d_type & DT_REG) != 0;
dynamic_string filename(ep->d_name);
if (is_directory)
{
if (flags & cFlagRecursive)
{
paths.push_back(filename);
}
}
if (((is_file) && (flags & cFlagAllowFiles)) || ((is_directory) && (flags & cFlagAllowDirs)))
{
if (0 == fnmatch(pFilespec, filename.get_ptr(), 0))
{
m_files.resize(m_files.size() + 1);
file_desc& file = m_files.back();
file.m_is_dir = is_directory;
file.m_base = pBasepath;
file.m_rel = pRelpath;
file.m_name = filename;
file.m_fullname = pathname + filename;
}
}
}
closedir(dp);
dp = NULL;
if (flags & cFlagRecursive)
{
for (uint i = 0; i < paths.size(); i++)
{
dynamic_string childpath;
if (strlen(pRelpath))
file_utils::combine_path(childpath, pRelpath, paths[i].get_ptr());
else
childpath = paths[i];
if (!find_internal(pBasepath, childpath.get_ptr(), pFilespec, flags, level + 1))
return false;
}
}
return true;
}
#else #else
#error Unimplemented #error Unimplemented
#endif #endif
} // namespace crnlib } // namespace crnlib
crnlib/crn_find_files.h
+37 -41
View File
@@ -2,59 +2,55 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
namespace crnlib namespace crnlib {
{ class find_files {
class find_files public:
{ struct file_desc {
public: inline file_desc()
struct file_desc : m_is_dir(false) {}
{
inline file_desc() : m_is_dir(false) { }
dynamic_string m_fullname; dynamic_string m_fullname;
dynamic_string m_base; dynamic_string m_base;
dynamic_string m_rel; dynamic_string m_rel;
dynamic_string m_name; dynamic_string m_name;
bool m_is_dir; bool m_is_dir;
inline bool operator== (const file_desc& other) const { return m_fullname == other.m_fullname; } inline bool operator==(const file_desc& other) const { return m_fullname == other.m_fullname; }
inline bool operator< (const file_desc& other) const { return m_fullname < other.m_fullname; } inline bool operator<(const file_desc& other) const { return m_fullname < other.m_fullname; }
inline operator size_t() const { return static_cast<size_t>(m_fullname); } inline operator size_t() const { return static_cast<size_t>(m_fullname); }
}; };
typedef crnlib::vector<file_desc> file_desc_vec; typedef crnlib::vector<file_desc> file_desc_vec;
inline find_files() inline find_files() {
{ m_last_error = 0; // S_OK;
m_last_error = 0; // S_OK; }
}
enum flags enum flags {
{ cFlagRecursive = 1,
cFlagRecursive = 1, cFlagAllowDirs = 2,
cFlagAllowDirs = 2, cFlagAllowFiles = 4,
cFlagAllowFiles = 4, cFlagAllowHidden = 8
cFlagAllowHidden = 8 };
};
bool find(const char* pBasepath, const char* pFilespec, uint flags = cFlagAllowFiles); bool find(const char* pBasepath, const char* pFilespec, uint flags = cFlagAllowFiles);
bool find(const char* pSpec, uint flags = cFlagAllowFiles); bool find(const char* pSpec, uint flags = cFlagAllowFiles);
// An HRESULT under Win32. FIXME: Abstract this better? // An HRESULT under Win32. FIXME: Abstract this better?
inline int64 get_last_error() const { return m_last_error; } inline int64 get_last_error() const { return m_last_error; }
const file_desc_vec& get_files() const { return m_files; } const file_desc_vec& get_files() const { return m_files; }
private: private:
file_desc_vec m_files; file_desc_vec m_files;
// A HRESULT under Win32 // A HRESULT under Win32
int64 m_last_error; int64 m_last_error;
bool find_internal(const char* pBasepath, const char* pRelpath, const char* pFilespec, uint flags, int level); bool find_internal(const char* pBasepath, const char* pRelpath, const char* pFilespec, uint flags, int level);
}; // class find_files }; // class find_files
} // namespace crnlib } // namespace crnlib
crnlib/crn_freeimage_image_utils.h
+110 -124
View File
@@ -6,153 +6,139 @@
#include "freeImagePlus.h" #include "freeImagePlus.h"
namespace crnlib namespace crnlib {
{ namespace freeimage_image_utils {
namespace freeimage_image_utils inline bool load_from_file(image_u8& dest, const wchar_t* pFilename, int fi_flag) {
{ fipImage src_image;
inline bool load_from_file(image_u8& dest, const wchar_t* pFilename, int fi_flag)
{
fipImage src_image;
if (!src_image.loadU(pFilename, fi_flag)) if (!src_image.loadU(pFilename, fi_flag))
return false; return false;
const uint orig_bits_per_pixel = src_image.getBitsPerPixel();
const FREE_IMAGE_COLOR_TYPE orig_color_type = src_image.getColorType(); const uint orig_bits_per_pixel = src_image.getBitsPerPixel();
if (!src_image.convertTo32Bits())
return false;
if (src_image.getBitsPerPixel() != 32) const FREE_IMAGE_COLOR_TYPE orig_color_type = src_image.getColorType();
return false;
uint width = src_image.getWidth(); if (!src_image.convertTo32Bits())
uint height = src_image.getHeight(); return false;
dest.resize(src_image.getWidth(), src_image.getHeight(), src_image.getWidth()); if (src_image.getBitsPerPixel() != 32)
return false;
color_quad_u8* pDst = dest.get_ptr(); uint width = src_image.getWidth();
uint height = src_image.getHeight();
bool grayscale = true; dest.resize(src_image.getWidth(), src_image.getHeight(), src_image.getWidth());
bool has_alpha = false;
for (uint y = 0; y < height; y++)
{
const BYTE* pSrc = src_image.getScanLine((WORD)(height - 1 - y));
color_quad_u8* pD = pDst;
for (uint x = width; x; x--) color_quad_u8* pDst = dest.get_ptr();
{
color_quad_u8 c;
c.r = pSrc[FI_RGBA_RED];
c.g = pSrc[FI_RGBA_GREEN];
c.b = pSrc[FI_RGBA_BLUE];
c.a = pSrc[FI_RGBA_ALPHA];
if (!c.is_grayscale())
grayscale = false;
has_alpha |= (c.a < 255);
pSrc += 4; bool grayscale = true;
*pD++ = c; bool has_alpha = false;
} for (uint y = 0; y < height; y++) {
const BYTE* pSrc = src_image.getScanLine((WORD)(height - 1 - y));
color_quad_u8* pD = pDst;
pDst += width; for (uint x = width; x; x--) {
} color_quad_u8 c;
c.r = pSrc[FI_RGBA_RED];
c.g = pSrc[FI_RGBA_GREEN];
c.b = pSrc[FI_RGBA_BLUE];
c.a = pSrc[FI_RGBA_ALPHA];
dest.reset_comp_flags(); if (!c.is_grayscale())
grayscale = false;
if (grayscale) has_alpha |= (c.a < 255);
dest.set_grayscale(true);
dest.set_component_valid(3, has_alpha || (orig_color_type == FIC_RGBALPHA) || (orig_bits_per_pixel == 32));
return true; pSrc += 4;
} *pD++ = c;
}
const int cSaveLuma = -1;
inline bool save_to_grayscale_file(const wchar_t* pFilename, const image_u8& src, int component, int fi_flag)
{
fipImage dst_image(FIT_BITMAP, (WORD)src.get_width(), (WORD)src.get_height(), 8);
RGBQUAD* p = dst_image.getPalette(); pDst += width;
for (uint i = 0; i < dst_image.getPaletteSize(); i++) }
{
p[i].rgbRed = (BYTE)i;
p[i].rgbGreen = (BYTE)i;
p[i].rgbBlue = (BYTE)i;
p[i].rgbReserved = 255;
}
for (uint y = 0; y < src.get_height(); y++) dest.reset_comp_flags();
{
const color_quad_u8* pSrc = src.get_scanline(y);
for (uint x = 0; x < src.get_width(); x++) if (grayscale)
{ dest.set_grayscale(true);
BYTE v;
if (component == cSaveLuma)
v = (BYTE)(*pSrc).get_luma();
else
v = (*pSrc)[component];
dst_image.setPixelIndex(x, src.get_height() - 1 - y, &v);
pSrc++; dest.set_component_valid(3, has_alpha || (orig_color_type == FIC_RGBALPHA) || (orig_bits_per_pixel == 32));
}
}
if (!dst_image.saveU(pFilename, fi_flag)) return true;
return false; }
return true; const int cSaveLuma = -1;
}
inline bool save_to_file(const wchar_t* pFilename, const image_u8& src, int fi_flag, bool ignore_alpha = false) inline bool save_to_grayscale_file(const wchar_t* pFilename, const image_u8& src, int component, int fi_flag) {
{ fipImage dst_image(FIT_BITMAP, (WORD)src.get_width(), (WORD)src.get_height(), 8);
const bool save_alpha = src.is_component_valid(3);
uint bpp = (save_alpha && !ignore_alpha) ? 32 : 24;
if (bpp == 32)
{
dynamic_wstring ext(pFilename);
get_extension(ext);
if ((ext == L"jpg") || (ext == L"jpeg") || (ext == L"gif") || (ext == L"jp2")) RGBQUAD* p = dst_image.getPalette();
bpp = 24; for (uint i = 0; i < dst_image.getPaletteSize(); i++) {
} p[i].rgbRed = (BYTE)i;
p[i].rgbGreen = (BYTE)i;
if ((bpp == 24) && (src.is_grayscale())) p[i].rgbBlue = (BYTE)i;
return save_to_grayscale_file(pFilename, src, cSaveLuma, fi_flag); p[i].rgbReserved = 255;
}
fipImage dst_image(FIT_BITMAP, (WORD)src.get_width(), (WORD)src.get_height(), (WORD)bpp);
for (uint y = 0; y < src.get_height(); y++) for (uint y = 0; y < src.get_height(); y++) {
{ const color_quad_u8* pSrc = src.get_scanline(y);
for (uint x = 0; x < src.get_width(); x++)
{
color_quad_u8 c(src(x, y));
RGBQUAD quad; for (uint x = 0; x < src.get_width(); x++) {
quad.rgbRed = c.r; BYTE v;
quad.rgbGreen = c.g; if (component == cSaveLuma)
quad.rgbBlue = c.b; v = (BYTE)(*pSrc).get_luma();
if (bpp == 32) else
quad.rgbReserved = c.a; v = (*pSrc)[component];
else dst_image.setPixelIndex(x, src.get_height() - 1 - y, &v);
quad.rgbReserved = 255;
dst_image.setPixelColor(x, src.get_height() - 1 - y, &quad); pSrc++;
} }
} }
if (!dst_image.saveU(pFilename, fi_flag)) if (!dst_image.saveU(pFilename, fi_flag))
return false; return false;
return true; return true;
} }
} // namespace freeimage_image_utils
} // namespace crnlib
inline bool save_to_file(const wchar_t* pFilename, const image_u8& src, int fi_flag, bool ignore_alpha = false) {
const bool save_alpha = src.is_component_valid(3);
uint bpp = (save_alpha && !ignore_alpha) ? 32 : 24;
if (bpp == 32) {
dynamic_wstring ext(pFilename);
get_extension(ext);
if ((ext == L"jpg") || (ext == L"jpeg") || (ext == L"gif") || (ext == L"jp2"))
bpp = 24;
}
if ((bpp == 24) && (src.is_grayscale()))
return save_to_grayscale_file(pFilename, src, cSaveLuma, fi_flag);
fipImage dst_image(FIT_BITMAP, (WORD)src.get_width(), (WORD)src.get_height(), (WORD)bpp);
for (uint y = 0; y < src.get_height(); y++) {
for (uint x = 0; x < src.get_width(); x++) {
color_quad_u8 c(src(x, y));
RGBQUAD quad;
quad.rgbRed = c.r;
quad.rgbGreen = c.g;
quad.rgbBlue = c.b;
if (bpp == 32)
quad.rgbReserved = c.a;
else
quad.rgbReserved = 255;
dst_image.setPixelColor(x, src.get_height() - 1 - y, &quad);
}
}
if (!dst_image.saveU(pFilename, fi_flag))
return false;
return true;
}
} // namespace freeimage_image_utils
} // namespace crnlib
crnlib/crn_hash.cpp
+48 -48
View File
@@ -5,64 +5,64 @@
#include "crn_core.h" #include "crn_core.h"
#undef get16bits #undef get16bits
#if (defined(__GNUC__) && defined(__i386__)) || defined(__WATCOMC__) \ #if (defined(__GNUC__) && defined(__i386__)) || defined(__WATCOMC__) || defined(_MSC_VER) || defined(__BORLANDC__) || defined(__TURBOC__)
|| defined(_MSC_VER) || defined (__BORLANDC__) || defined (__TURBOC__) #define get16bits(d) (*((const uint16*)(d)))
#define get16bits(d) (*((const uint16 *) (d)))
#endif #endif
#if !defined (get16bits) #if !defined(get16bits)
#define get16bits(d) ((((uint32)(((const uint8 *)(d))[1])) << 8)\ #define get16bits(d) ((((uint32)(((const uint8*)(d))[1])) << 8) + (uint32)(((const uint8*)(d))[0]))
+(uint32)(((const uint8 *)(d))[0]) )
#endif #endif
namespace crnlib namespace crnlib {
{ uint32 fast_hash(const void* p, int len) {
uint32 fast_hash (const void* p, int len) const char* data = static_cast<const char*>(p);
{
const char * data = static_cast<const char *>(p);
uint32 hash = len, tmp;
int rem;
if (len <= 0 || data == NULL) return 0; uint32 hash = len, tmp;
int rem;
rem = len & 3; if (len <= 0 || data == NULL)
len >>= 2; return 0;
/* Main loop */ rem = len & 3;
for (;len > 0; len--) { len >>= 2;
hash += get16bits (data);
tmp = (get16bits (data+2) << 11) ^ hash;
hash = (hash << 16) ^ tmp;
data += 2*sizeof (uint16);
hash += hash >> 11;
}
/* Handle end cases */ /* Main loop */
switch (rem) { for (; len > 0; len--) {
case 3: hash += get16bits (data); hash += get16bits(data);
hash ^= hash << 16; tmp = (get16bits(data + 2) << 11) ^ hash;
hash ^= data[sizeof (uint16)] << 18; hash = (hash << 16) ^ tmp;
hash += hash >> 11; data += 2 * sizeof(uint16);
break; hash += hash >> 11;
case 2: hash += get16bits (data); }
hash ^= hash << 11;
hash += hash >> 17;
break;
case 1: hash += *data;
hash ^= hash << 10;
hash += hash >> 1;
}
/* Force "avalanching" of final 127 bits */ /* Handle end cases */
hash ^= hash << 3; switch (rem) {
hash += hash >> 5; case 3:
hash ^= hash << 4; hash += get16bits(data);
hash ^= hash << 16;
hash ^= data[sizeof(uint16)] << 18;
hash += hash >> 11;
break;
case 2:
hash += get16bits(data);
hash ^= hash << 11;
hash += hash >> 17; hash += hash >> 17;
hash ^= hash << 25; break;
hash += hash >> 6; case 1:
hash += *data;
hash ^= hash << 10;
hash += hash >> 1;
}
return hash; /* Force "avalanching" of final 127 bits */
} hash ^= hash << 3;
hash += hash >> 5;
hash ^= hash << 4;
hash += hash >> 17;
hash ^= hash << 25;
hash += hash >> 6;
return hash;
}
} // namespace crnlib } // namespace crnlib
crnlib/crn_hash.h
+26 -29
View File
@@ -2,33 +2,30 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
namespace crnlib namespace crnlib {
{ uint32 fast_hash(const void* p, int len);
uint32 fast_hash (const void* p, int len);
// 4-byte integer hash, full avalanche
// 4-byte integer hash, full avalanche inline uint32 bitmix32c(uint32 a) {
inline uint32 bitmix32c(uint32 a) a = (a + 0x7ed55d16) + (a << 12);
{ a = (a ^ 0xc761c23c) ^ (a >> 19);
a = (a+0x7ed55d16) + (a<<12); a = (a + 0x165667b1) + (a << 5);
a = (a^0xc761c23c) ^ (a>>19); a = (a + 0xd3a2646c) ^ (a << 9);
a = (a+0x165667b1) + (a<<5); a = (a + 0xfd7046c5) + (a << 3);
a = (a+0xd3a2646c) ^ (a<<9); a = (a ^ 0xb55a4f09) ^ (a >> 16);
a = (a+0xfd7046c5) + (a<<3); return a;
a = (a^0xb55a4f09) ^ (a>>16); }
return a;
} // 4-byte integer hash, full avalanche, no constants
inline uint32 bitmix32(uint32 a) {
// 4-byte integer hash, full avalanche, no constants a -= (a << 6);
inline uint32 bitmix32(uint32 a) a ^= (a >> 17);
{ a -= (a << 9);
a -= (a<<6); a ^= (a << 4);
a ^= (a>>17); a -= (a << 3);
a -= (a<<9); a ^= (a << 10);
a ^= (a<<4); a ^= (a >> 15);
a -= (a<<3); return a;
a ^= (a<<10); }
a ^= (a>>15);
return a;
}
} // namespace crnlib } // namespace crnlib
crnlib/crn_hash_map.cpp
+2 -3
View File
@@ -4,8 +4,7 @@
#include "crn_hash_map.h" #include "crn_hash_map.h"
#include "crn_rand.h" #include "crn_rand.h"
namespace crnlib namespace crnlib {
{
#if 0 #if 0
class counted_obj class counted_obj
{ {
@@ -152,4 +151,4 @@ namespace crnlib
} }
#endif #endif
} // namespace crnlib } // namespace crnlib
crnlib/crn_hash_map.h
+681 -792
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_helpers.h
+53 -55
View File
@@ -2,63 +2,61 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
#define CRNLIB_NO_COPY_OR_ASSIGNMENT_OP(c) c(const c&); c& operator= (const c&); #define CRNLIB_NO_COPY_OR_ASSIGNMENT_OP(c) \
#define CRNLIB_NO_HEAP_ALLOC() private: static void* operator new(size_t); static void* operator new[](size_t); c(const c&); \
c& operator=(const c&);
#define CRNLIB_NO_HEAP_ALLOC() \
private: \
static void* operator new(size_t); \
static void* operator new[](size_t);
namespace crnlib namespace crnlib {
{ namespace helpers {
namespace helpers template <typename T>
{ struct rel_ops {
template<typename T> struct rel_ops friend bool operator!=(const T& x, const T& y) { return (!(x == y)); }
{ friend bool operator>(const T& x, const T& y) { return (y < x); }
friend bool operator!=(const T& x, const T& y) { return (!(x == y)); } friend bool operator<=(const T& x, const T& y) { return (!(y < x)); }
friend bool operator> (const T& x, const T& y) { return (y < x); } friend bool operator>=(const T& x, const T& y) { return (!(x < y)); }
friend bool operator<=(const T& x, const T& y) { return (!(y < x)); } };
friend bool operator>=(const T& x, const T& y) { return (!(x < y)); }
};
template <typename T>
inline T* construct(T* p)
{
return new (static_cast<void*>(p)) T;
}
template <typename T, typename U> template <typename T>
inline T* construct(T* p, const U& init) inline T* construct(T* p) {
{ return new (static_cast<void*>(p)) T;
return new (static_cast<void*>(p)) T(init); }
}
template <typename T> template <typename T, typename U>
inline void construct_array(T* p, uint n) inline T* construct(T* p, const U& init) {
{ return new (static_cast<void*>(p)) T(init);
T* q = p + n; }
for ( ; p != q; ++p)
new (static_cast<void*>(p)) T; template <typename T>
} inline void construct_array(T* p, uint n) {
T* q = p + n;
template <typename T, typename U> for (; p != q; ++p)
inline void construct_array(T* p, uint n, const U& init) new (static_cast<void*>(p)) T;
{ }
T* q = p + n;
for ( ; p != q; ++p) template <typename T, typename U>
new (static_cast<void*>(p)) T(init); inline void construct_array(T* p, uint n, const U& init) {
} T* q = p + n;
for (; p != q; ++p)
new (static_cast<void*>(p)) T(init);
}
template <typename T>
inline void destruct(T* p) {
(void)p;
p->~T();
}
template <typename T>
inline void destruct_array(T* p, uint n) {
T* q = p + n;
for (; p != q; ++p)
p->~T();
}
} // namespace helpers
template <typename T>
inline void destruct(T* p)
{
p;
p->~T();
}
template <typename T> inline void destruct_array(T* p, uint n)
{
T* q = p + n;
for ( ; p != q; ++p)
p->~T();
}
} // namespace helpers
} // namespace crnlib } // namespace crnlib
crnlib/crn_huffman_codes.cpp
+322 -343
View File
@@ -3,385 +3,364 @@
#include "crn_core.h" #include "crn_core.h"
#include "crn_huffman_codes.h" #include "crn_huffman_codes.h"
namespace crnlib namespace crnlib {
{ struct sym_freq {
struct sym_freq uint m_freq;
{ uint16 m_left;
uint m_freq; uint16 m_right;
uint16 m_left;
uint16 m_right;
inline bool operator< (const sym_freq& other) const inline bool operator<(const sym_freq& other) const {
{ return m_freq > other.m_freq;
return m_freq > other.m_freq; }
} };
};
static inline sym_freq* radix_sort_syms(uint num_syms, sym_freq* syms0, sym_freq* syms1)
{
const uint cMaxPasses = 2;
uint hist[256 * cMaxPasses];
memset(hist, 0, sizeof(hist[0]) * 256 * cMaxPasses);
sym_freq* p = syms0; static inline sym_freq* radix_sort_syms(uint num_syms, sym_freq* syms0, sym_freq* syms1) {
sym_freq* q = syms0 + (num_syms >> 1) * 2; const uint cMaxPasses = 2;
uint hist[256 * cMaxPasses];
for ( ; p != q; p += 2) memset(hist, 0, sizeof(hist[0]) * 256 * cMaxPasses);
{
const uint freq0 = p[0].m_freq;
const uint freq1 = p[1].m_freq;
hist[ freq0 & 0xFF]++; sym_freq* p = syms0;
hist[256 + ((freq0 >> 8) & 0xFF)]++; sym_freq* q = syms0 + (num_syms >> 1) * 2;
hist[ freq1 & 0xFF]++; for (; p != q; p += 2) {
hist[256 + ((freq1 >> 8) & 0xFF)]++; const uint freq0 = p[0].m_freq;
const uint freq1 = p[1].m_freq;
hist[freq0 & 0xFF]++;
hist[256 + ((freq0 >> 8) & 0xFF)]++;
hist[freq1 & 0xFF]++;
hist[256 + ((freq1 >> 8) & 0xFF)]++;
}
if (num_syms & 1) {
const uint freq = p->m_freq;
hist[freq & 0xFF]++;
hist[256 + ((freq >> 8) & 0xFF)]++;
}
sym_freq* pCur_syms = syms0;
sym_freq* pNew_syms = syms1;
for (uint pass = 0; pass < cMaxPasses; pass++) {
const uint* pHist = &hist[pass << 8];
uint offsets[256];
uint cur_ofs = 0;
for (uint i = 0; i < 256; i += 2) {
offsets[i] = cur_ofs;
cur_ofs += pHist[i];
offsets[i + 1] = cur_ofs;
cur_ofs += pHist[i + 1];
}
const uint pass_shift = pass << 3;
sym_freq* p = pCur_syms;
sym_freq* q = pCur_syms + (num_syms >> 1) * 2;
for (; p != q; p += 2) {
uint c0 = p[0].m_freq;
uint c1 = p[1].m_freq;
if (pass) {
c0 >>= 8;
c1 >>= 8;
} }
if (num_syms & 1) c0 &= 0xFF;
{ c1 &= 0xFF;
const uint freq = p->m_freq;
hist[ freq & 0xFF]++; if (c0 == c1) {
hist[256 + ((freq >> 8) & 0xFF)]++; uint dst_offset0 = offsets[c0];
offsets[c0] = dst_offset0 + 2;
pNew_syms[dst_offset0] = p[0];
pNew_syms[dst_offset0 + 1] = p[1];
} else {
uint dst_offset0 = offsets[c0]++;
uint dst_offset1 = offsets[c1]++;
pNew_syms[dst_offset0] = p[0];
pNew_syms[dst_offset1] = p[1];
} }
}
sym_freq* pCur_syms = syms0;
sym_freq* pNew_syms = syms1;
for (uint pass = 0; pass < cMaxPasses; pass++) if (num_syms & 1) {
{ uint c = ((p->m_freq) >> pass_shift) & 0xFF;
const uint* pHist = &hist[pass << 8];
uint offsets[256]; uint dst_offset = offsets[c];
offsets[c] = dst_offset + 1;
uint cur_ofs = 0; pNew_syms[dst_offset] = *p;
for (uint i = 0; i < 256; i += 2) }
{
offsets[i] = cur_ofs;
cur_ofs += pHist[i];
offsets[i+1] = cur_ofs; sym_freq* t = pCur_syms;
cur_ofs += pHist[i+1]; pCur_syms = pNew_syms;
} pNew_syms = t;
}
const uint pass_shift = pass << 3;
sym_freq* p = pCur_syms;
sym_freq* q = pCur_syms + (num_syms >> 1) * 2;
for ( ; p != q; p += 2)
{
uint c0 = p[0].m_freq;
uint c1 = p[1].m_freq;
if (pass)
{
c0 >>= 8;
c1 >>= 8;
}
c0 &= 0xFF;
c1 &= 0xFF;
if (c0 == c1)
{
uint dst_offset0 = offsets[c0];
offsets[c0] = dst_offset0 + 2;
pNew_syms[dst_offset0] = p[0];
pNew_syms[dst_offset0 + 1] = p[1];
}
else
{
uint dst_offset0 = offsets[c0]++;
uint dst_offset1 = offsets[c1]++;
pNew_syms[dst_offset0] = p[0];
pNew_syms[dst_offset1] = p[1];
}
}
if (num_syms & 1)
{
uint c = ((p->m_freq) >> pass_shift) & 0xFF;
uint dst_offset = offsets[c];
offsets[c] = dst_offset + 1;
pNew_syms[dst_offset] = *p;
}
sym_freq* t = pCur_syms;
pCur_syms = pNew_syms;
pNew_syms = t;
}
#ifdef CRNLIB_ASSERTS_ENABLED #ifdef CRNLIB_ASSERTS_ENABLED
uint prev_freq = 0; uint prev_freq = 0;
for (uint i = 0; i < num_syms; i++) for (uint i = 0; i < num_syms; i++) {
{ CRNLIB_ASSERT(!(pCur_syms[i].m_freq < prev_freq));
CRNLIB_ASSERT(!(pCur_syms[i].m_freq < prev_freq)); prev_freq = pCur_syms[i].m_freq;
prev_freq = pCur_syms[i].m_freq; }
}
#endif #endif
return pCur_syms;
}
struct huffman_work_tables
{
enum { cMaxInternalNodes = cHuffmanMaxSupportedSyms };
sym_freq syms0[cHuffmanMaxSupportedSyms + 1 + cMaxInternalNodes];
sym_freq syms1[cHuffmanMaxSupportedSyms + 1 + cMaxInternalNodes];
uint16 queue[cMaxInternalNodes];
};
void* create_generate_huffman_codes_tables()
{
return crnlib_new<huffman_work_tables>();
}
void free_generate_huffman_codes_tables(void* p)
{
crnlib_delete(static_cast<huffman_work_tables*>(p));
}
#if USE_CALCULATE_MINIMUM_REDUNDANCY return pCur_syms;
/* calculate_minimum_redundancy() written by }
struct huffman_work_tables {
enum { cMaxInternalNodes = cHuffmanMaxSupportedSyms };
sym_freq syms0[cHuffmanMaxSupportedSyms + 1 + cMaxInternalNodes];
sym_freq syms1[cHuffmanMaxSupportedSyms + 1 + cMaxInternalNodes];
uint16 queue[cMaxInternalNodes];
};
void* create_generate_huffman_codes_tables() {
return crnlib_new<huffman_work_tables>();
}
void free_generate_huffman_codes_tables(void* p) {
crnlib_delete(static_cast<huffman_work_tables*>(p));
}
#if USE_CALCULATE_MINIMUM_REDUNDANCY
/* calculate_minimum_redundancy() written by
Alistair Moffat, alistair@cs.mu.oz.au, Alistair Moffat, alistair@cs.mu.oz.au,
Jyrki Katajainen, jyrki@diku.dk Jyrki Katajainen, jyrki@diku.dk
November 1996. November 1996.
*/ */
static void calculate_minimum_redundancy(int A[], int n) { static void calculate_minimum_redundancy(int A[], int n) {
int root; /* next root node to be used */ int root; /* next root node to be used */
int leaf; /* next leaf to be used */ int leaf; /* next leaf to be used */
int next; /* next value to be assigned */ int next; /* next value to be assigned */
int avbl; /* number of available nodes */ int avbl; /* number of available nodes */
int used; /* number of internal nodes */ int used; /* number of internal nodes */
int dpth; /* current depth of leaves */ int dpth; /* current depth of leaves */
/* check for pathological cases */ /* check for pathological cases */
if (n==0) { return; } if (n == 0) {
if (n==1) { A[0] = 0; return; } return;
}
if (n == 1) {
A[0] = 0;
return;
}
/* first pass, left to right, setting parent pointers */ /* first pass, left to right, setting parent pointers */
A[0] += A[1]; root = 0; leaf = 2; A[0] += A[1];
for (next=1; next < n-1; next++) { root = 0;
/* select first item for a pairing */ leaf = 2;
if (leaf>=n || A[root]<A[leaf]) { for (next = 1; next < n - 1; next++) {
A[next] = A[root]; A[root++] = next; /* select first item for a pairing */
} else if (leaf >= n || A[root] < A[leaf]) {
A[next] = A[leaf++]; A[next] = A[root];
A[root++] = next;
} else
A[next] = A[leaf++];
/* add on the second item */ /* add on the second item */
if (leaf>=n || (root<next && A[root]<A[leaf])) { if (leaf >= n || (root < next && A[root] < A[leaf])) {
A[next] += A[root]; A[root++] = next; A[next] += A[root];
} else A[root++] = next;
A[next] += A[leaf++]; } else
} A[next] += A[leaf++];
}
/* second pass, right to left, setting internal depths */ /* second pass, right to left, setting internal depths */
A[n-2] = 0; A[n - 2] = 0;
for (next=n-3; next>=0; next--) for (next = n - 3; next >= 0; next--)
A[next] = A[A[next]]+1; A[next] = A[A[next]] + 1;
/* third pass, right to left, setting leaf depths */ /* third pass, right to left, setting leaf depths */
avbl = 1; used = dpth = 0; root = n-2; next = n-1; avbl = 1;
while (avbl>0) { used = dpth = 0;
while (root>=0 && A[root]==dpth) { root = n - 2;
used++; root--; next = n - 1;
} while (avbl > 0) {
while (avbl>used) { while (root >= 0 && A[root] == dpth) {
A[next--] = dpth; avbl--; used++;
} root--;
avbl = 2*used; dpth++; used = 0; }
} while (avbl > used) {
} A[next--] = dpth;
avbl--;
}
avbl = 2 * used;
dpth++;
used = 0;
}
}
#endif #endif
bool generate_huffman_codes(void* pContext, uint num_syms, const uint16* pFreq, uint8* pCodesizes, uint& max_code_size, uint& total_freq_ret)
{
if ((!num_syms) || (num_syms > cHuffmanMaxSupportedSyms))
return false;
huffman_work_tables& state = *static_cast<huffman_work_tables*>(pContext);;
uint max_freq = 0;
uint total_freq = 0;
uint num_used_syms = 0;
for (uint i = 0; i < num_syms; i++)
{
uint freq = pFreq[i];
if (!freq)
pCodesizes[i] = 0;
else
{
total_freq += freq;
max_freq = math::maximum(max_freq, freq);
sym_freq& sf = state.syms0[num_used_syms];
sf.m_left = (uint16)i;
sf.m_right = cUINT16_MAX;
sf.m_freq = freq;
num_used_syms++;
}
}
total_freq_ret = total_freq;
if (num_used_syms == 1) bool generate_huffman_codes(void* pContext, uint num_syms, const uint16* pFreq, uint8* pCodesizes, uint& max_code_size, uint& total_freq_ret) {
{ if ((!num_syms) || (num_syms > cHuffmanMaxSupportedSyms))
pCodesizes[state.syms0[0].m_left] = 1; return false;
return true;
}
sym_freq* syms = radix_sort_syms(num_used_syms, state.syms0, state.syms1); huffman_work_tables& state = *static_cast<huffman_work_tables*>(pContext);
;
uint max_freq = 0;
uint total_freq = 0;
uint num_used_syms = 0;
for (uint i = 0; i < num_syms; i++) {
uint freq = pFreq[i];
if (!freq)
pCodesizes[i] = 0;
else {
total_freq += freq;
max_freq = math::maximum(max_freq, freq);
sym_freq& sf = state.syms0[num_used_syms];
sf.m_left = (uint16)i;
sf.m_right = cUINT16_MAX;
sf.m_freq = freq;
num_used_syms++;
}
}
total_freq_ret = total_freq;
if (num_used_syms == 1) {
pCodesizes[state.syms0[0].m_left] = 1;
return true;
}
sym_freq* syms = radix_sort_syms(num_used_syms, state.syms0, state.syms1);
#if USE_CALCULATE_MINIMUM_REDUNDANCY #if USE_CALCULATE_MINIMUM_REDUNDANCY
int x[cHuffmanMaxSupportedSyms]; int x[cHuffmanMaxSupportedSyms];
for (uint i = 0; i < num_used_syms; i++) for (uint i = 0; i < num_used_syms; i++)
x[i] = state.syms0[i].m_freq; x[i] = state.syms0[i].m_freq;
calculate_minimum_redundancy(x, num_used_syms); calculate_minimum_redundancy(x, num_used_syms);
uint max_len = 0; uint max_len = 0;
for (uint i = 0; i < num_used_syms; i++) for (uint i = 0; i < num_used_syms; i++) {
{ uint len = x[i];
uint len = x[i]; max_len = math::maximum(len, max_len);
max_len = math::maximum(len, max_len); pCodesizes[state.syms0[i].m_left] = static_cast<uint8>(len);
pCodesizes[state.syms0[i].m_left] = static_cast<uint8>(len); }
return true;
#else
// Dummy node
sym_freq& sf = state.syms0[num_used_syms];
sf.m_left = cUINT16_MAX;
sf.m_right = cUINT16_MAX;
sf.m_freq = UINT_MAX;
uint next_internal_node = num_used_syms + 1;
uint queue_front = 0;
uint queue_end = 0;
uint next_lowest_sym = 0;
uint num_nodes_remaining = num_used_syms;
do {
uint left_freq = syms[next_lowest_sym].m_freq;
uint left_child = next_lowest_sym;
if ((queue_end > queue_front) && (syms[state.queue[queue_front]].m_freq < left_freq)) {
left_child = state.queue[queue_front];
left_freq = syms[left_child].m_freq;
queue_front++;
} else
next_lowest_sym++;
uint right_freq = syms[next_lowest_sym].m_freq;
uint right_child = next_lowest_sym;
if ((queue_end > queue_front) && (syms[state.queue[queue_front]].m_freq < right_freq)) {
right_child = state.queue[queue_front];
right_freq = syms[right_child].m_freq;
queue_front++;
} else
next_lowest_sym++;
const uint internal_node_index = next_internal_node;
next_internal_node++;
CRNLIB_ASSERT(next_internal_node < CRNLIB_ARRAYSIZE(state.syms0));
syms[internal_node_index].m_freq = left_freq + right_freq;
syms[internal_node_index].m_left = static_cast<uint16>(left_child);
syms[internal_node_index].m_right = static_cast<uint16>(right_child);
CRNLIB_ASSERT(queue_end < huffman_work_tables::cMaxInternalNodes);
state.queue[queue_end] = static_cast<uint16>(internal_node_index);
queue_end++;
num_nodes_remaining--;
} while (num_nodes_remaining > 1);
CRNLIB_ASSERT(next_lowest_sym == num_used_syms);
CRNLIB_ASSERT((queue_end - queue_front) == 1);
uint cur_node_index = state.queue[queue_front];
uint32* pStack = (syms == state.syms0) ? (uint32*)state.syms1 : (uint32*)state.syms0;
uint32* pStack_top = pStack;
uint max_level = 0;
for (;;) {
uint level = cur_node_index >> 16;
uint node_index = cur_node_index & 0xFFFF;
uint left_child = syms[node_index].m_left;
uint right_child = syms[node_index].m_right;
uint next_level = (cur_node_index + 0x10000) & 0xFFFF0000;
if (left_child < num_used_syms) {
max_level = math::maximum(max_level, level);
pCodesizes[syms[left_child].m_left] = static_cast<uint8>(level + 1);
if (right_child < num_used_syms) {
pCodesizes[syms[right_child].m_left] = static_cast<uint8>(level + 1);
if (pStack == pStack_top)
break;
cur_node_index = *--pStack;
} else {
cur_node_index = next_level | right_child;
} }
} else {
return true; if (right_child < num_used_syms) {
#else max_level = math::maximum(max_level, level);
// Dummy node
sym_freq& sf = state.syms0[num_used_syms];
sf.m_left = cUINT16_MAX;
sf.m_right = cUINT16_MAX;
sf.m_freq = UINT_MAX;
uint next_internal_node = num_used_syms + 1;
uint queue_front = 0;
uint queue_end = 0;
uint next_lowest_sym = 0;
uint num_nodes_remaining = num_used_syms;
do
{
uint left_freq = syms[next_lowest_sym].m_freq;
uint left_child = next_lowest_sym;
if ((queue_end > queue_front) && (syms[state.queue[queue_front]].m_freq < left_freq))
{
left_child = state.queue[queue_front];
left_freq = syms[left_child].m_freq;
queue_front++;
}
else
next_lowest_sym++;
uint right_freq = syms[next_lowest_sym].m_freq;
uint right_child = next_lowest_sym;
if ((queue_end > queue_front) && (syms[state.queue[queue_front]].m_freq < right_freq)) pCodesizes[syms[right_child].m_left] = static_cast<uint8>(level + 1);
{
right_child = state.queue[queue_front];
right_freq = syms[right_child].m_freq;
queue_front++;
}
else
next_lowest_sym++;
const uint internal_node_index = next_internal_node;
next_internal_node++;
CRNLIB_ASSERT(next_internal_node < CRNLIB_ARRAYSIZE(state.syms0)); cur_node_index = next_level | left_child;
} else {
syms[internal_node_index].m_freq = left_freq + right_freq; *pStack++ = next_level | left_child;
syms[internal_node_index].m_left = static_cast<uint16>(left_child);
syms[internal_node_index].m_right = static_cast<uint16>(right_child);
CRNLIB_ASSERT(queue_end < huffman_work_tables::cMaxInternalNodes);
state.queue[queue_end] = static_cast<uint16>(internal_node_index);
queue_end++;
num_nodes_remaining--;
} while (num_nodes_remaining > 1);
CRNLIB_ASSERT(next_lowest_sym == num_used_syms);
CRNLIB_ASSERT((queue_end - queue_front) == 1);
uint cur_node_index = state.queue[queue_front];
uint32* pStack = (syms == state.syms0) ? (uint32*)state.syms1 : (uint32*)state.syms0;
uint32* pStack_top = pStack;
uint max_level = 0; cur_node_index = next_level | right_child;
for ( ; ; )
{
uint level = cur_node_index >> 16;
uint node_index = cur_node_index & 0xFFFF;
uint left_child = syms[node_index].m_left;
uint right_child = syms[node_index].m_right;
uint next_level = (cur_node_index + 0x10000) & 0xFFFF0000;
if (left_child < num_used_syms)
{
max_level = math::maximum(max_level, level);
pCodesizes[syms[left_child].m_left] = static_cast<uint8>(level + 1);
if (right_child < num_used_syms)
{
pCodesizes[syms[right_child].m_left] = static_cast<uint8>(level + 1);
if (pStack == pStack_top) break;
cur_node_index = *--pStack;
}
else
{
cur_node_index = next_level | right_child;
}
}
else
{
if (right_child < num_used_syms)
{
max_level = math::maximum(max_level, level);
pCodesizes[syms[right_child].m_left] = static_cast<uint8>(level + 1);
cur_node_index = next_level | left_child;
}
else
{
*pStack++ = next_level | left_child;
cur_node_index = next_level | right_child;
}
}
} }
}
max_code_size = max_level + 1; }
max_code_size = max_level + 1;
#endif #endif
return true;
}
} // namespace crnlib return true;
}
} // namespace crnlib
crnlib/crn_huffman_codes.h
+7 -8
View File
@@ -2,13 +2,12 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
namespace crnlib namespace crnlib {
{ const uint cHuffmanMaxSupportedSyms = 8192;
const uint cHuffmanMaxSupportedSyms = 8192;
void* create_generate_huffman_codes_tables(); void* create_generate_huffman_codes_tables();
void free_generate_huffman_codes_tables(void* p); void free_generate_huffman_codes_tables(void* p);
bool generate_huffman_codes(void* pContext, uint num_syms, const uint16* pFreq, uint8* pCodesizes, uint& max_code_size, uint& total_freq_ret);
} // namespace crnlib bool generate_huffman_codes(void* pContext, uint num_syms, const uint16* pFreq, uint8* pCodesizes, uint& max_code_size, uint& total_freq_ret);
} // namespace crnlib
crnlib/crn_image.h
+601 -689
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_image_utils.cpp
+1111 -1261
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_image_utils.h
+130 -140
View File
@@ -4,190 +4,180 @@
#include "crn_image.h" #include "crn_image.h"
#include "crn_data_stream_serializer.h" #include "crn_data_stream_serializer.h"
namespace crnlib namespace crnlib {
{ enum pixel_format;
enum pixel_format;
namespace image_utils namespace image_utils {
{ enum read_flags_t {
enum read_flags_t cReadFlagForceSTB = 1,
{
cReadFlagForceSTB = 1,
cReadFlagsAllFlags = 1 cReadFlagsAllFlags = 1
}; };
bool read_from_stream_stb(data_stream_serializer& serializer, image_u8& img); bool read_from_stream_stb(data_stream_serializer& serializer, image_u8& img);
bool read_from_stream_jpgd(data_stream_serializer& serializer, image_u8& img); bool read_from_stream_jpgd(data_stream_serializer& serializer, image_u8& img);
bool read_from_stream(image_u8& dest, data_stream_serializer& serializer, uint read_flags = 0); bool read_from_stream(image_u8& dest, data_stream_serializer& serializer, uint read_flags = 0);
bool read_from_file(image_u8& dest, const char* pFilename, uint read_flags = 0); bool read_from_file(image_u8& dest, const char* pFilename, uint read_flags = 0);
// Reads texture from memory, results returned stb_image.c style. // Reads texture from memory, results returned stb_image.c style.
// *pActual_comps is set to 1, 3, or 4. req_comps must range from 1-4. // *pActual_comps is set to 1, 3, or 4. req_comps must range from 1-4.
uint8* read_from_memory(const uint8* pImage, int nSize, int* pWidth, int* pHeight, int* pActualComps, int req_comps, const char* pFilename); uint8* read_from_memory(const uint8* pImage, int nSize, int* pWidth, int* pHeight, int* pActualComps, int req_comps, const char* pFilename);
enum enum {
{ cWriteFlagIgnoreAlpha = 0x00000001,
cWriteFlagIgnoreAlpha = 0x00000001, cWriteFlagGrayscale = 0x00000002,
cWriteFlagGrayscale = 0x00000002,
cWriteFlagJPEGH1V1 = 0x00010000, cWriteFlagJPEGH1V1 = 0x00010000,
cWriteFlagJPEGH2V1 = 0x00020000, cWriteFlagJPEGH2V1 = 0x00020000,
cWriteFlagJPEGH2V2 = 0x00040000, cWriteFlagJPEGH2V2 = 0x00040000,
cWriteFlagJPEGTwoPass = 0x00080000, cWriteFlagJPEGTwoPass = 0x00080000,
cWriteFlagJPEGNoChromaDiscrim = 0x00100000, cWriteFlagJPEGNoChromaDiscrim = 0x00100000,
cWriteFlagJPEGQualityLevelMask = 0xFF000000, cWriteFlagJPEGQualityLevelMask = 0xFF000000,
cWriteFlagJPEGQualityLevelShift = 24, cWriteFlagJPEGQualityLevelShift = 24,
}; };
const int cLumaComponentIndex = -1; const int cLumaComponentIndex = -1;
inline uint create_jpeg_write_flags(uint base_flags, uint quality_level) { CRNLIB_ASSERT(quality_level <= 100); return base_flags | ((quality_level << cWriteFlagJPEGQualityLevelShift) & cWriteFlagJPEGQualityLevelMask); } inline uint create_jpeg_write_flags(uint base_flags, uint quality_level) {
CRNLIB_ASSERT(quality_level <= 100);
return base_flags | ((quality_level << cWriteFlagJPEGQualityLevelShift) & cWriteFlagJPEGQualityLevelMask);
}
bool write_to_file(const char* pFilename, const image_u8& img, uint write_flags = 0, int grayscale_comp_index = cLumaComponentIndex); bool write_to_file(const char* pFilename, const image_u8& img, uint write_flags = 0, int grayscale_comp_index = cLumaComponentIndex);
bool has_alpha(const image_u8& img); bool has_alpha(const image_u8& img);
bool is_normal_map(const image_u8& img, const char* pFilename = NULL); bool is_normal_map(const image_u8& img, const char* pFilename = NULL);
void renorm_normal_map(image_u8& img); void renorm_normal_map(image_u8& img);
struct resample_params struct resample_params {
{ resample_params()
resample_params() : : m_dst_width(0),
m_dst_width(0), m_dst_height(0),
m_dst_height(0), m_pFilter("lanczos4"),
m_pFilter("lanczos4"), m_filter_scale(1.0f),
m_filter_scale(1.0f), m_srgb(true),
m_srgb(true), m_wrapping(false),
m_wrapping(false), m_first_comp(0),
m_first_comp(0), m_num_comps(4),
m_num_comps(4), m_source_gamma(2.2f), // 1.75f
m_source_gamma(2.2f), // 1.75f m_multithreaded(true) {
m_multithreaded(true) }
{
}
uint m_dst_width; uint m_dst_width;
uint m_dst_height; uint m_dst_height;
const char* m_pFilter; const char* m_pFilter;
float m_filter_scale; float m_filter_scale;
bool m_srgb; bool m_srgb;
bool m_wrapping; bool m_wrapping;
uint m_first_comp; uint m_first_comp;
uint m_num_comps; uint m_num_comps;
float m_source_gamma; float m_source_gamma;
bool m_multithreaded; bool m_multithreaded;
}; };
bool resample_single_thread(const image_u8& src, image_u8& dst, const resample_params& params); bool resample_single_thread(const image_u8& src, image_u8& dst, const resample_params& params);
bool resample_multithreaded(const image_u8& src, image_u8& dst, const resample_params& params); bool resample_multithreaded(const image_u8& src, image_u8& dst, const resample_params& params);
bool resample(const image_u8& src, image_u8& dst, const resample_params& params); bool resample(const image_u8& src, image_u8& dst, const resample_params& params);
bool compute_delta(image_u8& dest, image_u8& a, image_u8& b, uint scale = 2); bool compute_delta(image_u8& dest, image_u8& a, image_u8& b, uint scale = 2);
class error_metrics class error_metrics {
{ public:
public: error_metrics() { utils::zero_this(this); }
error_metrics() { utils::zero_this(this); }
void print(const char* pName) const; void print(const char* pName) const;
// If num_channels==0, luma error is computed. // If num_channels==0, luma error is computed.
// If pHist != NULL, it must point to a 256 entry array. // If pHist != NULL, it must point to a 256 entry array.
bool compute(const image_u8& a, const image_u8& b, uint first_channel, uint num_channels, bool average_component_error = true); bool compute(const image_u8& a, const image_u8& b, uint first_channel, uint num_channels, bool average_component_error = true);
uint mMax; uint mMax;
double mMean; double mMean;
double mMeanSquared; double mMeanSquared;
double mRootMeanSquared; double mRootMeanSquared;
double mPeakSNR; double mPeakSNR;
inline bool operator== (const error_metrics& other) const inline bool operator==(const error_metrics& other) const {
{ return mPeakSNR == other.mPeakSNR;
return mPeakSNR == other.mPeakSNR; }
}
inline bool operator< (const error_metrics& other) const inline bool operator<(const error_metrics& other) const {
{ return mPeakSNR < other.mPeakSNR;
return mPeakSNR < other.mPeakSNR; }
}
inline bool operator> (const error_metrics& other) const inline bool operator>(const error_metrics& other) const {
{ return mPeakSNR > other.mPeakSNR;
return mPeakSNR > other.mPeakSNR; }
} };
};
void print_image_metrics(const image_u8& src_img, const image_u8& dst_img); void print_image_metrics(const image_u8& src_img, const image_u8& dst_img);
double compute_block_ssim(uint n, const uint8* pX, const uint8* pY); double compute_block_ssim(uint n, const uint8* pX, const uint8* pY);
double compute_ssim(const image_u8& a, const image_u8& b, int channel_index); double compute_ssim(const image_u8& a, const image_u8& b, int channel_index);
void print_ssim(const image_u8& src_img, const image_u8& dst_img); void print_ssim(const image_u8& src_img, const image_u8& dst_img);
enum conversion_type enum conversion_type {
{ cConversion_Invalid = -1,
cConversion_Invalid = -1,
cConversion_To_CCxY, cConversion_To_CCxY,
cConversion_From_CCxY, cConversion_From_CCxY,
cConversion_To_xGxR, cConversion_To_xGxR,
cConversion_From_xGxR, cConversion_From_xGxR,
cConversion_To_xGBR, cConversion_To_xGBR,
cConversion_From_xGBR, cConversion_From_xGBR,
cConversion_To_AGBR, cConversion_To_AGBR,
cConversion_From_AGBR, cConversion_From_AGBR,
cConversion_XY_to_XYZ, cConversion_XY_to_XYZ,
cConversion_Y_To_A, cConversion_Y_To_A,
cConversion_A_To_RGBA, cConversion_A_To_RGBA,
cConversion_Y_To_RGB, cConversion_Y_To_RGB,
cConversion_To_Y, cConversion_To_Y,
cConversionTotal cConversionTotal
}; };
void convert_image(image_u8& img, conversion_type conv_type); void convert_image(image_u8& img, conversion_type conv_type);
template<typename image_type> template <typename image_type>
inline uint8* pack_image(const image_type& img, const pixel_packer& packer, uint& n) inline uint8* pack_image(const image_type& img, const pixel_packer& packer, uint& n) {
{ n = 0;
n = 0;
if (!packer.is_valid()) if (!packer.is_valid())
return NULL; return NULL;
const uint width = img.get_width(), height = img.get_height(); const uint width = img.get_width(), height = img.get_height();
uint dst_pixel_stride = packer.get_pixel_stride(); uint dst_pixel_stride = packer.get_pixel_stride();
uint dst_pitch = width * dst_pixel_stride; uint dst_pitch = width * dst_pixel_stride;
n = dst_pitch * height; n = dst_pitch * height;
uint8* pImage = static_cast<uint8*>(crnlib_malloc(n)); uint8* pImage = static_cast<uint8*>(crnlib_malloc(n));
uint8* pDst = pImage; uint8* pDst = pImage;
for (uint y = 0; y < height; y++) for (uint y = 0; y < height; y++) {
{ const typename image_type::color_t* pSrc = img.get_scanline(y);
const typename image_type::color_t* pSrc = img.get_scanline(y); for (uint x = 0; x < width; x++)
for (uint x = 0; x < width; x++) pDst = (uint8*)packer.pack(*pSrc++, pDst);
pDst = (uint8*)packer.pack(*pSrc++, pDst); }
}
return pImage; return pImage;
} }
image_utils::conversion_type get_conversion_type(bool cooking, pixel_format fmt); image_utils::conversion_type get_conversion_type(bool cooking, pixel_format fmt);
image_utils::conversion_type get_image_conversion_type_from_crn_format(crn_format fmt); image_utils::conversion_type get_image_conversion_type_from_crn_format(crn_format fmt);
double compute_std_dev(uint n, const color_quad_u8* pPixels, uint first_channel, uint num_channels); double compute_std_dev(uint n, const color_quad_u8* pPixels, uint first_channel, uint num_channels);
uint8* read_image_from_memory(const uint8* pImage, int nSize, int* pWidth, int* pHeight, int* pActualComps, int req_comps, const char* pFilename); uint8* read_image_from_memory(const uint8* pImage, int nSize, int* pWidth, int* pHeight, int* pActualComps, int req_comps, const char* pFilename);
} // namespace image_utils } // namespace image_utils
} // namespace crnlib } // namespace crnlib
crnlib/crn_intersect.h
+88 -107
View File
@@ -3,121 +3,102 @@
#pragma once #pragma once
#include "crn_ray.h" #include "crn_ray.h"
namespace crnlib namespace crnlib {
{ namespace intersection {
namespace intersection enum result {
{ cBackfacing = -1,
enum result cFailure = 0,
{ cSuccess,
cBackfacing = -1, cParallel,
cFailure = 0, cInside,
cSuccess, };
cParallel,
cInside,
};
// Returns cInside, cSuccess, or cFailure.
// Algorithm: Graphics Gems 1
template<typename vector_type, typename scalar_type, typename ray_type, typename aabb_type>
result ray_aabb(vector_type& coord, scalar_type& t, const ray_type& ray, const aabb_type& box)
{
enum
{
cNumDim = vector_type::num_elements,
cRight = 0,
cLeft = 1,
cMiddle = 2
};
bool inside = true; // Returns cInside, cSuccess, or cFailure.
int quadrant[cNumDim]; // Algorithm: Graphics Gems 1
scalar_type candidate_plane[cNumDim]; template <typename vector_type, typename scalar_type, typename ray_type, typename aabb_type>
result ray_aabb(vector_type& coord, scalar_type& t, const ray_type& ray, const aabb_type& box) {
enum {
cNumDim = vector_type::num_elements,
cRight = 0,
cLeft = 1,
cMiddle = 2
};
for (int i = 0; i < cNumDim; i++) bool inside = true;
{ int quadrant[cNumDim];
if (ray.get_origin()[i] < box[0][i]) scalar_type candidate_plane[cNumDim];
{
quadrant[i] = cLeft;
candidate_plane[i] = box[0][i];
inside = false;
}
else if (ray.get_origin()[i] > box[1][i])
{
quadrant[i] = cRight;
candidate_plane[i] = box[1][i];
inside = false;
}
else
{
quadrant[i] = cMiddle;
}
}
if (inside) for (int i = 0; i < cNumDim; i++) {
{ if (ray.get_origin()[i] < box[0][i]) {
coord = ray.get_origin(); quadrant[i] = cLeft;
t = 0.0f; candidate_plane[i] = box[0][i];
return cInside; inside = false;
} } else if (ray.get_origin()[i] > box[1][i]) {
quadrant[i] = cRight;
candidate_plane[i] = box[1][i];
inside = false;
} else {
quadrant[i] = cMiddle;
}
}
scalar_type max_t[cNumDim]; if (inside) {
for (int i = 0; i < cNumDim; i++) coord = ray.get_origin();
{ t = 0.0f;
if ((quadrant[i] != cMiddle) && (ray.get_direction()[i] != 0.0f)) return cInside;
max_t[i] = (candidate_plane[i] - ray.get_origin()[i]) / ray.get_direction()[i]; }
else
max_t[i] = -1.0f;
}
int which_plane = 0; scalar_type max_t[cNumDim];
for (int i = 1; i < cNumDim; i++) for (int i = 0; i < cNumDim; i++) {
if (max_t[which_plane] < max_t[i]) if ((quadrant[i] != cMiddle) && (ray.get_direction()[i] != 0.0f))
which_plane = i; max_t[i] = (candidate_plane[i] - ray.get_origin()[i]) / ray.get_direction()[i];
else
max_t[i] = -1.0f;
}
if (max_t[which_plane] < 0.0f) int which_plane = 0;
return cFailure; for (int i = 1; i < cNumDim; i++)
if (max_t[which_plane] < max_t[i])
which_plane = i;
for (int i = 0; i < cNumDim; i++) if (max_t[which_plane] < 0.0f)
{ return cFailure;
if (i != which_plane)
{
coord[i] = ray.get_origin()[i] + max_t[which_plane] * ray.get_direction()[i];
if ( (coord[i] < box[0][i]) || (coord[i] > box[1][i]) ) for (int i = 0; i < cNumDim; i++) {
return cFailure; if (i != which_plane) {
} coord[i] = ray.get_origin()[i] + max_t[which_plane] * ray.get_direction()[i];
else
{
coord[i] = candidate_plane[i];
}
CRNLIB_ASSERT(coord[i] >= box[0][i] && coord[i] <= box[1][i]); if ((coord[i] < box[0][i]) || (coord[i] > box[1][i]))
} return cFailure;
} else {
coord[i] = candidate_plane[i];
}
t = max_t[which_plane]; CRNLIB_ASSERT(coord[i] >= box[0][i] && coord[i] <= box[1][i]);
return cSuccess; }
}
t = max_t[which_plane];
template<typename vector_type, typename scalar_type, typename ray_type, typename aabb_type> return cSuccess;
result ray_aabb(bool& started_within, vector_type& coord, scalar_type& t, const ray_type& ray, const aabb_type& box) }
{
if (!box.contains(ray.get_origin())) template <typename vector_type, typename scalar_type, typename ray_type, typename aabb_type>
{ result ray_aabb(bool& started_within, vector_type& coord, scalar_type& t, const ray_type& ray, const aabb_type& box) {
started_within = false; if (!box.contains(ray.get_origin())) {
return ray_aabb(coord, t, ray, box); started_within = false;
} return ray_aabb(coord, t, ray, box);
}
started_within = true;
started_within = true;
float diag_dist = box.diagonal_length() * 1.5f;
ray_type outside_ray(ray.eval(diag_dist), -ray.get_direction()); float diag_dist = box.diagonal_length() * 1.5f;
ray_type outside_ray(ray.eval(diag_dist), -ray.get_direction());
result res(ray_aabb(coord, t, outside_ray, box));
if (res != cSuccess) result res(ray_aabb(coord, t, outside_ray, box));
return res; if (res != cSuccess)
return res;
t = math::maximum(0.0f, diag_dist - t);
return cSuccess; t = math::maximum(0.0f, diag_dist - t);
} return cSuccess;
} }
}
} }
crnlib/crn_jpgd.cpp
+926 -1155
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_jpgd.h
+323 -293
View File
@@ -8,312 +8,342 @@
#include <setjmp.h> #include <setjmp.h>
#ifdef _MSC_VER #ifdef _MSC_VER
#define JPGD_NORETURN __declspec(noreturn) #define JPGD_NORETURN __declspec(noreturn)
#elif defined(__GNUC__) #elif defined(__GNUC__)
#define JPGD_NORETURN __attribute__ ((noreturn)) #define JPGD_NORETURN __attribute__((noreturn))
#else #else
#define JPGD_NORETURN #define JPGD_NORETURN
#endif #endif
namespace jpgd namespace jpgd {
{ typedef unsigned char uint8;
typedef unsigned char uint8; typedef signed short int16;
typedef signed short int16; typedef unsigned short uint16;
typedef unsigned short uint16; typedef unsigned int uint;
typedef unsigned int uint; typedef signed int int32;
typedef signed int int32;
// Loads a JPEG image from a memory buffer or a file. // Loads a JPEG image from a memory buffer or a file.
// req_comps can be 1 (grayscale), 3 (RGB), or 4 (RGBA). // req_comps can be 1 (grayscale), 3 (RGB), or 4 (RGBA).
// On return, width/height will be set to the image's dimensions, and actual_comps will be set to the either 1 (grayscale) or 3 (RGB). // On return, width/height will be set to the image's dimensions, and actual_comps will be set to the either 1 (grayscale) or 3 (RGB).
// Notes: For more control over where and how the source data is read, see the decompress_jpeg_image_from_stream() function below, or call the jpeg_decoder class directly. // Notes: For more control over where and how the source data is read, see the decompress_jpeg_image_from_stream() function below, or call the jpeg_decoder class directly.
// Requesting a 8 or 32bpp image is currently a little faster than 24bpp because the jpeg_decoder class itself currently always unpacks to either 8 or 32bpp. // Requesting a 8 or 32bpp image is currently a little faster than 24bpp because the jpeg_decoder class itself currently always unpacks to either 8 or 32bpp.
unsigned char *decompress_jpeg_image_from_memory(const unsigned char *pSrc_data, int src_data_size, int *width, int *height, int *actual_comps, int req_comps); unsigned char* decompress_jpeg_image_from_memory(const unsigned char* pSrc_data, int src_data_size, int* width, int* height, int* actual_comps, int req_comps);
unsigned char *decompress_jpeg_image_from_file(const char *pSrc_filename, int *width, int *height, int *actual_comps, int req_comps); unsigned char* decompress_jpeg_image_from_file(const char* pSrc_filename, int* width, int* height, int* actual_comps, int req_comps);
// Success/failure error codes. // Success/failure error codes.
enum jpgd_status enum jpgd_status {
{ JPGD_SUCCESS = 0,
JPGD_SUCCESS = 0, JPGD_FAILED = -1, JPGD_DONE = 1, JPGD_FAILED = -1,
JPGD_BAD_DHT_COUNTS = -256, JPGD_BAD_DHT_INDEX, JPGD_BAD_DHT_MARKER, JPGD_BAD_DQT_MARKER, JPGD_BAD_DQT_TABLE, JPGD_DONE = 1,
JPGD_BAD_PRECISION, JPGD_BAD_HEIGHT, JPGD_BAD_WIDTH, JPGD_TOO_MANY_COMPONENTS, JPGD_BAD_DHT_COUNTS = -256,
JPGD_BAD_SOF_LENGTH, JPGD_BAD_VARIABLE_MARKER, JPGD_BAD_DRI_LENGTH, JPGD_BAD_SOS_LENGTH, JPGD_BAD_DHT_INDEX,
JPGD_BAD_SOS_COMP_ID, JPGD_W_EXTRA_BYTES_BEFORE_MARKER, JPGD_NO_ARITHMITIC_SUPPORT, JPGD_UNEXPECTED_MARKER, JPGD_BAD_DHT_MARKER,
JPGD_NOT_JPEG, JPGD_UNSUPPORTED_MARKER, JPGD_BAD_DQT_LENGTH, JPGD_TOO_MANY_BLOCKS, JPGD_BAD_DQT_MARKER,
JPGD_UNDEFINED_QUANT_TABLE, JPGD_UNDEFINED_HUFF_TABLE, JPGD_NOT_SINGLE_SCAN, JPGD_UNSUPPORTED_COLORSPACE, JPGD_BAD_DQT_TABLE,
JPGD_UNSUPPORTED_SAMP_FACTORS, JPGD_DECODE_ERROR, JPGD_BAD_RESTART_MARKER, JPGD_ASSERTION_ERROR, JPGD_BAD_PRECISION,
JPGD_BAD_SOS_SPECTRAL, JPGD_BAD_SOS_SUCCESSIVE, JPGD_STREAM_READ, JPGD_NOTENOUGHMEM JPGD_BAD_HEIGHT,
}; JPGD_BAD_WIDTH,
JPGD_TOO_MANY_COMPONENTS,
// Input stream interface. JPGD_BAD_SOF_LENGTH,
// Derive from this class to read input data from sources other than files or memory. Set m_eof_flag to true when no more data is available. JPGD_BAD_VARIABLE_MARKER,
// The decoder is rather greedy: it will keep on calling this method until its internal input buffer is full, or until the EOF flag is set. JPGD_BAD_DRI_LENGTH,
// It the input stream contains data after the JPEG stream's EOI (end of image) marker it will probably be pulled into the internal buffer. JPGD_BAD_SOS_LENGTH,
// Call the get_total_bytes_read() method to determine the actual size of the JPEG stream after successful decoding. JPGD_BAD_SOS_COMP_ID,
class jpeg_decoder_stream JPGD_W_EXTRA_BYTES_BEFORE_MARKER,
{ JPGD_NO_ARITHMITIC_SUPPORT,
public: JPGD_UNEXPECTED_MARKER,
jpeg_decoder_stream() { } JPGD_NOT_JPEG,
virtual ~jpeg_decoder_stream() { } JPGD_UNSUPPORTED_MARKER,
JPGD_BAD_DQT_LENGTH,
JPGD_TOO_MANY_BLOCKS,
JPGD_UNDEFINED_QUANT_TABLE,
JPGD_UNDEFINED_HUFF_TABLE,
JPGD_NOT_SINGLE_SCAN,
JPGD_UNSUPPORTED_COLORSPACE,
JPGD_UNSUPPORTED_SAMP_FACTORS,
JPGD_DECODE_ERROR,
JPGD_BAD_RESTART_MARKER,
JPGD_ASSERTION_ERROR,
JPGD_BAD_SOS_SPECTRAL,
JPGD_BAD_SOS_SUCCESSIVE,
JPGD_STREAM_READ,
JPGD_NOTENOUGHMEM
};
// The read() method is called when the internal input buffer is empty. // Input stream interface.
// Parameters: // Derive from this class to read input data from sources other than files or memory. Set m_eof_flag to true when no more data is available.
// pBuf - input buffer // The decoder is rather greedy: it will keep on calling this method until its internal input buffer is full, or until the EOF flag is set.
// max_bytes_to_read - maximum bytes that can be written to pBuf // It the input stream contains data after the JPEG stream's EOI (end of image) marker it will probably be pulled into the internal buffer.
// pEOF_flag - set this to true if at end of stream (no more bytes remaining) // Call the get_total_bytes_read() method to determine the actual size of the JPEG stream after successful decoding.
// Returns -1 on error, otherwise return the number of bytes actually written to the buffer (which may be 0). class jpeg_decoder_stream {
// Notes: This method will be called in a loop until you set *pEOF_flag to true or the internal buffer is full. public:
virtual int read(uint8 *pBuf, int max_bytes_to_read, bool *pEOF_flag) = 0; jpeg_decoder_stream() {}
virtual ~jpeg_decoder_stream() {}
// The read() method is called when the internal input buffer is empty.
// Parameters:
// pBuf - input buffer
// max_bytes_to_read - maximum bytes that can be written to pBuf
// pEOF_flag - set this to true if at end of stream (no more bytes remaining)
// Returns -1 on error, otherwise return the number of bytes actually written to the buffer (which may be 0).
// Notes: This method will be called in a loop until you set *pEOF_flag to true or the internal buffer is full.
virtual int read(uint8* pBuf, int max_bytes_to_read, bool* pEOF_flag) = 0;
};
// stdio FILE stream class.
class jpeg_decoder_file_stream : public jpeg_decoder_stream {
jpeg_decoder_file_stream(const jpeg_decoder_file_stream&);
jpeg_decoder_file_stream& operator=(const jpeg_decoder_file_stream&);
FILE* m_pFile;
bool m_eof_flag, m_error_flag;
public:
jpeg_decoder_file_stream();
virtual ~jpeg_decoder_file_stream();
bool open(const char* Pfilename);
void close();
virtual int read(uint8* pBuf, int max_bytes_to_read, bool* pEOF_flag);
};
// Memory stream class.
class jpeg_decoder_mem_stream : public jpeg_decoder_stream {
const uint8* m_pSrc_data;
uint m_ofs, m_size;
public:
jpeg_decoder_mem_stream()
: m_pSrc_data(NULL), m_ofs(0), m_size(0) {}
jpeg_decoder_mem_stream(const uint8* pSrc_data, uint size)
: m_pSrc_data(pSrc_data), m_ofs(0), m_size(size) {}
virtual ~jpeg_decoder_mem_stream() {}
bool open(const uint8* pSrc_data, uint size);
void close() {
m_pSrc_data = NULL;
m_ofs = 0;
m_size = 0;
}
virtual int read(uint8* pBuf, int max_bytes_to_read, bool* pEOF_flag);
};
// Loads JPEG file from a jpeg_decoder_stream.
unsigned char* decompress_jpeg_image_from_stream(jpeg_decoder_stream* pStream, int* width, int* height, int* actual_comps, int req_comps);
enum {
JPGD_IN_BUF_SIZE = 8192,
JPGD_MAX_BLOCKS_PER_MCU = 10,
JPGD_MAX_HUFF_TABLES = 8,
JPGD_MAX_QUANT_TABLES = 4,
JPGD_MAX_COMPONENTS = 4,
JPGD_MAX_COMPS_IN_SCAN = 4,
JPGD_MAX_BLOCKS_PER_ROW = 8192,
JPGD_MAX_HEIGHT = 16384,
JPGD_MAX_WIDTH = 16384
};
typedef int16 jpgd_quant_t;
typedef int16 jpgd_block_t;
class jpeg_decoder {
public:
// Call get_error_code() after constructing to determine if the stream is valid or not. You may call the get_width(), get_height(), etc.
// methods after the constructor is called. You may then either destruct the object, or begin decoding the image by calling begin_decoding(), then decode() on each scanline.
jpeg_decoder(jpeg_decoder_stream* pStream);
~jpeg_decoder();
// Call this method after constructing the object to begin decompression.
// If JPGD_SUCCESS is returned you may then call decode() on each scanline.
int begin_decoding();
// Returns the next scan line.
// For grayscale images, pScan_line will point to a buffer containing 8-bit pixels (get_bytes_per_pixel() will return 1).
// Otherwise, it will always point to a buffer containing 32-bit RGBA pixels (A will always be 255, and get_bytes_per_pixel() will return 4).
// Returns JPGD_SUCCESS if a scan line has been returned.
// Returns JPGD_DONE if all scan lines have been returned.
// Returns JPGD_FAILED if an error occurred. Call get_error_code() for a more info.
int decode(const void** pScan_line, uint* pScan_line_len);
inline jpgd_status get_error_code() const { return m_error_code; }
inline int get_width() const { return m_image_x_size; }
inline int get_height() const { return m_image_y_size; }
inline int get_num_components() const { return m_comps_in_frame; }
inline int get_bytes_per_pixel() const { return m_dest_bytes_per_pixel; }
inline int get_bytes_per_scan_line() const { return m_image_x_size * get_bytes_per_pixel(); }
// Returns the total number of bytes actually consumed by the decoder (which should equal the actual size of the JPEG file).
inline int get_total_bytes_read() const { return m_total_bytes_read; }
private:
jpeg_decoder(const jpeg_decoder&);
jpeg_decoder& operator=(const jpeg_decoder&);
typedef void (*pDecode_block_func)(jpeg_decoder*, int, int, int);
struct huff_tables {
bool ac_table;
uint look_up[256];
uint look_up2[256];
uint8 code_size[256];
uint tree[512];
}; };
// stdio FILE stream class. struct coeff_buf {
class jpeg_decoder_file_stream : public jpeg_decoder_stream uint8* pData;
{ int block_num_x, block_num_y;
jpeg_decoder_file_stream(const jpeg_decoder_file_stream &); int block_len_x, block_len_y;
jpeg_decoder_file_stream &operator =(const jpeg_decoder_file_stream &); int block_size;
FILE *m_pFile;
bool m_eof_flag, m_error_flag;
public:
jpeg_decoder_file_stream();
virtual ~jpeg_decoder_file_stream();
bool open(const char *Pfilename);
void close();
virtual int read(uint8 *pBuf, int max_bytes_to_read, bool *pEOF_flag);
}; };
// Memory stream class. struct mem_block {
class jpeg_decoder_mem_stream : public jpeg_decoder_stream mem_block* m_pNext;
{ size_t m_used_count;
const uint8 *m_pSrc_data; size_t m_size;
uint m_ofs, m_size; char m_data[1];
public:
jpeg_decoder_mem_stream() : m_pSrc_data(NULL), m_ofs(0), m_size(0) { }
jpeg_decoder_mem_stream(const uint8 *pSrc_data, uint size) : m_pSrc_data(pSrc_data), m_ofs(0), m_size(size) { }
virtual ~jpeg_decoder_mem_stream() { }
bool open(const uint8 *pSrc_data, uint size);
void close() { m_pSrc_data = NULL; m_ofs = 0; m_size = 0; }
virtual int read(uint8 *pBuf, int max_bytes_to_read, bool *pEOF_flag);
}; };
// Loads JPEG file from a jpeg_decoder_stream. jmp_buf m_jmp_state;
unsigned char *decompress_jpeg_image_from_stream(jpeg_decoder_stream *pStream, int *width, int *height, int *actual_comps, int req_comps); mem_block* m_pMem_blocks;
int m_image_x_size;
int m_image_y_size;
jpeg_decoder_stream* m_pStream;
int m_progressive_flag;
uint8 m_huff_ac[JPGD_MAX_HUFF_TABLES];
uint8* m_huff_num[JPGD_MAX_HUFF_TABLES]; // pointer to number of Huffman codes per bit size
uint8* m_huff_val[JPGD_MAX_HUFF_TABLES]; // pointer to Huffman codes per bit size
jpgd_quant_t* m_quant[JPGD_MAX_QUANT_TABLES]; // pointer to quantization tables
int m_scan_type; // Gray, Yh1v1, Yh1v2, Yh2v1, Yh2v2 (CMYK111, CMYK4114 no longer supported)
int m_comps_in_frame; // # of components in frame
int m_comp_h_samp[JPGD_MAX_COMPONENTS]; // component's horizontal sampling factor
int m_comp_v_samp[JPGD_MAX_COMPONENTS]; // component's vertical sampling factor
int m_comp_quant[JPGD_MAX_COMPONENTS]; // component's quantization table selector
int m_comp_ident[JPGD_MAX_COMPONENTS]; // component's ID
int m_comp_h_blocks[JPGD_MAX_COMPONENTS];
int m_comp_v_blocks[JPGD_MAX_COMPONENTS];
int m_comps_in_scan; // # of components in scan
int m_comp_list[JPGD_MAX_COMPS_IN_SCAN]; // components in this scan
int m_comp_dc_tab[JPGD_MAX_COMPONENTS]; // component's DC Huffman coding table selector
int m_comp_ac_tab[JPGD_MAX_COMPONENTS]; // component's AC Huffman coding table selector
int m_spectral_start; // spectral selection start
int m_spectral_end; // spectral selection end
int m_successive_low; // successive approximation low
int m_successive_high; // successive approximation high
int m_max_mcu_x_size; // MCU's max. X size in pixels
int m_max_mcu_y_size; // MCU's max. Y size in pixels
int m_blocks_per_mcu;
int m_max_blocks_per_row;
int m_mcus_per_row, m_mcus_per_col;
int m_mcu_org[JPGD_MAX_BLOCKS_PER_MCU];
int m_total_lines_left; // total # lines left in image
int m_mcu_lines_left; // total # lines left in this MCU
int m_real_dest_bytes_per_scan_line;
int m_dest_bytes_per_scan_line; // rounded up
int m_dest_bytes_per_pixel; // 4 (RGB) or 1 (Y)
huff_tables* m_pHuff_tabs[JPGD_MAX_HUFF_TABLES];
coeff_buf* m_dc_coeffs[JPGD_MAX_COMPONENTS];
coeff_buf* m_ac_coeffs[JPGD_MAX_COMPONENTS];
int m_eob_run;
int m_block_y_mcu[JPGD_MAX_COMPONENTS];
uint8* m_pIn_buf_ofs;
int m_in_buf_left;
int m_tem_flag;
bool m_eof_flag;
uint8 m_in_buf_pad_start[128];
uint8 m_in_buf[JPGD_IN_BUF_SIZE + 128];
uint8 m_in_buf_pad_end[128];
int m_bits_left;
uint m_bit_buf;
int m_restart_interval;
int m_restarts_left;
int m_next_restart_num;
int m_max_mcus_per_row;
int m_max_blocks_per_mcu;
int m_expanded_blocks_per_mcu;
int m_expanded_blocks_per_row;
int m_expanded_blocks_per_component;
bool m_freq_domain_chroma_upsample;
int m_max_mcus_per_col;
uint m_last_dc_val[JPGD_MAX_COMPONENTS];
jpgd_block_t* m_pMCU_coefficients;
int m_mcu_block_max_zag[JPGD_MAX_BLOCKS_PER_MCU];
uint8* m_pSample_buf;
int m_crr[256];
int m_cbb[256];
int m_crg[256];
int m_cbg[256];
uint8* m_pScan_line_0;
uint8* m_pScan_line_1;
jpgd_status m_error_code;
bool m_ready_flag;
int m_total_bytes_read;
enum void free_all_blocks();
{ JPGD_NORETURN void stop_decoding(jpgd_status status);
JPGD_IN_BUF_SIZE = 8192, JPGD_MAX_BLOCKS_PER_MCU = 10, JPGD_MAX_HUFF_TABLES = 8, JPGD_MAX_QUANT_TABLES = 4, void* alloc(size_t n, bool zero = false);
JPGD_MAX_COMPONENTS = 4, JPGD_MAX_COMPS_IN_SCAN = 4, JPGD_MAX_BLOCKS_PER_ROW = 8192, JPGD_MAX_HEIGHT = 16384, JPGD_MAX_WIDTH = 16384 void word_clear(void* p, uint16 c, uint n);
}; void prep_in_buffer();
void read_dht_marker();
typedef int16 jpgd_quant_t; void read_dqt_marker();
typedef int16 jpgd_block_t; void read_sof_marker();
void skip_variable_marker();
void read_dri_marker();
void read_sos_marker();
int next_marker();
int process_markers();
void locate_soi_marker();
void locate_sof_marker();
int locate_sos_marker();
void init(jpeg_decoder_stream* pStream);
void create_look_ups();
void fix_in_buffer();
void transform_mcu(int mcu_row);
void transform_mcu_expand(int mcu_row);
coeff_buf* coeff_buf_open(int block_num_x, int block_num_y, int block_len_x, int block_len_y);
inline jpgd_block_t* coeff_buf_getp(coeff_buf* cb, int block_x, int block_y);
void load_next_row();
void decode_next_row();
void make_huff_table(int index, huff_tables* pH);
void check_quant_tables();
void check_huff_tables();
void calc_mcu_block_order();
int init_scan();
void init_frame();
void process_restart();
void decode_scan(pDecode_block_func decode_block_func);
void init_progressive();
void init_sequential();
void decode_start();
void decode_init(jpeg_decoder_stream* pStream);
void H2V2Convert();
void H2V1Convert();
void H1V2Convert();
void H1V1Convert();
void gray_convert();
void expanded_convert();
void find_eoi();
inline uint get_char();
inline uint get_char(bool* pPadding_flag);
inline void stuff_char(uint8 q);
inline uint8 get_octet();
inline uint get_bits(int num_bits);
inline uint get_bits_no_markers(int numbits);
inline int huff_decode(huff_tables* pH);
inline int huff_decode(huff_tables* pH, int& extrabits);
static inline uint8 clamp(int i);
static void decode_block_dc_first(jpeg_decoder* pD, int component_id, int block_x, int block_y);
static void decode_block_dc_refine(jpeg_decoder* pD, int component_id, int block_x, int block_y);
static void decode_block_ac_first(jpeg_decoder* pD, int component_id, int block_x, int block_y);
static void decode_block_ac_refine(jpeg_decoder* pD, int component_id, int block_x, int block_y);
};
class jpeg_decoder } // namespace jpgd
{
public:
// Call get_error_code() after constructing to determine if the stream is valid or not. You may call the get_width(), get_height(), etc.
// methods after the constructor is called. You may then either destruct the object, or begin decoding the image by calling begin_decoding(), then decode() on each scanline.
jpeg_decoder(jpeg_decoder_stream *pStream);
~jpeg_decoder(); #endif // JPEG_DECODER_H
// Call this method after constructing the object to begin decompression.
// If JPGD_SUCCESS is returned you may then call decode() on each scanline.
int begin_decoding();
// Returns the next scan line.
// For grayscale images, pScan_line will point to a buffer containing 8-bit pixels (get_bytes_per_pixel() will return 1).
// Otherwise, it will always point to a buffer containing 32-bit RGBA pixels (A will always be 255, and get_bytes_per_pixel() will return 4).
// Returns JPGD_SUCCESS if a scan line has been returned.
// Returns JPGD_DONE if all scan lines have been returned.
// Returns JPGD_FAILED if an error occurred. Call get_error_code() for a more info.
int decode(const void** pScan_line, uint* pScan_line_len);
inline jpgd_status get_error_code() const { return m_error_code; }
inline int get_width() const { return m_image_x_size; }
inline int get_height() const { return m_image_y_size; }
inline int get_num_components() const { return m_comps_in_frame; }
inline int get_bytes_per_pixel() const { return m_dest_bytes_per_pixel; }
inline int get_bytes_per_scan_line() const { return m_image_x_size * get_bytes_per_pixel(); }
// Returns the total number of bytes actually consumed by the decoder (which should equal the actual size of the JPEG file).
inline int get_total_bytes_read() const { return m_total_bytes_read; }
private:
jpeg_decoder(const jpeg_decoder &);
jpeg_decoder &operator =(const jpeg_decoder &);
typedef void (*pDecode_block_func)(jpeg_decoder *, int, int, int);
struct huff_tables
{
bool ac_table;
uint look_up[256];
uint look_up2[256];
uint8 code_size[256];
uint tree[512];
};
struct coeff_buf
{
uint8 *pData;
int block_num_x, block_num_y;
int block_len_x, block_len_y;
int block_size;
};
struct mem_block
{
mem_block *m_pNext;
size_t m_used_count;
size_t m_size;
char m_data[1];
};
jmp_buf m_jmp_state;
mem_block *m_pMem_blocks;
int m_image_x_size;
int m_image_y_size;
jpeg_decoder_stream *m_pStream;
int m_progressive_flag;
uint8 m_huff_ac[JPGD_MAX_HUFF_TABLES];
uint8* m_huff_num[JPGD_MAX_HUFF_TABLES]; // pointer to number of Huffman codes per bit size
uint8* m_huff_val[JPGD_MAX_HUFF_TABLES]; // pointer to Huffman codes per bit size
jpgd_quant_t* m_quant[JPGD_MAX_QUANT_TABLES]; // pointer to quantization tables
int m_scan_type; // Gray, Yh1v1, Yh1v2, Yh2v1, Yh2v2 (CMYK111, CMYK4114 no longer supported)
int m_comps_in_frame; // # of components in frame
int m_comp_h_samp[JPGD_MAX_COMPONENTS]; // component's horizontal sampling factor
int m_comp_v_samp[JPGD_MAX_COMPONENTS]; // component's vertical sampling factor
int m_comp_quant[JPGD_MAX_COMPONENTS]; // component's quantization table selector
int m_comp_ident[JPGD_MAX_COMPONENTS]; // component's ID
int m_comp_h_blocks[JPGD_MAX_COMPONENTS];
int m_comp_v_blocks[JPGD_MAX_COMPONENTS];
int m_comps_in_scan; // # of components in scan
int m_comp_list[JPGD_MAX_COMPS_IN_SCAN]; // components in this scan
int m_comp_dc_tab[JPGD_MAX_COMPONENTS]; // component's DC Huffman coding table selector
int m_comp_ac_tab[JPGD_MAX_COMPONENTS]; // component's AC Huffman coding table selector
int m_spectral_start; // spectral selection start
int m_spectral_end; // spectral selection end
int m_successive_low; // successive approximation low
int m_successive_high; // successive approximation high
int m_max_mcu_x_size; // MCU's max. X size in pixels
int m_max_mcu_y_size; // MCU's max. Y size in pixels
int m_blocks_per_mcu;
int m_max_blocks_per_row;
int m_mcus_per_row, m_mcus_per_col;
int m_mcu_org[JPGD_MAX_BLOCKS_PER_MCU];
int m_total_lines_left; // total # lines left in image
int m_mcu_lines_left; // total # lines left in this MCU
int m_real_dest_bytes_per_scan_line;
int m_dest_bytes_per_scan_line; // rounded up
int m_dest_bytes_per_pixel; // 4 (RGB) or 1 (Y)
huff_tables* m_pHuff_tabs[JPGD_MAX_HUFF_TABLES];
coeff_buf* m_dc_coeffs[JPGD_MAX_COMPONENTS];
coeff_buf* m_ac_coeffs[JPGD_MAX_COMPONENTS];
int m_eob_run;
int m_block_y_mcu[JPGD_MAX_COMPONENTS];
uint8* m_pIn_buf_ofs;
int m_in_buf_left;
int m_tem_flag;
bool m_eof_flag;
uint8 m_in_buf_pad_start[128];
uint8 m_in_buf[JPGD_IN_BUF_SIZE + 128];
uint8 m_in_buf_pad_end[128];
int m_bits_left;
uint m_bit_buf;
int m_restart_interval;
int m_restarts_left;
int m_next_restart_num;
int m_max_mcus_per_row;
int m_max_blocks_per_mcu;
int m_expanded_blocks_per_mcu;
int m_expanded_blocks_per_row;
int m_expanded_blocks_per_component;
bool m_freq_domain_chroma_upsample;
int m_max_mcus_per_col;
uint m_last_dc_val[JPGD_MAX_COMPONENTS];
jpgd_block_t* m_pMCU_coefficients;
int m_mcu_block_max_zag[JPGD_MAX_BLOCKS_PER_MCU];
uint8* m_pSample_buf;
int m_crr[256];
int m_cbb[256];
int m_crg[256];
int m_cbg[256];
uint8* m_pScan_line_0;
uint8* m_pScan_line_1;
jpgd_status m_error_code;
bool m_ready_flag;
int m_total_bytes_read;
void free_all_blocks();
JPGD_NORETURN void stop_decoding(jpgd_status status);
void *alloc(size_t n, bool zero = false);
void word_clear(void *p, uint16 c, uint n);
void prep_in_buffer();
void read_dht_marker();
void read_dqt_marker();
void read_sof_marker();
void skip_variable_marker();
void read_dri_marker();
void read_sos_marker();
int next_marker();
int process_markers();
void locate_soi_marker();
void locate_sof_marker();
int locate_sos_marker();
void init(jpeg_decoder_stream * pStream);
void create_look_ups();
void fix_in_buffer();
void transform_mcu(int mcu_row);
void transform_mcu_expand(int mcu_row);
coeff_buf* coeff_buf_open(int block_num_x, int block_num_y, int block_len_x, int block_len_y);
inline jpgd_block_t *coeff_buf_getp(coeff_buf *cb, int block_x, int block_y);
void load_next_row();
void decode_next_row();
void make_huff_table(int index, huff_tables *pH);
void check_quant_tables();
void check_huff_tables();
void calc_mcu_block_order();
int init_scan();
void init_frame();
void process_restart();
void decode_scan(pDecode_block_func decode_block_func);
void init_progressive();
void init_sequential();
void decode_start();
void decode_init(jpeg_decoder_stream * pStream);
void H2V2Convert();
void H2V1Convert();
void H1V2Convert();
void H1V1Convert();
void gray_convert();
void expanded_convert();
void find_eoi();
inline uint get_char();
inline uint get_char(bool *pPadding_flag);
inline void stuff_char(uint8 q);
inline uint8 get_octet();
inline uint get_bits(int num_bits);
inline uint get_bits_no_markers(int numbits);
inline int huff_decode(huff_tables *pH);
inline int huff_decode(huff_tables *pH, int& extrabits);
static inline uint8 clamp(int i);
static void decode_block_dc_first(jpeg_decoder *pD, int component_id, int block_x, int block_y);
static void decode_block_dc_refine(jpeg_decoder *pD, int component_id, int block_x, int block_y);
static void decode_block_ac_first(jpeg_decoder *pD, int component_id, int block_x, int block_y);
static void decode_block_ac_refine(jpeg_decoder *pD, int component_id, int block_x, int block_y);
};
} // namespace jpgd
#endif // JPEG_DECODER_H
crnlib/crn_jpge.cpp
+1010 -934
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_jpge.h
+143 -141
View File
@@ -4,166 +4,168 @@
#ifndef JPEG_ENCODER_H #ifndef JPEG_ENCODER_H
#define JPEG_ENCODER_H #define JPEG_ENCODER_H
namespace jpge namespace jpge {
{ typedef unsigned char uint8;
typedef unsigned char uint8; typedef signed short int16;
typedef signed short int16; typedef signed int int32;
typedef signed int int32; typedef unsigned short uint16;
typedef unsigned short uint16; typedef unsigned int uint32;
typedef unsigned int uint32; typedef unsigned int uint;
typedef unsigned int uint;
// JPEG chroma subsampling factors. Y_ONLY (grayscale images) and H2V2 (color images) are the most common. // JPEG chroma subsampling factors. Y_ONLY (grayscale images) and H2V2 (color images) are the most common.
enum subsampling_t { Y_ONLY = 0, H1V1 = 1, H2V1 = 2, H2V2 = 3 }; enum subsampling_t { Y_ONLY = 0,
H1V1 = 1,
H2V1 = 2,
H2V2 = 3 };
// JPEG compression parameters structure. // JPEG compression parameters structure.
struct params struct params {
{ inline params()
inline params() : m_quality(85), m_subsampling(H2V2), m_no_chroma_discrim_flag(false), m_two_pass_flag(false) { } : m_quality(85), m_subsampling(H2V2), m_no_chroma_discrim_flag(false), m_two_pass_flag(false) {}
inline bool check() const inline bool check() const {
{ if ((m_quality < 1) || (m_quality > 100))
if ((m_quality < 1) || (m_quality > 100)) return false; return false;
if ((uint)m_subsampling > (uint)H2V2) return false; if ((uint)m_subsampling > (uint)H2V2)
return true; return false;
} return true;
}
// Quality: 1-100, higher is better. Typical values are around 50-95. // Quality: 1-100, higher is better. Typical values are around 50-95.
int m_quality; int m_quality;
// m_subsampling: // m_subsampling:
// 0 = Y (grayscale) only // 0 = Y (grayscale) only
// 1 = YCbCr, no subsampling (H1V1, YCbCr 1x1x1, 3 blocks per MCU) // 1 = YCbCr, no subsampling (H1V1, YCbCr 1x1x1, 3 blocks per MCU)
// 2 = YCbCr, H2V1 subsampling (YCbCr 2x1x1, 4 blocks per MCU) // 2 = YCbCr, H2V1 subsampling (YCbCr 2x1x1, 4 blocks per MCU)
// 3 = YCbCr, H2V2 subsampling (YCbCr 4x1x1, 6 blocks per MCU-- very common) // 3 = YCbCr, H2V2 subsampling (YCbCr 4x1x1, 6 blocks per MCU-- very common)
subsampling_t m_subsampling; subsampling_t m_subsampling;
// Disables CbCr discrimination - only intended for testing. // Disables CbCr discrimination - only intended for testing.
// If true, the Y quantization table is also used for the CbCr channels. // If true, the Y quantization table is also used for the CbCr channels.
bool m_no_chroma_discrim_flag; bool m_no_chroma_discrim_flag;
bool m_two_pass_flag; bool m_two_pass_flag;
}; };
// Writes JPEG image to a file. // Writes JPEG image to a file.
// num_channels must be 1 (Y) or 3 (RGB), image pitch must be width*num_channels. // num_channels must be 1 (Y) or 3 (RGB), image pitch must be width*num_channels.
bool compress_image_to_jpeg_file(const char *pFilename, int width, int height, int num_channels, const uint8 *pImage_data, const params &comp_params = params()); bool compress_image_to_jpeg_file(const char* pFilename, int width, int height, int num_channels, const uint8* pImage_data, const params& comp_params = params());
// Writes JPEG image to memory buffer. // Writes JPEG image to memory buffer.
// On entry, buf_size is the size of the output buffer pointed at by pBuf, which should be at least ~1024 bytes. // On entry, buf_size is the size of the output buffer pointed at by pBuf, which should be at least ~1024 bytes.
// If return value is true, buf_size will be set to the size of the compressed data. // If return value is true, buf_size will be set to the size of the compressed data.
bool compress_image_to_jpeg_file_in_memory(void *pBuf, int &buf_size, int width, int height, int num_channels, const uint8 *pImage_data, const params &comp_params = params()); bool compress_image_to_jpeg_file_in_memory(void* pBuf, int& buf_size, int width, int height, int num_channels, const uint8* pImage_data, const params& comp_params = params());
// Output stream abstract class - used by the jpeg_encoder class to write to the output stream. // Output stream abstract class - used by the jpeg_encoder class to write to the output stream.
// put_buf() is generally called with len==JPGE_OUT_BUF_SIZE bytes, but for headers it'll be called with smaller amounts. // put_buf() is generally called with len==JPGE_OUT_BUF_SIZE bytes, but for headers it'll be called with smaller amounts.
class output_stream class output_stream {
{ public:
public: virtual ~output_stream(){};
virtual ~output_stream() { }; virtual bool put_buf(const void* Pbuf, int len) = 0;
virtual bool put_buf(const void* Pbuf, int len) = 0; template <class T>
template<class T> inline bool put_obj(const T& obj) { return put_buf(&obj, sizeof(T)); } inline bool put_obj(const T& obj) { return put_buf(&obj, sizeof(T)); }
}; };
// Lower level jpeg_encoder class - useful if more control is needed than the above helper functions. // Lower level jpeg_encoder class - useful if more control is needed than the above helper functions.
class jpeg_encoder class jpeg_encoder {
{ public:
public: jpeg_encoder();
jpeg_encoder(); ~jpeg_encoder();
~jpeg_encoder();
// Initializes the compressor. // Initializes the compressor.
// pStream: The stream object to use for writing compressed data. // pStream: The stream object to use for writing compressed data.
// params - Compression parameters structure, defined above. // params - Compression parameters structure, defined above.
// width, height - Image dimensions. // width, height - Image dimensions.
// channels - May be 1, or 3. 1 indicates grayscale, 3 indicates RGB source data. // channels - May be 1, or 3. 1 indicates grayscale, 3 indicates RGB source data.
// Returns false on out of memory or if a stream write fails. // Returns false on out of memory or if a stream write fails.
bool init(output_stream *pStream, int width, int height, int src_channels, const params &comp_params = params()); bool init(output_stream* pStream, int width, int height, int src_channels, const params& comp_params = params());
const params &get_params() const { return m_params; } const params& get_params() const { return m_params; }
// Deinitializes the compressor, freeing any allocated memory. May be called at any time. // Deinitializes the compressor, freeing any allocated memory. May be called at any time.
void deinit(); void deinit();
uint get_total_passes() const { return m_params.m_two_pass_flag ? 2 : 1; } uint get_total_passes() const { return m_params.m_two_pass_flag ? 2 : 1; }
inline uint get_cur_pass() { return m_pass_num; } inline uint get_cur_pass() { return m_pass_num; }
// Call this method with each source scanline. // Call this method with each source scanline.
// width * src_channels bytes per scanline is expected (RGB or Y format). // width * src_channels bytes per scanline is expected (RGB or Y format).
// You must call with NULL after all scanlines are processed to finish compression. // You must call with NULL after all scanlines are processed to finish compression.
// Returns false on out of memory or if a stream write fails. // Returns false on out of memory or if a stream write fails.
bool process_scanline(const void* pScanline); bool process_scanline(const void* pScanline);
private: private:
jpeg_encoder(const jpeg_encoder &); jpeg_encoder(const jpeg_encoder&);
jpeg_encoder &operator =(const jpeg_encoder &); jpeg_encoder& operator=(const jpeg_encoder&);
typedef int32 sample_array_t; typedef int32 sample_array_t;
output_stream *m_pStream; output_stream* m_pStream;
params m_params; params m_params;
uint8 m_num_components; uint8 m_num_components;
uint8 m_comp_h_samp[3], m_comp_v_samp[3]; uint8 m_comp_h_samp[3], m_comp_v_samp[3];
int m_image_x, m_image_y, m_image_bpp, m_image_bpl; int m_image_x, m_image_y, m_image_bpp, m_image_bpl;
int m_image_x_mcu, m_image_y_mcu; int m_image_x_mcu, m_image_y_mcu;
int m_image_bpl_xlt, m_image_bpl_mcu; int m_image_bpl_xlt, m_image_bpl_mcu;
int m_mcus_per_row; int m_mcus_per_row;
int m_mcu_x, m_mcu_y; int m_mcu_x, m_mcu_y;
uint8 *m_mcu_lines[16]; uint8* m_mcu_lines[16];
uint8 m_mcu_y_ofs; uint8 m_mcu_y_ofs;
sample_array_t m_sample_array[64]; sample_array_t m_sample_array[64];
int16 m_coefficient_array[64]; int16 m_coefficient_array[64];
int32 m_quantization_tables[2][64]; int32 m_quantization_tables[2][64];
uint m_huff_codes[4][256]; uint m_huff_codes[4][256];
uint8 m_huff_code_sizes[4][256]; uint8 m_huff_code_sizes[4][256];
uint8 m_huff_bits[4][17]; uint8 m_huff_bits[4][17];
uint8 m_huff_val[4][256]; uint8 m_huff_val[4][256];
uint32 m_huff_count[4][256]; uint32 m_huff_count[4][256];
int m_last_dc_val[3]; int m_last_dc_val[3];
enum { JPGE_OUT_BUF_SIZE = 2048 }; enum { JPGE_OUT_BUF_SIZE = 2048 };
uint8 m_out_buf[JPGE_OUT_BUF_SIZE]; uint8 m_out_buf[JPGE_OUT_BUF_SIZE];
uint8 *m_pOut_buf; uint8* m_pOut_buf;
uint m_out_buf_left; uint m_out_buf_left;
uint32 m_bit_buffer; uint32 m_bit_buffer;
uint m_bits_in; uint m_bits_in;
uint8 m_pass_num; uint8 m_pass_num;
bool m_all_stream_writes_succeeded; bool m_all_stream_writes_succeeded;
void optimize_huffman_table(int table_num, int table_len); void optimize_huffman_table(int table_num, int table_len);
void emit_byte(uint8 i); void emit_byte(uint8 i);
void emit_word(uint i); void emit_word(uint i);
void emit_marker(int marker); void emit_marker(int marker);
void emit_jfif_app0(); void emit_jfif_app0();
void emit_dqt(); void emit_dqt();
void emit_sof(); void emit_sof();
void emit_dht(uint8 *bits, uint8 *val, int index, bool ac_flag); void emit_dht(uint8* bits, uint8* val, int index, bool ac_flag);
void emit_dhts(); void emit_dhts();
void emit_sos(); void emit_sos();
void emit_markers(); void emit_markers();
void compute_huffman_table(uint *codes, uint8 *code_sizes, uint8 *bits, uint8 *val); void compute_huffman_table(uint* codes, uint8* code_sizes, uint8* bits, uint8* val);
void compute_quant_table(int32 *dst, int16 *src); void compute_quant_table(int32* dst, int16* src);
void adjust_quant_table(int32 *dst, int32 *src); void adjust_quant_table(int32* dst, int32* src);
void first_pass_init(); void first_pass_init();
bool second_pass_init(); bool second_pass_init();
bool jpg_open(int p_x_res, int p_y_res, int src_channels); bool jpg_open(int p_x_res, int p_y_res, int src_channels);
void load_block_8_8_grey(int x); void load_block_8_8_grey(int x);
void load_block_8_8(int x, int y, int c); void load_block_8_8(int x, int y, int c);
void load_block_16_8(int x, int c); void load_block_16_8(int x, int c);
void load_block_16_8_8(int x, int c); void load_block_16_8_8(int x, int c);
void load_quantized_coefficients(int component_num); void load_quantized_coefficients(int component_num);
void flush_output_buffer(); void flush_output_buffer();
void put_bits(uint bits, uint len); void put_bits(uint bits, uint len);
void code_coefficients_pass_one(int component_num); void code_coefficients_pass_one(int component_num);
void code_coefficients_pass_two(int component_num); void code_coefficients_pass_two(int component_num);
void code_block(int component_num); void code_block(int component_num);
void process_mcu_row(); void process_mcu_row();
bool terminate_pass_one(); bool terminate_pass_one();
bool terminate_pass_two(); bool terminate_pass_two();
bool process_end_of_image(); bool process_end_of_image();
void load_mcu(const void* src); void load_mcu(const void* src);
void clear(); void clear();
void init(); void init();
}; };
} // namespace jpge } // namespace jpge
#endif // JPEG_ENCODER #endif // JPEG_ENCODER
crnlib/crn_ktx_texture.cpp
+741 -827
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_ktx_texture.h
+244 -197
View File
@@ -10,235 +10,282 @@
#define KTX_ENDIAN 0x04030201 #define KTX_ENDIAN 0x04030201
#define KTX_OPPOSITE_ENDIAN 0x01020304 #define KTX_OPPOSITE_ENDIAN 0x01020304
namespace crnlib namespace crnlib {
{ extern const uint8 s_ktx_file_id[12];
extern const uint8 s_ktx_file_id[12];
struct ktx_header struct ktx_header {
{ uint8 m_identifier[12];
uint8 m_identifier[12]; uint32 m_endianness;
uint32 m_endianness; uint32 m_glType;
uint32 m_glType; uint32 m_glTypeSize;
uint32 m_glTypeSize; uint32 m_glFormat;
uint32 m_glFormat; uint32 m_glInternalFormat;
uint32 m_glInternalFormat; uint32 m_glBaseInternalFormat;
uint32 m_glBaseInternalFormat; uint32 m_pixelWidth;
uint32 m_pixelWidth; uint32 m_pixelHeight;
uint32 m_pixelHeight; uint32 m_pixelDepth;
uint32 m_pixelDepth; uint32 m_numberOfArrayElements;
uint32 m_numberOfArrayElements; uint32 m_numberOfFaces;
uint32 m_numberOfFaces; uint32 m_numberOfMipmapLevels;
uint32 m_numberOfMipmapLevels; uint32 m_bytesOfKeyValueData;
uint32 m_bytesOfKeyValueData;
void clear() void clear() {
{ memset(this, 0, sizeof(*this));
memset(this, 0, sizeof(*this)); }
}
void endian_swap() void endian_swap() {
{ utils::endian_swap_mem32(&m_endianness, (sizeof(*this) - sizeof(m_identifier)) / sizeof(uint32));
utils::endian_swap_mem32(&m_endianness, (sizeof(*this) - sizeof(m_identifier)) / sizeof(uint32)); }
} };
};
typedef crnlib::vector<uint8_vec> ktx_key_value_vec; typedef crnlib::vector<uint8_vec> ktx_key_value_vec;
typedef crnlib::vector<uint8_vec> ktx_image_data_vec; typedef crnlib::vector<uint8_vec> ktx_image_data_vec;
// Compressed pixel data formats: ETC1, DXT1, DXT3, DXT5 // Compressed pixel data formats: ETC1, DXT1, DXT3, DXT5
enum enum {
{ KTX_ETC1_RGB8_OES = 0x8D64,
KTX_ETC1_RGB8_OES = 0x8D64, KTX_RGB_S3TC = 0x83A0, KTX_RGB4_S3TC = 0x83A1, KTX_COMPRESSED_RGB_S3TC_DXT1_EXT = 0x83F0, KTX_COMPRESSED_RGB8_ETC2 = 0x9274,
KTX_COMPRESSED_RGBA_S3TC_DXT1_EXT = 0x83F1, KTX_COMPRESSED_SRGB_S3TC_DXT1_EXT = 0x8C4C, KTX_COMPRESSED_SRGB_ALPHA_S3TC_DXT1_EXT = 0x8C4D, KTX_COMPRESSED_RGBA8_ETC2_EAC = 0x9278,
KTX_RGBA_S3TC = 0x83A2, KTX_RGBA4_S3TC = 0x83A3, KTX_COMPRESSED_RGBA_S3TC_DXT3_EXT = 0x83F2, KTX_COMPRESSED_SRGB_ALPHA_S3TC_DXT3_EXT = 0x8C4E, KTX_RGB_S3TC = 0x83A0,
KTX_COMPRESSED_RGBA_S3TC_DXT5_EXT = 0x83F3, KTX_COMPRESSED_SRGB_ALPHA_S3TC_DXT5_EXT = 0x8C4F, KTX_RGBA_DXT5_S3TC = 0x83A4, KTX_RGBA4_DXT5_S3TC = 0x83A5, KTX_RGB4_S3TC = 0x83A1,
KTX_COMPRESSED_RED_RGTC1_EXT = 0x8DBB, KTX_COMPRESSED_SIGNED_RED_RGTC1_EXT = 0x8DBC, KTX_COMPRESSED_RED_GREEN_RGTC2_EXT = 0x8DBD, KTX_COMPRESSED_SIGNED_RED_GREEN_RGTC2_EXT = 0x8DBE, KTX_COMPRESSED_RGB_S3TC_DXT1_EXT = 0x83F0,
KTX_COMPRESSED_LUMINANCE_LATC1_EXT = 0x8C70, KTX_COMPRESSED_SIGNED_LUMINANCE_LATC1_EXT = 0x8C71, KTX_COMPRESSED_LUMINANCE_ALPHA_LATC2_EXT = 0x8C72, KTX_COMPRESSED_SIGNED_LUMINANCE_ALPHA_LATC2_EXT = 0x8C73 KTX_COMPRESSED_RGBA_S3TC_DXT1_EXT = 0x83F1,
}; KTX_COMPRESSED_SRGB_S3TC_DXT1_EXT = 0x8C4C,
KTX_COMPRESSED_SRGB_ALPHA_S3TC_DXT1_EXT = 0x8C4D,
KTX_RGBA_S3TC = 0x83A2,
KTX_RGBA4_S3TC = 0x83A3,
KTX_COMPRESSED_RGBA_S3TC_DXT3_EXT = 0x83F2,
KTX_COMPRESSED_SRGB_ALPHA_S3TC_DXT3_EXT = 0x8C4E,
KTX_COMPRESSED_RGBA_S3TC_DXT5_EXT = 0x83F3,
KTX_COMPRESSED_SRGB_ALPHA_S3TC_DXT5_EXT = 0x8C4F,
KTX_RGBA_DXT5_S3TC = 0x83A4,
KTX_RGBA4_DXT5_S3TC = 0x83A5,
KTX_COMPRESSED_RED_RGTC1_EXT = 0x8DBB,
KTX_COMPRESSED_SIGNED_RED_RGTC1_EXT = 0x8DBC,
KTX_COMPRESSED_RED_GREEN_RGTC2_EXT = 0x8DBD,
KTX_COMPRESSED_SIGNED_RED_GREEN_RGTC2_EXT = 0x8DBE,
KTX_COMPRESSED_LUMINANCE_LATC1_EXT = 0x8C70,
KTX_COMPRESSED_SIGNED_LUMINANCE_LATC1_EXT = 0x8C71,
KTX_COMPRESSED_LUMINANCE_ALPHA_LATC2_EXT = 0x8C72,
KTX_COMPRESSED_SIGNED_LUMINANCE_ALPHA_LATC2_EXT = 0x8C73
};
// Pixel formats (various internal, base, and base internal formats) // Pixel formats (various internal, base, and base internal formats)
enum enum {
{ KTX_R8 = 0x8229,
KTX_R8 = 0x8229, KTX_R8UI = 0x8232, KTX_RGB8 = 0x8051, KTX_SRGB8 = 0x8C41, KTX_SRGB = 0x8C40, KTX_SRGB_ALPHA = 0x8C42, KTX_R8UI = 0x8232,
KTX_SRGB8_ALPHA8 = 0x8C43, KTX_RGBA8 = 0x8058, KTX_STENCIL_INDEX = 0x1901, KTX_DEPTH_COMPONENT = 0x1902, KTX_DEPTH_STENCIL = 0x84F9, KTX_RED = 0x1903, KTX_RGB8 = 0x8051,
KTX_GREEN = 0x1904, KTX_BLUE = 0x1905, KTX_ALPHA = 0x1906, KTX_RG = 0x8227, KTX_RGB = 0x1907, KTX_RGBA = 0x1908, KTX_BGR = 0x80E0, KTX_BGRA = 0x80E1, KTX_SRGB8 = 0x8C41,
KTX_RED_INTEGER = 0x8D94, KTX_GREEN_INTEGER = 0x8D95, KTX_BLUE_INTEGER = 0x8D96, KTX_ALPHA_INTEGER = 0x8D97, KTX_RGB_INTEGER = 0x8D98, KTX_RGBA_INTEGER = 0x8D99, KTX_SRGB = 0x8C40,
KTX_BGR_INTEGER = 0x8D9A, KTX_BGRA_INTEGER = 0x8D9B, KTX_LUMINANCE = 0x1909, KTX_LUMINANCE_ALPHA = 0x190A, KTX_RG_INTEGER = 0x8228, KTX_RG8 = 0x822B, KTX_SRGB_ALPHA = 0x8C42,
KTX_ALPHA8 = 0x803C, KTX_LUMINANCE8 = 0x8040, KTX_LUMINANCE8_ALPHA8 = 0x8045 KTX_SRGB8_ALPHA8 = 0x8C43,
}; KTX_RGBA8 = 0x8058,
KTX_STENCIL_INDEX = 0x1901,
KTX_DEPTH_COMPONENT = 0x1902,
KTX_DEPTH_STENCIL = 0x84F9,
KTX_RED = 0x1903,
KTX_GREEN = 0x1904,
KTX_BLUE = 0x1905,
KTX_ALPHA = 0x1906,
KTX_RG = 0x8227,
KTX_RGB = 0x1907,
KTX_RGBA = 0x1908,
KTX_BGR = 0x80E0,
KTX_BGRA = 0x80E1,
KTX_RED_INTEGER = 0x8D94,
KTX_GREEN_INTEGER = 0x8D95,
KTX_BLUE_INTEGER = 0x8D96,
KTX_ALPHA_INTEGER = 0x8D97,
KTX_RGB_INTEGER = 0x8D98,
KTX_RGBA_INTEGER = 0x8D99,
KTX_BGR_INTEGER = 0x8D9A,
KTX_BGRA_INTEGER = 0x8D9B,
KTX_LUMINANCE = 0x1909,
KTX_LUMINANCE_ALPHA = 0x190A,
KTX_RG_INTEGER = 0x8228,
KTX_RG8 = 0x822B,
KTX_ALPHA8 = 0x803C,
KTX_LUMINANCE8 = 0x8040,
KTX_LUMINANCE8_ALPHA8 = 0x8045
};
// Pixel data types // Pixel data types
enum enum {
{ KTX_UNSIGNED_BYTE = 0x1401,
KTX_UNSIGNED_BYTE = 0x1401, KTX_BYTE = 0x1400, KTX_UNSIGNED_SHORT = 0x1403, KTX_SHORT = 0x1402, KTX_BYTE = 0x1400,
KTX_UNSIGNED_INT = 0x1405, KTX_INT = 0x1404, KTX_HALF_FLOAT = 0x140B, KTX_FLOAT = 0x1406, KTX_UNSIGNED_SHORT = 0x1403,
KTX_UNSIGNED_BYTE_3_3_2 = 0x8032, KTX_UNSIGNED_BYTE_2_3_3_REV = 0x8362, KTX_UNSIGNED_SHORT_5_6_5 = 0x8363, KTX_SHORT = 0x1402,
KTX_UNSIGNED_SHORT_5_6_5_REV = 0x8364, KTX_UNSIGNED_SHORT_4_4_4_4 = 0x8033, KTX_UNSIGNED_SHORT_4_4_4_4_REV = 0x8365, KTX_UNSIGNED_INT = 0x1405,
KTX_UNSIGNED_SHORT_5_5_5_1 = 0x8034, KTX_UNSIGNED_SHORT_1_5_5_5_REV = 0x8366, KTX_UNSIGNED_INT_8_8_8_8 = 0x8035, KTX_INT = 0x1404,
KTX_UNSIGNED_INT_8_8_8_8_REV = 0x8367, KTX_UNSIGNED_INT_10_10_10_2 = 0x8036, KTX_UNSIGNED_INT_2_10_10_10_REV = 0x8368, KTX_HALF_FLOAT = 0x140B,
KTX_UNSIGNED_INT_24_8 = 0x84FA, KTX_UNSIGNED_INT_10F_11F_11F_REV = 0x8C3B, KTX_UNSIGNED_INT_5_9_9_9_REV = 0x8C3E, KTX_FLOAT = 0x1406,
KTX_FLOAT_32_UNSIGNED_INT_24_8_REV = 0x8DAD KTX_UNSIGNED_BYTE_3_3_2 = 0x8032,
}; KTX_UNSIGNED_BYTE_2_3_3_REV = 0x8362,
KTX_UNSIGNED_SHORT_5_6_5 = 0x8363,
bool is_packed_pixel_ogl_type(uint32 ogl_type); KTX_UNSIGNED_SHORT_5_6_5_REV = 0x8364,
uint get_ogl_type_size(uint32 ogl_type); KTX_UNSIGNED_SHORT_4_4_4_4 = 0x8033,
bool get_ogl_fmt_desc(uint32 ogl_fmt, uint32 ogl_type, uint& block_dim, uint& bytes_per_block); KTX_UNSIGNED_SHORT_4_4_4_4_REV = 0x8365,
uint get_ogl_type_size(uint32 ogl_type); KTX_UNSIGNED_SHORT_5_5_5_1 = 0x8034,
uint32 get_ogl_base_internal_fmt(uint32 ogl_fmt); KTX_UNSIGNED_SHORT_1_5_5_5_REV = 0x8366,
KTX_UNSIGNED_INT_8_8_8_8 = 0x8035,
KTX_UNSIGNED_INT_8_8_8_8_REV = 0x8367,
KTX_UNSIGNED_INT_10_10_10_2 = 0x8036,
KTX_UNSIGNED_INT_2_10_10_10_REV = 0x8368,
KTX_UNSIGNED_INT_24_8 = 0x84FA,
KTX_UNSIGNED_INT_10F_11F_11F_REV = 0x8C3B,
KTX_UNSIGNED_INT_5_9_9_9_REV = 0x8C3E,
KTX_FLOAT_32_UNSIGNED_INT_24_8_REV = 0x8DAD
};
class ktx_texture bool is_packed_pixel_ogl_type(uint32 ogl_type);
{ uint get_ogl_type_size(uint32 ogl_type);
public: bool get_ogl_fmt_desc(uint32 ogl_fmt, uint32 ogl_type, uint& block_dim, uint& bytes_per_block);
ktx_texture() uint get_ogl_type_size(uint32 ogl_type);
{ uint32 get_ogl_base_internal_fmt(uint32 ogl_fmt);
clear();
}
ktx_texture(const ktx_texture& other) class ktx_texture {
{ public:
*this = other; ktx_texture() {
} clear();
}
ktx_texture& operator= (const ktx_texture& rhs) ktx_texture(const ktx_texture& other) {
{ *this = other;
if (this == &rhs) }
return *this;
clear(); ktx_texture& operator=(const ktx_texture& rhs) {
if (this == &rhs)
return *this;
m_header = rhs.m_header; clear();
m_key_values = rhs.m_key_values;
m_image_data = rhs.m_image_data;
m_block_dim = rhs.m_block_dim;
m_bytes_per_block = rhs.m_bytes_per_block;
m_opposite_endianness = rhs.m_opposite_endianness;
return *this; m_header = rhs.m_header;
} m_key_values = rhs.m_key_values;
m_image_data = rhs.m_image_data;
m_block_dim = rhs.m_block_dim;
m_bytes_per_block = rhs.m_bytes_per_block;
m_opposite_endianness = rhs.m_opposite_endianness;
void clear() return *this;
{ }
m_header.clear();
m_key_values.clear();
m_image_data.clear();
m_block_dim = 0;
m_bytes_per_block = 0;
m_opposite_endianness = false; void clear() {
} m_header.clear();
m_key_values.clear();
m_image_data.clear();
// High level methods m_block_dim = 0;
bool read_from_stream(data_stream_serializer& serializer); m_bytes_per_block = 0;
bool write_to_stream(data_stream_serializer& serializer, bool no_keyvalue_data = false);
bool init_2D(uint width, uint height, uint num_mips, uint32 ogl_internal_fmt, uint32 ogl_fmt, uint32 ogl_type);
bool init_2D_array(uint width, uint height, uint num_mips, uint array_size, uint32 ogl_internal_fmt, uint32 ogl_fmt, uint32 ogl_type);
bool init_3D(uint width, uint height, uint depth, uint num_mips, uint32 ogl_internal_fmt, uint32 ogl_fmt, uint32 ogl_type);
bool init_cubemap(uint dim, uint num_mips, uint32 ogl_internal_fmt, uint32 ogl_fmt, uint32 ogl_type);
bool check_header() const; m_opposite_endianness = false;
bool consistency_check() const; }
// General info // High level methods
bool read_from_stream(data_stream_serializer& serializer);
bool write_to_stream(data_stream_serializer& serializer, bool no_keyvalue_data = false);
bool is_valid() const { return (m_header.m_pixelWidth > 0) && (m_image_data.size() > 0); } bool init_2D(uint width, uint height, uint num_mips, uint32 ogl_internal_fmt, uint32 ogl_fmt, uint32 ogl_type);
bool init_2D_array(uint width, uint height, uint num_mips, uint array_size, uint32 ogl_internal_fmt, uint32 ogl_fmt, uint32 ogl_type);
bool init_3D(uint width, uint height, uint depth, uint num_mips, uint32 ogl_internal_fmt, uint32 ogl_fmt, uint32 ogl_type);
bool init_cubemap(uint dim, uint num_mips, uint32 ogl_internal_fmt, uint32 ogl_fmt, uint32 ogl_type);
uint get_width() const { return m_header.m_pixelWidth; } bool check_header() const;
uint get_height() const { return CRNLIB_MAX(m_header.m_pixelHeight, 1); } bool consistency_check() const;
uint get_depth() const { return CRNLIB_MAX(m_header.m_pixelDepth, 1); }
uint get_num_mips() const { return CRNLIB_MAX(m_header.m_numberOfMipmapLevels, 1); }
uint get_array_size() const { return CRNLIB_MAX(m_header.m_numberOfArrayElements, 1); }
uint get_num_faces() const { return m_header.m_numberOfFaces; }
uint32 get_ogl_type() const { return m_header.m_glType; } // General info
uint32 get_ogl_fmt() const { return m_header.m_glFormat; }
uint32 get_ogl_base_fmt() const { return m_header.m_glBaseInternalFormat; }
uint32 get_ogl_internal_fmt() const { return m_header.m_glInternalFormat; }
uint get_total_images() const { return get_num_mips() * (get_depth() * get_num_faces() * get_array_size()); }
bool is_compressed() const { return m_block_dim > 1; } bool is_valid() const { return (m_header.m_pixelWidth > 0) && (m_image_data.size() > 0); }
bool is_uncompressed() const { return !is_compressed(); }
bool get_opposite_endianness() const { return m_opposite_endianness; }
void set_opposite_endianness(bool flag) { m_opposite_endianness = flag; }
uint32 get_block_dim() const { return m_block_dim; } uint get_width() const { return m_header.m_pixelWidth; }
uint32 get_bytes_per_block() const { return m_bytes_per_block; } uint get_height() const { return CRNLIB_MAX(m_header.m_pixelHeight, 1); }
uint get_depth() const { return CRNLIB_MAX(m_header.m_pixelDepth, 1); }
uint get_num_mips() const { return CRNLIB_MAX(m_header.m_numberOfMipmapLevels, 1); }
uint get_array_size() const { return CRNLIB_MAX(m_header.m_numberOfArrayElements, 1); }
uint get_num_faces() const { return m_header.m_numberOfFaces; }
const ktx_header& get_header() const { return m_header; } uint32 get_ogl_type() const { return m_header.m_glType; }
uint32 get_ogl_fmt() const { return m_header.m_glFormat; }
uint32 get_ogl_base_fmt() const { return m_header.m_glBaseInternalFormat; }
uint32 get_ogl_internal_fmt() const { return m_header.m_glInternalFormat; }
// Key values uint get_total_images() const { return get_num_mips() * (get_depth() * get_num_faces() * get_array_size()); }
const ktx_key_value_vec& get_key_value_vec() const { return m_key_values; }
ktx_key_value_vec& get_key_value_vec() { return m_key_values; }
const uint8_vec* find_key(const char* pKey) const; bool is_compressed() const { return m_block_dim > 1; }
bool get_key_value_as_string(const char* pKey, dynamic_string& str) const; bool is_uncompressed() const { return !is_compressed(); }
uint add_key_value(const char* pKey, const void* pVal, uint val_size); bool get_opposite_endianness() const { return m_opposite_endianness; }
uint add_key_value(const char* pKey, const char* pVal) { return add_key_value(pKey, pVal, static_cast<uint>(strlen(pVal)) + 1); } void set_opposite_endianness(bool flag) { m_opposite_endianness = flag; }
// Image data uint32 get_block_dim() const { return m_block_dim; }
uint get_num_images() const { return m_image_data.size(); } uint32 get_bytes_per_block() const { return m_bytes_per_block; }
const uint8_vec& get_image_data(uint image_index) const { return m_image_data[image_index]; } const ktx_header& get_header() const { return m_header; }
uint8_vec& get_image_data(uint image_index) { return m_image_data[image_index]; }
const uint8_vec& get_image_data(uint mip_index, uint array_index, uint face_index, uint zslice_index) const { return get_image_data(get_image_index(mip_index, array_index, face_index, zslice_index)); } // Key values
uint8_vec& get_image_data(uint mip_index, uint array_index, uint face_index, uint zslice_index) { return get_image_data(get_image_index(mip_index, array_index, face_index, zslice_index)); } const ktx_key_value_vec& get_key_value_vec() const { return m_key_values; }
ktx_key_value_vec& get_key_value_vec() { return m_key_values; }
const ktx_image_data_vec& get_image_data_vec() const { return m_image_data; }
ktx_image_data_vec& get_image_data_vec() { return m_image_data; }
void add_image(uint face_index, uint mip_index, const void* pImage, uint image_size) const uint8_vec* find_key(const char* pKey) const;
{ bool get_key_value_as_string(const char* pKey, dynamic_string& str) const;
const uint image_index = get_image_index(mip_index, 0, face_index, 0);
if (image_index >= m_image_data.size())
m_image_data.resize(image_index + 1);
if (image_size)
{
uint8_vec& v = m_image_data[image_index];
v.resize(image_size);
memcpy(&v[0], pImage, image_size);
}
}
uint get_image_index(uint mip_index, uint array_index, uint face_index, uint zslice_index) const uint add_key_value(const char* pKey, const void* pVal, uint val_size);
{ uint add_key_value(const char* pKey, const char* pVal) { return add_key_value(pKey, pVal, static_cast<uint>(strlen(pVal)) + 1); }
CRNLIB_ASSERT((mip_index < get_num_mips()) && (array_index < get_array_size()) && (face_index < get_num_faces()) && (zslice_index < get_depth()));
return zslice_index + (face_index * get_depth()) + (array_index * (get_depth() * get_num_faces())) + (mip_index * (get_depth() * get_num_faces() * get_array_size()));
}
void get_mip_dim(uint mip_index, uint& mip_width, uint& mip_height) const
{
CRNLIB_ASSERT(mip_index < get_num_mips());
mip_width = CRNLIB_MAX(get_width() >> mip_index, 1);
mip_height = CRNLIB_MAX(get_height() >> mip_index, 1);
}
void get_mip_dim(uint mip_index, uint& mip_width, uint& mip_height, uint& mip_depth) const // Image data
{ uint get_num_images() const { return m_image_data.size(); }
CRNLIB_ASSERT(mip_index < get_num_mips());
mip_width = CRNLIB_MAX(get_width() >> mip_index, 1);
mip_height = CRNLIB_MAX(get_height() >> mip_index, 1);
mip_depth = CRNLIB_MAX(get_depth() >> mip_index, 1);
}
private:
ktx_header m_header;
ktx_key_value_vec m_key_values;
ktx_image_data_vec m_image_data;
uint32 m_block_dim; const uint8_vec& get_image_data(uint image_index) const { return m_image_data[image_index]; }
uint32 m_bytes_per_block; uint8_vec& get_image_data(uint image_index) { return m_image_data[image_index]; }
bool m_opposite_endianness;
bool compute_pixel_info();
};
} // namespace crnlib
#endif // #ifndef _KTX_TEXTURE_H_ const uint8_vec& get_image_data(uint mip_index, uint array_index, uint face_index, uint zslice_index) const { return get_image_data(get_image_index(mip_index, array_index, face_index, zslice_index)); }
uint8_vec& get_image_data(uint mip_index, uint array_index, uint face_index, uint zslice_index) { return get_image_data(get_image_index(mip_index, array_index, face_index, zslice_index)); }
const ktx_image_data_vec& get_image_data_vec() const { return m_image_data; }
ktx_image_data_vec& get_image_data_vec() { return m_image_data; }
void add_image(uint face_index, uint mip_index, const void* pImage, uint image_size) {
const uint image_index = get_image_index(mip_index, 0, face_index, 0);
if (image_index >= m_image_data.size())
m_image_data.resize(image_index + 1);
if (image_size) {
uint8_vec& v = m_image_data[image_index];
v.resize(image_size);
memcpy(&v[0], pImage, image_size);
}
}
uint get_image_index(uint mip_index, uint array_index, uint face_index, uint zslice_index) const {
CRNLIB_ASSERT((mip_index < get_num_mips()) && (array_index < get_array_size()) && (face_index < get_num_faces()) && (zslice_index < get_depth()));
return zslice_index + (face_index * get_depth()) + (array_index * (get_depth() * get_num_faces())) + (mip_index * (get_depth() * get_num_faces() * get_array_size()));
}
void get_mip_dim(uint mip_index, uint& mip_width, uint& mip_height) const {
CRNLIB_ASSERT(mip_index < get_num_mips());
mip_width = CRNLIB_MAX(get_width() >> mip_index, 1);
mip_height = CRNLIB_MAX(get_height() >> mip_index, 1);
}
void get_mip_dim(uint mip_index, uint& mip_width, uint& mip_height, uint& mip_depth) const {
CRNLIB_ASSERT(mip_index < get_num_mips());
mip_width = CRNLIB_MAX(get_width() >> mip_index, 1);
mip_height = CRNLIB_MAX(get_height() >> mip_index, 1);
mip_depth = CRNLIB_MAX(get_depth() >> mip_index, 1);
}
private:
ktx_header m_header;
ktx_key_value_vec m_key_values;
ktx_image_data_vec m_image_data;
uint32 m_block_dim;
uint32 m_bytes_per_block;
bool m_opposite_endianness;
bool compute_pixel_info();
};
} // namespace crnlib
#endif // #ifndef _KTX_TEXTURE_H_
crnlib/crn_lzma_codec.cpp
+89 -99
View File
@@ -7,136 +7,126 @@
#include "lzma_LzmaLib.h" #include "lzma_LzmaLib.h"
#include "crn_threading.h" #include "crn_threading.h"
namespace crnlib namespace crnlib {
{ lzma_codec::lzma_codec()
lzma_codec::lzma_codec() : : m_pCompress(LzmaCompress),
m_pCompress(LzmaCompress), m_pUncompress(LzmaUncompress) {
m_pUncompress(LzmaUncompress) CRNLIB_ASSUME(cLZMAPropsSize == LZMA_PROPS_SIZE);
{ }
CRNLIB_ASSUME(cLZMAPropsSize == LZMA_PROPS_SIZE);
}
lzma_codec::~lzma_codec() lzma_codec::~lzma_codec() {
{ }
}
bool lzma_codec::pack(const void* p, uint n, crnlib::vector<uint8>& buf) bool lzma_codec::pack(const void* p, uint n, crnlib::vector<uint8>& buf) {
{ if (n > 1024U * 1024U * 1024U)
if (n > 1024U*1024U*1024U) return false;
return false;
uint max_comp_size = n + math::maximum<uint>(128, n >> 8); uint max_comp_size = n + math::maximum<uint>(128, n >> 8);
buf.resize(sizeof(header) + max_comp_size); buf.resize(sizeof(header) + max_comp_size);
header* pHDR = reinterpret_cast<header*>(&buf[0]); header* pHDR = reinterpret_cast<header*>(&buf[0]);
uint8* pComp_data = &buf[sizeof(header)]; uint8* pComp_data = &buf[sizeof(header)];
utils::zero_object(*pHDR); utils::zero_object(*pHDR);
pHDR->m_uncomp_size = n; pHDR->m_uncomp_size = n;
pHDR->m_adler32 = adler32(p, n); pHDR->m_adler32 = adler32(p, n);
if (n) if (n) {
{ size_t destLen = 0;
size_t destLen = 0; size_t outPropsSize = 0;
size_t outPropsSize = 0; int status = SZ_ERROR_INPUT_EOF;
int status = SZ_ERROR_INPUT_EOF;
for (uint trial = 0; trial < 3; trial++) for (uint trial = 0; trial < 3; trial++) {
{ destLen = max_comp_size;
destLen = max_comp_size; outPropsSize = cLZMAPropsSize;
outPropsSize = cLZMAPropsSize;
status = (*m_pCompress)(pComp_data, &destLen, reinterpret_cast<const unsigned char*>(p), n, status = (*m_pCompress)(pComp_data, &destLen, reinterpret_cast<const unsigned char*>(p), n,
pHDR->m_lzma_props, &outPropsSize, pHDR->m_lzma_props, &outPropsSize,
-1, /* 0 <= level <= 9, default = 5 */ -1, /* 0 <= level <= 9, default = 5 */
0, /* default = (1 << 24) */ 0, /* default = (1 << 24) */
-1, /* 0 <= lc <= 8, default = 3 */ -1, /* 0 <= lc <= 8, default = 3 */
-1, /* 0 <= lp <= 4, default = 0 */ -1, /* 0 <= lp <= 4, default = 0 */
-1, /* 0 <= pb <= 4, default = 2 */ -1, /* 0 <= pb <= 4, default = 2 */
-1, /* 5 <= fb <= 273, default = 32 */ -1, /* 5 <= fb <= 273, default = 32 */
#ifdef WIN32 #ifdef WIN32
(g_number_of_processors > 1) ? 2 : 1 (g_number_of_processors > 1) ? 2 : 1
#else #else
1 1
#endif #endif
); );
if (status != SZ_ERROR_OUTPUT_EOF) if (status != SZ_ERROR_OUTPUT_EOF)
break; break;
max_comp_size += ((n+1)/2); max_comp_size += ((n + 1) / 2);
buf.resize(sizeof(header) + max_comp_size); buf.resize(sizeof(header) + max_comp_size);
pHDR = reinterpret_cast<header*>(&buf[0]); pHDR = reinterpret_cast<header*>(&buf[0]);
pComp_data = &buf[sizeof(header)]; pComp_data = &buf[sizeof(header)];
} }
if (status != SZ_OK) if (status != SZ_OK) {
{ buf.clear();
buf.clear(); return false;
return false; }
}
pHDR->m_comp_size = static_cast<uint>(destLen); pHDR->m_comp_size = static_cast<uint>(destLen);
buf.resize(CRNLIB_SIZEOF_U32(header) + static_cast<uint32>(destLen)); buf.resize(CRNLIB_SIZEOF_U32(header) + static_cast<uint32>(destLen));
} }
pHDR->m_sig = header::cSig; pHDR->m_sig = header::cSig;
pHDR->m_checksum = static_cast<uint8>(adler32((uint8*)pHDR + header::cChecksumSkipBytes, sizeof(header) - header::cChecksumSkipBytes)); pHDR->m_checksum = static_cast<uint8>(adler32((uint8*)pHDR + header::cChecksumSkipBytes, sizeof(header) - header::cChecksumSkipBytes));
return true; return true;
} }
bool lzma_codec::unpack(const void* p, uint n, crnlib::vector<uint8>& buf) bool lzma_codec::unpack(const void* p, uint n, crnlib::vector<uint8>& buf) {
{ buf.resize(0);
buf.resize(0);
if (n < sizeof(header)) if (n < sizeof(header))
return false; return false;
const header& hdr = *static_cast<const header*>(p); const header& hdr = *static_cast<const header*>(p);
if (hdr.m_sig != header::cSig) if (hdr.m_sig != header::cSig)
return false; return false;
if (static_cast<uint8>(adler32((const uint8*)&hdr + header::cChecksumSkipBytes, sizeof(hdr) - header::cChecksumSkipBytes)) != hdr.m_checksum) if (static_cast<uint8>(adler32((const uint8*)&hdr + header::cChecksumSkipBytes, sizeof(hdr) - header::cChecksumSkipBytes)) != hdr.m_checksum)
return false; return false;
if (!hdr.m_uncomp_size) if (!hdr.m_uncomp_size)
return true; return true;
if (!hdr.m_comp_size) if (!hdr.m_comp_size)
return false; return false;
if (hdr.m_uncomp_size > 1024U*1024U*1024U) if (hdr.m_uncomp_size > 1024U * 1024U * 1024U)
return false; return false;
if (!buf.try_resize(hdr.m_uncomp_size)) if (!buf.try_resize(hdr.m_uncomp_size))
return false; return false;
const uint8* pComp_data = static_cast<const uint8*>(p) + sizeof(header); const uint8* pComp_data = static_cast<const uint8*>(p) + sizeof(header);
size_t srcLen = n - sizeof(header); size_t srcLen = n - sizeof(header);
if (srcLen < hdr.m_comp_size) if (srcLen < hdr.m_comp_size)
return false; return false;
size_t destLen = hdr.m_uncomp_size; size_t destLen = hdr.m_uncomp_size;
int status = (*m_pUncompress)(&buf[0], &destLen, pComp_data, &srcLen, int status = (*m_pUncompress)(&buf[0], &destLen, pComp_data, &srcLen,
hdr.m_lzma_props, cLZMAPropsSize); hdr.m_lzma_props, cLZMAPropsSize);
if ((status != SZ_OK) || (destLen != hdr.m_uncomp_size)) if ((status != SZ_OK) || (destLen != hdr.m_uncomp_size)) {
{ buf.clear();
buf.clear(); return false;
return false; }
}
if (adler32(&buf[0], buf.size()) != hdr.m_adler32) if (adler32(&buf[0], buf.size()) != hdr.m_adler32) {
{ buf.clear();
buf.clear(); return false;
return false; }
}
return true; return true;
} }
} // namespace crnlib } // namespace crnlib
crnlib/crn_lzma_codec.h
+37 -40
View File
@@ -3,58 +3,55 @@
#pragma once #pragma once
#include "crn_packed_uint.h" #include "crn_packed_uint.h"
namespace crnlib namespace crnlib {
{ class lzma_codec {
class lzma_codec public:
{ lzma_codec();
public: ~lzma_codec();
lzma_codec();
~lzma_codec();
// Always available, because we're statically linking in lzmalib now vs. dynamically loading the DLL. // Always available, because we're statically linking in lzmalib now vs. dynamically loading the DLL.
bool is_initialized() const { return true; } bool is_initialized() const { return true; }
bool pack(const void* p, uint n, crnlib::vector<uint8>& buf); bool pack(const void* p, uint n, crnlib::vector<uint8>& buf);
bool unpack(const void* p, uint n, crnlib::vector<uint8>& buf); bool unpack(const void* p, uint n, crnlib::vector<uint8>& buf);
private: private:
typedef int (CRNLIB_STDCALL *LzmaCompressFuncPtr)(unsigned char *dest, size_t *destLen, const unsigned char *src, size_t srcLen, typedef int(CRNLIB_STDCALL* LzmaCompressFuncPtr)(unsigned char* dest, size_t* destLen, const unsigned char* src, size_t srcLen,
unsigned char *outProps, size_t *outPropsSize, /* *outPropsSize must be = 5 */ unsigned char* outProps, size_t* outPropsSize, /* *outPropsSize must be = 5 */
int level, /* 0 <= level <= 9, default = 5 */ int level, /* 0 <= level <= 9, default = 5 */
unsigned dictSize, /* default = (1 << 24) */ unsigned dictSize, /* default = (1 << 24) */
int lc, /* 0 <= lc <= 8, default = 3 */ int lc, /* 0 <= lc <= 8, default = 3 */
int lp, /* 0 <= lp <= 4, default = 0 */ int lp, /* 0 <= lp <= 4, default = 0 */
int pb, /* 0 <= pb <= 4, default = 2 */ int pb, /* 0 <= pb <= 4, default = 2 */
int fb, /* 5 <= fb <= 273, default = 32 */ int fb, /* 5 <= fb <= 273, default = 32 */
int numThreads /* 1 or 2, default = 2 */ int numThreads /* 1 or 2, default = 2 */
); );
typedef int (CRNLIB_STDCALL *LzmaUncompressFuncPtr)(unsigned char *dest, size_t *destLen, const unsigned char *src, size_t *srcLen, typedef int(CRNLIB_STDCALL* LzmaUncompressFuncPtr)(unsigned char* dest, size_t* destLen, const unsigned char* src, size_t* srcLen,
const unsigned char *props, size_t propsSize); const unsigned char* props, size_t propsSize);
LzmaCompressFuncPtr m_pCompress; LzmaCompressFuncPtr m_pCompress;
LzmaUncompressFuncPtr m_pUncompress; LzmaUncompressFuncPtr m_pUncompress;
enum { cLZMAPropsSize = 5 }; enum { cLZMAPropsSize = 5 };
#pragma pack(push) #pragma pack(push)
#pragma pack(1) #pragma pack(1)
struct header struct header {
{ enum { cSig = 'L' | ('0' << 8),
enum { cSig = 'L' | ('0' << 8), cChecksumSkipBytes = 3 }; cChecksumSkipBytes = 3 };
packed_uint<2> m_sig; packed_uint<2> m_sig;
uint8 m_checksum; uint8 m_checksum;
uint8 m_lzma_props[cLZMAPropsSize]; uint8 m_lzma_props[cLZMAPropsSize];
packed_uint<4> m_comp_size; packed_uint<4> m_comp_size;
packed_uint<4> m_uncomp_size; packed_uint<4> m_uncomp_size;
packed_uint<4> m_adler32; packed_uint<4> m_adler32;
}; };
#pragma pack(pop) #pragma pack(pop)
};
}; } // namespace crnlib
} // namespace crnlib
crnlib/crn_math.cpp
+58 -67
View File
@@ -2,75 +2,66 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#include "crn_core.h" #include "crn_core.h"
namespace crnlib namespace crnlib {
{ namespace math {
namespace math uint g_bitmasks[32] =
{ {
uint g_bitmasks[32] = 1U << 0U, 1U << 1U, 1U << 2U, 1U << 3U,
{ 1U << 4U, 1U << 5U, 1U << 6U, 1U << 7U,
1U << 0U, 1U << 1U, 1U << 2U, 1U << 3U, 1U << 8U, 1U << 9U, 1U << 10U, 1U << 11U,
1U << 4U, 1U << 5U, 1U << 6U, 1U << 7U, 1U << 12U, 1U << 13U, 1U << 14U, 1U << 15U,
1U << 8U, 1U << 9U, 1U << 10U, 1U << 11U, 1U << 16U, 1U << 17U, 1U << 18U, 1U << 19U,
1U << 12U, 1U << 13U, 1U << 14U, 1U << 15U, 1U << 20U, 1U << 21U, 1U << 22U, 1U << 23U,
1U << 16U, 1U << 17U, 1U << 18U, 1U << 19U, 1U << 24U, 1U << 25U, 1U << 26U, 1U << 27U,
1U << 20U, 1U << 21U, 1U << 22U, 1U << 23U, 1U << 28U, 1U << 29U, 1U << 30U, 1U << 31U};
1U << 24U, 1U << 25U, 1U << 26U, 1U << 27U,
1U << 28U, 1U << 29U, 1U << 30U, 1U << 31U
};
double compute_entropy(const uint8* p, uint n)
{
uint hist[256];
utils::zero_object(hist);
for (uint i = 0; i < n; i++)
hist[*p++]++;
double entropy = 0.0f;
const double invln2 = 1.0f/log(2.0f);
for (uint i = 0; i < 256; i++)
{
if (!hist[i])
continue;
double prob = static_cast<double>(hist[i]) / n;
entropy += (-log(prob) * invln2) * hist[i];
}
return entropy;
}
void compute_lower_pow2_dim(int& width, int& height) double compute_entropy(const uint8* p, uint n) {
{ uint hist[256];
const int tex_width = width; utils::zero_object(hist);
const int tex_height = height;
width = 1; for (uint i = 0; i < n; i++)
for ( ; ; ) hist[*p++]++;
{
if ((width * 2) > tex_width)
break;
width *= 2;
}
height = 1; double entropy = 0.0f;
for ( ; ; )
{
if ((height * 2) > tex_height)
break;
height *= 2;
}
}
void compute_upper_pow2_dim(int& width, int& height) const double invln2 = 1.0f / log(2.0f);
{ for (uint i = 0; i < 256; i++) {
if (!math::is_power_of_2((uint32)width)) if (!hist[i])
width = math::next_pow2((uint32)width); continue;
if (!math::is_power_of_2((uint32)height)) double prob = static_cast<double>(hist[i]) / n;
height = math::next_pow2((uint32)height); entropy += (-log(prob) * invln2) * hist[i];
} }
} // namespace math return entropy;
} // namespace crnlib }
void compute_lower_pow2_dim(int& width, int& height) {
const int tex_width = width;
const int tex_height = height;
width = 1;
for (;;) {
if ((width * 2) > tex_width)
break;
width *= 2;
}
height = 1;
for (;;) {
if ((height * 2) > tex_height)
break;
height *= 2;
}
}
void compute_upper_pow2_dim(int& width, int& height) {
if (!math::is_power_of_2((uint32)width))
width = math::next_pow2((uint32)width);
if (!math::is_power_of_2((uint32)height))
height = math::next_pow2((uint32)height);
}
} // namespace math
} // namespace crnlib
crnlib/crn_math.h
+226 -192
View File
@@ -3,244 +3,278 @@
#pragma once #pragma once
#if defined(_M_IX86) && defined(_MSC_VER) #if defined(_M_IX86) && defined(_MSC_VER)
#include <intrin.h> #include <intrin.h>
#pragma intrinsic(__emulu) #pragma intrinsic(__emulu)
unsigned __int64 __emulu(unsigned int a,unsigned int b ); unsigned __int64 __emulu(unsigned int a, unsigned int b);
#endif #endif
namespace crnlib namespace crnlib {
{ namespace math {
namespace math const float cNearlyInfinite = 1.0e+37f;
{
const float cNearlyInfinite = 1.0e+37f;
const float cDegToRad = 0.01745329252f;
const float cRadToDeg = 57.29577951f;
extern uint g_bitmasks[32]; const float cDegToRad = 0.01745329252f;
const float cRadToDeg = 57.29577951f;
template<typename T> inline bool within_closed_range(T a, T b, T c) { return (a >= b) && (a <= c); } extern uint g_bitmasks[32];
template<typename T> inline bool within_open_range(T a, T b, T c) { return (a >= b) && (a < c); } template <typename T>
inline bool within_closed_range(T a, T b, T c) {
return (a >= b) && (a <= c);
}
// Yes I know these should probably be pass by ref, not val: template <typename T>
// http://www.stepanovpapers.com/notes.pdf inline bool within_open_range(T a, T b, T c) {
// Just don't use them on non-simple (non built-in) types! return (a >= b) && (a < c);
template<typename T> inline T minimum(T a, T b) { return (a < b) ? a : b; } }
template<typename T> inline T minimum(T a, T b, T c) { return minimum(minimum(a, b), c); } // Yes I know these should probably be pass by ref, not val:
// http://www.stepanovpapers.com/notes.pdf
// Just don't use them on non-simple (non built-in) types!
template <typename T>
inline T minimum(T a, T b) {
return (a < b) ? a : b;
}
template<typename T> inline T maximum(T a, T b) { return (a > b) ? a : b; } template <typename T>
inline T minimum(T a, T b, T c) {
return minimum(minimum(a, b), c);
}
template<typename T> inline T maximum(T a, T b, T c) { return maximum(maximum(a, b), c); } template <typename T>
inline T maximum(T a, T b) {
return (a > b) ? a : b;
}
template<typename T, typename U> inline T lerp(T a, T b, U c) { return a + (b - a) * c; } template <typename T>
inline T maximum(T a, T b, T c) {
return maximum(maximum(a, b), c);
}
template<typename T> inline T clamp(T value, T low, T high) { return (value < low) ? low : ((value > high) ? high : value); } template <typename T, typename U>
inline T lerp(T a, T b, U c) {
return a + (b - a) * c;
}
template<typename T> inline T saturate(T value) { return (value < 0.0f) ? 0.0f : ((value > 1.0f) ? 1.0f : value); } template <typename T>
inline T clamp(T value, T low, T high) {
return (value < low) ? low : ((value > high) ? high : value);
}
inline int float_to_int(float f) { return static_cast<int>(f); } template <typename T>
inline T saturate(T value) {
return (value < 0.0f) ? 0.0f : ((value > 1.0f) ? 1.0f : value);
}
inline uint float_to_uint(float f) { return static_cast<uint>(f); } inline int float_to_int(float f) {
return static_cast<int>(f);
}
inline int float_to_int(double f) { return static_cast<int>(f); } inline uint float_to_uint(float f) {
return static_cast<uint>(f);
}
inline uint float_to_uint(double f) { return static_cast<uint>(f); } inline int float_to_int(double f) {
return static_cast<int>(f);
}
inline int float_to_int_round(float f) { return static_cast<int>((f < 0.0f) ? -floor(-f + .5f) : floor(f + .5f)); } inline uint float_to_uint(double f) {
return static_cast<uint>(f);
}
inline uint float_to_uint_round(float f) { return static_cast<uint>((f < 0.0f) ? 0.0f : floor(f + .5f)); } inline int float_to_int_round(float f) {
return static_cast<int>((f < 0.0f) ? -floor(-f + .5f) : floor(f + .5f));
}
template<typename T> inline int sign(T value) { return (value < 0) ? -1 : ((value > 0) ? 1 : 0); } inline uint float_to_uint_round(float f) {
return static_cast<uint>((f < 0.0f) ? 0.0f : floor(f + .5f));
}
template<typename T> inline T square(T value) { return value * value; } template <typename T>
inline int sign(T value) {
return (value < 0) ? -1 : ((value > 0) ? 1 : 0);
}
#ifdef __GNUC__ template <typename T>
#if CRNLIB_64BIT_POINTERS==0 inline T square(T value) {
inline bool is_power_of_2(uint32 x) { return x && ((x & (x - 1U)) == 0U); } return value * value;
#endif }
#endif
inline bool is_power_of_2(uint64 x) { return x && ((x & (x - 1U)) == 0U); } inline bool is_power_of_2(uint32 x) {
return x && ((x & (x - 1U)) == 0U);
}
inline bool is_power_of_2(uint64 x) {
return x && ((x & (x - 1U)) == 0U);
}
template<typename T> inline T align_up_value(T x, uint alignment) template <typename T>
{ inline T align_up_value(T x, uint alignment) {
CRNLIB_ASSERT(is_power_of_2(alignment)); CRNLIB_ASSERT(is_power_of_2(alignment));
uint q = static_cast<uint>(x); uint q = static_cast<uint>(x);
q = (q + alignment - 1) & (~(alignment - 1)); q = (q + alignment - 1) & (~(alignment - 1));
return static_cast<T>(q); return static_cast<T>(q);
} }
template<typename T> inline T align_down_value(T x, uint alignment) template <typename T>
{ inline T align_down_value(T x, uint alignment) {
CRNLIB_ASSERT(is_power_of_2(alignment)); CRNLIB_ASSERT(is_power_of_2(alignment));
uint q = static_cast<uint>(x); uint q = static_cast<uint>(x);
q = q & (~(alignment - 1)); q = q & (~(alignment - 1));
return static_cast<T>(q); return static_cast<T>(q);
} }
template<typename T> inline T get_align_up_value_delta(T x, uint alignment) template <typename T>
{ inline T get_align_up_value_delta(T x, uint alignment) {
return align_up_value(x, alignment) - x; return align_up_value(x, alignment) - x;
} }
#ifdef __GNUC__ // From "Hackers Delight"
#if CRNLIB_64BIT_POINTERS==0 inline uint32 next_pow2(uint32 val) {
// From "Hackers Delight" val--;
inline uint32 next_pow2(uint32 val) val |= val >> 16;
{ val |= val >> 8;
val--; val |= val >> 4;
val |= val >> 16; val |= val >> 2;
val |= val >> 8; val |= val >> 1;
val |= val >> 4; return val + 1;
val |= val >> 2; }
val |= val >> 1;
return val + 1;
}
#endif
#endif
inline uint64 next_pow2(uint64 val) inline uint64 next_pow2(uint64 val) {
{ val--;
val--; val |= val >> 32;
val |= val >> 32; val |= val >> 16;
val |= val >> 16; val |= val >> 8;
val |= val >> 8; val |= val >> 4;
val |= val >> 4; val |= val >> 2;
val |= val >> 2; val |= val >> 1;
val |= val >> 1; return val + 1;
return val + 1; }
}
inline uint floor_log2i(uint v)
{
uint l = 0;
while (v > 1U)
{
v >>= 1;
l++;
}
return l;
}
inline uint ceil_log2i(uint v) inline uint floor_log2i(uint v) {
{ uint l = 0;
uint l = floor_log2i(v); while (v > 1U) {
if ((l != cIntBits) && (v > (1U << l))) v >>= 1;
l++; l++;
return l; }
} return l;
}
// Returns the total number of bits needed to encode v. inline uint ceil_log2i(uint v) {
inline uint total_bits(uint v) uint l = floor_log2i(v);
{ if ((l != cIntBits) && (v > (1U << l)))
uint l = 0; l++;
while (v > 0U) return l;
{ }
v >>= 1;
l++;
}
return l;
}
// Actually counts the number of set bits, but hey // Returns the total number of bits needed to encode v.
inline uint bitmask_size(uint mask) inline uint total_bits(uint v) {
{ uint l = 0;
uint size = 0; while (v > 0U) {
while (mask) v >>= 1;
{ l++;
mask &= (mask - 1U); }
size++; return l;
} }
return size;
}
inline uint bitmask_ofs(uint mask) // Actually counts the number of set bits, but hey
{ inline uint bitmask_size(uint mask) {
if (!mask) uint size = 0;
return 0; while (mask) {
uint ofs = 0; mask &= (mask - 1U);
while ((mask & 1U) == 0) size++;
{ }
mask >>= 1U; return size;
ofs++; }
}
return ofs;
}
// See Bit Twiddling Hacks (public domain) inline uint bitmask_ofs(uint mask) {
// http://www-graphics.stanford.edu/~seander/bithacks.html if (!mask)
inline uint count_trailing_zero_bits(uint v) return 0;
{ uint ofs = 0;
uint c = 32; // c will be the number of zero bits on the right while ((mask & 1U) == 0) {
mask >>= 1U;
ofs++;
}
return ofs;
}
static const unsigned int B[] = { 0x55555555, 0x33333333, 0x0F0F0F0F, 0x00FF00FF, 0x0000FFFF }; // See Bit Twiddling Hacks (public domain)
static const unsigned int S[] = { 1, 2, 4, 8, 16 }; // Our Magic Binary Numbers // http://www-graphics.stanford.edu/~seander/bithacks.html
inline uint count_trailing_zero_bits(uint v) {
uint c = 32; // c will be the number of zero bits on the right
for (int i = 4; i >= 0; --i) // unroll for more speed static const unsigned int B[] = {0x55555555, 0x33333333, 0x0F0F0F0F, 0x00FF00FF, 0x0000FFFF};
{ static const unsigned int S[] = {1, 2, 4, 8, 16}; // Our Magic Binary Numbers
if (v & B[i])
{
v <<= S[i];
c -= S[i];
}
}
if (v) for (int i = 4; i >= 0; --i) // unroll for more speed
{ {
c--; if (v & B[i]) {
} v <<= S[i];
c -= S[i];
}
}
return c; if (v) {
} c--;
}
inline uint count_leading_zero_bits(uint v) return c;
{ }
uint temp;
uint result = 32U;
temp = (v >> 16U); if (temp) { result -= 16U; v = temp; } inline uint count_leading_zero_bits(uint v) {
temp = (v >> 8U); if (temp) { result -= 8U; v = temp; } uint temp;
temp = (v >> 4U); if (temp) { result -= 4U; v = temp; } uint result = 32U;
temp = (v >> 2U); if (temp) { result -= 2U; v = temp; }
temp = (v >> 1U); if (temp) { result -= 1U; v = temp; }
if (v & 1U) temp = (v >> 16U);
result--; if (temp) {
result -= 16U;
v = temp;
}
temp = (v >> 8U);
if (temp) {
result -= 8U;
v = temp;
}
temp = (v >> 4U);
if (temp) {
result -= 4U;
v = temp;
}
temp = (v >> 2U);
if (temp) {
result -= 2U;
v = temp;
}
temp = (v >> 1U);
if (temp) {
result -= 1U;
v = temp;
}
return result; if (v & 1U)
} result--;
inline uint64 emulu(uint32 a, uint32 b) return result;
{ }
inline uint64 emulu(uint32 a, uint32 b) {
#if defined(_M_IX86) && defined(_MSC_VER) #if defined(_M_IX86) && defined(_MSC_VER)
return __emulu(a, b); return __emulu(a, b);
#else #else
return static_cast<uint64>(a) * static_cast<uint64>(b); return static_cast<uint64>(a) * static_cast<uint64>(b);
#endif #endif
} }
double compute_entropy(const uint8* p, uint n);
void compute_lower_pow2_dim(int& width, int& height);
void compute_upper_pow2_dim(int& width, int& height);
inline bool equal_tol(float a, float b, float t)
{
return fabs(a - b) < ((maximum(fabs(a), fabs(b)) + 1.0f) * t);
}
inline bool equal_tol(double a, double b, double t)
{
return fabs(a - b) < ((maximum(fabs(a), fabs(b)) + 1.0f) * t);
}
}
} // namespace crnlib
double compute_entropy(const uint8* p, uint n);
void compute_lower_pow2_dim(int& width, int& height);
void compute_upper_pow2_dim(int& width, int& height);
inline bool equal_tol(float a, float b, float t) {
return fabs(a - b) < ((maximum(fabs(a), fabs(b)) + 1.0f) * t);
}
inline bool equal_tol(double a, double b, double t) {
return fabs(a - b) < ((maximum(fabs(a), fabs(b)) + 1.0f) * t);
}
}
} // namespace crnlib
crnlib/crn_matrix.h
+446 -517
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_mem.cpp
+234 -356
View File
@@ -14,366 +14,244 @@
#define _msize malloc_usable_size #define _msize malloc_usable_size
#endif #endif
namespace crnlib namespace crnlib {
{
#if CRNLIB_MEM_STATS #if CRNLIB_MEM_STATS
#if CRNLIB_64BIT_POINTERS #if CRNLIB_64BIT_POINTERS
typedef LONGLONG mem_stat_t; typedef LONGLONG mem_stat_t;
#define CRNLIB_MEM_COMPARE_EXCHANGE InterlockedCompareExchange64 #define CRNLIB_MEM_COMPARE_EXCHANGE InterlockedCompareExchange64
#else
typedef LONG mem_stat_t;
#define CRNLIB_MEM_COMPARE_EXCHANGE InterlockedCompareExchange
#endif
static volatile mem_stat_t g_total_blocks;
static volatile mem_stat_t g_total_allocated;
static volatile mem_stat_t g_max_allocated;
static mem_stat_t update_total_allocated(int block_delta, mem_stat_t byte_delta)
{
mem_stat_t cur_total_blocks;
for ( ; ; )
{
cur_total_blocks = (mem_stat_t)g_total_blocks;
mem_stat_t new_total_blocks = static_cast<mem_stat_t>(cur_total_blocks + block_delta);
CRNLIB_ASSERT(new_total_blocks >= 0);
if (CRNLIB_MEM_COMPARE_EXCHANGE(&g_total_blocks, new_total_blocks, cur_total_blocks) == cur_total_blocks)
break;
}
mem_stat_t cur_total_allocated, new_total_allocated;
for ( ; ; )
{
cur_total_allocated = g_total_allocated;
new_total_allocated = static_cast<mem_stat_t>(cur_total_allocated + byte_delta);
CRNLIB_ASSERT(new_total_allocated >= 0);
if (CRNLIB_MEM_COMPARE_EXCHANGE(&g_total_allocated, new_total_allocated, cur_total_allocated) == cur_total_allocated)
break;
}
for ( ; ; )
{
mem_stat_t cur_max_allocated = g_max_allocated;
mem_stat_t new_max_allocated = CRNLIB_MAX(new_total_allocated, cur_max_allocated);
if (CRNLIB_MEM_COMPARE_EXCHANGE(&g_max_allocated, new_max_allocated, cur_max_allocated) == cur_max_allocated)
break;
}
return new_total_allocated;
}
#endif // CRNLIB_MEM_STATS
static void* crnlib_default_realloc(void* p, size_t size, size_t* pActual_size, bool movable, void* pUser_data)
{
pUser_data;
void* p_new;
if (!p)
{
p_new = ::malloc(size);
CRNLIB_ASSERT( (reinterpret_cast<ptr_bits_t>(p_new) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) == 0 );
if (!p_new)
{
printf("WARNING: ::malloc() of size %u failed!\n", (uint)size);
}
if (pActual_size)
*pActual_size = p_new ? ::_msize(p_new) : 0;
}
else if (!size)
{
::free(p);
p_new = NULL;
if (pActual_size)
*pActual_size = 0;
}
else
{
void* p_final_block = p;
#ifdef WIN32
p_new = ::_expand(p, size);
#else #else
p_new = NULL; typedef LONG mem_stat_t;
#define CRNLIB_MEM_COMPARE_EXCHANGE InterlockedCompareExchange
#endif #endif
if (p_new) static volatile mem_stat_t g_total_blocks;
{ static volatile mem_stat_t g_total_allocated;
CRNLIB_ASSERT( (reinterpret_cast<ptr_bits_t>(p_new) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) == 0 ); static volatile mem_stat_t g_max_allocated;
p_final_block = p_new;
}
else if (movable)
{
p_new = ::realloc(p, size);
if (p_new) static mem_stat_t update_total_allocated(int block_delta, mem_stat_t byte_delta) {
{ mem_stat_t cur_total_blocks;
CRNLIB_ASSERT( (reinterpret_cast<ptr_bits_t>(p_new) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) == 0 ); for (;;) {
p_final_block = p_new; cur_total_blocks = (mem_stat_t)g_total_blocks;
} mem_stat_t new_total_blocks = static_cast<mem_stat_t>(cur_total_blocks + block_delta);
else CRNLIB_ASSERT(new_total_blocks >= 0);
{ if (CRNLIB_MEM_COMPARE_EXCHANGE(&g_total_blocks, new_total_blocks, cur_total_blocks) == cur_total_blocks)
printf("WARNING: ::realloc() of size %u failed!\n", (uint)size); break;
} }
}
if (pActual_size) mem_stat_t cur_total_allocated, new_total_allocated;
*pActual_size = ::_msize(p_final_block); for (;;) {
} cur_total_allocated = g_total_allocated;
new_total_allocated = static_cast<mem_stat_t>(cur_total_allocated + byte_delta);
return p_new; CRNLIB_ASSERT(new_total_allocated >= 0);
} if (CRNLIB_MEM_COMPARE_EXCHANGE(&g_total_allocated, new_total_allocated, cur_total_allocated) == cur_total_allocated)
break;
static size_t crnlib_default_msize(void* p, void* pUser_data) }
{ for (;;) {
pUser_data; mem_stat_t cur_max_allocated = g_max_allocated;
return p ? _msize(p) : 0; mem_stat_t new_max_allocated = CRNLIB_MAX(new_total_allocated, cur_max_allocated);
} if (CRNLIB_MEM_COMPARE_EXCHANGE(&g_max_allocated, new_max_allocated, cur_max_allocated) == cur_max_allocated)
break;
#if 0 }
static __declspec(thread) void *g_pBuf; return new_total_allocated;
static __declspec(thread) size_t g_buf_size; }
static __declspec(thread) size_t g_buf_ofs; #endif // CRNLIB_MEM_STATS
static size_t crnlib_nofree_msize(void* p, void* pUser_data) static void* crnlib_default_realloc(void* p, size_t size, size_t* pActual_size, bool movable, void*) {
{ void* p_new;
pUser_data;
return p ? ((const size_t*)p)[-1] : 0; if (!p) {
} p_new = ::malloc(size);
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(p_new) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) == 0);
static void* crnlib_nofree_realloc(void* p, size_t size, size_t* pActual_size, bool movable, void* pUser_data)
{ if (!p_new) {
pUser_data; printf("WARNING: ::malloc() of size %u failed!\n", (uint)size);
}
void* p_new;
if (pActual_size)
if (!p) *pActual_size = p_new ? ::_msize(p_new) : 0;
{ } else if (!size) {
size = math::align_up_value(size, CRNLIB_MIN_ALLOC_ALIGNMENT); ::free(p);
size_t actual_size = sizeof(size_t)*2 + size; p_new = NULL;
size_t num_remaining = g_buf_size - g_buf_ofs;
if (num_remaining < actual_size) if (pActual_size)
{ *pActual_size = 0;
g_buf_size = CRNLIB_MAX(actual_size, 32*1024*1024); } else {
g_buf_ofs = 0; void* p_final_block = p;
g_pBuf = malloc(g_buf_size); #ifdef WIN32
if (!g_pBuf) p_new = ::_expand(p, size);
return NULL; #else
} p_new = NULL;
#endif
p_new = (uint8*)g_pBuf + g_buf_ofs;
((size_t*)p_new)[1] = size; if (p_new) {
p_new = (size_t*)p_new + 2; CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(p_new) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) == 0);
g_buf_ofs += actual_size; p_final_block = p_new;
} else if (movable) {
if (pActual_size) p_new = ::realloc(p, size);
*pActual_size = size;
if (p_new) {
CRNLIB_ASSERT(crnlib_nofree_msize(p_new, NULL) == size); CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(p_new) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) == 0);
} p_final_block = p_new;
else if (!size) } else {
{ printf("WARNING: ::realloc() of size %u failed!\n", (uint)size);
if (pActual_size) }
*pActual_size = 0; }
p_new = NULL;
} if (pActual_size)
else *pActual_size = ::_msize(p_final_block);
{ }
size_t cur_size = crnlib_nofree_msize(p, NULL);
p_new = p; return p_new;
}
if (!movable)
return NULL; static size_t crnlib_default_msize(void* p, void*) {
return p ? _msize(p) : 0;
if (size > cur_size) }
{
p_new = crnlib_nofree_realloc(NULL, size, NULL, true, NULL); static crn_realloc_func g_pRealloc = crnlib_default_realloc;
if (!p_new) static crn_msize_func g_pMSize = crnlib_default_msize;
return NULL; static void* g_pUser_data;
memcpy(p_new, p, cur_size); void crnlib_mem_error(const char* p_msg) {
crnlib_assert(p_msg, __FILE__, __LINE__);
cur_size = size; }
} void* crnlib_malloc(size_t size) {
return crnlib_malloc(size, NULL);
if (pActual_size) }
*pActual_size = cur_size;
} void* crnlib_malloc(size_t size, size_t* pActual_size) {
size = (size + sizeof(uint32) - 1U) & ~(sizeof(uint32) - 1U);
return p_new; if (!size)
} size = sizeof(uint32);
static crn_realloc_func g_pRealloc = crnlib_nofree_realloc; if (size > CRNLIB_MAX_POSSIBLE_BLOCK_SIZE) {
static crn_msize_func g_pMSize = crnlib_nofree_msize; crnlib_mem_error("crnlib_malloc: size too big");
#else return NULL;
static crn_realloc_func g_pRealloc = crnlib_default_realloc; }
static crn_msize_func g_pMSize = crnlib_default_msize;
#endif size_t actual_size = size;
static void* g_pUser_data; uint8* p_new = static_cast<uint8*>((*g_pRealloc)(NULL, size, &actual_size, true, g_pUser_data));
void crnlib_mem_error(const char* p_msg) if (pActual_size)
{ *pActual_size = actual_size;
crnlib_assert(p_msg, __FILE__, __LINE__);
} if ((!p_new) || (actual_size < size)) {
void* crnlib_malloc(size_t size) crnlib_mem_error("crnlib_malloc: out of memory");
{ return NULL;
return crnlib_malloc(size, NULL); }
}
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(p_new) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) == 0);
void* crnlib_malloc(size_t size, size_t* pActual_size)
{ #if CRNLIB_MEM_STATS
size = (size + sizeof(uint32) - 1U) & ~(sizeof(uint32) - 1U); CRNLIB_ASSERT((*g_pMSize)(p_new, g_pUser_data) == actual_size);
if (!size) update_total_allocated(1, static_cast<mem_stat_t>(actual_size));
size = sizeof(uint32); #endif
if (size > CRNLIB_MAX_POSSIBLE_BLOCK_SIZE) return p_new;
{ }
crnlib_mem_error("crnlib_malloc: size too big");
return NULL; void* crnlib_realloc(void* p, size_t size, size_t* pActual_size, bool movable) {
} if ((ptr_bits_t)p & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) {
crnlib_mem_error("crnlib_realloc: bad ptr");
size_t actual_size = size; return NULL;
uint8* p_new = static_cast<uint8*>((*g_pRealloc)(NULL, size, &actual_size, true, g_pUser_data)); }
if (pActual_size) if (size > CRNLIB_MAX_POSSIBLE_BLOCK_SIZE) {
*pActual_size = actual_size; crnlib_mem_error("crnlib_malloc: size too big");
return NULL;
if ((!p_new) || (actual_size < size)) }
{
crnlib_mem_error("crnlib_malloc: out of memory"); #if CRNLIB_MEM_STATS
return NULL; size_t cur_size = p ? (*g_pMSize)(p, g_pUser_data) : 0;
} CRNLIB_ASSERT(!p || (cur_size >= sizeof(uint32)));
#endif
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(p_new) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) == 0); if ((size) && (size < sizeof(uint32)))
size = sizeof(uint32);
#if CRNLIB_MEM_STATS
CRNLIB_ASSERT((*g_pMSize)(p_new, g_pUser_data) == actual_size); size_t actual_size = size;
update_total_allocated(1, static_cast<mem_stat_t>(actual_size)); void* p_new = (*g_pRealloc)(p, size, &actual_size, movable, g_pUser_data);
#endif
if (pActual_size)
return p_new; *pActual_size = actual_size;
}
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(p_new) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) == 0);
void* crnlib_realloc(void* p, size_t size, size_t* pActual_size, bool movable)
{ #if CRNLIB_MEM_STATS
if ((ptr_bits_t)p & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) CRNLIB_ASSERT(!p_new || ((*g_pMSize)(p_new, g_pUser_data) == actual_size));
{
crnlib_mem_error("crnlib_realloc: bad ptr"); int num_new_blocks = 0;
return NULL; if (p) {
} if (!p_new)
num_new_blocks = -1;
if (size > CRNLIB_MAX_POSSIBLE_BLOCK_SIZE) } else if (p_new) {
{ num_new_blocks = 1;
crnlib_mem_error("crnlib_malloc: size too big"); }
return NULL; update_total_allocated(num_new_blocks, static_cast<mem_stat_t>(actual_size) - static_cast<mem_stat_t>(cur_size));
} #endif
#if CRNLIB_MEM_STATS return p_new;
size_t cur_size = p ? (*g_pMSize)(p, g_pUser_data) : 0; }
CRNLIB_ASSERT(!p || (cur_size >= sizeof(uint32)));
#endif void* crnlib_calloc(size_t count, size_t size, size_t* pActual_size) {
if ((size) && (size < sizeof(uint32))) size_t total = count * size;
size = sizeof(uint32); void* p = crnlib_malloc(total, pActual_size);
if (p)
size_t actual_size = size; memset(p, 0, total);
void* p_new = (*g_pRealloc)(p, size, &actual_size, movable, g_pUser_data); return p;
}
if (pActual_size)
*pActual_size = actual_size; void crnlib_free(void* p) {
if (!p)
CRNLIB_ASSERT((reinterpret_cast<ptr_bits_t>(p_new) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) == 0); return;
#if CRNLIB_MEM_STATS if (reinterpret_cast<ptr_bits_t>(p) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) {
CRNLIB_ASSERT(!p_new || ((*g_pMSize)(p_new, g_pUser_data) == actual_size)); crnlib_mem_error("crnlib_free: bad ptr");
return;
int num_new_blocks = 0; }
if (p)
{ #if CRNLIB_MEM_STATS
if (!p_new) size_t cur_size = (*g_pMSize)(p, g_pUser_data);
num_new_blocks = -1; CRNLIB_ASSERT(cur_size >= sizeof(uint32));
} update_total_allocated(-1, -static_cast<mem_stat_t>(cur_size));
else if (p_new) #endif
{
num_new_blocks = 1; (*g_pRealloc)(p, 0, NULL, true, g_pUser_data);
} }
update_total_allocated(num_new_blocks, static_cast<mem_stat_t>(actual_size) - static_cast<mem_stat_t>(cur_size));
#endif size_t crnlib_msize(void* p) {
if (!p)
return p_new; return 0;
}
if (reinterpret_cast<ptr_bits_t>(p) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) {
void* crnlib_calloc(size_t count, size_t size, size_t* pActual_size) crnlib_mem_error("crnlib_msize: bad ptr");
{ return 0;
size_t total = count * size; }
void *p = crnlib_malloc(total, pActual_size);
if (p) memset(p, 0, total); return (*g_pMSize)(p, g_pUser_data);
return p; }
}
void crnlib_print_mem_stats() {
void crnlib_free(void* p) #if CRNLIB_MEM_STATS
{ if (console::is_initialized()) {
if (!p) console::debug("crnlib_print_mem_stats:");
return; console::debug("Current blocks: %u, allocated: " CRNLIB_INT64_FORMAT_SPECIFIER ", max ever allocated: " CRNLIB_INT64_FORMAT_SPECIFIER, g_total_blocks, (int64)g_total_allocated, (int64)g_max_allocated);
} else {
if (reinterpret_cast<ptr_bits_t>(p) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1)) printf("crnlib_print_mem_stats:\n");
{ printf("Current blocks: %u, allocated: " CRNLIB_INT64_FORMAT_SPECIFIER ", max ever allocated: " CRNLIB_INT64_FORMAT_SPECIFIER "\n", g_total_blocks, (int64)g_total_allocated, (int64)g_max_allocated);
crnlib_mem_error("crnlib_free: bad ptr"); }
return; #endif
} }
#if CRNLIB_MEM_STATS } // namespace crnlib
size_t cur_size = (*g_pMSize)(p, g_pUser_data);
CRNLIB_ASSERT(cur_size >= sizeof(uint32)); void crn_set_memory_callbacks(crn_realloc_func pRealloc, crn_msize_func pMSize, void* pUser_data) {
update_total_allocated(-1, -static_cast<mem_stat_t>(cur_size)); if ((!pRealloc) || (!pMSize)) {
#endif crnlib::g_pRealloc = crnlib::crnlib_default_realloc;
crnlib::g_pMSize = crnlib::crnlib_default_msize;
(*g_pRealloc)(p, 0, NULL, true, g_pUser_data); crnlib::g_pUser_data = NULL;
} } else {
crnlib::g_pRealloc = pRealloc;
size_t crnlib_msize(void* p) crnlib::g_pMSize = pMSize;
{ crnlib::g_pUser_data = pUser_data;
if (!p) }
return 0;
if (reinterpret_cast<ptr_bits_t>(p) & (CRNLIB_MIN_ALLOC_ALIGNMENT - 1))
{
crnlib_mem_error("crnlib_msize: bad ptr");
return 0;
}
return (*g_pMSize)(p, g_pUser_data);
}
void crnlib_print_mem_stats()
{
#if CRNLIB_MEM_STATS
if (console::is_initialized())
{
console::debug("crnlib_print_mem_stats:");
console::debug("Current blocks: %u, allocated: " CRNLIB_INT64_FORMAT_SPECIFIER ", max ever allocated: " CRNLIB_INT64_FORMAT_SPECIFIER, g_total_blocks, (int64)g_total_allocated, (int64)g_max_allocated);
}
else
{
printf("crnlib_print_mem_stats:\n");
printf("Current blocks: %u, allocated: " CRNLIB_INT64_FORMAT_SPECIFIER ", max ever allocated: " CRNLIB_INT64_FORMAT_SPECIFIER "\n", g_total_blocks, (int64)g_total_allocated, (int64)g_max_allocated);
}
#endif
}
} // namespace crnlib
void crn_set_memory_callbacks(crn_realloc_func pRealloc, crn_msize_func pMSize, void* pUser_data)
{
if ((!pRealloc) || (!pMSize))
{
crnlib::g_pRealloc = crnlib::crnlib_default_realloc;
crnlib::g_pMSize = crnlib::crnlib_default_msize;
crnlib::g_pUser_data = NULL;
}
else
{
crnlib::g_pRealloc = pRealloc;
crnlib::g_pMSize = pMSize;
crnlib::g_pUser_data = pUser_data;
}
} }
crnlib/crn_mem.h
+151 -179
View File
@@ -6,204 +6,176 @@
#define CRNLIB_MIN_ALLOC_ALIGNMENT sizeof(size_t) * 2 #define CRNLIB_MIN_ALLOC_ALIGNMENT sizeof(size_t) * 2
#endif #endif
namespace crnlib namespace crnlib {
{
#if CRNLIB_64BIT_POINTERS #if CRNLIB_64BIT_POINTERS
const uint64 CRNLIB_MAX_POSSIBLE_BLOCK_SIZE = 0x400000000ULL; const uint64 CRNLIB_MAX_POSSIBLE_BLOCK_SIZE = 0x400000000ULL;
#else #else
const uint32 CRNLIB_MAX_POSSIBLE_BLOCK_SIZE = 0x7FFF0000U; const uint32 CRNLIB_MAX_POSSIBLE_BLOCK_SIZE = 0x7FFF0000U;
#endif #endif
void* crnlib_malloc(size_t size); void* crnlib_malloc(size_t size);
void* crnlib_malloc(size_t size, size_t* pActual_size); void* crnlib_malloc(size_t size, size_t* pActual_size);
void* crnlib_realloc(void* p, size_t size, size_t* pActual_size = NULL, bool movable = true); void* crnlib_realloc(void* p, size_t size, size_t* pActual_size = NULL, bool movable = true);
void* crnlib_calloc(size_t count, size_t size, size_t* pActual_size = NULL); void* crnlib_calloc(size_t count, size_t size, size_t* pActual_size = NULL);
void crnlib_free(void* p); void crnlib_free(void* p);
size_t crnlib_msize(void* p); size_t crnlib_msize(void* p);
void crnlib_print_mem_stats(); void crnlib_print_mem_stats();
void crnlib_mem_error(const char* p_msg); void crnlib_mem_error(const char* p_msg);
// omfg - there must be a better way
template<typename T>
inline T* crnlib_new()
{
T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
if (CRNLIB_IS_SCALAR_TYPE(T))
return p;
return helpers::construct(p);
}
template<typename T, typename A> // omfg - there must be a better way
inline T* crnlib_new(const A& init0)
{
T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
return new (static_cast<void*>(p)) T(init0);
}
template<typename T, typename A> template <typename T>
inline T* crnlib_new(A& init0) inline T* crnlib_new() {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); if (CRNLIB_IS_SCALAR_TYPE(T))
return new (static_cast<void*>(p)) T(init0); return p;
} return helpers::construct(p);
}
template<typename T, typename A, typename B> template <typename T, typename A>
inline T* crnlib_new(const A& init0, const B& init1) inline T* crnlib_new(const A& init0) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); return new (static_cast<void*>(p)) T(init0);
return new (static_cast<void*>(p)) T(init0, init1); }
}
template<typename T, typename A, typename B, typename C> template <typename T, typename A>
inline T* crnlib_new(const A& init0, const B& init1, const C& init2) inline T* crnlib_new(A& init0) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); return new (static_cast<void*>(p)) T(init0);
return new (static_cast<void*>(p)) T(init0, init1, init2); }
}
template<typename T, typename A, typename B, typename C, typename D> template <typename T, typename A, typename B>
inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3) inline T* crnlib_new(const A& init0, const B& init1) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); return new (static_cast<void*>(p)) T(init0, init1);
return new (static_cast<void*>(p)) T(init0, init1, init2, init3); }
}
template<typename T, typename A, typename B, typename C, typename D, typename E> template <typename T, typename A, typename B, typename C>
inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4) inline T* crnlib_new(const A& init0, const B& init1, const C& init2) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); return new (static_cast<void*>(p)) T(init0, init1, init2);
return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4); }
}
template<typename T, typename A, typename B, typename C, typename D, typename E, typename F> template <typename T, typename A, typename B, typename C, typename D>
inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5) inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); return new (static_cast<void*>(p)) T(init0, init1, init2, init3);
return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5); }
}
template<typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G> template <typename T, typename A, typename B, typename C, typename D, typename E>
inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6) inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4);
return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6); }
}
template<typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G, typename H> template <typename T, typename A, typename B, typename C, typename D, typename E, typename F>
inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6, const H& init7) inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5);
return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6, init7); }
}
template<typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G, typename H, typename I> template <typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G>
inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6, const H& init7, const I& init8) inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6);
return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6, init7, init8); }
}
template<typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G, typename H, typename I, typename J> template <typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G, typename H>
inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6, const H& init7, const I& init8, const J& init9) inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6, const H& init7) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6, init7);
return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6, init7, init8, init9); }
}
template<typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G, typename H, typename I, typename J, typename K> template <typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G, typename H, typename I>
inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6, const H& init7, const I& init8, const J& init9, const K& init10) inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6, const H& init7, const I& init8) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6, init7, init8);
return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6, init7, init8, init9, init10); }
}
template<typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G, typename H, typename I, typename J, typename K, typename L> template <typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G, typename H, typename I, typename J>
inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6, const H& init7, const I& init8, const J& init9, const K& init10, const L& init11) inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6, const H& init7, const I& init8, const J& init9) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
T* p = static_cast<T*>(crnlib_malloc(sizeof(T))); return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6, init7, init8, init9);
return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6, init7, init8, init9, init10, init11); }
}
template<typename T> template <typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G, typename H, typename I, typename J, typename K>
inline T* crnlib_new_array(uint32 num) inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6, const H& init7, const I& init8, const J& init9, const K& init10) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
if (!num) num = 1; return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6, init7, init8, init9, init10);
}
uint64 total = CRNLIB_MIN_ALLOC_ALIGNMENT + sizeof(T) * num; template <typename T, typename A, typename B, typename C, typename D, typename E, typename F, typename G, typename H, typename I, typename J, typename K, typename L>
if (total > CRNLIB_MAX_POSSIBLE_BLOCK_SIZE) inline T* crnlib_new(const A& init0, const B& init1, const C& init2, const D& init3, const E& init4, const F& init5, const G& init6, const H& init7, const I& init8, const J& init9, const K& init10, const L& init11) {
{ T* p = static_cast<T*>(crnlib_malloc(sizeof(T)));
crnlib_mem_error("crnlib_new_array: Array too large!"); return new (static_cast<void*>(p)) T(init0, init1, init2, init3, init4, init5, init6, init7, init8, init9, init10, init11);
return NULL; }
template <typename T>
inline T* crnlib_new_array(uint32 num) {
if (!num)
num = 1;
uint64 total = CRNLIB_MIN_ALLOC_ALIGNMENT + sizeof(T) * num;
if (total > CRNLIB_MAX_POSSIBLE_BLOCK_SIZE) {
crnlib_mem_error("crnlib_new_array: Array too large!");
return NULL;
}
uint8* q = static_cast<uint8*>(crnlib_malloc(static_cast<size_t>(total)));
T* p = reinterpret_cast<T*>(q + CRNLIB_MIN_ALLOC_ALIGNMENT);
reinterpret_cast<uint32*>(p)[-1] = num;
reinterpret_cast<uint32*>(p)[-2] = ~num;
if (!CRNLIB_IS_SCALAR_TYPE(T)) {
helpers::construct_array(p, num);
}
return p;
}
template <typename T>
inline void crnlib_delete(T* p) {
if (p) {
if (!CRNLIB_IS_SCALAR_TYPE(T)) {
helpers::destruct(p);
}
crnlib_free(p);
}
}
template <typename T>
inline void crnlib_delete_array(T* p) {
if (p) {
const uint32 num = reinterpret_cast<uint32*>(p)[-1];
const uint32 num_check = reinterpret_cast<uint32*>(p)[-2];
CRNLIB_ASSERT(num && (num == ~num_check));
if (num == ~num_check) {
if (!CRNLIB_IS_SCALAR_TYPE(T)) {
helpers::destruct_array(p, num);
} }
uint8* q = static_cast<uint8*>(crnlib_malloc(static_cast<size_t>(total)));
T* p = reinterpret_cast<T*>(q + CRNLIB_MIN_ALLOC_ALIGNMENT); crnlib_free(reinterpret_cast<uint8*>(p) - CRNLIB_MIN_ALLOC_ALIGNMENT);
}
}
}
reinterpret_cast<uint32*>(p)[-1] = num; } // namespace crnlib
reinterpret_cast<uint32*>(p)[-2] = ~num; #define CRNLIB_DEFINE_NEW_DELETE \
void* operator new(size_t size) { \
if (!CRNLIB_IS_SCALAR_TYPE(T)) void* p = crnlib::crnlib_malloc(size); \
{ if (!p) \
helpers::construct_array(p, num); crnlib_fail("new: Out of memory!", __FILE__, __LINE__); \
} return p; \
return p; } \
} void* operator new[](size_t size) { \
void* p = crnlib::crnlib_malloc(size); \
template<typename T> if (!p) \
inline void crnlib_delete(T* p) crnlib_fail("new[]: Out of memory!", __FILE__, __LINE__); \
{ return p; \
if (p) } \
{ void operator delete(void* p_block) { \
if (!CRNLIB_IS_SCALAR_TYPE(T)) crnlib::crnlib_free(p_block); \
{ } \
helpers::destruct(p); void operator delete[](void* p_block) { \
} crnlib::crnlib_free(p_block); \
crnlib_free(p); }
}
}
template<typename T>
inline void crnlib_delete_array(T* p)
{
if (p)
{
const uint32 num = reinterpret_cast<uint32*>(p)[-1];
const uint32 num_check = reinterpret_cast<uint32*>(p)[-2];
CRNLIB_ASSERT(num && (num == ~num_check));
if (num == ~num_check)
{
if (!CRNLIB_IS_SCALAR_TYPE(T))
{
helpers::destruct_array(p, num);
}
crnlib_free(reinterpret_cast<uint8*>(p) - CRNLIB_MIN_ALLOC_ALIGNMENT);
}
}
}
} // namespace crnlib
#define CRNLIB_DEFINE_NEW_DELETE \
void* operator new (size_t size) \
{ \
void* p = crnlib::crnlib_malloc(size); \
if (!p) \
crnlib_fail("new: Out of memory!", __FILE__, __LINE__); \
return p; \
} \
void* operator new[] (size_t size) \
{ \
void* p = crnlib::crnlib_malloc(size); \
if (!p) \
crnlib_fail("new[]: Out of memory!", __FILE__, __LINE__); \
return p; \
} \
void operator delete (void* p_block) \
{ \
crnlib::crnlib_free(p_block); \
} \
void operator delete[] (void* p_block) \
{ \
crnlib::crnlib_free(p_block); \
}
crnlib/crn_miniz.cpp
+1876 -1502
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_miniz.h
+262 -221
View File
@@ -174,7 +174,7 @@
#define MINIZ_X86_OR_X64_CPU 1 #define MINIZ_X86_OR_X64_CPU 1
#endif #endif
#if (__BYTE_ORDER__==__ORDER_LITTLE_ENDIAN__) || MINIZ_X86_OR_X64_CPU #if (__BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__) || MINIZ_X86_OR_X64_CPU
// Set MINIZ_LITTLE_ENDIAN to 1 if the processor is little endian. // Set MINIZ_LITTLE_ENDIAN to 1 if the processor is little endian.
#define MINIZ_LITTLE_ENDIAN 1 #define MINIZ_LITTLE_ENDIAN 1
#endif #endif
@@ -199,18 +199,22 @@ extern "C" {
typedef unsigned long mz_ulong; typedef unsigned long mz_ulong;
// mz_free() internally uses the MZ_FREE() macro (which by default calls free() unless you've modified the MZ_MALLOC macro) to release a block allocated from the heap. // mz_free() internally uses the MZ_FREE() macro (which by default calls free() unless you've modified the MZ_MALLOC macro) to release a block allocated from the heap.
void mz_free(void *p); void mz_free(void* p);
#define MZ_ADLER32_INIT (1) #define MZ_ADLER32_INIT (1)
// mz_adler32() returns the initial adler-32 value to use when called with ptr==NULL. // mz_adler32() returns the initial adler-32 value to use when called with ptr==NULL.
mz_ulong mz_adler32(mz_ulong adler, const unsigned char *ptr, size_t buf_len); mz_ulong mz_adler32(mz_ulong adler, const unsigned char* ptr, size_t buf_len);
#define MZ_CRC32_INIT (0) #define MZ_CRC32_INIT (0)
// mz_crc32() returns the initial CRC-32 value to use when called with ptr==NULL. // mz_crc32() returns the initial CRC-32 value to use when called with ptr==NULL.
mz_ulong mz_crc32(mz_ulong crc, const unsigned char *ptr, size_t buf_len); mz_ulong mz_crc32(mz_ulong crc, const unsigned char* ptr, size_t buf_len);
// Compression strategies. // Compression strategies.
enum { MZ_DEFAULT_STRATEGY = 0, MZ_FILTERED = 1, MZ_HUFFMAN_ONLY = 2, MZ_RLE = 3, MZ_FIXED = 4 }; enum { MZ_DEFAULT_STRATEGY = 0,
MZ_FILTERED = 1,
MZ_HUFFMAN_ONLY = 2,
MZ_RLE = 3,
MZ_FIXED = 4 };
// Method // Method
#define MZ_DEFLATED 8 #define MZ_DEFLATED 8
@@ -219,25 +223,44 @@ enum { MZ_DEFAULT_STRATEGY = 0, MZ_FILTERED = 1, MZ_HUFFMAN_ONLY = 2, MZ_RLE = 3
// Heap allocation callbacks. // Heap allocation callbacks.
// Note that mz_alloc_func parameter types purpsosely differ from zlib's: items/size is size_t, not unsigned long. // Note that mz_alloc_func parameter types purpsosely differ from zlib's: items/size is size_t, not unsigned long.
typedef void *(*mz_alloc_func)(void *opaque, size_t items, size_t size); typedef void* (*mz_alloc_func)(void* opaque, size_t items, size_t size);
typedef void (*mz_free_func)(void *opaque, void *address); typedef void (*mz_free_func)(void* opaque, void* address);
typedef void *(*mz_realloc_func)(void *opaque, void *address, size_t items, size_t size); typedef void* (*mz_realloc_func)(void* opaque, void* address, size_t items, size_t size);
#define MZ_VERSION "9.1.14" #define MZ_VERSION "9.1.14"
#define MZ_VERNUM 0x91E0 #define MZ_VERNUM 0x91E0
#define MZ_VER_MAJOR 9 #define MZ_VER_MAJOR 9
#define MZ_VER_MINOR 1 #define MZ_VER_MINOR 1
#define MZ_VER_REVISION 14 #define MZ_VER_REVISION 14
#define MZ_VER_SUBREVISION 0 #define MZ_VER_SUBREVISION 0
// Flush values. For typical usage you only need MZ_NO_FLUSH and MZ_FINISH. The other values are for advanced use (refer to the zlib docs). // Flush values. For typical usage you only need MZ_NO_FLUSH and MZ_FINISH. The other values are for advanced use (refer to the zlib docs).
enum { MZ_NO_FLUSH = 0, MZ_PARTIAL_FLUSH = 1, MZ_SYNC_FLUSH = 2, MZ_FULL_FLUSH = 3, MZ_FINISH = 4, MZ_BLOCK = 5 }; enum { MZ_NO_FLUSH = 0,
MZ_PARTIAL_FLUSH = 1,
MZ_SYNC_FLUSH = 2,
MZ_FULL_FLUSH = 3,
MZ_FINISH = 4,
MZ_BLOCK = 5 };
// Return status codes. MZ_PARAM_ERROR is non-standard. // Return status codes. MZ_PARAM_ERROR is non-standard.
enum { MZ_OK = 0, MZ_STREAM_END = 1, MZ_NEED_DICT = 2, MZ_ERRNO = -1, MZ_STREAM_ERROR = -2, MZ_DATA_ERROR = -3, MZ_MEM_ERROR = -4, MZ_BUF_ERROR = -5, MZ_VERSION_ERROR = -6, MZ_PARAM_ERROR = -10000 }; enum { MZ_OK = 0,
MZ_STREAM_END = 1,
MZ_NEED_DICT = 2,
MZ_ERRNO = -1,
MZ_STREAM_ERROR = -2,
MZ_DATA_ERROR = -3,
MZ_MEM_ERROR = -4,
MZ_BUF_ERROR = -5,
MZ_VERSION_ERROR = -6,
MZ_PARAM_ERROR = -10000 };
// Compression levels: 0-9 are the standard zlib-style levels, 10 is best possible compression (not zlib compatible, and may be very slow), MZ_DEFAULT_COMPRESSION=MZ_DEFAULT_LEVEL. // Compression levels: 0-9 are the standard zlib-style levels, 10 is best possible compression (not zlib compatible, and may be very slow), MZ_DEFAULT_COMPRESSION=MZ_DEFAULT_LEVEL.
enum { MZ_NO_COMPRESSION = 0, MZ_BEST_SPEED = 1, MZ_BEST_COMPRESSION = 9, MZ_UBER_COMPRESSION = 10, MZ_DEFAULT_LEVEL = 6, MZ_DEFAULT_COMPRESSION = -1 }; enum { MZ_NO_COMPRESSION = 0,
MZ_BEST_SPEED = 1,
MZ_BEST_COMPRESSION = 9,
MZ_UBER_COMPRESSION = 10,
MZ_DEFAULT_LEVEL = 6,
MZ_DEFAULT_COMPRESSION = -1 };
// Window bits // Window bits
#define MZ_DEFAULT_WINDOW_BITS 15 #define MZ_DEFAULT_WINDOW_BITS 15
@@ -245,32 +268,31 @@ enum { MZ_NO_COMPRESSION = 0, MZ_BEST_SPEED = 1, MZ_BEST_COMPRESSION = 9, MZ_UBE
struct mz_internal_state; struct mz_internal_state;
// Compression/decompression stream struct. // Compression/decompression stream struct.
typedef struct mz_stream_s typedef struct mz_stream_s {
{ const unsigned char* next_in; // pointer to next byte to read
const unsigned char *next_in; // pointer to next byte to read unsigned int avail_in; // number of bytes available at next_in
unsigned int avail_in; // number of bytes available at next_in mz_ulong total_in; // total number of bytes consumed so far
mz_ulong total_in; // total number of bytes consumed so far
unsigned char *next_out; // pointer to next byte to write unsigned char* next_out; // pointer to next byte to write
unsigned int avail_out; // number of bytes that can be written to next_out unsigned int avail_out; // number of bytes that can be written to next_out
mz_ulong total_out; // total number of bytes produced so far mz_ulong total_out; // total number of bytes produced so far
char *msg; // error msg (unused) char* msg; // error msg (unused)
struct mz_internal_state *state; // internal state, allocated by zalloc/zfree struct mz_internal_state* state; // internal state, allocated by zalloc/zfree
mz_alloc_func zalloc; // optional heap allocation function (defaults to malloc) mz_alloc_func zalloc; // optional heap allocation function (defaults to malloc)
mz_free_func zfree; // optional heap free function (defaults to free) mz_free_func zfree; // optional heap free function (defaults to free)
void *opaque; // heap alloc function user pointer void* opaque; // heap alloc function user pointer
int data_type; // data_type (unused) int data_type; // data_type (unused)
mz_ulong adler; // adler32 of the source or uncompressed data mz_ulong adler; // adler32 of the source or uncompressed data
mz_ulong reserved; // not used mz_ulong reserved; // not used
} mz_stream; } mz_stream;
typedef mz_stream *mz_streamp; typedef mz_stream* mz_streamp;
// Returns the version string of miniz.c. // Returns the version string of miniz.c.
const char *mz_version(void); const char* mz_version(void);
// mz_deflateInit() initializes a compressor with default options: // mz_deflateInit() initializes a compressor with default options:
// Parameters: // Parameters:
@@ -318,8 +340,8 @@ mz_ulong mz_deflateBound(mz_streamp pStream, mz_ulong source_len);
// Single-call compression functions mz_compress() and mz_compress2(): // Single-call compression functions mz_compress() and mz_compress2():
// Returns MZ_OK on success, or one of the error codes from mz_deflate() on failure. // Returns MZ_OK on success, or one of the error codes from mz_deflate() on failure.
int mz_compress(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len); int mz_compress(unsigned char* pDest, mz_ulong* pDest_len, const unsigned char* pSource, mz_ulong source_len);
int mz_compress2(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len, int level); int mz_compress2(unsigned char* pDest, mz_ulong* pDest_len, const unsigned char* pSource, mz_ulong source_len, int level);
// mz_compressBound() returns a (very) conservative upper bound on the amount of data that could be generated by calling mz_compress(). // mz_compressBound() returns a (very) conservative upper bound on the amount of data that could be generated by calling mz_compress().
mz_ulong mz_compressBound(mz_ulong source_len); mz_ulong mz_compressBound(mz_ulong source_len);
@@ -352,87 +374,87 @@ int mz_inflateEnd(mz_streamp pStream);
// Single-call decompression. // Single-call decompression.
// Returns MZ_OK on success, or one of the error codes from mz_inflate() on failure. // Returns MZ_OK on success, or one of the error codes from mz_inflate() on failure.
int mz_uncompress(unsigned char *pDest, mz_ulong *pDest_len, const unsigned char *pSource, mz_ulong source_len); int mz_uncompress(unsigned char* pDest, mz_ulong* pDest_len, const unsigned char* pSource, mz_ulong source_len);
// Returns a string description of the specified error code, or NULL if the error code is invalid. // Returns a string description of the specified error code, or NULL if the error code is invalid.
const char *mz_error(int err); const char* mz_error(int err);
// Redefine zlib-compatible names to miniz equivalents, so miniz.c can be used as a drop-in replacement for the subset of zlib that miniz.c supports. // Redefine zlib-compatible names to miniz equivalents, so miniz.c can be used as a drop-in replacement for the subset of zlib that miniz.c supports.
// Define MINIZ_NO_ZLIB_COMPATIBLE_NAMES to disable zlib-compatibility if you use zlib in the same project. // Define MINIZ_NO_ZLIB_COMPATIBLE_NAMES to disable zlib-compatibility if you use zlib in the same project.
#ifndef MINIZ_NO_ZLIB_COMPATIBLE_NAMES #ifndef MINIZ_NO_ZLIB_COMPATIBLE_NAMES
typedef unsigned char Byte; typedef unsigned char Byte;
typedef unsigned int uInt; typedef unsigned int uInt;
typedef mz_ulong uLong; typedef mz_ulong uLong;
typedef Byte Bytef; typedef Byte Bytef;
typedef uInt uIntf; typedef uInt uIntf;
typedef char charf; typedef char charf;
typedef int intf; typedef int intf;
typedef void *voidpf; typedef void* voidpf;
typedef uLong uLongf; typedef uLong uLongf;
typedef void *voidp; typedef void* voidp;
typedef void *const voidpc; typedef void* const voidpc;
#define Z_NULL 0 #define Z_NULL 0
#define Z_NO_FLUSH MZ_NO_FLUSH #define Z_NO_FLUSH MZ_NO_FLUSH
#define Z_PARTIAL_FLUSH MZ_PARTIAL_FLUSH #define Z_PARTIAL_FLUSH MZ_PARTIAL_FLUSH
#define Z_SYNC_FLUSH MZ_SYNC_FLUSH #define Z_SYNC_FLUSH MZ_SYNC_FLUSH
#define Z_FULL_FLUSH MZ_FULL_FLUSH #define Z_FULL_FLUSH MZ_FULL_FLUSH
#define Z_FINISH MZ_FINISH #define Z_FINISH MZ_FINISH
#define Z_BLOCK MZ_BLOCK #define Z_BLOCK MZ_BLOCK
#define Z_OK MZ_OK #define Z_OK MZ_OK
#define Z_STREAM_END MZ_STREAM_END #define Z_STREAM_END MZ_STREAM_END
#define Z_NEED_DICT MZ_NEED_DICT #define Z_NEED_DICT MZ_NEED_DICT
#define Z_ERRNO MZ_ERRNO #define Z_ERRNO MZ_ERRNO
#define Z_STREAM_ERROR MZ_STREAM_ERROR #define Z_STREAM_ERROR MZ_STREAM_ERROR
#define Z_DATA_ERROR MZ_DATA_ERROR #define Z_DATA_ERROR MZ_DATA_ERROR
#define Z_MEM_ERROR MZ_MEM_ERROR #define Z_MEM_ERROR MZ_MEM_ERROR
#define Z_BUF_ERROR MZ_BUF_ERROR #define Z_BUF_ERROR MZ_BUF_ERROR
#define Z_VERSION_ERROR MZ_VERSION_ERROR #define Z_VERSION_ERROR MZ_VERSION_ERROR
#define Z_PARAM_ERROR MZ_PARAM_ERROR #define Z_PARAM_ERROR MZ_PARAM_ERROR
#define Z_NO_COMPRESSION MZ_NO_COMPRESSION #define Z_NO_COMPRESSION MZ_NO_COMPRESSION
#define Z_BEST_SPEED MZ_BEST_SPEED #define Z_BEST_SPEED MZ_BEST_SPEED
#define Z_BEST_COMPRESSION MZ_BEST_COMPRESSION #define Z_BEST_COMPRESSION MZ_BEST_COMPRESSION
#define Z_DEFAULT_COMPRESSION MZ_DEFAULT_COMPRESSION #define Z_DEFAULT_COMPRESSION MZ_DEFAULT_COMPRESSION
#define Z_DEFAULT_STRATEGY MZ_DEFAULT_STRATEGY #define Z_DEFAULT_STRATEGY MZ_DEFAULT_STRATEGY
#define Z_FILTERED MZ_FILTERED #define Z_FILTERED MZ_FILTERED
#define Z_HUFFMAN_ONLY MZ_HUFFMAN_ONLY #define Z_HUFFMAN_ONLY MZ_HUFFMAN_ONLY
#define Z_RLE MZ_RLE #define Z_RLE MZ_RLE
#define Z_FIXED MZ_FIXED #define Z_FIXED MZ_FIXED
#define Z_DEFLATED MZ_DEFLATED #define Z_DEFLATED MZ_DEFLATED
#define Z_DEFAULT_WINDOW_BITS MZ_DEFAULT_WINDOW_BITS #define Z_DEFAULT_WINDOW_BITS MZ_DEFAULT_WINDOW_BITS
#define alloc_func mz_alloc_func #define alloc_func mz_alloc_func
#define free_func mz_free_func #define free_func mz_free_func
#define internal_state mz_internal_state #define internal_state mz_internal_state
#define z_stream mz_stream #define z_stream mz_stream
#define deflateInit mz_deflateInit #define deflateInit mz_deflateInit
#define deflateInit2 mz_deflateInit2 #define deflateInit2 mz_deflateInit2
#define deflateReset mz_deflateReset #define deflateReset mz_deflateReset
#define deflate mz_deflate #define deflate mz_deflate
#define deflateEnd mz_deflateEnd #define deflateEnd mz_deflateEnd
#define deflateBound mz_deflateBound #define deflateBound mz_deflateBound
#define compress mz_compress #define compress mz_compress
#define compress2 mz_compress2 #define compress2 mz_compress2
#define compressBound mz_compressBound #define compressBound mz_compressBound
#define inflateInit mz_inflateInit #define inflateInit mz_inflateInit
#define inflateInit2 mz_inflateInit2 #define inflateInit2 mz_inflateInit2
#define inflate mz_inflate #define inflate mz_inflate
#define inflateEnd mz_inflateEnd #define inflateEnd mz_inflateEnd
#define uncompress mz_uncompress #define uncompress mz_uncompress
#define crc32 mz_crc32 #define crc32 mz_crc32
#define adler32 mz_adler32 #define adler32 mz_adler32
#define MAX_WBITS 15 #define MAX_WBITS 15
#define MAX_MEM_LEVEL 9 #define MAX_MEM_LEVEL 9
#define zError mz_error #define zError mz_error
#define ZLIB_VERSION MZ_VERSION #define ZLIB_VERSION MZ_VERSION
#define ZLIB_VERNUM MZ_VERNUM #define ZLIB_VERNUM MZ_VERNUM
#define ZLIB_VER_MAJOR MZ_VER_MAJOR #define ZLIB_VER_MAJOR MZ_VER_MAJOR
#define ZLIB_VER_MINOR MZ_VER_MINOR #define ZLIB_VER_MINOR MZ_VER_MINOR
#define ZLIB_VER_REVISION MZ_VER_REVISION #define ZLIB_VER_REVISION MZ_VER_REVISION
#define ZLIB_VER_SUBREVISION MZ_VER_SUBREVISION #define ZLIB_VER_SUBREVISION MZ_VER_SUBREVISION
#define zlibVersion mz_version #define zlibVersion mz_version
#define zlib_version mz_version() #define zlib_version mz_version()
#endif // #ifndef MINIZ_NO_ZLIB_COMPATIBLE_NAMES #endif // #ifndef MINIZ_NO_ZLIB_COMPATIBLE_NAMES
#endif // MINIZ_NO_ZLIB_APIS #endif // MINIZ_NO_ZLIB_APIS
// ------------------- Types and macros // ------------------- Types and macros
@@ -450,18 +472,17 @@ typedef int mz_bool;
// Works around MSVC's spammy "warning C4127: conditional expression is constant" message. // Works around MSVC's spammy "warning C4127: conditional expression is constant" message.
#ifdef _MSC_VER #ifdef _MSC_VER
#define MZ_MACRO_END while (0, 0) #define MZ_MACRO_END while (0, 0)
#else #else
#define MZ_MACRO_END while (0) #define MZ_MACRO_END while (0)
#endif #endif
// ------------------- ZIP archive reading/writing // ------------------- ZIP archive reading/writing
#ifndef MINIZ_NO_ARCHIVE_APIS #ifndef MINIZ_NO_ARCHIVE_APIS
enum enum {
{ MZ_ZIP_MAX_IO_BUF_SIZE = 64 * 1024,
MZ_ZIP_MAX_IO_BUF_SIZE = 64*1024,
MZ_ZIP_MAX_ARCHIVE_FILENAME_SIZE = 260, MZ_ZIP_MAX_ARCHIVE_FILENAME_SIZE = 260,
MZ_ZIP_MAX_ARCHIVE_FILE_COMMENT_SIZE = 256 MZ_ZIP_MAX_ARCHIVE_FILE_COMMENT_SIZE = 256
}; };
@@ -488,14 +509,13 @@ typedef struct
char m_comment[MZ_ZIP_MAX_ARCHIVE_FILE_COMMENT_SIZE]; char m_comment[MZ_ZIP_MAX_ARCHIVE_FILE_COMMENT_SIZE];
} mz_zip_archive_file_stat; } mz_zip_archive_file_stat;
typedef size_t (*mz_file_read_func)(void *pOpaque, mz_uint64 file_ofs, void *pBuf, size_t n); typedef size_t (*mz_file_read_func)(void* pOpaque, mz_uint64 file_ofs, void* pBuf, size_t n);
typedef size_t (*mz_file_write_func)(void *pOpaque, mz_uint64 file_ofs, const void *pBuf, size_t n); typedef size_t (*mz_file_write_func)(void* pOpaque, mz_uint64 file_ofs, const void* pBuf, size_t n);
struct mz_zip_internal_state_tag; struct mz_zip_internal_state_tag;
typedef struct mz_zip_internal_state_tag mz_zip_internal_state; typedef struct mz_zip_internal_state_tag mz_zip_internal_state;
typedef enum typedef enum {
{
MZ_ZIP_MODE_INVALID = 0, MZ_ZIP_MODE_INVALID = 0,
MZ_ZIP_MODE_READING = 1, MZ_ZIP_MODE_READING = 1,
MZ_ZIP_MODE_WRITING = 2, MZ_ZIP_MODE_WRITING = 2,
@@ -514,21 +534,20 @@ typedef struct
mz_alloc_func m_pAlloc; mz_alloc_func m_pAlloc;
mz_free_func m_pFree; mz_free_func m_pFree;
mz_realloc_func m_pRealloc; mz_realloc_func m_pRealloc;
void *m_pAlloc_opaque; void* m_pAlloc_opaque;
mz_file_read_func m_pRead; mz_file_read_func m_pRead;
mz_file_write_func m_pWrite; mz_file_write_func m_pWrite;
void *m_pIO_opaque; void* m_pIO_opaque;
mz_zip_internal_state *m_pState; mz_zip_internal_state* m_pState;
} mz_zip_archive; } mz_zip_archive;
typedef enum typedef enum {
{ MZ_ZIP_FLAG_CASE_SENSITIVE = 0x0100,
MZ_ZIP_FLAG_CASE_SENSITIVE = 0x0100, MZ_ZIP_FLAG_IGNORE_PATH = 0x0200,
MZ_ZIP_FLAG_IGNORE_PATH = 0x0200, MZ_ZIP_FLAG_COMPRESSED_DATA = 0x0400,
MZ_ZIP_FLAG_COMPRESSED_DATA = 0x0400,
MZ_ZIP_FLAG_DO_NOT_SORT_CENTRAL_DIRECTORY = 0x0800 MZ_ZIP_FLAG_DO_NOT_SORT_CENTRAL_DIRECTORY = 0x0800
} mz_zip_flags; } mz_zip_flags;
@@ -536,68 +555,68 @@ typedef enum
// Inits a ZIP archive reader. // Inits a ZIP archive reader.
// These functions read and validate the archive's central directory. // These functions read and validate the archive's central directory.
mz_bool mz_zip_reader_init(mz_zip_archive *pZip, mz_uint64 size, mz_uint32 flags); mz_bool mz_zip_reader_init(mz_zip_archive* pZip, mz_uint64 size, mz_uint32 flags);
mz_bool mz_zip_reader_init_mem(mz_zip_archive *pZip, const void *pMem, size_t size, mz_uint32 flags); mz_bool mz_zip_reader_init_mem(mz_zip_archive* pZip, const void* pMem, size_t size, mz_uint32 flags);
#ifndef MINIZ_NO_STDIO #ifndef MINIZ_NO_STDIO
mz_bool mz_zip_reader_init_file(mz_zip_archive *pZip, const char *pFilename, mz_uint32 flags); mz_bool mz_zip_reader_init_file(mz_zip_archive* pZip, const char* pFilename, mz_uint32 flags);
#endif #endif
// Returns the total number of files in the archive. // Returns the total number of files in the archive.
mz_uint mz_zip_reader_get_num_files(mz_zip_archive *pZip); mz_uint mz_zip_reader_get_num_files(mz_zip_archive* pZip);
// Returns detailed information about an archive file entry. // Returns detailed information about an archive file entry.
mz_bool mz_zip_reader_file_stat(mz_zip_archive *pZip, mz_uint file_index, mz_zip_archive_file_stat *pStat); mz_bool mz_zip_reader_file_stat(mz_zip_archive* pZip, mz_uint file_index, mz_zip_archive_file_stat* pStat);
// Determines if an archive file entry is a directory entry. // Determines if an archive file entry is a directory entry.
mz_bool mz_zip_reader_is_file_a_directory(mz_zip_archive *pZip, mz_uint file_index); mz_bool mz_zip_reader_is_file_a_directory(mz_zip_archive* pZip, mz_uint file_index);
mz_bool mz_zip_reader_is_file_encrypted(mz_zip_archive *pZip, mz_uint file_index); mz_bool mz_zip_reader_is_file_encrypted(mz_zip_archive* pZip, mz_uint file_index);
// Retrieves the filename of an archive file entry. // Retrieves the filename of an archive file entry.
// Returns the number of bytes written to pFilename, or if filename_buf_size is 0 this function returns the number of bytes needed to fully store the filename. // Returns the number of bytes written to pFilename, or if filename_buf_size is 0 this function returns the number of bytes needed to fully store the filename.
mz_uint mz_zip_reader_get_filename(mz_zip_archive *pZip, mz_uint file_index, char *pFilename, mz_uint filename_buf_size); mz_uint mz_zip_reader_get_filename(mz_zip_archive* pZip, mz_uint file_index, char* pFilename, mz_uint filename_buf_size);
// Attempts to locates a file in the archive's central directory. // Attempts to locates a file in the archive's central directory.
// Valid flags: MZ_ZIP_FLAG_CASE_SENSITIVE, MZ_ZIP_FLAG_IGNORE_PATH // Valid flags: MZ_ZIP_FLAG_CASE_SENSITIVE, MZ_ZIP_FLAG_IGNORE_PATH
// Returns -1 if the file cannot be found. // Returns -1 if the file cannot be found.
int mz_zip_reader_locate_file(mz_zip_archive *pZip, const char *pName, const char *pComment, mz_uint flags); int mz_zip_reader_locate_file(mz_zip_archive* pZip, const char* pName, const char* pComment, mz_uint flags);
// Extracts a archive file to a memory buffer using no memory allocation. // Extracts a archive file to a memory buffer using no memory allocation.
mz_bool mz_zip_reader_extract_to_mem_no_alloc(mz_zip_archive *pZip, mz_uint file_index, void *pBuf, size_t buf_size, mz_uint flags, void *pUser_read_buf, size_t user_read_buf_size); mz_bool mz_zip_reader_extract_to_mem_no_alloc(mz_zip_archive* pZip, mz_uint file_index, void* pBuf, size_t buf_size, mz_uint flags, void* pUser_read_buf, size_t user_read_buf_size);
mz_bool mz_zip_reader_extract_file_to_mem_no_alloc(mz_zip_archive *pZip, const char *pFilename, void *pBuf, size_t buf_size, mz_uint flags, void *pUser_read_buf, size_t user_read_buf_size); mz_bool mz_zip_reader_extract_file_to_mem_no_alloc(mz_zip_archive* pZip, const char* pFilename, void* pBuf, size_t buf_size, mz_uint flags, void* pUser_read_buf, size_t user_read_buf_size);
// Extracts a archive file to a memory buffer. // Extracts a archive file to a memory buffer.
mz_bool mz_zip_reader_extract_to_mem(mz_zip_archive *pZip, mz_uint file_index, void *pBuf, size_t buf_size, mz_uint flags); mz_bool mz_zip_reader_extract_to_mem(mz_zip_archive* pZip, mz_uint file_index, void* pBuf, size_t buf_size, mz_uint flags);
mz_bool mz_zip_reader_extract_file_to_mem(mz_zip_archive *pZip, const char *pFilename, void *pBuf, size_t buf_size, mz_uint flags); mz_bool mz_zip_reader_extract_file_to_mem(mz_zip_archive* pZip, const char* pFilename, void* pBuf, size_t buf_size, mz_uint flags);
// Extracts a archive file to a dynamically allocated heap buffer. // Extracts a archive file to a dynamically allocated heap buffer.
void *mz_zip_reader_extract_to_heap(mz_zip_archive *pZip, mz_uint file_index, size_t *pSize, mz_uint flags); void* mz_zip_reader_extract_to_heap(mz_zip_archive* pZip, mz_uint file_index, size_t* pSize, mz_uint flags);
void *mz_zip_reader_extract_file_to_heap(mz_zip_archive *pZip, const char *pFilename, size_t *pSize, mz_uint flags); void* mz_zip_reader_extract_file_to_heap(mz_zip_archive* pZip, const char* pFilename, size_t* pSize, mz_uint flags);
// Extracts a archive file using a callback function to output the file's data. // Extracts a archive file using a callback function to output the file's data.
mz_bool mz_zip_reader_extract_to_callback(mz_zip_archive *pZip, mz_uint file_index, mz_file_write_func pCallback, void *pOpaque, mz_uint flags); mz_bool mz_zip_reader_extract_to_callback(mz_zip_archive* pZip, mz_uint file_index, mz_file_write_func pCallback, void* pOpaque, mz_uint flags);
mz_bool mz_zip_reader_extract_file_to_callback(mz_zip_archive *pZip, const char *pFilename, mz_file_write_func pCallback, void *pOpaque, mz_uint flags); mz_bool mz_zip_reader_extract_file_to_callback(mz_zip_archive* pZip, const char* pFilename, mz_file_write_func pCallback, void* pOpaque, mz_uint flags);
#ifndef MINIZ_NO_STDIO #ifndef MINIZ_NO_STDIO
// Extracts a archive file to a disk file and sets its last accessed and modified times. // Extracts a archive file to a disk file and sets its last accessed and modified times.
// This function only extracts files, not archive directory records. // This function only extracts files, not archive directory records.
mz_bool mz_zip_reader_extract_to_file(mz_zip_archive *pZip, mz_uint file_index, const char *pDst_filename, mz_uint flags); mz_bool mz_zip_reader_extract_to_file(mz_zip_archive* pZip, mz_uint file_index, const char* pDst_filename, mz_uint flags);
mz_bool mz_zip_reader_extract_file_to_file(mz_zip_archive *pZip, const char *pArchive_filename, const char *pDst_filename, mz_uint flags); mz_bool mz_zip_reader_extract_file_to_file(mz_zip_archive* pZip, const char* pArchive_filename, const char* pDst_filename, mz_uint flags);
#endif #endif
// Ends archive reading, freeing all allocations, and closing the input archive file if mz_zip_reader_init_file() was used. // Ends archive reading, freeing all allocations, and closing the input archive file if mz_zip_reader_init_file() was used.
mz_bool mz_zip_reader_end(mz_zip_archive *pZip); mz_bool mz_zip_reader_end(mz_zip_archive* pZip);
// ZIP archive writing // ZIP archive writing
#ifndef MINIZ_NO_ARCHIVE_WRITING_APIS #ifndef MINIZ_NO_ARCHIVE_WRITING_APIS
// Inits a ZIP archive writer. // Inits a ZIP archive writer.
mz_bool mz_zip_writer_init(mz_zip_archive *pZip, mz_uint64 existing_size); mz_bool mz_zip_writer_init(mz_zip_archive* pZip, mz_uint64 existing_size);
mz_bool mz_zip_writer_init_heap(mz_zip_archive *pZip, size_t size_to_reserve_at_beginning, size_t initial_allocation_size); mz_bool mz_zip_writer_init_heap(mz_zip_archive* pZip, size_t size_to_reserve_at_beginning, size_t initial_allocation_size);
#ifndef MINIZ_NO_STDIO #ifndef MINIZ_NO_STDIO
mz_bool mz_zip_writer_init_file(mz_zip_archive *pZip, const char *pFilename, mz_uint64 size_to_reserve_at_beginning); mz_bool mz_zip_writer_init_file(mz_zip_archive* pZip, const char* pFilename, mz_uint64 size_to_reserve_at_beginning);
#endif #endif
// Converts a ZIP archive reader object into a writer object, to allow efficient in-place file appends to occur on an existing archive. // Converts a ZIP archive reader object into a writer object, to allow efficient in-place file appends to occur on an existing archive.
@@ -606,48 +625,48 @@ mz_bool mz_zip_writer_init_file(mz_zip_archive *pZip, const char *pFilename, mz_
// Finally, for archives opened using mz_zip_reader_init, the mz_zip_archive's user provided m_pWrite function cannot be NULL. // Finally, for archives opened using mz_zip_reader_init, the mz_zip_archive's user provided m_pWrite function cannot be NULL.
// Note: In-place archive modification is not recommended unless you know what you're doing, because if execution stops or something goes wrong before // Note: In-place archive modification is not recommended unless you know what you're doing, because if execution stops or something goes wrong before
// the archive is finalized the file's central directory will be hosed. // the archive is finalized the file's central directory will be hosed.
mz_bool mz_zip_writer_init_from_reader(mz_zip_archive *pZip, const char *pFilename); mz_bool mz_zip_writer_init_from_reader(mz_zip_archive* pZip, const char* pFilename);
// Adds the contents of a memory buffer to an archive. These functions record the current local time into the archive. // Adds the contents of a memory buffer to an archive. These functions record the current local time into the archive.
// To add a directory entry, call this method with an archive name ending in a forwardslash with empty buffer. // To add a directory entry, call this method with an archive name ending in a forwardslash with empty buffer.
// level_and_flags - compression level (0-10, see MZ_BEST_SPEED, MZ_BEST_COMPRESSION, etc.) logically OR'd with zero or more mz_zip_flags, or just set to MZ_DEFAULT_COMPRESSION. // level_and_flags - compression level (0-10, see MZ_BEST_SPEED, MZ_BEST_COMPRESSION, etc.) logically OR'd with zero or more mz_zip_flags, or just set to MZ_DEFAULT_COMPRESSION.
mz_bool mz_zip_writer_add_mem(mz_zip_archive *pZip, const char *pArchive_name, const void *pBuf, size_t buf_size, mz_uint level_and_flags); mz_bool mz_zip_writer_add_mem(mz_zip_archive* pZip, const char* pArchive_name, const void* pBuf, size_t buf_size, mz_uint level_and_flags);
mz_bool mz_zip_writer_add_mem_ex(mz_zip_archive *pZip, const char *pArchive_name, const void *pBuf, size_t buf_size, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags, mz_uint64 uncomp_size, mz_uint32 uncomp_crc32); mz_bool mz_zip_writer_add_mem_ex(mz_zip_archive* pZip, const char* pArchive_name, const void* pBuf, size_t buf_size, const void* pComment, mz_uint16 comment_size, mz_uint level_and_flags, mz_uint64 uncomp_size, mz_uint32 uncomp_crc32);
#ifndef MINIZ_NO_STDIO #ifndef MINIZ_NO_STDIO
// Adds the contents of a disk file to an archive. This function also records the disk file's modified time into the archive. // Adds the contents of a disk file to an archive. This function also records the disk file's modified time into the archive.
// level_and_flags - compression level (0-10, see MZ_BEST_SPEED, MZ_BEST_COMPRESSION, etc.) logically OR'd with zero or more mz_zip_flags, or just set to MZ_DEFAULT_COMPRESSION. // level_and_flags - compression level (0-10, see MZ_BEST_SPEED, MZ_BEST_COMPRESSION, etc.) logically OR'd with zero or more mz_zip_flags, or just set to MZ_DEFAULT_COMPRESSION.
mz_bool mz_zip_writer_add_file(mz_zip_archive *pZip, const char *pArchive_name, const char *pSrc_filename, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags); mz_bool mz_zip_writer_add_file(mz_zip_archive* pZip, const char* pArchive_name, const char* pSrc_filename, const void* pComment, mz_uint16 comment_size, mz_uint level_and_flags);
#endif #endif
// Adds a file to an archive by fully cloning the data from another archive. // Adds a file to an archive by fully cloning the data from another archive.
// This function fully clones the source file's compressed data (no recompression), along with its full filename, extra data, and comment fields. // This function fully clones the source file's compressed data (no recompression), along with its full filename, extra data, and comment fields.
mz_bool mz_zip_writer_add_from_zip_reader(mz_zip_archive *pZip, mz_zip_archive *pSource_zip, mz_uint file_index); mz_bool mz_zip_writer_add_from_zip_reader(mz_zip_archive* pZip, mz_zip_archive* pSource_zip, mz_uint file_index);
// Finalizes the archive by writing the central directory records followed by the end of central directory record. // Finalizes the archive by writing the central directory records followed by the end of central directory record.
// After an archive is finalized, the only valid call on the mz_zip_archive struct is mz_zip_writer_end(). // After an archive is finalized, the only valid call on the mz_zip_archive struct is mz_zip_writer_end().
// An archive must be manually finalized by calling this function for it to be valid. // An archive must be manually finalized by calling this function for it to be valid.
mz_bool mz_zip_writer_finalize_archive(mz_zip_archive *pZip); mz_bool mz_zip_writer_finalize_archive(mz_zip_archive* pZip);
mz_bool mz_zip_writer_finalize_heap_archive(mz_zip_archive *pZip, void **pBuf, size_t *pSize); mz_bool mz_zip_writer_finalize_heap_archive(mz_zip_archive* pZip, void** pBuf, size_t* pSize);
// Ends archive writing, freeing all allocations, and closing the output file if mz_zip_writer_init_file() was used. // Ends archive writing, freeing all allocations, and closing the output file if mz_zip_writer_init_file() was used.
// Note for the archive to be valid, it must have been finalized before ending. // Note for the archive to be valid, it must have been finalized before ending.
mz_bool mz_zip_writer_end(mz_zip_archive *pZip); mz_bool mz_zip_writer_end(mz_zip_archive* pZip);
// Misc. high-level helper functions: // Misc. high-level helper functions:
// mz_zip_add_mem_to_archive_file_in_place() efficiently (but not atomically) appends a memory blob to a ZIP archive. // mz_zip_add_mem_to_archive_file_in_place() efficiently (but not atomically) appends a memory blob to a ZIP archive.
// level_and_flags - compression level (0-10, see MZ_BEST_SPEED, MZ_BEST_COMPRESSION, etc.) logically OR'd with zero or more mz_zip_flags, or just set to MZ_DEFAULT_COMPRESSION. // level_and_flags - compression level (0-10, see MZ_BEST_SPEED, MZ_BEST_COMPRESSION, etc.) logically OR'd with zero or more mz_zip_flags, or just set to MZ_DEFAULT_COMPRESSION.
mz_bool mz_zip_add_mem_to_archive_file_in_place(const char *pZip_filename, const char *pArchive_name, const void *pBuf, size_t buf_size, const void *pComment, mz_uint16 comment_size, mz_uint level_and_flags); mz_bool mz_zip_add_mem_to_archive_file_in_place(const char* pZip_filename, const char* pArchive_name, const void* pBuf, size_t buf_size, const void* pComment, mz_uint16 comment_size, mz_uint level_and_flags);
// Reads a single file from an archive into a heap block. // Reads a single file from an archive into a heap block.
// If pComment is not NULL, only the file with the specified comment will be extracted. // If pComment is not NULL, only the file with the specified comment will be extracted.
// Returns NULL on failure. // Returns NULL on failure.
void *mz_zip_extract_archive_file_to_heap(const char *pZip_filename, const char *pArchive_name, const char *pComment, size_t *pSize, mz_uint flags); void* mz_zip_extract_archive_file_to_heap(const char* pZip_filename, const char* pArchive_name, const char* pComment, size_t* pSize, mz_uint flags);
#endif // #ifndef MINIZ_NO_ARCHIVE_WRITING_APIS #endif // #ifndef MINIZ_NO_ARCHIVE_WRITING_APIS
#endif // #ifndef MINIZ_NO_ARCHIVE_APIS #endif // #ifndef MINIZ_NO_ARCHIVE_APIS
// ------------------- Low-level Decompression API Definitions // ------------------- Low-level Decompression API Definitions
@@ -656,8 +675,7 @@ void *mz_zip_extract_archive_file_to_heap(const char *pZip_filename, const char
// TINFL_FLAG_HAS_MORE_INPUT: If set, there are more input bytes available beyond the end of the supplied input buffer. If clear, the input buffer contains all remaining input. // TINFL_FLAG_HAS_MORE_INPUT: If set, there are more input bytes available beyond the end of the supplied input buffer. If clear, the input buffer contains all remaining input.
// TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF: If set, the output buffer is large enough to hold the entire decompressed stream. If clear, the output buffer is at least the size of the dictionary (typically 32KB). // TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF: If set, the output buffer is large enough to hold the entire decompressed stream. If clear, the output buffer is at least the size of the dictionary (typically 32KB).
// TINFL_FLAG_COMPUTE_ADLER32: Force adler-32 checksum computation of the decompressed bytes. // TINFL_FLAG_COMPUTE_ADLER32: Force adler-32 checksum computation of the decompressed bytes.
enum enum {
{
TINFL_FLAG_PARSE_ZLIB_HEADER = 1, TINFL_FLAG_PARSE_ZLIB_HEADER = 1,
TINFL_FLAG_HAS_MORE_INPUT = 2, TINFL_FLAG_HAS_MORE_INPUT = 2,
TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF = 4, TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF = 4,
@@ -672,26 +690,26 @@ enum
// Function returns a pointer to the decompressed data, or NULL on failure. // Function returns a pointer to the decompressed data, or NULL on failure.
// *pOut_len will be set to the decompressed data's size, which could be larger than src_buf_len on uncompressible data. // *pOut_len will be set to the decompressed data's size, which could be larger than src_buf_len on uncompressible data.
// The caller must call mz_free() on the returned block when it's no longer needed. // The caller must call mz_free() on the returned block when it's no longer needed.
void *tinfl_decompress_mem_to_heap(const void *pSrc_buf, size_t src_buf_len, size_t *pOut_len, int flags); void* tinfl_decompress_mem_to_heap(const void* pSrc_buf, size_t src_buf_len, size_t* pOut_len, int flags);
// tinfl_decompress_mem_to_mem() decompresses a block in memory to another block in memory. // tinfl_decompress_mem_to_mem() decompresses a block in memory to another block in memory.
// Returns TINFL_DECOMPRESS_MEM_TO_MEM_FAILED on failure, or the number of bytes written on success. // Returns TINFL_DECOMPRESS_MEM_TO_MEM_FAILED on failure, or the number of bytes written on success.
#define TINFL_DECOMPRESS_MEM_TO_MEM_FAILED ((size_t)(-1)) #define TINFL_DECOMPRESS_MEM_TO_MEM_FAILED ((size_t)(-1))
size_t tinfl_decompress_mem_to_mem(void *pOut_buf, size_t out_buf_len, const void *pSrc_buf, size_t src_buf_len, int flags); size_t tinfl_decompress_mem_to_mem(void* pOut_buf, size_t out_buf_len, const void* pSrc_buf, size_t src_buf_len, int flags);
// tinfl_decompress_mem_to_callback() decompresses a block in memory to an internal 32KB buffer, and a user provided callback function will be called to flush the buffer. // tinfl_decompress_mem_to_callback() decompresses a block in memory to an internal 32KB buffer, and a user provided callback function will be called to flush the buffer.
// Returns 1 on success or 0 on failure. // Returns 1 on success or 0 on failure.
typedef int (*tinfl_put_buf_func_ptr)(const void* pBuf, int len, void *pUser); typedef int (*tinfl_put_buf_func_ptr)(const void* pBuf, int len, void* pUser);
int tinfl_decompress_mem_to_callback(const void *pIn_buf, size_t *pIn_buf_size, tinfl_put_buf_func_ptr pPut_buf_func, void *pPut_buf_user, int flags); int tinfl_decompress_mem_to_callback(const void* pIn_buf, size_t* pIn_buf_size, tinfl_put_buf_func_ptr pPut_buf_func, void* pPut_buf_user, int flags);
struct tinfl_decompressor_tag; typedef struct tinfl_decompressor_tag tinfl_decompressor; struct tinfl_decompressor_tag;
typedef struct tinfl_decompressor_tag tinfl_decompressor;
// Max size of LZ dictionary. // Max size of LZ dictionary.
#define TINFL_LZ_DICT_SIZE 32768 #define TINFL_LZ_DICT_SIZE 32768
// Return status. // Return status.
typedef enum typedef enum {
{
TINFL_STATUS_BAD_PARAM = -3, TINFL_STATUS_BAD_PARAM = -3,
TINFL_STATUS_ADLER32_MISMATCH = -2, TINFL_STATUS_ADLER32_MISMATCH = -2,
TINFL_STATUS_FAILED = -1, TINFL_STATUS_FAILED = -1,
@@ -701,18 +719,25 @@ typedef enum
} tinfl_status; } tinfl_status;
// Initializes the decompressor to its initial state. // Initializes the decompressor to its initial state.
#define tinfl_init(r) do { (r)->m_state = 0; } MZ_MACRO_END #define tinfl_init(r) \
do { \
(r)->m_state = 0; \
} \
MZ_MACRO_END
#define tinfl_get_adler32(r) (r)->m_check_adler32 #define tinfl_get_adler32(r) (r)->m_check_adler32
// Main low-level decompressor coroutine function. This is the only function actually needed for decompression. All the other functions are just high-level helpers for improved usability. // Main low-level decompressor coroutine function. This is the only function actually needed for decompression. All the other functions are just high-level helpers for improved usability.
// This is a universal API, i.e. it can be used as a building block to build any desired higher level decompression API. In the limit case, it can be called once per every byte input or output. // This is a universal API, i.e. it can be used as a building block to build any desired higher level decompression API. In the limit case, it can be called once per every byte input or output.
tinfl_status tinfl_decompress(tinfl_decompressor *r, const mz_uint8 *pIn_buf_next, size_t *pIn_buf_size, mz_uint8 *pOut_buf_start, mz_uint8 *pOut_buf_next, size_t *pOut_buf_size, const mz_uint32 decomp_flags); tinfl_status tinfl_decompress(tinfl_decompressor* r, const mz_uint8* pIn_buf_next, size_t* pIn_buf_size, mz_uint8* pOut_buf_start, mz_uint8* pOut_buf_next, size_t* pOut_buf_size, const mz_uint32 decomp_flags);
// Internal/private bits follow. // Internal/private bits follow.
enum enum {
{ TINFL_MAX_HUFF_TABLES = 3,
TINFL_MAX_HUFF_TABLES = 3, TINFL_MAX_HUFF_SYMBOLS_0 = 288, TINFL_MAX_HUFF_SYMBOLS_1 = 32, TINFL_MAX_HUFF_SYMBOLS_2 = 19, TINFL_MAX_HUFF_SYMBOLS_0 = 288,
TINFL_FAST_LOOKUP_BITS = 10, TINFL_FAST_LOOKUP_SIZE = 1 << TINFL_FAST_LOOKUP_BITS TINFL_MAX_HUFF_SYMBOLS_1 = 32,
TINFL_MAX_HUFF_SYMBOLS_2 = 19,
TINFL_FAST_LOOKUP_BITS = 10,
TINFL_FAST_LOOKUP_SIZE = 1 << TINFL_FAST_LOOKUP_BITS
}; };
typedef struct typedef struct
@@ -722,19 +747,18 @@ typedef struct
} tinfl_huff_table; } tinfl_huff_table;
#if MINIZ_HAS_64BIT_REGISTERS #if MINIZ_HAS_64BIT_REGISTERS
#define TINFL_USE_64BIT_BITBUF 1 #define TINFL_USE_64BIT_BITBUF 1
#endif #endif
#if TINFL_USE_64BIT_BITBUF #if TINFL_USE_64BIT_BITBUF
typedef mz_uint64 tinfl_bit_buf_t; typedef mz_uint64 tinfl_bit_buf_t;
#define TINFL_BITBUF_SIZE (64) #define TINFL_BITBUF_SIZE (64)
#else #else
typedef mz_uint32 tinfl_bit_buf_t; typedef mz_uint32 tinfl_bit_buf_t;
#define TINFL_BITBUF_SIZE (32) #define TINFL_BITBUF_SIZE (32)
#endif #endif
struct tinfl_decompressor_tag struct tinfl_decompressor_tag {
{
mz_uint32 m_state, m_num_bits, m_zhdr0, m_zhdr1, m_z_adler32, m_final, m_type, m_check_adler32, m_dist, m_counter, m_num_extra, m_table_sizes[TINFL_MAX_HUFF_TABLES]; mz_uint32 m_state, m_num_bits, m_zhdr0, m_zhdr1, m_z_adler32, m_final, m_type, m_check_adler32, m_dist, m_counter, m_num_extra, m_table_sizes[TINFL_MAX_HUFF_TABLES];
tinfl_bit_buf_t m_bit_buf; tinfl_bit_buf_t m_bit_buf;
size_t m_dist_from_out_buf_start; size_t m_dist_from_out_buf_start;
@@ -749,9 +773,10 @@ struct tinfl_decompressor_tag
// tdefl_init() compression flags logically OR'd together (low 12 bits contain the max. number of probes per dictionary search): // tdefl_init() compression flags logically OR'd together (low 12 bits contain the max. number of probes per dictionary search):
// TDEFL_DEFAULT_MAX_PROBES: The compressor defaults to 128 dictionary probes per dictionary search. 0=Huffman only, 1=Huffman+LZ (fastest/crap compression), 4095=Huffman+LZ (slowest/best compression). // TDEFL_DEFAULT_MAX_PROBES: The compressor defaults to 128 dictionary probes per dictionary search. 0=Huffman only, 1=Huffman+LZ (fastest/crap compression), 4095=Huffman+LZ (slowest/best compression).
enum enum {
{ TDEFL_HUFFMAN_ONLY = 0,
TDEFL_HUFFMAN_ONLY = 0, TDEFL_DEFAULT_MAX_PROBES = 128, TDEFL_MAX_PROBES_MASK = 0xFFF TDEFL_DEFAULT_MAX_PROBES = 128,
TDEFL_MAX_PROBES_MASK = 0xFFF
}; };
// TDEFL_WRITE_ZLIB_HEADER: If set, the compressor outputs a zlib header before the deflate data, and the Adler-32 of the source data at the end. Otherwise, you'll get raw deflate data. // TDEFL_WRITE_ZLIB_HEADER: If set, the compressor outputs a zlib header before the deflate data, and the Adler-32 of the source data at the end. Otherwise, you'll get raw deflate data.
@@ -762,16 +787,15 @@ enum
// TDEFL_FILTER_MATCHES: Discards matches <= 5 chars if enabled. // TDEFL_FILTER_MATCHES: Discards matches <= 5 chars if enabled.
// TDEFL_FORCE_ALL_STATIC_BLOCKS: Disable usage of optimized Huffman tables. // TDEFL_FORCE_ALL_STATIC_BLOCKS: Disable usage of optimized Huffman tables.
// TDEFL_FORCE_ALL_RAW_BLOCKS: Only use raw (uncompressed) deflate blocks. // TDEFL_FORCE_ALL_RAW_BLOCKS: Only use raw (uncompressed) deflate blocks.
enum enum {
{ TDEFL_WRITE_ZLIB_HEADER = 0x01000,
TDEFL_WRITE_ZLIB_HEADER = 0x01000, TDEFL_COMPUTE_ADLER32 = 0x02000,
TDEFL_COMPUTE_ADLER32 = 0x02000, TDEFL_GREEDY_PARSING_FLAG = 0x04000,
TDEFL_GREEDY_PARSING_FLAG = 0x04000,
TDEFL_NONDETERMINISTIC_PARSING_FLAG = 0x08000, TDEFL_NONDETERMINISTIC_PARSING_FLAG = 0x08000,
TDEFL_RLE_MATCHES = 0x10000, TDEFL_RLE_MATCHES = 0x10000,
TDEFL_FILTER_MATCHES = 0x20000, TDEFL_FILTER_MATCHES = 0x20000,
TDEFL_FORCE_ALL_STATIC_BLOCKS = 0x40000, TDEFL_FORCE_ALL_STATIC_BLOCKS = 0x40000,
TDEFL_FORCE_ALL_RAW_BLOCKS = 0x80000 TDEFL_FORCE_ALL_RAW_BLOCKS = 0x80000
}; };
// High level compression functions: // High level compression functions:
@@ -783,40 +807,58 @@ enum
// Function returns a pointer to the compressed data, or NULL on failure. // Function returns a pointer to the compressed data, or NULL on failure.
// *pOut_len will be set to the compressed data's size, which could be larger than src_buf_len on uncompressible data. // *pOut_len will be set to the compressed data's size, which could be larger than src_buf_len on uncompressible data.
// The caller must free() the returned block when it's no longer needed. // The caller must free() the returned block when it's no longer needed.
void *tdefl_compress_mem_to_heap(const void *pSrc_buf, size_t src_buf_len, size_t *pOut_len, int flags); void* tdefl_compress_mem_to_heap(const void* pSrc_buf, size_t src_buf_len, size_t* pOut_len, int flags);
// tdefl_compress_mem_to_mem() compresses a block in memory to another block in memory. // tdefl_compress_mem_to_mem() compresses a block in memory to another block in memory.
// Returns 0 on failure. // Returns 0 on failure.
size_t tdefl_compress_mem_to_mem(void *pOut_buf, size_t out_buf_len, const void *pSrc_buf, size_t src_buf_len, int flags); size_t tdefl_compress_mem_to_mem(void* pOut_buf, size_t out_buf_len, const void* pSrc_buf, size_t src_buf_len, int flags);
// Compresses an image to a compressed PNG file in memory. // Compresses an image to a compressed PNG file in memory.
// On entry: // On entry:
// pImage, w, h, and num_chans describe the image to compress. num_chans may be 1, 2, 3, or 4. // pImage, w, h, and num_chans describe the image to compress. num_chans may be 1, 2, 3, or 4.
// The image pitch in bytes per scanline will be w*num_chans. The leftmost pixel on the top scanline is stored first in memory. // The image pitch in bytes per scanline will be w*num_chans. The leftmost pixel on the top scanline is stored first in memory.
// On return: // On return:
// Function returns a pointer to the compressed data, or NULL on failure. // Function returns a pointer to the compressed data, or NULL on failure.
// *pLen_out will be set to the size of the PNG image file. // *pLen_out will be set to the size of the PNG image file.
// The caller must mz_free() the returned heap block (which will typically be larger than *pLen_out) when it's no longer needed. // The caller must mz_free() the returned heap block (which will typically be larger than *pLen_out) when it's no longer needed.
void *tdefl_write_image_to_png_file_in_memory(const void *pImage, int w, int h, int num_chans, size_t *pLen_out); void* tdefl_write_image_to_png_file_in_memory(const void* pImage, int w, int h, int num_chans, size_t* pLen_out);
// Output stream interface. The compressor uses this interface to write compressed data. It'll typically be called TDEFL_OUT_BUF_SIZE at a time. // Output stream interface. The compressor uses this interface to write compressed data. It'll typically be called TDEFL_OUT_BUF_SIZE at a time.
typedef mz_bool (*tdefl_put_buf_func_ptr)(const void* pBuf, int len, void *pUser); typedef mz_bool (*tdefl_put_buf_func_ptr)(const void* pBuf, int len, void* pUser);
// tdefl_compress_mem_to_output() compresses a block to an output stream. The above helpers use this function internally. // tdefl_compress_mem_to_output() compresses a block to an output stream. The above helpers use this function internally.
mz_bool tdefl_compress_mem_to_output(const void *pBuf, size_t buf_len, tdefl_put_buf_func_ptr pPut_buf_func, void *pPut_buf_user, int flags); mz_bool tdefl_compress_mem_to_output(const void* pBuf, size_t buf_len, tdefl_put_buf_func_ptr pPut_buf_func, void* pPut_buf_user, int flags);
enum { TDEFL_MAX_HUFF_TABLES = 3, TDEFL_MAX_HUFF_SYMBOLS_0 = 288, TDEFL_MAX_HUFF_SYMBOLS_1 = 32, TDEFL_MAX_HUFF_SYMBOLS_2 = 19, TDEFL_LZ_DICT_SIZE = 32768, TDEFL_LZ_DICT_SIZE_MASK = TDEFL_LZ_DICT_SIZE - 1, TDEFL_MIN_MATCH_LEN = 3, TDEFL_MAX_MATCH_LEN = 258 }; enum { TDEFL_MAX_HUFF_TABLES = 3,
TDEFL_MAX_HUFF_SYMBOLS_0 = 288,
TDEFL_MAX_HUFF_SYMBOLS_1 = 32,
TDEFL_MAX_HUFF_SYMBOLS_2 = 19,
TDEFL_LZ_DICT_SIZE = 32768,
TDEFL_LZ_DICT_SIZE_MASK = TDEFL_LZ_DICT_SIZE - 1,
TDEFL_MIN_MATCH_LEN = 3,
TDEFL_MAX_MATCH_LEN = 258 };
// TDEFL_OUT_BUF_SIZE MUST be large enough to hold a single entire compressed output block (using static/fixed Huffman codes). // TDEFL_OUT_BUF_SIZE MUST be large enough to hold a single entire compressed output block (using static/fixed Huffman codes).
#if TDEFL_LESS_MEMORY #if TDEFL_LESS_MEMORY
enum { TDEFL_LZ_CODE_BUF_SIZE = 24 * 1024, TDEFL_OUT_BUF_SIZE = (TDEFL_LZ_CODE_BUF_SIZE * 13 ) / 10, TDEFL_MAX_HUFF_SYMBOLS = 288, TDEFL_LZ_HASH_BITS = 12, TDEFL_LEVEL1_HASH_SIZE_MASK = 4095, TDEFL_LZ_HASH_SHIFT = (TDEFL_LZ_HASH_BITS + 2) / 3, TDEFL_LZ_HASH_SIZE = 1 << TDEFL_LZ_HASH_BITS }; enum { TDEFL_LZ_CODE_BUF_SIZE = 24 * 1024,
TDEFL_OUT_BUF_SIZE = (TDEFL_LZ_CODE_BUF_SIZE * 13) / 10,
TDEFL_MAX_HUFF_SYMBOLS = 288,
TDEFL_LZ_HASH_BITS = 12,
TDEFL_LEVEL1_HASH_SIZE_MASK = 4095,
TDEFL_LZ_HASH_SHIFT = (TDEFL_LZ_HASH_BITS + 2) / 3,
TDEFL_LZ_HASH_SIZE = 1 << TDEFL_LZ_HASH_BITS };
#else #else
enum { TDEFL_LZ_CODE_BUF_SIZE = 64 * 1024, TDEFL_OUT_BUF_SIZE = (TDEFL_LZ_CODE_BUF_SIZE * 13 ) / 10, TDEFL_MAX_HUFF_SYMBOLS = 288, TDEFL_LZ_HASH_BITS = 15, TDEFL_LEVEL1_HASH_SIZE_MASK = 4095, TDEFL_LZ_HASH_SHIFT = (TDEFL_LZ_HASH_BITS + 2) / 3, TDEFL_LZ_HASH_SIZE = 1 << TDEFL_LZ_HASH_BITS }; enum { TDEFL_LZ_CODE_BUF_SIZE = 64 * 1024,
TDEFL_OUT_BUF_SIZE = (TDEFL_LZ_CODE_BUF_SIZE * 13) / 10,
TDEFL_MAX_HUFF_SYMBOLS = 288,
TDEFL_LZ_HASH_BITS = 15,
TDEFL_LEVEL1_HASH_SIZE_MASK = 4095,
TDEFL_LZ_HASH_SHIFT = (TDEFL_LZ_HASH_BITS + 2) / 3,
TDEFL_LZ_HASH_SIZE = 1 << TDEFL_LZ_HASH_BITS };
#endif #endif
// The low-level tdefl functions below may be used directly if the above helper functions aren't flexible enough. The low-level functions don't make any heap allocations, unlike the above helper functions. // The low-level tdefl functions below may be used directly if the above helper functions aren't flexible enough. The low-level functions don't make any heap allocations, unlike the above helper functions.
typedef enum typedef enum {
{
TDEFL_STATUS_BAD_PARAM = -2, TDEFL_STATUS_BAD_PARAM = -2,
TDEFL_STATUS_PUT_BUF_FAILED = -1, TDEFL_STATUS_PUT_BUF_FAILED = -1,
TDEFL_STATUS_OKAY = 0, TDEFL_STATUS_OKAY = 0,
@@ -824,8 +866,7 @@ typedef enum
} tdefl_status; } tdefl_status;
// Must map to MZ_NO_FLUSH, MZ_SYNC_FLUSH, etc. enums // Must map to MZ_NO_FLUSH, MZ_SYNC_FLUSH, etc. enums
typedef enum typedef enum {
{
TDEFL_NO_FLUSH = 0, TDEFL_NO_FLUSH = 0,
TDEFL_SYNC_FLUSH = 2, TDEFL_SYNC_FLUSH = 2,
TDEFL_FULL_FLUSH = 3, TDEFL_FULL_FLUSH = 3,
@@ -836,7 +877,7 @@ typedef enum
typedef struct typedef struct
{ {
tdefl_put_buf_func_ptr m_pPut_buf_func; tdefl_put_buf_func_ptr m_pPut_buf_func;
void *m_pPut_buf_user; void* m_pPut_buf_user;
mz_uint m_flags, m_max_probes[2]; mz_uint m_flags, m_max_probes[2];
int m_greedy_parsing; int m_greedy_parsing;
mz_uint m_adler32, m_lookahead_pos, m_lookahead_size, m_dict_size; mz_uint m_adler32, m_lookahead_pos, m_lookahead_size, m_dict_size;
@@ -844,11 +885,11 @@ typedef struct
mz_uint m_num_flags_left, m_total_lz_bytes, m_lz_code_buf_dict_pos, m_bits_in, m_bit_buffer; mz_uint m_num_flags_left, m_total_lz_bytes, m_lz_code_buf_dict_pos, m_bits_in, m_bit_buffer;
mz_uint m_saved_match_dist, m_saved_match_len, m_saved_lit, m_output_flush_ofs, m_output_flush_remaining, m_finished, m_block_index, m_wants_to_finish; mz_uint m_saved_match_dist, m_saved_match_len, m_saved_lit, m_output_flush_ofs, m_output_flush_remaining, m_finished, m_block_index, m_wants_to_finish;
tdefl_status m_prev_return_status; tdefl_status m_prev_return_status;
const void *m_pIn_buf; const void* m_pIn_buf;
void *m_pOut_buf; void* m_pOut_buf;
size_t *m_pIn_buf_size, *m_pOut_buf_size; size_t *m_pIn_buf_size, *m_pOut_buf_size;
tdefl_flush m_flush; tdefl_flush m_flush;
const mz_uint8 *m_pSrc; const mz_uint8* m_pSrc;
size_t m_src_buf_left, m_out_buf_ofs; size_t m_src_buf_left, m_out_buf_ofs;
mz_uint8 m_dict[TDEFL_LZ_DICT_SIZE + TDEFL_MAX_MATCH_LEN - 1]; mz_uint8 m_dict[TDEFL_LZ_DICT_SIZE + TDEFL_MAX_MATCH_LEN - 1];
mz_uint16 m_huff_count[TDEFL_MAX_HUFF_TABLES][TDEFL_MAX_HUFF_SYMBOLS]; mz_uint16 m_huff_count[TDEFL_MAX_HUFF_TABLES][TDEFL_MAX_HUFF_SYMBOLS];
@@ -865,17 +906,17 @@ typedef struct
// pBut_buf_func: If NULL, output data will be supplied to the specified callback. In this case, the user should call the tdefl_compress_buffer() API for compression. // pBut_buf_func: If NULL, output data will be supplied to the specified callback. In this case, the user should call the tdefl_compress_buffer() API for compression.
// If pBut_buf_func is NULL the user should always call the tdefl_compress() API. // If pBut_buf_func is NULL the user should always call the tdefl_compress() API.
// flags: See the above enums (TDEFL_HUFFMAN_ONLY, TDEFL_WRITE_ZLIB_HEADER, etc.) // flags: See the above enums (TDEFL_HUFFMAN_ONLY, TDEFL_WRITE_ZLIB_HEADER, etc.)
tdefl_status tdefl_init(tdefl_compressor *d, tdefl_put_buf_func_ptr pPut_buf_func, void *pPut_buf_user, int flags); tdefl_status tdefl_init(tdefl_compressor* d, tdefl_put_buf_func_ptr pPut_buf_func, void* pPut_buf_user, int flags);
// Compresses a block of data, consuming as much of the specified input buffer as possible, and writing as much compressed data to the specified output buffer as possible. // Compresses a block of data, consuming as much of the specified input buffer as possible, and writing as much compressed data to the specified output buffer as possible.
tdefl_status tdefl_compress(tdefl_compressor *d, const void *pIn_buf, size_t *pIn_buf_size, void *pOut_buf, size_t *pOut_buf_size, tdefl_flush flush); tdefl_status tdefl_compress(tdefl_compressor* d, const void* pIn_buf, size_t* pIn_buf_size, void* pOut_buf, size_t* pOut_buf_size, tdefl_flush flush);
// tdefl_compress_buffer() is only usable when the tdefl_init() is called with a non-NULL tdefl_put_buf_func_ptr. // tdefl_compress_buffer() is only usable when the tdefl_init() is called with a non-NULL tdefl_put_buf_func_ptr.
// tdefl_compress_buffer() always consumes the entire input buffer. // tdefl_compress_buffer() always consumes the entire input buffer.
tdefl_status tdefl_compress_buffer(tdefl_compressor *d, const void *pIn_buf, size_t in_buf_size, tdefl_flush flush); tdefl_status tdefl_compress_buffer(tdefl_compressor* d, const void* pIn_buf, size_t in_buf_size, tdefl_flush flush);
tdefl_status tdefl_get_prev_return_status(tdefl_compressor *d); tdefl_status tdefl_get_prev_return_status(tdefl_compressor* d);
mz_uint32 tdefl_get_adler32(tdefl_compressor *d); mz_uint32 tdefl_get_adler32(tdefl_compressor* d);
// Can't use tdefl_create_comp_flags_from_zip_params if MINIZ_NO_ZLIB_APIS isn't defined, because it uses some of its macros. // Can't use tdefl_create_comp_flags_from_zip_params if MINIZ_NO_ZLIB_APIS isn't defined, because it uses some of its macros.
#ifndef MINIZ_NO_ZLIB_APIS #ifndef MINIZ_NO_ZLIB_APIS
@@ -884,10 +925,10 @@ mz_uint32 tdefl_get_adler32(tdefl_compressor *d);
// window_bits may be -15 (raw deflate) or 15 (zlib) // window_bits may be -15 (raw deflate) or 15 (zlib)
// strategy may be either MZ_DEFAULT_STRATEGY, MZ_FILTERED, MZ_HUFFMAN_ONLY, MZ_RLE, or MZ_FIXED // strategy may be either MZ_DEFAULT_STRATEGY, MZ_FILTERED, MZ_HUFFMAN_ONLY, MZ_RLE, or MZ_FIXED
mz_uint tdefl_create_comp_flags_from_zip_params(int level, int window_bits, int strategy); mz_uint tdefl_create_comp_flags_from_zip_params(int level, int window_bits, int strategy);
#endif // #ifndef MINIZ_NO_ZLIB_APIS #endif // #ifndef MINIZ_NO_ZLIB_APIS
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif
#endif // MINIZ_HEADER_INCLUDED #endif // MINIZ_HEADER_INCLUDED
crnlib/crn_mipmapped_texture.cpp
+2758 -3027
View File
File diff suppressed because it is too large Load Diff
crnlib/crn_mipmapped_texture.h
+285 -287
View File
@@ -12,328 +12,326 @@
#include "crn_texture_file_types.h" #include "crn_texture_file_types.h"
#include "crn_image_utils.h" #include "crn_image_utils.h"
namespace crnlib namespace crnlib {
{ extern const vec2I g_vertical_cross_image_offsets[6];
extern const vec2I g_vertical_cross_image_offsets[6];
enum orientation_flags_t enum orientation_flags_t {
{ cOrientationFlagXFlipped = 1,
cOrientationFlagXFlipped = 1, cOrientationFlagYFlipped = 2,
cOrientationFlagYFlipped = 2,
cDefaultOrientationFlags = 0 cDefaultOrientationFlags = 0
}; };
enum unpack_flags_t enum unpack_flags_t {
{ cUnpackFlagUncook = 1,
cUnpackFlagUncook = 1, cUnpackFlagUnflip = 2
cUnpackFlagUnflip = 2 };
};
class mip_level class mip_level {
{ friend class mipmapped_texture;
friend class mipmapped_texture;
public: public:
mip_level(); mip_level();
~mip_level(); ~mip_level();
mip_level(const mip_level& other); mip_level(const mip_level& other);
mip_level& operator= (const mip_level& rhs); mip_level& operator=(const mip_level& rhs);
// Assumes ownership. // Assumes ownership.
void assign(image_u8* p, pixel_format fmt = PIXEL_FMT_INVALID, orientation_flags_t orient_flags = cDefaultOrientationFlags); void assign(image_u8* p, pixel_format fmt = PIXEL_FMT_INVALID, orientation_flags_t orient_flags = cDefaultOrientationFlags);
void assign(dxt_image* p, pixel_format fmt = PIXEL_FMT_INVALID, orientation_flags_t orient_flags = cDefaultOrientationFlags); void assign(dxt_image* p, pixel_format fmt = PIXEL_FMT_INVALID, orientation_flags_t orient_flags = cDefaultOrientationFlags);
void clear(); void clear();
inline uint get_width() const { return m_width; } inline uint get_width() const { return m_width; }
inline uint get_height() const { return m_height; } inline uint get_height() const { return m_height; }
inline uint get_total_pixels() const { return m_width * m_height; } inline uint get_total_pixels() const { return m_width * m_height; }
orientation_flags_t get_orientation_flags() const { return m_orient_flags; }
void set_orientation_flags(orientation_flags_t flags) { m_orient_flags = flags; }
inline image_u8* get_image() const { return m_pImage; } orientation_flags_t get_orientation_flags() const { return m_orient_flags; }
inline dxt_image* get_dxt_image() const { return m_pDXTImage; } void set_orientation_flags(orientation_flags_t flags) { m_orient_flags = flags; }
image_u8* get_unpacked_image(image_u8& tmp, uint unpack_flags) const;
inline bool is_packed() const { return m_pDXTImage != NULL; } inline image_u8* get_image() const { return m_pImage; }
inline dxt_image* get_dxt_image() const { return m_pDXTImage; }
inline bool is_valid() const { return (m_pImage != NULL) || (m_pDXTImage != NULL); } image_u8* get_unpacked_image(image_u8& tmp, uint unpack_flags) const;
inline pixel_format_helpers::component_flags get_comp_flags() const { return m_comp_flags; } inline bool is_packed() const { return m_pDXTImage != NULL; }
inline void set_comp_flags(pixel_format_helpers::component_flags comp_flags) { m_comp_flags = comp_flags; }
inline pixel_format get_format() const { return m_format; } inline bool is_valid() const { return (m_pImage != NULL) || (m_pDXTImage != NULL); }
inline void set_format(pixel_format fmt) { m_format = fmt; }
bool convert(pixel_format fmt, bool cook, const dxt_image::pack_params& p); inline pixel_format_helpers::component_flags get_comp_flags() const { return m_comp_flags; }
inline void set_comp_flags(pixel_format_helpers::component_flags comp_flags) { m_comp_flags = comp_flags; }
bool pack_to_dxt(const image_u8& img, pixel_format fmt, bool cook, const dxt_image::pack_params& p, orientation_flags_t orient_flags = cDefaultOrientationFlags); inline pixel_format get_format() const { return m_format; }
bool pack_to_dxt(pixel_format fmt, bool cook, const dxt_image::pack_params& p); inline void set_format(pixel_format fmt) { m_format = fmt; }
bool unpack_from_dxt(bool uncook = true); bool convert(pixel_format fmt, bool cook, const dxt_image::pack_params& p);
// Returns true if flipped on either axis. bool pack_to_dxt(const image_u8& img, pixel_format fmt, bool cook, const dxt_image::pack_params& p, orientation_flags_t orient_flags = cDefaultOrientationFlags);
bool is_flipped() const; bool pack_to_dxt(pixel_format fmt, bool cook, const dxt_image::pack_params& p);
bool is_x_flipped() const;
bool is_y_flipped() const;
bool can_unflip_without_unpacking() const;
// Returns true if unflipped on either axis.
// Will try to flip packed (DXT/ETC) data in-place, if this isn't possible it'll unpack/uncook the mip level then unflip.
bool unflip(bool allow_unpacking_to_flip, bool uncook_during_unpack);
bool set_alpha_to_luma();
bool convert(image_utils::conversion_type conv_type);
bool flip_x(); bool unpack_from_dxt(bool uncook = true);
bool flip_y();
private: // Returns true if flipped on either axis.
uint m_width; bool is_flipped() const;
uint m_height;
pixel_format_helpers::component_flags m_comp_flags; bool is_x_flipped() const;
pixel_format m_format; bool is_y_flipped() const;
image_u8* m_pImage; bool can_unflip_without_unpacking() const;
dxt_image* m_pDXTImage;
orientation_flags_t m_orient_flags; // Returns true if unflipped on either axis.
// Will try to flip packed (DXT/ETC) data in-place, if this isn't possible it'll unpack/uncook the mip level then unflip.
bool unflip(bool allow_unpacking_to_flip, bool uncook_during_unpack);
void cook_image(image_u8& img) const; bool set_alpha_to_luma();
void uncook_image(image_u8& img) const; bool convert(image_utils::conversion_type conv_type);
};
// A face is an array of mip_level ptr's. bool flip_x();
typedef crnlib::vector<mip_level*> mip_ptr_vec; bool flip_y();
// And an array of one, six, or N faces make up a texture. private:
typedef crnlib::vector<mip_ptr_vec> face_vec; uint m_width;
uint m_height;
class mipmapped_texture pixel_format_helpers::component_flags m_comp_flags;
{ pixel_format m_format;
public:
// Construction/destruction
mipmapped_texture();
~mipmapped_texture();
mipmapped_texture(const mipmapped_texture& other); image_u8* m_pImage;
mipmapped_texture& operator= (const mipmapped_texture& rhs); dxt_image* m_pDXTImage;
void clear(); orientation_flags_t m_orient_flags;
void init(uint width, uint height, uint levels, uint faces, pixel_format fmt, const char* pName, orientation_flags_t orient_flags); void cook_image(image_u8& img) const;
void uncook_image(image_u8& img) const;
};
// Assumes ownership. // A face is an array of mip_level ptr's.
void assign(face_vec& faces); typedef crnlib::vector<mip_level*> mip_ptr_vec;
void assign(mip_level* pLevel);
void assign(image_u8* p, pixel_format fmt = PIXEL_FMT_INVALID, orientation_flags_t orient_flags = cDefaultOrientationFlags);
void assign(dxt_image* p, pixel_format fmt = PIXEL_FMT_INVALID, orientation_flags_t orient_flags = cDefaultOrientationFlags);
void set(texture_file_types::format source_file_type, const mipmapped_texture& mipmapped_texture); // And an array of one, six, or N faces make up a texture.
typedef crnlib::vector<mip_ptr_vec> face_vec;
// Accessors class mipmapped_texture {
image_u8* get_level_image(uint face, uint level, image_u8& img, uint unpack_flags = cUnpackFlagUncook | cUnpackFlagUnflip) const; public:
// Construction/destruction
mipmapped_texture();
~mipmapped_texture();
inline bool is_valid() const { return m_faces.size() > 0; } mipmapped_texture(const mipmapped_texture& other);
mipmapped_texture& operator=(const mipmapped_texture& rhs);
const dynamic_string& get_name() const { return m_name; }
void set_name(const dynamic_string& name) { m_name = name; }
const dynamic_string& get_source_filename() const { return get_name(); }
texture_file_types::format get_source_file_type() const { return m_source_file_type; }
inline uint get_width() const { return m_width; }
inline uint get_height() const { return m_height; }
inline uint get_total_pixels() const { return m_width * m_height; }
uint get_total_pixels_in_all_faces_and_mips() const;
inline uint get_num_faces() const { return m_faces.size(); } void clear();
inline uint get_num_levels() const { if (m_faces.empty()) return 0; else return m_faces[0].size(); }
inline pixel_format_helpers::component_flags get_comp_flags() const { return m_comp_flags; } void init(uint width, uint height, uint levels, uint faces, pixel_format fmt, const char* pName, orientation_flags_t orient_flags);
inline pixel_format get_format() const { return m_format; }
inline bool is_unpacked() const { if (get_num_faces()) { return get_level(0, 0)->get_image() != NULL; } return false; } // Assumes ownership.
void assign(face_vec& faces);
void assign(mip_level* pLevel);
void assign(image_u8* p, pixel_format fmt = PIXEL_FMT_INVALID, orientation_flags_t orient_flags = cDefaultOrientationFlags);
void assign(dxt_image* p, pixel_format fmt = PIXEL_FMT_INVALID, orientation_flags_t orient_flags = cDefaultOrientationFlags);
inline const mip_ptr_vec& get_face(uint face) const { return m_faces[face]; } void set(texture_file_types::format source_file_type, const mipmapped_texture& mipmapped_texture);
inline mip_ptr_vec& get_face(uint face) { return m_faces[face]; }
// Accessors
inline const mip_level* get_level(uint face, uint mip) const { return m_faces[face][mip]; } image_u8* get_level_image(uint face, uint level, image_u8& img, uint unpack_flags = cUnpackFlagUncook | cUnpackFlagUnflip) const;
inline mip_level* get_level(uint face, uint mip) { return m_faces[face][mip]; }
inline bool is_valid() const { return m_faces.size() > 0; }
bool has_alpha() const;
bool is_normal_map() const; const dynamic_string& get_name() const { return m_name; }
bool is_vertical_cross() const; void set_name(const dynamic_string& name) { m_name = name; }
bool is_packed() const;
texture_type determine_texture_type() const; const dynamic_string& get_source_filename() const { return get_name(); }
texture_file_types::format get_source_file_type() const { return m_source_file_type; }
const dynamic_string& get_last_error() const { return m_last_error; }
void clear_last_error() { m_last_error.clear(); } inline uint get_width() const { return m_width; }
inline uint get_height() const { return m_height; }
// Reading/writing inline uint get_total_pixels() const { return m_width * m_height; }
bool read_dds(data_stream_serializer& serializer); uint get_total_pixels_in_all_faces_and_mips() const;
bool write_dds(data_stream_serializer& serializer) const;
inline uint get_num_faces() const { return m_faces.size(); }
bool read_ktx(data_stream_serializer& serializer); inline uint get_num_levels() const {
bool write_ktx(data_stream_serializer& serializer) const; if (m_faces.empty())
return 0;
bool read_crn(data_stream_serializer& serializer); else
bool read_crn_from_memory(const void *pData, uint data_size, const char* pFilename); return m_faces[0].size();
}
// If file_format is texture_file_types::cFormatInvalid, the format will be determined from the filename's extension.
bool read_from_file(const char* pFilename, texture_file_types::format file_format = texture_file_types::cFormatInvalid); inline pixel_format_helpers::component_flags get_comp_flags() const { return m_comp_flags; }
bool read_from_stream(data_stream_serializer& serializer, texture_file_types::format file_format = texture_file_types::cFormatInvalid); inline pixel_format get_format() const { return m_format; }
bool write_to_file( inline bool is_unpacked() const {
const char* pFilename, if (get_num_faces()) {
texture_file_types::format file_format = texture_file_types::cFormatInvalid, return get_level(0, 0)->get_image() != NULL;
crn_comp_params* pComp_params = NULL, }
uint32* pActual_quality_level = NULL, float* pActual_bitrate = NULL, return false;
uint32 image_write_flags = 0); }
// Conversion inline const mip_ptr_vec& get_face(uint face) const { return m_faces[face]; }
bool convert(pixel_format fmt, bool cook, const dxt_image::pack_params& p); inline mip_ptr_vec& get_face(uint face) { return m_faces[face]; }
bool convert(pixel_format fmt, const dxt_image::pack_params& p);
bool convert(pixel_format fmt, bool cook, const dxt_image::pack_params& p, int qdxt_quality, bool hierarchical = true); inline const mip_level* get_level(uint face, uint mip) const { return m_faces[face][mip]; }
bool convert(image_utils::conversion_type conv_type); inline mip_level* get_level(uint face, uint mip) { return m_faces[face][mip]; }
bool unpack_from_dxt(bool uncook = true); bool has_alpha() const;
bool is_normal_map() const;
bool set_alpha_to_luma(); bool is_vertical_cross() const;
bool is_packed() const;
void discard_mipmaps(); texture_type determine_texture_type() const;
void discard_mips(); const dynamic_string& get_last_error() const { return m_last_error; }
void clear_last_error() { m_last_error.clear(); }
struct resample_params
{ // Reading/writing
resample_params() : bool read_dds(data_stream_serializer& serializer);
m_pFilter("kaiser"), bool write_dds(data_stream_serializer& serializer) const;
m_wrapping(false),
m_srgb(false), bool read_ktx(data_stream_serializer& serializer);
m_renormalize(false), bool write_ktx(data_stream_serializer& serializer) const;
m_filter_scale(.9f),
m_gamma(1.75f), // or 2.2f bool read_crn(data_stream_serializer& serializer);
m_multithreaded(true) bool read_crn_from_memory(const void* pData, uint data_size, const char* pFilename);
{
} // If file_format is texture_file_types::cFormatInvalid, the format will be determined from the filename's extension.
bool read_from_file(const char* pFilename, texture_file_types::format file_format = texture_file_types::cFormatInvalid);
const char* m_pFilter; bool read_from_stream(data_stream_serializer& serializer, texture_file_types::format file_format = texture_file_types::cFormatInvalid);
bool m_wrapping;
bool m_srgb; bool write_to_file(
bool m_renormalize; const char* pFilename,
float m_filter_scale; texture_file_types::format file_format = texture_file_types::cFormatInvalid,
float m_gamma; crn_comp_params* pComp_params = NULL,
bool m_multithreaded; uint32* pActual_quality_level = NULL, float* pActual_bitrate = NULL,
}; uint32 image_write_flags = 0);
bool resize(uint new_width, uint new_height, const resample_params& params); // Conversion
bool convert(pixel_format fmt, bool cook, const dxt_image::pack_params& p);
struct generate_mipmap_params : public resample_params bool convert(pixel_format fmt, const dxt_image::pack_params& p);
{ bool convert(pixel_format fmt, bool cook, const dxt_image::pack_params& p, int qdxt_quality, bool hierarchical = true);
generate_mipmap_params() : bool convert(image_utils::conversion_type conv_type);
resample_params(),
m_min_mip_size(1), bool unpack_from_dxt(bool uncook = true);
m_max_mips(0)
{ bool set_alpha_to_luma();
}
void discard_mipmaps();
uint m_min_mip_size;
uint m_max_mips; // actually the max # of total levels void discard_mips();
};
struct resample_params {
bool generate_mipmaps(const generate_mipmap_params& params, bool force); resample_params()
: m_pFilter("kaiser"),
bool crop(uint x, uint y, uint width, uint height); m_wrapping(false),
m_srgb(false),
bool vertical_cross_to_cubemap(); m_renormalize(false),
m_filter_scale(.9f),
// Low-level clustered DXT (QDXT) compression m_gamma(1.75f), // or 2.2f
struct qdxt_state m_multithreaded(true) {
{ }
qdxt_state(task_pool& tp) : m_fmt(PIXEL_FMT_INVALID), m_qdxt1(tp), m_qdxt5a(tp), m_qdxt5b(tp)
{ const char* m_pFilter;
} bool m_wrapping;
bool m_srgb;
pixel_format m_fmt; bool m_renormalize;
qdxt1 m_qdxt1; float m_filter_scale;
qdxt5 m_qdxt5a; float m_gamma;
qdxt5 m_qdxt5b; bool m_multithreaded;
crnlib::vector<dxt_pixel_block> m_pixel_blocks; };
qdxt1_params m_qdxt1_params; bool resize(uint new_width, uint new_height, const resample_params& params);
qdxt5_params m_qdxt5_params[2];
bool m_has_blocks[3]; struct generate_mipmap_params : public resample_params {
generate_mipmap_params()
void clear() : resample_params(),
{ m_min_mip_size(1),
m_fmt = PIXEL_FMT_INVALID; m_max_mips(0) {
m_qdxt1.clear(); }
m_qdxt5a.clear();
m_qdxt5b.clear(); uint m_min_mip_size;
m_pixel_blocks.clear(); uint m_max_mips; // actually the max # of total levels
m_qdxt1_params.clear(); };
m_qdxt5_params[0].clear();
m_qdxt5_params[1].clear(); bool generate_mipmaps(const generate_mipmap_params& params, bool force);
utils::zero_object(m_has_blocks);
} bool crop(uint x, uint y, uint width, uint height);
};
bool qdxt_pack_init(qdxt_state& state, mipmapped_texture& dst_tex, const qdxt1_params& dxt1_params, const qdxt5_params& dxt5_params, pixel_format fmt, bool cook); bool vertical_cross_to_cubemap();
bool qdxt_pack(qdxt_state& state, mipmapped_texture& dst_tex, const qdxt1_params& dxt1_params, const qdxt5_params& dxt5_params);
// Low-level clustered DXT (QDXT) compression
void swap(mipmapped_texture& img); struct qdxt_state {
qdxt_state(task_pool& tp)
bool check() const; : m_fmt(PIXEL_FMT_INVALID), m_qdxt1(tp), m_qdxt5a(tp), m_qdxt5b(tp) {
}
void set_orientation_flags(orientation_flags_t flags);
pixel_format m_fmt;
// Returns true if any face/miplevel is flipped. qdxt1 m_qdxt1;
bool is_flipped() const; qdxt5 m_qdxt5a;
bool is_x_flipped() const; qdxt5 m_qdxt5b;
bool is_y_flipped() const; crnlib::vector<dxt_pixel_block> m_pixel_blocks;
bool can_unflip_without_unpacking() const;
bool unflip(bool allow_unpacking_to_flip, bool uncook_if_necessary_to_unpack); qdxt1_params m_qdxt1_params;
qdxt5_params m_qdxt5_params[2];
bool flip_y(bool update_orientation_flags); bool m_has_blocks[3];
private: void clear() {
dynamic_string m_name; m_fmt = PIXEL_FMT_INVALID;
m_qdxt1.clear();
uint m_width; m_qdxt5a.clear();
uint m_height; m_qdxt5b.clear();
m_pixel_blocks.clear();
pixel_format_helpers::component_flags m_comp_flags; m_qdxt1_params.clear();
pixel_format m_format; m_qdxt5_params[0].clear();
m_qdxt5_params[1].clear();
face_vec m_faces; utils::zero_object(m_has_blocks);
}
texture_file_types::format m_source_file_type; };
bool qdxt_pack_init(qdxt_state& state, mipmapped_texture& dst_tex, const qdxt1_params& dxt1_params, const qdxt5_params& dxt5_params, pixel_format fmt, bool cook);
mutable dynamic_string m_last_error; bool qdxt_pack(qdxt_state& state, mipmapped_texture& dst_tex, const qdxt1_params& dxt1_params, const qdxt5_params& dxt5_params);
inline void clear_last_error() const { m_last_error.clear(); } void swap(mipmapped_texture& img);
inline void set_last_error(const char* p) const { m_last_error = p; }
bool check() const;
void free_all_mips();
bool read_regular_image(data_stream_serializer &serializer, texture_file_types::format file_format); void set_orientation_flags(orientation_flags_t flags);
bool write_regular_image(const char* pFilename, uint32 image_write_flags);
bool read_dds_internal(data_stream_serializer& serializer); // Returns true if any face/miplevel is flipped.
void print_crn_comp_params(const crn_comp_params& p); bool is_flipped() const;
bool write_comp_texture(const char* pFilename, const crn_comp_params &comp_params, uint32 *pActual_quality_level, float *pActual_bitrate); bool is_x_flipped() const;
void change_dxt1_to_dxt1a(); bool is_y_flipped() const;
bool flip_y_helper(); bool can_unflip_without_unpacking() const;
}; bool unflip(bool allow_unpacking_to_flip, bool uncook_if_necessary_to_unpack);
inline void swap(mipmapped_texture& a, mipmapped_texture& b) bool flip_y(bool update_orientation_flags);
{
a.swap(b); private:
} dynamic_string m_name;
} // namespace crnlib uint m_width;
uint m_height;
pixel_format_helpers::component_flags m_comp_flags;
pixel_format m_format;
face_vec m_faces;
texture_file_types::format m_source_file_type;
mutable dynamic_string m_last_error;
inline void clear_last_error() const { m_last_error.clear(); }
inline void set_last_error(const char* p) const { m_last_error = p; }
void free_all_mips();
bool read_regular_image(data_stream_serializer& serializer);
bool write_regular_image(const char* pFilename, uint32 image_write_flags);
bool read_dds_internal(data_stream_serializer& serializer);
void print_crn_comp_params(const crn_comp_params& p);
bool write_comp_texture(const char* pFilename, const crn_comp_params& comp_params, uint32* pActual_quality_level, float* pActual_bitrate);
void change_dxt1_to_dxt1a();
bool flip_y_helper();
};
inline void swap(mipmapped_texture& a, mipmapped_texture& b) {
a.swap(b);
}
} // namespace crnlib
crnlib/crn_packed_uint.h
+65 -77
View File
@@ -2,90 +2,78 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
namespace crnlib namespace crnlib {
{ template <unsigned int N>
template<unsigned int N> struct packed_uint {
struct packed_uint inline packed_uint() {}
{
inline packed_uint() { }
inline packed_uint(unsigned int val) { *this = val; } inline packed_uint(unsigned int val) { *this = val; }
inline packed_uint(const packed_uint& other) { *this = other; } inline packed_uint(const packed_uint& other) { *this = other; }
inline packed_uint& operator= (const packed_uint& rhs) inline packed_uint& operator=(const packed_uint& rhs) {
{ if (this != &rhs)
if (this != &rhs) memcpy(m_buf, rhs.m_buf, sizeof(m_buf));
memcpy(m_buf, rhs.m_buf, sizeof(m_buf)); return *this;
return *this; }
}
inline packed_uint& operator= (unsigned int val) inline packed_uint& operator=(unsigned int val) {
{
#ifdef CRNLIB_BUILD_DEBUG #ifdef CRNLIB_BUILD_DEBUG
if (N == 1) if (N == 1) {
{ CRNLIB_ASSERT(val <= 0xFFU);
CRNLIB_ASSERT(val <= 0xFFU); } else if (N == 2) {
} CRNLIB_ASSERT(val <= 0xFFFFU);
else if (N == 2) } else if (N == 3) {
{ CRNLIB_ASSERT(val <= 0xFFFFFFU);
CRNLIB_ASSERT(val <= 0xFFFFU); }
} #endif
else if (N == 3)
{
CRNLIB_ASSERT(val <= 0xFFFFFFU);
}
#endif
val <<= (8U * (4U - N));
for (unsigned int i = 0; i < N; i++) val <<= (8U * (4U - N));
{
m_buf[i] = static_cast<unsigned char>(val >> 24U);
val <<= 8U;
}
return *this; for (unsigned int i = 0; i < N; i++) {
} m_buf[i] = static_cast<unsigned char>(val >> 24U);
val <<= 8U;
}
inline operator unsigned int() const return *this;
{ }
switch (N)
{
case 1: return m_buf[0];
case 2: return (m_buf[0] << 8U) | m_buf[1];
case 3: return (m_buf[0] << 16U) | (m_buf[1] << 8U) | (m_buf[2]);
default: return (m_buf[0] << 24U) | (m_buf[1] << 16U) | (m_buf[2] << 8U) | (m_buf[3]);
}
}
unsigned char m_buf[N]; inline operator unsigned int() const {
}; switch (N) {
template<typename T> case 1:
class packed_value return m_buf[0];
{ case 2:
public: return (m_buf[0] << 8U) | m_buf[1];
packed_value() { } case 3:
packed_value(T val) { *this = val; } return (m_buf[0] << 16U) | (m_buf[1] << 8U) | (m_buf[2]);
default:
return (m_buf[0] << 24U) | (m_buf[1] << 16U) | (m_buf[2] << 8U) | (m_buf[3]);
}
}
inline operator T() const unsigned char m_buf[N];
{ };
T result = 0; template <typename T>
for (int i = sizeof(T) - 1; i >= 0; i--) class packed_value {
result = static_cast<T>((result << 8) | m_bytes[i]); public:
return result; packed_value() {}
} packed_value(T val) { *this = val; }
packed_value& operator= (T val)
{ inline operator T() const {
for (int i = 0; i < sizeof(T); i++) T result = 0;
{ for (int i = sizeof(T) - 1; i >= 0; i--)
m_bytes[i] = static_cast<uint8>(val); result = static_cast<T>((result << 8) | m_bytes[i]);
val >>= 8; return result;
} }
return *this; packed_value& operator=(T val) {
} for (int i = 0; i < sizeof(T); i++) {
private: m_bytes[i] = static_cast<uint8>(val);
uint8 m_bytes[sizeof(T)]; val >>= 8;
}; }
} // namespace crnlib return *this;
}
private:
uint8 m_bytes[sizeof(T)];
};
} // namespace crnlib
crnlib/crn_pixel_format.cpp
+316 -263
View File
@@ -4,277 +4,330 @@
#include "crn_pixel_format.h" #include "crn_pixel_format.h"
#include "crn_image.h" #include "crn_image.h"
namespace crnlib namespace crnlib {
{ namespace pixel_format_helpers {
namespace pixel_format_helpers const pixel_format g_all_pixel_formats[] =
{ {
const pixel_format g_all_pixel_formats[] = PIXEL_FMT_DXT1,
{ PIXEL_FMT_DXT2,
PIXEL_FMT_DXT1, PIXEL_FMT_DXT3,
PIXEL_FMT_DXT2, PIXEL_FMT_DXT4,
PIXEL_FMT_DXT3, PIXEL_FMT_DXT5,
PIXEL_FMT_DXT4, PIXEL_FMT_3DC,
PIXEL_FMT_DXT5, PIXEL_FMT_DXN,
PIXEL_FMT_3DC, PIXEL_FMT_DXT5A,
PIXEL_FMT_DXN, PIXEL_FMT_DXT5_CCxY,
PIXEL_FMT_DXT5A, PIXEL_FMT_DXT5_xGxR,
PIXEL_FMT_DXT5_CCxY, PIXEL_FMT_DXT5_xGBR,
PIXEL_FMT_DXT5_xGxR, PIXEL_FMT_DXT5_AGBR,
PIXEL_FMT_DXT5_xGBR, PIXEL_FMT_DXT1A,
PIXEL_FMT_DXT5_AGBR, PIXEL_FMT_ETC1,
PIXEL_FMT_DXT1A, PIXEL_FMT_ETC2,
PIXEL_FMT_ETC1, PIXEL_FMT_ETC2A,
PIXEL_FMT_R8G8B8, PIXEL_FMT_ETC1S,
PIXEL_FMT_L8, PIXEL_FMT_ETC2AS,
PIXEL_FMT_A8, PIXEL_FMT_R8G8B8,
PIXEL_FMT_A8L8, PIXEL_FMT_L8,
PIXEL_FMT_A8R8G8B8 PIXEL_FMT_A8,
}; PIXEL_FMT_A8L8,
PIXEL_FMT_A8R8G8B8};
uint get_num_formats() uint get_num_formats() {
{ return sizeof(g_all_pixel_formats) / sizeof(g_all_pixel_formats[0]);
return sizeof(g_all_pixel_formats) / sizeof(g_all_pixel_formats[0]); }
}
pixel_format get_pixel_format_by_index(uint index) pixel_format get_pixel_format_by_index(uint index) {
{ CRNLIB_ASSERT(index < get_num_formats());
CRNLIB_ASSERT(index < get_num_formats()); return g_all_pixel_formats[index];
return g_all_pixel_formats[index]; }
}
const char* get_pixel_format_string(pixel_format fmt) const char* get_pixel_format_string(pixel_format fmt) {
{ switch (fmt) {
switch (fmt) case PIXEL_FMT_INVALID:
{ return "INVALID";
case PIXEL_FMT_INVALID: return "INVALID"; case PIXEL_FMT_DXT1:
case PIXEL_FMT_DXT1: return "DXT1"; return "DXT1";
case PIXEL_FMT_DXT1A: return "DXT1A"; case PIXEL_FMT_DXT1A:
case PIXEL_FMT_DXT2: return "DXT2"; return "DXT1A";
case PIXEL_FMT_DXT3: return "DXT3"; case PIXEL_FMT_DXT2:
case PIXEL_FMT_DXT4: return "DXT4"; return "DXT2";
case PIXEL_FMT_DXT5: return "DXT5"; case PIXEL_FMT_DXT3:
case PIXEL_FMT_3DC: return "3DC"; return "DXT3";
case PIXEL_FMT_DXN: return "DXN"; case PIXEL_FMT_DXT4:
case PIXEL_FMT_DXT5A: return "DXT5A"; return "DXT4";
case PIXEL_FMT_DXT5_CCxY: return "DXT5_CCxY"; case PIXEL_FMT_DXT5:
case PIXEL_FMT_DXT5_xGxR: return "DXT5_xGxR"; return "DXT5";
case PIXEL_FMT_DXT5_xGBR: return "DXT5_xGBR"; case PIXEL_FMT_3DC:
case PIXEL_FMT_DXT5_AGBR: return "DXT5_AGBR"; return "3DC";
case PIXEL_FMT_ETC1: return "ETC1"; case PIXEL_FMT_DXN:
case PIXEL_FMT_R8G8B8: return "R8G8B8"; return "DXN";
case PIXEL_FMT_A8R8G8B8: return "A8R8G8B8"; case PIXEL_FMT_DXT5A:
case PIXEL_FMT_A8: return "A8"; return "DXT5A";
case PIXEL_FMT_L8: return "L8"; case PIXEL_FMT_DXT5_CCxY:
case PIXEL_FMT_A8L8: return "A8L8"; return "DXT5_CCxY";
default: break; case PIXEL_FMT_DXT5_xGxR:
} return "DXT5_xGxR";
CRNLIB_ASSERT(false); case PIXEL_FMT_DXT5_xGBR:
return "?"; return "DXT5_xGBR";
} case PIXEL_FMT_DXT5_AGBR:
return "DXT5_AGBR";
case PIXEL_FMT_ETC1:
return "ETC1";
case PIXEL_FMT_ETC2:
return "ETC2";
case PIXEL_FMT_ETC2A:
return "ETC2A";
case PIXEL_FMT_ETC1S:
return "ETC1S";
case PIXEL_FMT_ETC2AS:
return "ETC2AS";
case PIXEL_FMT_R8G8B8:
return "R8G8B8";
case PIXEL_FMT_A8R8G8B8:
return "A8R8G8B8";
case PIXEL_FMT_A8:
return "A8";
case PIXEL_FMT_L8:
return "L8";
case PIXEL_FMT_A8L8:
return "A8L8";
default:
break;
}
CRNLIB_ASSERT(false);
return "?";
}
const char* get_crn_format_string(crn_format fmt) const char* get_crn_format_string(crn_format fmt) {
{ switch (fmt) {
switch (fmt) case cCRNFmtDXT1:
{ return "DXT1";
case cCRNFmtDXT1: return "DXT1"; case cCRNFmtDXT3:
case cCRNFmtDXT3: return "DXT3"; return "DXT3";
case cCRNFmtDXT5: return "DXT5"; case cCRNFmtDXT5:
case cCRNFmtDXT5_CCxY: return "DXT5_CCxY"; return "DXT5";
case cCRNFmtDXT5_xGBR: return "DXT5_xGBR"; case cCRNFmtDXT5_CCxY:
case cCRNFmtDXT5_AGBR: return "DXT5_AGBR"; return "DXT5_CCxY";
case cCRNFmtDXT5_xGxR: return "DXT5_xGxR"; case cCRNFmtDXT5_xGBR:
case cCRNFmtDXN_XY: return "DXN_XY"; return "DXT5_xGBR";
case cCRNFmtDXN_YX: return "DXN_YX"; case cCRNFmtDXT5_AGBR:
case cCRNFmtDXT5A: return "DXT5A"; return "DXT5_AGBR";
case cCRNFmtETC1: return "ETC1"; case cCRNFmtDXT5_xGxR:
default: break; return "DXT5_xGxR";
} case cCRNFmtDXN_XY:
CRNLIB_ASSERT(false); return "DXN_XY";
return "?"; case cCRNFmtDXN_YX:
} return "DXN_YX";
case cCRNFmtDXT5A:
return "DXT5A";
case cCRNFmtETC1:
return "ETC1";
case cCRNFmtETC2:
return "ETC2";
case cCRNFmtETC2A:
return "ETC2A";
case cCRNFmtETC1S:
return "ETC1S";
case cCRNFmtETC2AS:
return "ETC2AS";
default:
break;
}
CRNLIB_ASSERT(false);
return "?";
}
component_flags get_component_flags(pixel_format fmt) component_flags get_component_flags(pixel_format fmt) {
{ // These flags are for *uncooked* pixels, i.e. after after adding Z to DXN maps, or converting YCC maps to RGB, etc.
// These flags are for *uncooked* pixels, i.e. after after adding Z to DXN maps, or converting YCC maps to RGB, etc.
uint flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid | cCompFlagGrayscale; uint flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid | cCompFlagGrayscale;
switch (fmt) switch (fmt) {
{ case PIXEL_FMT_DXT1:
case PIXEL_FMT_DXT1: case PIXEL_FMT_ETC1:
case PIXEL_FMT_ETC1: case PIXEL_FMT_ETC2:
{ case PIXEL_FMT_ETC1S: {
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid; flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid;
break; break;
} }
case PIXEL_FMT_DXT1A: case PIXEL_FMT_DXT1A: {
{ flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid;
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid; break;
break; }
} case PIXEL_FMT_DXT2:
case PIXEL_FMT_DXT2: case PIXEL_FMT_DXT3: {
case PIXEL_FMT_DXT3: flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid;
{ break;
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid; }
break; case PIXEL_FMT_DXT4:
} case PIXEL_FMT_DXT5:
case PIXEL_FMT_DXT4: case PIXEL_FMT_ETC2A:
case PIXEL_FMT_DXT5: case PIXEL_FMT_ETC2AS: {
{ flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid;
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid; break;
break; }
} case PIXEL_FMT_DXT5A: {
case PIXEL_FMT_DXT5A: flags = cCompFlagAValid;
{ break;
flags = cCompFlagAValid; }
break; case PIXEL_FMT_DXT5_CCxY: {
} flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagLumaChroma;
case PIXEL_FMT_DXT5_CCxY: break;
{ }
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagLumaChroma; case PIXEL_FMT_DXT5_xGBR: {
break; flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagNormalMap;
} break;
case PIXEL_FMT_DXT5_xGBR: }
{ case PIXEL_FMT_DXT5_AGBR: {
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagNormalMap; flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid | cCompFlagNormalMap;
break; break;
} }
case PIXEL_FMT_DXT5_AGBR: case PIXEL_FMT_DXT5_xGxR: {
{ flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagNormalMap;
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid | cCompFlagNormalMap; break;
break; }
} case PIXEL_FMT_3DC: {
case PIXEL_FMT_DXT5_xGxR: flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagNormalMap;
{ break;
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagNormalMap; }
break; case PIXEL_FMT_DXN: {
} flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagNormalMap;
case PIXEL_FMT_3DC: break;
{ }
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagNormalMap; case PIXEL_FMT_R8G8B8: {
break; flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid;
} break;
case PIXEL_FMT_DXN: }
{ case PIXEL_FMT_A8R8G8B8: {
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagNormalMap; flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid;
break; break;
} }
case PIXEL_FMT_R8G8B8: case PIXEL_FMT_A8: {
{ flags = cCompFlagAValid;
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid; break;
break; }
} case PIXEL_FMT_L8: {
case PIXEL_FMT_A8R8G8B8: flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagGrayscale;
{ break;
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid; }
break; case PIXEL_FMT_A8L8: {
} flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid | cCompFlagGrayscale;
case PIXEL_FMT_A8: break;
{ }
flags = cCompFlagAValid; default: {
break; CRNLIB_ASSERT(0);
} break;
case PIXEL_FMT_L8: }
{ }
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagGrayscale; return static_cast<component_flags>(flags);
break; }
}
case PIXEL_FMT_A8L8:
{
flags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid | cCompFlagGrayscale;
break;
}
default:
{
CRNLIB_ASSERT(0);
break;
}
}
return static_cast<component_flags>(flags);
}
crn_format convert_pixel_format_to_best_crn_format(pixel_format crn_fmt) crn_format convert_pixel_format_to_best_crn_format(pixel_format crn_fmt) {
{ crn_format fmt = cCRNFmtDXT1;
crn_format fmt = cCRNFmtDXT1; switch (crn_fmt) {
switch (crn_fmt) case PIXEL_FMT_DXT1:
{ case PIXEL_FMT_DXT1A:
case PIXEL_FMT_DXT1: fmt = cCRNFmtDXT1;
case PIXEL_FMT_DXT1A: break;
fmt = cCRNFmtDXT1; case PIXEL_FMT_DXT2:
break; case PIXEL_FMT_DXT3:
case PIXEL_FMT_DXT2: case PIXEL_FMT_DXT4:
case PIXEL_FMT_DXT3: case PIXEL_FMT_DXT5:
case PIXEL_FMT_DXT4: fmt = cCRNFmtDXT5;
case PIXEL_FMT_DXT5: break;
fmt = cCRNFmtDXT5; case PIXEL_FMT_3DC:
break; fmt = cCRNFmtDXN_YX;
case PIXEL_FMT_3DC: break;
fmt = cCRNFmtDXN_YX; case PIXEL_FMT_DXN:
break; fmt = cCRNFmtDXN_XY;
case PIXEL_FMT_DXN: break;
fmt = cCRNFmtDXN_XY; case PIXEL_FMT_DXT5A:
break; fmt = cCRNFmtDXT5A;
case PIXEL_FMT_DXT5A: break;
fmt = cCRNFmtDXT5A; case PIXEL_FMT_R8G8B8:
break; case PIXEL_FMT_L8:
case PIXEL_FMT_R8G8B8: fmt = cCRNFmtDXT1;
case PIXEL_FMT_L8: break;
fmt = cCRNFmtDXT1; case PIXEL_FMT_A8R8G8B8:
break; case PIXEL_FMT_A8:
case PIXEL_FMT_A8R8G8B8: case PIXEL_FMT_A8L8:
case PIXEL_FMT_A8: fmt = cCRNFmtDXT5;
case PIXEL_FMT_A8L8: break;
fmt = cCRNFmtDXT5; case PIXEL_FMT_DXT5_CCxY:
break; fmt = cCRNFmtDXT5_CCxY;
case PIXEL_FMT_DXT5_CCxY: break;
fmt = cCRNFmtDXT5_CCxY; case PIXEL_FMT_DXT5_xGBR:
break; fmt = cCRNFmtDXT5_xGBR;
case PIXEL_FMT_DXT5_xGBR: break;
fmt = cCRNFmtDXT5_xGBR; case PIXEL_FMT_DXT5_AGBR:
break; fmt = cCRNFmtDXT5_AGBR;
case PIXEL_FMT_DXT5_AGBR: break;
fmt = cCRNFmtDXT5_AGBR; case PIXEL_FMT_DXT5_xGxR:
break; fmt = cCRNFmtDXT5_xGxR;
case PIXEL_FMT_DXT5_xGxR: break;
fmt = cCRNFmtDXT5_xGxR; case PIXEL_FMT_ETC1:
break; fmt = cCRNFmtETC1;
case PIXEL_FMT_ETC1: break;
fmt = cCRNFmtETC1; case PIXEL_FMT_ETC2:
break; fmt = cCRNFmtETC2;
default: break;
{ case PIXEL_FMT_ETC2A:
CRNLIB_ASSERT(false); fmt = cCRNFmtETC2A;
break; break;
} case PIXEL_FMT_ETC1S:
} fmt = cCRNFmtETC1S;
return fmt; break;
} case PIXEL_FMT_ETC2AS:
fmt = cCRNFmtETC2AS;
break;
default: {
CRNLIB_ASSERT(false);
break;
}
}
return fmt;
}
pixel_format convert_crn_format_to_pixel_format(crn_format fmt) pixel_format convert_crn_format_to_pixel_format(crn_format fmt) {
{ switch (fmt) {
switch (fmt) case cCRNFmtDXT1:
{ return PIXEL_FMT_DXT1;
case cCRNFmtDXT1: return PIXEL_FMT_DXT1; case cCRNFmtDXT3:
case cCRNFmtDXT3: return PIXEL_FMT_DXT3; return PIXEL_FMT_DXT3;
case cCRNFmtDXT5: return PIXEL_FMT_DXT5; case cCRNFmtDXT5:
case cCRNFmtDXT5_CCxY: return PIXEL_FMT_DXT5_CCxY; return PIXEL_FMT_DXT5;
case cCRNFmtDXT5_xGxR: return PIXEL_FMT_DXT5_xGxR; case cCRNFmtDXT5_CCxY:
case cCRNFmtDXT5_xGBR: return PIXEL_FMT_DXT5_xGBR; return PIXEL_FMT_DXT5_CCxY;
case cCRNFmtDXT5_AGBR: return PIXEL_FMT_DXT5_AGBR; case cCRNFmtDXT5_xGxR:
case cCRNFmtDXN_XY: return PIXEL_FMT_DXN; return PIXEL_FMT_DXT5_xGxR;
case cCRNFmtDXN_YX: return PIXEL_FMT_3DC; case cCRNFmtDXT5_xGBR:
case cCRNFmtDXT5A: return PIXEL_FMT_DXT5A; return PIXEL_FMT_DXT5_xGBR;
case cCRNFmtETC1: return PIXEL_FMT_ETC1; case cCRNFmtDXT5_AGBR:
default: return PIXEL_FMT_DXT5_AGBR;
{ case cCRNFmtDXN_XY:
CRNLIB_ASSERT(false); return PIXEL_FMT_DXN;
break; case cCRNFmtDXN_YX:
} return PIXEL_FMT_3DC;
} case cCRNFmtDXT5A:
return PIXEL_FMT_DXT5A;
case cCRNFmtETC1:
return PIXEL_FMT_ETC1;
case cCRNFmtETC2:
return PIXEL_FMT_ETC2;
case cCRNFmtETC2A:
return PIXEL_FMT_ETC2A;
case cCRNFmtETC1S:
return PIXEL_FMT_ETC1S;
case cCRNFmtETC2AS:
return PIXEL_FMT_ETC2AS;
default: {
CRNLIB_ASSERT(false);
break;
}
}
return PIXEL_FMT_INVALID; return PIXEL_FMT_INVALID;
} }
} // namespace pixel_format } // namespace pixel_format
} // namespace crnlib
} // namespace crnlib
crnlib/crn_pixel_format.h
+322 -254
View File
@@ -5,280 +5,348 @@
#include "../inc/crnlib.h" #include "../inc/crnlib.h"
#include "../inc/dds_defs.h" #include "../inc/dds_defs.h"
namespace crnlib namespace crnlib {
{ namespace pixel_format_helpers {
namespace pixel_format_helpers uint get_num_formats();
{ pixel_format get_pixel_format_by_index(uint index);
uint get_num_formats();
pixel_format get_pixel_format_by_index(uint index);
const char* get_pixel_format_string(pixel_format fmt); const char* get_pixel_format_string(pixel_format fmt);
const char* get_crn_format_string(crn_format fmt); const char* get_crn_format_string(crn_format fmt);
inline bool is_grayscale(pixel_format fmt) inline bool is_grayscale(pixel_format fmt) {
{ switch (fmt) {
switch (fmt) case PIXEL_FMT_L8:
{ case PIXEL_FMT_A8L8:
case PIXEL_FMT_L8: return true;
case PIXEL_FMT_A8L8: default:
return true; break;
default: break; }
} return false;
return false; }
}
inline bool is_dxt1(pixel_format fmt) inline bool is_dxt1(pixel_format fmt) {
{ return (fmt == PIXEL_FMT_DXT1) || (fmt == PIXEL_FMT_DXT1A);
return (fmt == PIXEL_FMT_DXT1) || (fmt == PIXEL_FMT_DXT1A); }
}
// has_alpha() should probably be called "has_opacity()" - it indicates if the format encodes opacity // has_alpha() should probably be called "has_opacity()" - it indicates if the format encodes opacity
// because some swizzled DXT5 formats do not encode opacity. // because some swizzled DXT5 formats do not encode opacity.
inline bool has_alpha(pixel_format fmt) inline bool has_alpha(pixel_format fmt) {
{ switch (fmt) {
switch (fmt) case PIXEL_FMT_DXT1A:
{ case PIXEL_FMT_DXT2:
case PIXEL_FMT_DXT1A: case PIXEL_FMT_DXT3:
case PIXEL_FMT_DXT2: case PIXEL_FMT_DXT4:
case PIXEL_FMT_DXT3: case PIXEL_FMT_DXT5:
case PIXEL_FMT_DXT4: case PIXEL_FMT_DXT5A:
case PIXEL_FMT_DXT5: case PIXEL_FMT_A8R8G8B8:
case PIXEL_FMT_DXT5A: case PIXEL_FMT_A8:
case PIXEL_FMT_A8R8G8B8: case PIXEL_FMT_A8L8:
case PIXEL_FMT_A8: case PIXEL_FMT_DXT5_AGBR:
case PIXEL_FMT_A8L8: case PIXEL_FMT_ETC2A:
case PIXEL_FMT_DXT5_AGBR: case PIXEL_FMT_ETC2AS:
return true; return true;
default: break; default:
} break;
return false; }
} return false;
}
inline bool is_alpha_only(pixel_format fmt) inline bool is_alpha_only(pixel_format fmt) {
{ switch (fmt) {
switch (fmt) case PIXEL_FMT_A8:
{ case PIXEL_FMT_DXT5A:
case PIXEL_FMT_A8: return true;
case PIXEL_FMT_DXT5A: default:
return true; break;
default: break; }
} return false;
return false; }
}
inline bool is_normal_map(pixel_format fmt) inline bool is_normal_map(pixel_format fmt) {
{ switch (fmt) {
switch (fmt) case PIXEL_FMT_3DC:
{ case PIXEL_FMT_DXN:
case PIXEL_FMT_3DC: case PIXEL_FMT_DXT5_xGBR:
case PIXEL_FMT_DXN: case PIXEL_FMT_DXT5_xGxR:
case PIXEL_FMT_DXT5_xGBR: case PIXEL_FMT_DXT5_AGBR:
case PIXEL_FMT_DXT5_xGxR: return true;
case PIXEL_FMT_DXT5_AGBR: default:
return true; break;
default: break; }
} return false;
return false; }
}
inline int is_dxt(pixel_format fmt) inline int is_dxt(pixel_format fmt) {
{ switch (fmt) {
switch (fmt) case PIXEL_FMT_DXT1:
{ case PIXEL_FMT_DXT1A:
case PIXEL_FMT_DXT1: case PIXEL_FMT_DXT2:
case PIXEL_FMT_DXT1A: case PIXEL_FMT_DXT3:
case PIXEL_FMT_DXT2: case PIXEL_FMT_DXT4:
case PIXEL_FMT_DXT3: case PIXEL_FMT_DXT5:
case PIXEL_FMT_DXT4: case PIXEL_FMT_3DC:
case PIXEL_FMT_DXT5: case PIXEL_FMT_DXT5A:
case PIXEL_FMT_3DC: case PIXEL_FMT_DXN:
case PIXEL_FMT_DXT5A: case PIXEL_FMT_DXT5_CCxY:
case PIXEL_FMT_DXN: case PIXEL_FMT_DXT5_xGxR:
case PIXEL_FMT_DXT5_CCxY: case PIXEL_FMT_DXT5_xGBR:
case PIXEL_FMT_DXT5_xGxR: case PIXEL_FMT_DXT5_AGBR:
case PIXEL_FMT_DXT5_xGBR: case PIXEL_FMT_ETC1:
case PIXEL_FMT_DXT5_AGBR: case PIXEL_FMT_ETC2:
case PIXEL_FMT_ETC1: case PIXEL_FMT_ETC2A:
return true; case PIXEL_FMT_ETC1S:
default: break; case PIXEL_FMT_ETC2AS:
} return true;
return false; default:
} break;
}
return false;
}
inline int get_fundamental_format(pixel_format fmt) inline int get_fundamental_format(pixel_format fmt) {
{ switch (fmt) {
switch (fmt) case PIXEL_FMT_DXT1A:
{ return PIXEL_FMT_DXT1;
case PIXEL_FMT_DXT1A: case PIXEL_FMT_DXT5_CCxY:
return PIXEL_FMT_DXT1; case PIXEL_FMT_DXT5_xGxR:
case PIXEL_FMT_DXT5_CCxY: case PIXEL_FMT_DXT5_xGBR:
case PIXEL_FMT_DXT5_xGxR: case PIXEL_FMT_DXT5_AGBR:
case PIXEL_FMT_DXT5_xGBR: return PIXEL_FMT_DXT5;
case PIXEL_FMT_DXT5_AGBR: default:
return PIXEL_FMT_DXT5; break;
default: break; }
} return fmt;
return fmt; }
}
inline dxt_format get_dxt_format(pixel_format fmt) inline dxt_format get_dxt_format(pixel_format fmt) {
{ switch (fmt) {
switch (fmt) case PIXEL_FMT_DXT1:
{ return cDXT1;
case PIXEL_FMT_DXT1: return cDXT1; case PIXEL_FMT_DXT1A:
case PIXEL_FMT_DXT1A: return cDXT1A; return cDXT1A;
case PIXEL_FMT_DXT2: return cDXT3; case PIXEL_FMT_DXT2:
case PIXEL_FMT_DXT3: return cDXT3; return cDXT3;
case PIXEL_FMT_DXT4: return cDXT5; case PIXEL_FMT_DXT3:
case PIXEL_FMT_DXT5: return cDXT5; return cDXT3;
case PIXEL_FMT_3DC: return cDXN_YX; case PIXEL_FMT_DXT4:
case PIXEL_FMT_DXT5A: return cDXT5A; return cDXT5;
case PIXEL_FMT_DXN: return cDXN_XY; case PIXEL_FMT_DXT5:
case PIXEL_FMT_DXT5_CCxY: return cDXT5; return cDXT5;
case PIXEL_FMT_DXT5_xGxR: return cDXT5; case PIXEL_FMT_3DC:
case PIXEL_FMT_DXT5_xGBR: return cDXT5; return cDXN_YX;
case PIXEL_FMT_DXT5_AGBR: return cDXT5; case PIXEL_FMT_DXT5A:
case PIXEL_FMT_ETC1: return cETC1; return cDXT5A;
default: break; case PIXEL_FMT_DXN:
} return cDXN_XY;
return cDXTInvalid; case PIXEL_FMT_DXT5_CCxY:
} return cDXT5;
case PIXEL_FMT_DXT5_xGxR:
return cDXT5;
case PIXEL_FMT_DXT5_xGBR:
return cDXT5;
case PIXEL_FMT_DXT5_AGBR:
return cDXT5;
case PIXEL_FMT_ETC1:
return cETC1;
case PIXEL_FMT_ETC2:
return cETC2;
case PIXEL_FMT_ETC2A:
return cETC2A;
case PIXEL_FMT_ETC1S:
return cETC1S;
case PIXEL_FMT_ETC2AS:
return cETC2AS;
default:
break;
}
return cDXTInvalid;
}
inline pixel_format from_dxt_format(dxt_format dxt_fmt) inline pixel_format from_dxt_format(dxt_format dxt_fmt) {
{ switch (dxt_fmt) {
switch (dxt_fmt) case cDXT1:
{ return PIXEL_FMT_DXT1;
case cDXT1: case cDXT1A:
return PIXEL_FMT_DXT1; return PIXEL_FMT_DXT1A;
case cDXT1A: case cDXT3:
return PIXEL_FMT_DXT1A; return PIXEL_FMT_DXT3;
case cDXT3: case cDXT5:
return PIXEL_FMT_DXT3; return PIXEL_FMT_DXT5;
case cDXT5: case cDXN_XY:
return PIXEL_FMT_DXT5; return PIXEL_FMT_DXN;
case cDXN_XY: case cDXN_YX:
return PIXEL_FMT_DXN; return PIXEL_FMT_3DC;
case cDXN_YX: case cDXT5A:
return PIXEL_FMT_3DC; return PIXEL_FMT_DXT5A;
case cDXT5A: case cETC1:
return PIXEL_FMT_DXT5A; return PIXEL_FMT_ETC1;
case cETC1: case cETC2:
return PIXEL_FMT_ETC1; return PIXEL_FMT_ETC2;
default: break; case cETC2A:
} return PIXEL_FMT_ETC2A;
CRNLIB_ASSERT(false); case cETC1S:
return PIXEL_FMT_INVALID; return PIXEL_FMT_ETC1S;
} case cETC2AS:
return PIXEL_FMT_ETC2AS;
default:
break;
}
CRNLIB_ASSERT(false);
return PIXEL_FMT_INVALID;
}
inline bool is_pixel_format_non_srgb(pixel_format fmt) inline bool is_pixel_format_non_srgb(pixel_format fmt) {
{ switch (fmt) {
switch (fmt) case PIXEL_FMT_3DC:
{ case PIXEL_FMT_DXN:
case PIXEL_FMT_3DC: case PIXEL_FMT_DXT5A:
case PIXEL_FMT_DXN: case PIXEL_FMT_DXT5_CCxY:
case PIXEL_FMT_DXT5A: case PIXEL_FMT_DXT5_xGxR:
case PIXEL_FMT_DXT5_CCxY: case PIXEL_FMT_DXT5_xGBR:
case PIXEL_FMT_DXT5_xGxR: case PIXEL_FMT_DXT5_AGBR:
case PIXEL_FMT_DXT5_xGBR: return true;
case PIXEL_FMT_DXT5_AGBR: default:
return true; break;
default: break; }
} return false;
return false; }
}
inline bool is_crn_format_non_srgb(crn_format fmt) inline bool is_crn_format_non_srgb(crn_format fmt) {
{ switch (fmt) {
switch (fmt) case cCRNFmtDXN_XY:
{ case cCRNFmtDXN_YX:
case cCRNFmtDXN_XY: case cCRNFmtDXT5A:
case cCRNFmtDXN_YX: case cCRNFmtDXT5_CCxY:
case cCRNFmtDXT5A: case cCRNFmtDXT5_xGxR:
case cCRNFmtDXT5_CCxY: case cCRNFmtDXT5_xGBR:
case cCRNFmtDXT5_xGxR: case cCRNFmtDXT5_AGBR:
case cCRNFmtDXT5_xGBR: return true;
case cCRNFmtDXT5_AGBR: default:
return true; break;
default: break; }
} return false;
return false; }
}
inline uint get_bpp(pixel_format fmt) inline uint get_bpp(pixel_format fmt) {
{ switch (fmt) {
switch (fmt) case PIXEL_FMT_DXT1:
{ return 4;
case PIXEL_FMT_DXT1: return 4; case PIXEL_FMT_DXT1A:
case PIXEL_FMT_DXT1A: return 4; return 4;
case PIXEL_FMT_ETC1: return 4; case PIXEL_FMT_ETC1:
case PIXEL_FMT_DXT2: return 8; return 4;
case PIXEL_FMT_DXT3: return 8; case PIXEL_FMT_ETC2:
case PIXEL_FMT_DXT4: return 8; return 4;
case PIXEL_FMT_DXT5: return 8; case PIXEL_FMT_ETC2A:
case PIXEL_FMT_3DC: return 8; return 8;
case PIXEL_FMT_DXT5A: return 4; case PIXEL_FMT_ETC1S:
case PIXEL_FMT_R8G8B8: return 24; return 4;
case PIXEL_FMT_A8R8G8B8: return 32; case PIXEL_FMT_ETC2AS:
case PIXEL_FMT_A8: return 8; return 8;
case PIXEL_FMT_L8: return 8; case PIXEL_FMT_DXT2:
case PIXEL_FMT_A8L8: return 16; return 8;
case PIXEL_FMT_DXN: return 8; case PIXEL_FMT_DXT3:
case PIXEL_FMT_DXT5_CCxY: return 8; return 8;
case PIXEL_FMT_DXT5_xGxR: return 8; case PIXEL_FMT_DXT4:
case PIXEL_FMT_DXT5_xGBR: return 8; return 8;
case PIXEL_FMT_DXT5_AGBR: return 8; case PIXEL_FMT_DXT5:
default: break; return 8;
} case PIXEL_FMT_3DC:
CRNLIB_ASSERT(false); return 8;
return 0; case PIXEL_FMT_DXT5A:
}; return 4;
case PIXEL_FMT_R8G8B8:
return 24;
case PIXEL_FMT_A8R8G8B8:
return 32;
case PIXEL_FMT_A8:
return 8;
case PIXEL_FMT_L8:
return 8;
case PIXEL_FMT_A8L8:
return 16;
case PIXEL_FMT_DXN:
return 8;
case PIXEL_FMT_DXT5_CCxY:
return 8;
case PIXEL_FMT_DXT5_xGxR:
return 8;
case PIXEL_FMT_DXT5_xGBR:
return 8;
case PIXEL_FMT_DXT5_AGBR:
return 8;
default:
break;
}
CRNLIB_ASSERT(false);
return 0;
};
inline uint get_dxt_bytes_per_block(pixel_format fmt) inline uint get_dxt_bytes_per_block(pixel_format fmt) {
{ switch (fmt) {
switch (fmt) case PIXEL_FMT_DXT1:
{ return 8;
case PIXEL_FMT_DXT1: return 8; case PIXEL_FMT_DXT1A:
case PIXEL_FMT_DXT1A: return 8; return 8;
case PIXEL_FMT_DXT5A: return 8; case PIXEL_FMT_DXT5A:
case PIXEL_FMT_ETC1: return 8; return 8;
case PIXEL_FMT_DXT2: return 16; case PIXEL_FMT_ETC1:
case PIXEL_FMT_DXT3: return 16; return 8;
case PIXEL_FMT_DXT4: return 16; case PIXEL_FMT_ETC2:
case PIXEL_FMT_DXT5: return 16; return 8;
case PIXEL_FMT_3DC: return 16; case PIXEL_FMT_ETC2A:
case PIXEL_FMT_DXN: return 16; return 16;
case PIXEL_FMT_DXT5_CCxY: return 16; case PIXEL_FMT_ETC1S:
case PIXEL_FMT_DXT5_xGxR: return 16; return 8;
case PIXEL_FMT_DXT5_xGBR: return 16; case PIXEL_FMT_ETC2AS:
case PIXEL_FMT_DXT5_AGBR: return 16; return 16;
default: break; case PIXEL_FMT_DXT2:
} return 16;
CRNLIB_ASSERT(false); case PIXEL_FMT_DXT3:
return 0; return 16;
} case PIXEL_FMT_DXT4:
return 16;
case PIXEL_FMT_DXT5:
return 16;
case PIXEL_FMT_3DC:
return 16;
case PIXEL_FMT_DXN:
return 16;
case PIXEL_FMT_DXT5_CCxY:
return 16;
case PIXEL_FMT_DXT5_xGxR:
return 16;
case PIXEL_FMT_DXT5_xGBR:
return 16;
case PIXEL_FMT_DXT5_AGBR:
return 16;
default:
break;
}
CRNLIB_ASSERT(false);
return 0;
}
enum component_flags enum component_flags {
{ cCompFlagRValid = 1,
cCompFlagRValid = 1, cCompFlagGValid = 2,
cCompFlagGValid = 2, cCompFlagBValid = 4,
cCompFlagBValid = 4, cCompFlagAValid = 8,
cCompFlagAValid = 8,
cCompFlagGrayscale = 16, cCompFlagGrayscale = 16,
cCompFlagNormalMap = 32, cCompFlagNormalMap = 32,
cCompFlagLumaChroma = 64, cCompFlagLumaChroma = 64,
cDefaultCompFlags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid cDefaultCompFlags = cCompFlagRValid | cCompFlagGValid | cCompFlagBValid | cCompFlagAValid
}; };
component_flags get_component_flags(pixel_format fmt); component_flags get_component_flags(pixel_format fmt);
crn_format convert_pixel_format_to_best_crn_format(pixel_format crn_fmt); crn_format convert_pixel_format_to_best_crn_format(pixel_format crn_fmt);
pixel_format convert_crn_format_to_pixel_format(crn_format fmt); pixel_format convert_crn_format_to_pixel_format(crn_format fmt);
} // namespace pixel_format_helpers } // namespace pixel_format_helpers
} // namespace crnlib
} // namespace crnlib
crnlib/crn_platform.cpp
+45 -56
View File
@@ -6,87 +6,76 @@
#include "crn_winhdr.h" #include "crn_winhdr.h"
#endif #endif
#ifndef _MSC_VER #ifndef _MSC_VER
int sprintf_s(char *buffer, size_t sizeOfBuffer, const char *format, ...) int sprintf_s(char* buffer, size_t sizeOfBuffer, const char* format, ...) {
{ if (!sizeOfBuffer)
if (!sizeOfBuffer) return 0;
return 0;
va_list args; va_list args;
va_start(args, format); va_start(args, format);
int c = vsnprintf(buffer, sizeOfBuffer, format, args); int c = vsnprintf(buffer, sizeOfBuffer, format, args);
va_end(args); va_end(args);
buffer[sizeOfBuffer - 1] = '\0'; buffer[sizeOfBuffer - 1] = '\0';
if (c < 0) if (c < 0)
return sizeOfBuffer - 1; return sizeOfBuffer - 1;
return CRNLIB_MIN(c, (int)sizeOfBuffer - 1); return CRNLIB_MIN(c, (int)sizeOfBuffer - 1);
} }
int vsprintf_s(char *buffer, size_t sizeOfBuffer, const char *format, va_list args) int vsprintf_s(char* buffer, size_t sizeOfBuffer, const char* format, va_list args) {
{ if (!sizeOfBuffer)
if (!sizeOfBuffer) return 0;
return 0;
int c = vsnprintf(buffer, sizeOfBuffer, format, args); int c = vsnprintf(buffer, sizeOfBuffer, format, args);
buffer[sizeOfBuffer - 1] = '\0'; buffer[sizeOfBuffer - 1] = '\0';
if (c < 0) if (c < 0)
return sizeOfBuffer - 1; return sizeOfBuffer - 1;
return CRNLIB_MIN(c, (int)sizeOfBuffer - 1); return CRNLIB_MIN(c, (int)sizeOfBuffer - 1);
} }
char* strlwr(char* p) char* strlwr(char* p) {
{ char* q = p;
char *q = p; while (*q) {
while (*q) char c = *q;
{ *q++ = tolower(c);
char c = *q; }
*q++ = tolower(c); return p;
}
return p;
} }
char* strupr(char *p) char* strupr(char* p) {
{ char* q = p;
char *q = p; while (*q) {
while (*q) char c = *q;
{ *q++ = toupper(c);
char c = *q; }
*q++ = toupper(c); return p;
}
return p;
} }
#endif // __GNUC__ #endif // __GNUC__
void crnlib_debug_break(void) void crnlib_debug_break(void) {
{ CRNLIB_BREAKPOINT
CRNLIB_BREAKPOINT
} }
#if CRNLIB_USE_WIN32_API #if CRNLIB_USE_WIN32_API
#include "crn_winhdr.h" #include "crn_winhdr.h"
bool crnlib_is_debugger_present(void) bool crnlib_is_debugger_present(void) {
{ return IsDebuggerPresent() != 0;
return IsDebuggerPresent() != 0;
} }
void crnlib_output_debug_string(const char* p) void crnlib_output_debug_string(const char* p) {
{ OutputDebugStringA(p);
OutputDebugStringA(p);
} }
#else #else
bool crnlib_is_debugger_present(void) bool crnlib_is_debugger_present(void) {
{ return false;
return false;
} }
void crnlib_output_debug_string(const char* p) void crnlib_output_debug_string(const char* p) {
{ puts(p);
puts(p);
} }
#endif // CRNLIB_USE_WIN32_API #endif // CRNLIB_USE_WIN32_API
crnlib/crn_platform.h
+45 -44
View File
@@ -11,86 +11,87 @@ void crnlib_assert(const char* pExp, const char* pFile, unsigned line);
void crnlib_fail(const char* pExp, const char* pFile, unsigned line); void crnlib_fail(const char* pExp, const char* pFile, unsigned line);
#if CRNLIB_LITTLE_ENDIAN_CPU #if CRNLIB_LITTLE_ENDIAN_CPU
const bool c_crnlib_little_endian_platform = true; const bool c_crnlib_little_endian_platform = true;
#else #else
const bool c_crnlib_little_endian_platform = false; const bool c_crnlib_little_endian_platform = false;
#endif #endif
const bool c_crnlib_big_endian_platform = !c_crnlib_little_endian_platform; const bool c_crnlib_big_endian_platform = !c_crnlib_little_endian_platform;
#ifdef __GNUC__ #ifdef __GNUC__
#define crn_fopen(pDstFile, f, m) *(pDstFile) = fopen64(f, m) #define crn_fopen(pDstFile, f, m) *(pDstFile) = fopen64(f, m)
#define crn_fseek fseeko64 #define crn_fseek fseeko64
#define crn_ftell ftello64 #define crn_ftell ftello64
#elif defined( _MSC_VER ) #elif defined(_MSC_VER)
#define crn_fopen(pDstFile, f, m) fopen_s(pDstFile, f, m) #define crn_fopen(pDstFile, f, m) fopen_s(pDstFile, f, m)
#define crn_fseek _fseeki64 #define crn_fseek _fseeki64
#define crn_ftell _ftelli64 #define crn_ftell _ftelli64
#else #else
#define crn_fopen(pDstFile, f, m) *(pDstFile) = fopen(f, m) #define crn_fopen(pDstFile, f, m) *(pDstFile) = fopen(f, m)
#define crn_fseek(s, o, w) fseek(s, static_cast<long>(o), w) #define crn_fseek(s, o, w) fseek(s, static_cast<long>(o), w)
#define crn_ftell ftell #define crn_ftell ftell
#endif #endif
#if CRNLIB_USE_WIN32_API #if CRNLIB_USE_WIN32_API
#define CRNLIB_BREAKPOINT DebugBreak(); #define CRNLIB_BREAKPOINT DebugBreak();
#define CRNLIB_BUILTIN_EXPECT(c, v) c #define CRNLIB_BUILTIN_EXPECT(c, v) c
#elif defined(__GNUC__) #elif defined(__GNUC__)
#define CRNLIB_BREAKPOINT asm("int $3"); #define CRNLIB_BREAKPOINT asm("int $3");
#define CRNLIB_BUILTIN_EXPECT(c, v) __builtin_expect(c, v) #define CRNLIB_BUILTIN_EXPECT(c, v) __builtin_expect(c, v)
#else #else
#define CRNLIB_BREAKPOINT #define CRNLIB_BREAKPOINT
#define CRNLIB_BUILTIN_EXPECT(c, v) c #define CRNLIB_BUILTIN_EXPECT(c, v) c
#endif #endif
#if defined(__GNUC__) #if defined(__GNUC__)
#define CRNLIB_ALIGNED(x) __attribute__((aligned(x))) #define CRNLIB_ALIGNED(x) __attribute__((aligned(x)))
#define CRNLIB_NOINLINE __attribute__((noinline)) #define CRNLIB_NOINLINE __attribute__((noinline))
#elif defined(_MSC_VER) #elif defined(_MSC_VER)
#define CRNLIB_ALIGNED(x) __declspec(align(x)) #define CRNLIB_ALIGNED(x) __declspec(align(x))
#define CRNLIB_NOINLINE __declspec(noinline) #define CRNLIB_NOINLINE __declspec(noinline)
#else #else
#define CRNLIB_ALIGNED(x) #define CRNLIB_ALIGNED(x)
#define CRNLIB_NOINLINE #define CRNLIB_NOINLINE
#endif #endif
#define CRNLIB_GET_ALIGNMENT(v) ((!sizeof(v)) ? 1 : (__alignof(v) ? __alignof(v) : sizeof(uint32))) #define CRNLIB_GET_ALIGNMENT(v) ((!sizeof(v)) ? 1 : (__alignof(v) ? __alignof(v) : sizeof(uint32)))
#ifndef _MSC_VER #ifndef _MSC_VER
int sprintf_s(char *buffer, size_t sizeOfBuffer, const char *format, ...); int sprintf_s(char* buffer, size_t sizeOfBuffer, const char* format, ...);
int vsprintf_s(char *buffer, size_t sizeOfBuffer, const char *format, va_list args); int vsprintf_s(char* buffer, size_t sizeOfBuffer, const char* format, va_list args);
char* strlwr(char* p); char* strlwr(char* p);
char* strupr(char *p); char* strupr(char* p);
#define _stricmp strcasecmp #define _stricmp strcasecmp
#define _strnicmp strncasecmp #define _strnicmp strncasecmp
#endif #endif
inline bool crnlib_is_little_endian() { return c_crnlib_little_endian_platform; } inline bool crnlib_is_little_endian() {
inline bool crnlib_is_big_endian() { return c_crnlib_big_endian_platform; } return c_crnlib_little_endian_platform;
}
inline bool crnlib_is_big_endian() {
return c_crnlib_big_endian_platform;
}
inline bool crnlib_is_pc() inline bool crnlib_is_pc() {
{
#ifdef CRNLIB_PLATFORM_PC #ifdef CRNLIB_PLATFORM_PC
return true; return true;
#else #else
return false; return false;
#endif #endif
} }
inline bool crnlib_is_x86() inline bool crnlib_is_x86() {
{
#ifdef CRNLIB_PLATFORM_PC_X86 #ifdef CRNLIB_PLATFORM_PC_X86
return true; return true;
#else #else
return false; return false;
#endif #endif
} }
inline bool crnlib_is_x64() inline bool crnlib_is_x64() {
{
#ifdef CRNLIB_PLATFORM_PC_X64 #ifdef CRNLIB_PLATFORM_PC_X64
return true; return true;
#else #else
return false; return false;
#endif #endif
} }
crnlib/crn_prefix_coding.cpp
+284 -327
View File
@@ -5,352 +5,309 @@
//#include "rand.h" //#include "rand.h"
#ifdef CRNLIB_BUILD_DEBUG #ifdef CRNLIB_BUILD_DEBUG
//#define TEST_DECODER_TABLES //#define TEST_DECODER_TABLES
#endif #endif
namespace crnlib namespace crnlib {
{
namespace prefix_coding namespace prefix_coding {
{ bool limit_max_code_size(uint num_syms, uint8* pCodesizes, uint max_code_size) {
bool limit_max_code_size(uint num_syms, uint8* pCodesizes, uint max_code_size) const uint cMaxEverCodeSize = 34;
{
const uint cMaxEverCodeSize = 34;
if ((!num_syms) || (num_syms > cMaxSupportedSyms) || (max_code_size < 1) || (max_code_size > cMaxEverCodeSize))
return false;
uint num_codes[cMaxEverCodeSize + 1];
utils::zero_object(num_codes);
bool should_limit = false; if ((!num_syms) || (num_syms > cMaxSupportedSyms) || (max_code_size < 1) || (max_code_size > cMaxEverCodeSize))
return false;
for (uint i = 0; i < num_syms; i++)
{
uint c = pCodesizes[i];
if (c)
{
CRNLIB_ASSERT(c <= cMaxEverCodeSize);
num_codes[c]++;
if (c > max_code_size)
should_limit = true;
}
}
if (!should_limit)
return true;
uint ofs = 0;
uint next_sorted_ofs[cMaxEverCodeSize + 1];
for (uint i = 1; i <= cMaxEverCodeSize; i++)
{
next_sorted_ofs[i] = ofs;
ofs += num_codes[i];
}
if ((ofs < 2) || (ofs > cMaxSupportedSyms))
return true;
if (ofs > (1U << max_code_size))
return false;
for (uint i = max_code_size + 1; i <= cMaxEverCodeSize; i++)
num_codes[max_code_size] += num_codes[i];
// Technique of adjusting tree to enforce maximum code size from LHArc.
uint total = 0;
for (uint i = max_code_size; i; --i)
total += (num_codes[i] << (max_code_size - i));
if (total == (1U << max_code_size)) uint num_codes[cMaxEverCodeSize + 1];
return true; utils::zero_object(num_codes);
do
{
num_codes[max_code_size]--;
uint i; bool should_limit = false;
for (i = max_code_size - 1; i; --i)
{
if (!num_codes[i])
continue;
num_codes[i]--;
num_codes[i + 1] += 2;
break;
}
if (!i)
return false;
total--; for (uint i = 0; i < num_syms; i++) {
} while (total != (1U << max_code_size)); uint c = pCodesizes[i];
if (c) {
uint8 new_codesizes[cMaxSupportedSyms]; CRNLIB_ASSERT(c <= cMaxEverCodeSize);
uint8* p = new_codesizes;
for (uint i = 1; i <= max_code_size; i++) num_codes[c]++;
{ if (c > max_code_size)
uint n = num_codes[i]; should_limit = true;
if (n) }
{ }
memset(p, i, n);
p += n; if (!should_limit)
} return true;
}
uint ofs = 0;
for (uint i = 0; i < num_syms; i++) uint next_sorted_ofs[cMaxEverCodeSize + 1];
{ for (uint i = 1; i <= cMaxEverCodeSize; i++) {
const uint c = pCodesizes[i]; next_sorted_ofs[i] = ofs;
if (c) ofs += num_codes[i];
{ }
uint ofs = next_sorted_ofs[c];
next_sorted_ofs[c] = ofs + 1; if ((ofs < 2) || (ofs > cMaxSupportedSyms))
return true;
pCodesizes[i] = static_cast<uint8>(new_codesizes[ofs]);
} if (ofs > (1U << max_code_size))
} return false;
return true; for (uint i = max_code_size + 1; i <= cMaxEverCodeSize; i++)
num_codes[max_code_size] += num_codes[i];
// Technique of adjusting tree to enforce maximum code size from LHArc.
uint total = 0;
for (uint i = max_code_size; i; --i)
total += (num_codes[i] << (max_code_size - i));
if (total == (1U << max_code_size))
return true;
do {
num_codes[max_code_size]--;
uint i;
for (i = max_code_size - 1; i; --i) {
if (!num_codes[i])
continue;
num_codes[i]--;
num_codes[i + 1] += 2;
break;
}
if (!i)
return false;
total--;
} while (total != (1U << max_code_size));
uint8 new_codesizes[cMaxSupportedSyms];
uint8* p = new_codesizes;
for (uint i = 1; i <= max_code_size; i++) {
uint n = num_codes[i];
if (n) {
memset(p, i, n);
p += n;
}
}
for (uint i = 0; i < num_syms; i++) {
const uint c = pCodesizes[i];
if (c) {
uint ofs = next_sorted_ofs[c];
next_sorted_ofs[c] = ofs + 1;
pCodesizes[i] = static_cast<uint8>(new_codesizes[ofs]);
}
}
return true;
}
bool generate_codes(uint num_syms, const uint8* pCodesizes, uint16* pCodes) {
uint num_codes[cMaxExpectedCodeSize + 1];
utils::zero_object(num_codes);
for (uint i = 0; i < num_syms; i++) {
uint c = pCodesizes[i];
if (c) {
CRNLIB_ASSERT(c <= cMaxExpectedCodeSize);
num_codes[c]++;
}
}
uint code = 0;
uint next_code[cMaxExpectedCodeSize + 1];
next_code[0] = 0;
for (uint i = 1; i <= cMaxExpectedCodeSize; i++) {
next_code[i] = code;
code = (code + num_codes[i]) << 1;
}
if (code != (1 << (cMaxExpectedCodeSize + 1))) {
uint t = 0;
for (uint i = 1; i <= cMaxExpectedCodeSize; i++) {
t += num_codes[i];
if (t > 1)
return false;
}
}
for (uint i = 0; i < num_syms; i++) {
uint c = pCodesizes[i];
if (c) {
CRNLIB_ASSERT(next_code[c] <= cUINT16_MAX);
pCodes[i] = static_cast<uint16>(next_code[c]++);
CRNLIB_ASSERT(math::total_bits(pCodes[i]) <= pCodesizes[i]);
}
}
return true;
}
bool generate_decoder_tables(uint num_syms, const uint8* pCodesizes, decoder_tables* pTables, uint table_bits) {
uint min_codes[cMaxExpectedCodeSize];
if ((!num_syms) || (table_bits > cMaxTableBits))
return false;
pTables->m_num_syms = num_syms;
uint num_codes[cMaxExpectedCodeSize + 1];
utils::zero_object(num_codes);
for (uint i = 0; i < num_syms; i++) {
uint c = pCodesizes[i];
if (c)
num_codes[c]++;
}
uint sorted_positions[cMaxExpectedCodeSize + 1];
uint code = 0;
uint total_used_syms = 0;
uint max_code_size = 0;
uint min_code_size = UINT_MAX;
for (uint i = 1; i <= cMaxExpectedCodeSize; i++) {
const uint n = num_codes[i];
if (!n)
pTables->m_max_codes[i - 1] = 0; //UINT_MAX;
else {
min_code_size = math::minimum(min_code_size, i);
max_code_size = math::maximum(max_code_size, i);
min_codes[i - 1] = code;
pTables->m_max_codes[i - 1] = code + n - 1;
pTables->m_max_codes[i - 1] = 1 + ((pTables->m_max_codes[i - 1] << (16 - i)) | ((1 << (16 - i)) - 1));
pTables->m_val_ptrs[i - 1] = total_used_syms;
sorted_positions[i] = total_used_syms;
code += n;
total_used_syms += n;
}
code <<= 1;
}
pTables->m_total_used_syms = total_used_syms;
if (total_used_syms > pTables->m_cur_sorted_symbol_order_size) {
pTables->m_cur_sorted_symbol_order_size = total_used_syms;
if (!math::is_power_of_2(total_used_syms))
pTables->m_cur_sorted_symbol_order_size = math::minimum<uint>(num_syms, math::next_pow2(total_used_syms));
if (pTables->m_sorted_symbol_order) {
crnlib_delete_array(pTables->m_sorted_symbol_order);
pTables->m_sorted_symbol_order = NULL;
}
pTables->m_sorted_symbol_order = crnlib_new_array<uint16>(pTables->m_cur_sorted_symbol_order_size);
}
pTables->m_min_code_size = static_cast<uint8>(min_code_size);
pTables->m_max_code_size = static_cast<uint8>(max_code_size);
for (uint i = 0; i < num_syms; i++) {
uint c = pCodesizes[i];
if (c) {
CRNLIB_ASSERT(num_codes[c]);
uint sorted_pos = sorted_positions[c]++;
CRNLIB_ASSERT(sorted_pos < total_used_syms);
pTables->m_sorted_symbol_order[sorted_pos] = static_cast<uint16>(i);
}
}
if (table_bits <= pTables->m_min_code_size)
table_bits = 0;
pTables->m_table_bits = table_bits;
if (table_bits) {
uint table_size = 1 << table_bits;
if (table_size > pTables->m_cur_lookup_size) {
pTables->m_cur_lookup_size = table_size;
if (pTables->m_lookup) {
crnlib_delete_array(pTables->m_lookup);
pTables->m_lookup = NULL;
} }
bool generate_codes(uint num_syms, const uint8* pCodesizes, uint16* pCodes)
{
uint num_codes[cMaxExpectedCodeSize + 1];
utils::zero_object(num_codes);
for (uint i = 0; i < num_syms; i++) pTables->m_lookup = crnlib_new_array<uint32>(table_size);
{ }
uint c = pCodesizes[i];
if (c)
{
CRNLIB_ASSERT(c <= cMaxExpectedCodeSize);
num_codes[c]++;
}
}
uint code = 0; memset(pTables->m_lookup, 0xFF, static_cast<uint>(sizeof(pTables->m_lookup[0])) * (1UL << table_bits));
uint next_code[cMaxExpectedCodeSize + 1]; for (uint codesize = 1; codesize <= table_bits; codesize++) {
next_code[0] = 0; if (!num_codes[codesize])
continue;
for (uint i = 1; i <= cMaxExpectedCodeSize; i++)
{
next_code[i] = code;
code = (code + num_codes[i]) << 1;
}
if (code != (1 << (cMaxExpectedCodeSize + 1))) const uint fillsize = table_bits - codesize;
{ const uint fillnum = 1 << fillsize;
uint t = 0;
for (uint i = 1; i <= cMaxExpectedCodeSize; i++)
{
t += num_codes[i];
if (t > 1)
return false;
}
}
for (uint i = 0; i < num_syms; i++) const uint min_code = min_codes[codesize - 1];
{ const uint max_code = pTables->get_unshifted_max_code(codesize);
uint c = pCodesizes[i]; const uint val_ptr = pTables->m_val_ptrs[codesize - 1];
if (c)
{ for (uint code = min_code; code <= max_code; code++) {
CRNLIB_ASSERT(next_code[c] <= cUINT16_MAX); const uint sym_index = pTables->m_sorted_symbol_order[val_ptr + code - min_code];
pCodes[i] = static_cast<uint16>(next_code[c]++); CRNLIB_ASSERT(pCodesizes[sym_index] == codesize);
CRNLIB_ASSERT(math::total_bits(pCodes[i]) <= pCodesizes[i]); for (uint j = 0; j < fillnum; j++) {
} const uint t = j + (code << fillsize);
}
CRNLIB_ASSERT(t < (1U << table_bits));
return true;
CRNLIB_ASSERT(pTables->m_lookup[t] == cUINT32_MAX);
pTables->m_lookup[t] = sym_index | (codesize << 16U);
}
} }
}
bool generate_decoder_tables(uint num_syms, const uint8* pCodesizes, decoder_tables* pTables, uint table_bits) }
{
uint min_codes[cMaxExpectedCodeSize];
if ((!num_syms) || (table_bits > cMaxTableBits))
return false;
pTables->m_num_syms = num_syms;
uint num_codes[cMaxExpectedCodeSize + 1];
utils::zero_object(num_codes);
for (uint i = 0; i < num_syms; i++) for (uint i = 0; i < cMaxExpectedCodeSize; i++)
{ pTables->m_val_ptrs[i] -= min_codes[i];
uint c = pCodesizes[i];
if (c)
num_codes[c]++;
}
uint sorted_positions[cMaxExpectedCodeSize + 1]; pTables->m_table_max_code = 0;
pTables->m_decode_start_code_size = pTables->m_min_code_size;
uint code = 0;
uint total_used_syms = 0; if (table_bits) {
uint max_code_size = 0; uint i;
uint min_code_size = UINT_MAX; for (i = table_bits; i >= 1; i--) {
for (uint i = 1; i <= cMaxExpectedCodeSize; i++) if (num_codes[i]) {
{ pTables->m_table_max_code = pTables->m_max_codes[i - 1];
const uint n = num_codes[i]; break;
if (!n)
pTables->m_max_codes[i - 1] = 0;//UINT_MAX;
else
{
min_code_size = math::minimum(min_code_size, i);
max_code_size = math::maximum(max_code_size, i);
min_codes[i - 1] = code;
pTables->m_max_codes[i - 1] = code + n - 1;
pTables->m_max_codes[i - 1] = 1 + ((pTables->m_max_codes[i - 1] << (16 - i)) | ((1 << (16 - i)) - 1));
pTables->m_val_ptrs[i - 1] = total_used_syms;
sorted_positions[i] = total_used_syms;
code += n;
total_used_syms += n;
}
code <<= 1;
}
pTables->m_total_used_syms = total_used_syms;
if (total_used_syms > pTables->m_cur_sorted_symbol_order_size)
{
pTables->m_cur_sorted_symbol_order_size = total_used_syms;
if (!math::is_power_of_2(total_used_syms))
pTables->m_cur_sorted_symbol_order_size = math::minimum<uint>(num_syms, math::next_pow2(total_used_syms));
if (pTables->m_sorted_symbol_order)
{
crnlib_delete_array(pTables->m_sorted_symbol_order);
pTables->m_sorted_symbol_order = NULL;
}
pTables->m_sorted_symbol_order = crnlib_new_array<uint16>(pTables->m_cur_sorted_symbol_order_size);
}
pTables->m_min_code_size = static_cast<uint8>(min_code_size);
pTables->m_max_code_size = static_cast<uint8>(max_code_size);
for (uint i = 0; i < num_syms; i++)
{
uint c = pCodesizes[i];
if (c)
{
CRNLIB_ASSERT(num_codes[c]);
uint sorted_pos = sorted_positions[c]++;
CRNLIB_ASSERT(sorted_pos < total_used_syms);
pTables->m_sorted_symbol_order[sorted_pos] = static_cast<uint16>(i);
}
}
if (table_bits <= pTables->m_min_code_size)
table_bits = 0;
pTables->m_table_bits = table_bits;
if (table_bits)
{
uint table_size = 1 << table_bits;
if (table_size > pTables->m_cur_lookup_size)
{
pTables->m_cur_lookup_size = table_size;
if (pTables->m_lookup)
{
crnlib_delete_array(pTables->m_lookup);
pTables->m_lookup = NULL;
}
pTables->m_lookup = crnlib_new_array<uint32>(table_size);
}
memset(pTables->m_lookup, 0xFF, static_cast<uint>(sizeof(pTables->m_lookup[0])) * (1UL << table_bits));
for (uint codesize = 1; codesize <= table_bits; codesize++)
{
if (!num_codes[codesize])
continue;
const uint fillsize = table_bits - codesize;
const uint fillnum = 1 << fillsize;
const uint min_code = min_codes[codesize - 1];
const uint max_code = pTables->get_unshifted_max_code(codesize);
const uint val_ptr = pTables->m_val_ptrs[codesize - 1];
for (uint code = min_code; code <= max_code; code++)
{
const uint sym_index = pTables->m_sorted_symbol_order[ val_ptr + code - min_code ];
CRNLIB_ASSERT( pCodesizes[sym_index] == codesize );
for (uint j = 0; j < fillnum; j++)
{
const uint t = j + (code << fillsize);
CRNLIB_ASSERT(t < (1U << table_bits));
CRNLIB_ASSERT(pTables->m_lookup[t] == cUINT32_MAX);
pTables->m_lookup[t] = sym_index | (codesize << 16U);
}
}
}
}
for (uint i = 0; i < cMaxExpectedCodeSize; i++)
pTables->m_val_ptrs[i] -= min_codes[i];
pTables->m_table_max_code = 0;
pTables->m_decode_start_code_size = pTables->m_min_code_size;
if (table_bits)
{
uint i;
for (i = table_bits; i >= 1; i--)
{
if (num_codes[i])
{
pTables->m_table_max_code = pTables->m_max_codes[i - 1];
break;
}
}
if (i >= 1)
{
pTables->m_decode_start_code_size = table_bits + 1;
for (uint i = table_bits + 1; i <= max_code_size; i++)
{
if (num_codes[i])
{
pTables->m_decode_start_code_size = i;
break;
}
}
}
}
// sentinels
pTables->m_max_codes[cMaxExpectedCodeSize] = UINT_MAX;
pTables->m_val_ptrs[cMaxExpectedCodeSize] = 0xFFFFF;
pTables->m_table_shift = 32 - pTables->m_table_bits;
return true;
} }
}
} // namespace prefix_codig if (i >= 1) {
pTables->m_decode_start_code_size = table_bits + 1;
for (uint i = table_bits + 1; i <= max_code_size; i++) {
if (num_codes[i]) {
pTables->m_decode_start_code_size = i;
break;
}
}
}
}
// sentinels
pTables->m_max_codes[cMaxExpectedCodeSize] = UINT_MAX;
pTables->m_val_ptrs[cMaxExpectedCodeSize] = 0xFFFFF;
} // namespace crnlib pTables->m_table_shift = 32 - pTables->m_table_bits;
return true;
}
} // namespace prefix_codig
} // namespace crnlib
crnlib/crn_prefix_coding.h
+75 -88
View File
@@ -2,115 +2,102 @@
// See Copyright Notice and license at the end of inc/crnlib.h // See Copyright Notice and license at the end of inc/crnlib.h
#pragma once #pragma once
namespace crnlib namespace crnlib {
{ namespace prefix_coding {
namespace prefix_coding const uint cMaxExpectedCodeSize = 16;
{ const uint cMaxSupportedSyms = 8192;
const uint cMaxExpectedCodeSize = 16; const uint cMaxTableBits = 11;
const uint cMaxSupportedSyms = 8192;
const uint cMaxTableBits = 11;
bool limit_max_code_size(uint num_syms, uint8* pCodesizes, uint max_code_size); bool limit_max_code_size(uint num_syms, uint8* pCodesizes, uint max_code_size);
bool generate_codes(uint num_syms, const uint8* pCodesizes, uint16* pCodes); bool generate_codes(uint num_syms, const uint8* pCodesizes, uint16* pCodes);
class decoder_tables class decoder_tables {
{ public:
public: inline decoder_tables()
inline decoder_tables() : : m_table_shift(0), m_table_max_code(0), m_decode_start_code_size(0), m_cur_lookup_size(0), m_lookup(NULL), m_cur_sorted_symbol_order_size(0), m_sorted_symbol_order(NULL) {
m_table_shift(0), m_table_max_code(0), m_decode_start_code_size(0), m_cur_lookup_size(0), m_lookup(NULL), m_cur_sorted_symbol_order_size(0), m_sorted_symbol_order(NULL) }
{
}
inline decoder_tables(const decoder_tables& other) : inline decoder_tables(const decoder_tables& other)
m_table_shift(0), m_table_max_code(0), m_decode_start_code_size(0), m_cur_lookup_size(0), m_lookup(NULL), m_cur_sorted_symbol_order_size(0), m_sorted_symbol_order(NULL) : m_table_shift(0), m_table_max_code(0), m_decode_start_code_size(0), m_cur_lookup_size(0), m_lookup(NULL), m_cur_sorted_symbol_order_size(0), m_sorted_symbol_order(NULL) {
{ *this = other;
*this = other; }
}
decoder_tables& operator= (const decoder_tables& other) decoder_tables& operator=(const decoder_tables& other) {
{ if (this == &other)
if (this == &other) return *this;
return *this;
clear(); clear();
memcpy(this, &other, sizeof(*this)); memcpy(this, &other, sizeof(*this));
if (other.m_lookup) if (other.m_lookup) {
{ m_lookup = crnlib_new_array<uint32>(m_cur_lookup_size);
m_lookup = crnlib_new_array<uint32>(m_cur_lookup_size); memcpy(m_lookup, other.m_lookup, sizeof(m_lookup[0]) * m_cur_lookup_size);
memcpy(m_lookup, other.m_lookup, sizeof(m_lookup[0]) * m_cur_lookup_size); }
}
if (other.m_sorted_symbol_order) if (other.m_sorted_symbol_order) {
{ m_sorted_symbol_order = crnlib_new_array<uint16>(m_cur_sorted_symbol_order_size);
m_sorted_symbol_order = crnlib_new_array<uint16>(m_cur_sorted_symbol_order_size); memcpy(m_sorted_symbol_order, other.m_sorted_symbol_order, sizeof(m_sorted_symbol_order[0]) * m_cur_sorted_symbol_order_size);
memcpy(m_sorted_symbol_order, other.m_sorted_symbol_order, sizeof(m_sorted_symbol_order[0]) * m_cur_sorted_symbol_order_size); }
}
return *this; return *this;
} }
inline void clear() inline void clear() {
{ if (m_lookup) {
if (m_lookup) crnlib_delete_array(m_lookup);
{ m_lookup = 0;
crnlib_delete_array(m_lookup); m_cur_lookup_size = 0;
m_lookup = 0; }
m_cur_lookup_size = 0;
}
if (m_sorted_symbol_order) if (m_sorted_symbol_order) {
{ crnlib_delete_array(m_sorted_symbol_order);
crnlib_delete_array(m_sorted_symbol_order); m_sorted_symbol_order = NULL;
m_sorted_symbol_order = NULL; m_cur_sorted_symbol_order_size = 0;
m_cur_sorted_symbol_order_size = 0; }
} }
}
inline ~decoder_tables() inline ~decoder_tables() {
{ if (m_lookup)
if (m_lookup) crnlib_delete_array(m_lookup);
crnlib_delete_array(m_lookup);
if (m_sorted_symbol_order) if (m_sorted_symbol_order)
crnlib_delete_array(m_sorted_symbol_order); crnlib_delete_array(m_sorted_symbol_order);
} }
// DO NOT use any complex classes here - it is bitwise copied. // DO NOT use any complex classes here - it is bitwise copied.
uint m_num_syms; uint m_num_syms;
uint m_total_used_syms; uint m_total_used_syms;
uint m_table_bits; uint m_table_bits;
uint m_table_shift; uint m_table_shift;
uint m_table_max_code; uint m_table_max_code;
uint m_decode_start_code_size; uint m_decode_start_code_size;
uint8 m_min_code_size; uint8 m_min_code_size;
uint8 m_max_code_size; uint8 m_max_code_size;
uint m_max_codes[cMaxExpectedCodeSize + 1]; uint m_max_codes[cMaxExpectedCodeSize + 1];
int m_val_ptrs[cMaxExpectedCodeSize + 1]; int m_val_ptrs[cMaxExpectedCodeSize + 1];
uint m_cur_lookup_size; uint m_cur_lookup_size;
uint32* m_lookup; uint32* m_lookup;
uint m_cur_sorted_symbol_order_size; uint m_cur_sorted_symbol_order_size;
uint16* m_sorted_symbol_order; uint16* m_sorted_symbol_order;
inline uint get_unshifted_max_code(uint len) const inline uint get_unshifted_max_code(uint len) const {
{ CRNLIB_ASSERT((len >= 1) && (len <= cMaxExpectedCodeSize));
CRNLIB_ASSERT( (len >= 1) && (len <= cMaxExpectedCodeSize) ); uint k = m_max_codes[len - 1];
uint k = m_max_codes[len - 1]; if (!k)
if (!k) return UINT_MAX;
return UINT_MAX; return (k - 1) >> (16 - len);
return (k - 1) >> (16 - len); }
} };
};
bool generate_decoder_tables(uint num_syms, const uint8* pCodesizes, decoder_tables* pTables, uint table_bits); bool generate_decoder_tables(uint num_syms, const uint8* pCodesizes, decoder_tables* pTables, uint table_bits);
} // namespace prefix_coding } // namespace prefix_coding
} // namespace crnlib } // namespace crnlib
crnlib/crn_qdxt1.cpp
+684 -754
View File
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More