* Optimize Shader LineDictionary with Variable-Length 3-Streams This commit completely reorganizes the string dictionary compression pipeline used by compiled Material Text Chunks and improves matinfo dictionary output. 1. Multi-Base Variable-Length Scaling: We replaced the static 16-bit indices overhead with a bounded payload. Now, indices scale dynamically: - 0 to 239 evaluate in 1 byte. - 240 to 3584 leverage 0xF0-0xFD escapes evaluating in 2 bytes. - 3584+ are locked behind a 0xFF marker to 3 bytes. This natively eradicated the massive monolithic lengths and zero-padding issues previously dominating shader packages. 2. Variable-Length 3-Stream Decoding: To solve the Zstandard/Zlib entropy fragmentation that conventionally plagues interleaved variable byte lengths (which previously inflated our `filament.aar` boundary constraint by +2KB), we segregated the encoded payloads. By grouping high-entropy string boundaries into a `Base Stream` and isolating offset digits inside an `Extension Stream`, predictive LZ77 ZIP sliding-windows perfectly map over both arrays independently without disruption. 3. Optimize Numeric Stream using LEB128 Prior to this change, numerical suffixes split from shader variables (e.g., `param_1024` -> `param_` + `1024`) were fed back into the localized String Dictionary. Because high-frequency numbers were assigned disjointed localized IDs per shader variant, LZ77 failed to cross-reference their repetitive structures across shipped `.aar` archives, fracturing compression sequences. This patch implements a unified 3-Stream topology. It extracts numerical primitives (< 32768) away from the baseline String Dictionary, writing them into an isolated, contiguous LEB128 array. By using a dedicated `[254]` Escape Token within the primary stream, numerical variables maintain exact 1-byte (`< 128`) or 2-byte (`>= 128`) geometric layouts across all permutations. The resulting deterministic alignment guarantees that Zlib sliding windows can deduplicate highly repetitive variables across the entire application binary block. 4. We use the ShaderStage information to create distinct index ranges, which further help use 1-byte indices. Verification Metrics: `filament-android.aar`: -7,938 B `gltfio-android.aar`: -290,939 B `libfilament.a`: -18,464 B * Optimize shader dictionary by decoding '_' for numeric literals Most numbers extracted from the shader text are preceded by an underscore (e.g., from `_`, `hp_copy_`), which previously caused standalone `_` strings to heavily pollute the LineDictionary. This change removes the standalone `_` from the dictionary index: - `MaterialChunk` rehydrates the `_` prefix when decoding these numeric literals. This frees up dictionary indices, yielding massive byte savings across uncompressed binaries (e.g., -28.4 KB for volume_masked.filamat). * Optimize ShaderMinifier to strip explicit spacing Spirv-cross outputs GLSL with explicit spacing around generic operators (e.g., ` = `, `, `, ` ) * `). This padding consumes a significant amount of uncompressed bytes across large ubershaders. By applying targeted string replacements at the end of the `ShaderMinifier` pass, we strip this extraneous padding down to its raw tokens (e.g., `a=b`, `a,b`, `a*b`). This optimization preserves isolating spaces where valuable, ensuring line-dictionary tokens (such as raw `=` or `,`) remain deduplicated instead of fusing into unpredictable variables. Impact: This saves roughly ~9.1 KB in `libfilament.a` and ~3.2 KB in `volume_masked.filamat` uncompressed, with proportional gains across the downstream LZ4 compressed archives.
Filamat
Filamat allows for generating materials programatically on the device as opposed to with the matc
tool on the host machine. The cost is a binary size increase of your app due to the relatively
larger size of the filamat library.
For a smaller-sized library, see filamat_lite. It has no dependencies on
glslang, but can only compile materials for OpenGL and does no shader code optimization.
The filamat package is included in the releases available on GitHub.
Libraries
Filamat is distributed as a set of static libraries you must link against:
filamat, Filamat main libraryfilabridge, Support library for Filament / Filamatshaders, Shader text for material generationutils, Support library for Filament / Filamatsmol-v, SPIR-V compression library
To use Filamat from Java you must use the following two libraries instead:
filamat-java.jar, Contains Filamat's Java classesfilamat-jni, Filamat's JNI bindings
Linking against Filamat
This walkthrough will get you successfully compiling and linking native code against Filamat with minimum dependencies.
To start, download Filament's latest binary release
and extract into a directory of your choosing. Binary releases are suffixed with the platform name,
for example, filament-20181009-linux.tgz.
Create a file, main.cpp, in the same directory with the following contents:
#include <filamat/MaterialBuilder.h>
#include <iostream>
using namespace filamat;
int main(int argc, char** argv)
{
// Must be called before any materials can be built.
MaterialBuilder::init();
MaterialBuilder builder;
builder
.name("My material")
.material("void material (inout MaterialInputs material) {"
" prepareMaterial(material);"
" material.baseColor.rgb = float3(1.0, 0.0, 0.0);"
"}")
.shading(MaterialBuilder::Shading::LIT)
.targetApi(MaterialBuilder::TargetApi::ALL)
.platform(MaterialBuilder::Platform::ALL);
Package package = builder.build();
if (package.isValid()) {
std::cout << "Success!" << std::endl;
}
// Call when finished building all materials to release internal MaterialBuilder resources.
MaterialBuilder::shutdown();
return 0;
}
The directory should look like:
|-- README.md
|-- bin
|-- docs
|-- include
|-- lib
|-- main.cpp
We'll use a platform-specific Makefile to compile and link main.cpp with Filamat's libraries.
Copy your platform's Makefile below into a Makefile inside the same directory.
Linux
FILAMENT_LIBS=-lfilamat -lfilabridge -lshaders -lutils -lsmol-v
CC=clang++
main: main.o
$(CC) -Llib/x86_64/ -stdlib=libc++ main.o $(FILAMENT_LIBS) -lpthread -ldl -o main
main.o: main.cpp
$(CC) -Iinclude/ -std=c++20 -stdlib=libc++ -pthread -c main.cpp
clean:
rm -f main main.o
.PHONY: clean
macOS
FILAMENT_LIBS=-lfilamat -lfilabridge -lshaders -lutils -lsmol-v
CC=clang++
main: main.o
$(CC) -Llib/x86_64/ main.o $(FILAMENT_LIBS) -o main
main.o: main.cpp
$(CC) -Iinclude/ -std=c++20 -c main.cpp
clean:
rm -f main main.o
.PHONY: clean
Windows
Note that the static libraries distributed for Windows include several
variants: mt, md, mtd, mdd. These correspond to the run-time library
flags
/MT, /MD, /MTd, and /MDd, respectively. Here we use the mt variant.
When building Filamat from source, the USE_STATIC_CRT CMake option can be
used to change the run-time library version.
FILAMENT_LIBS=lib/x86_64/mt/filamat.lib lib/x86_64/mt/filabridge.lib lib/x86_64/mt/shaders.lib \
lib/x86_64/mt/utils.lib lib/x86_64/mt/smol-v.lib
CC=clang-cl.exe
main.exe: main.obj
$(CC) main.obj $(FILAMENT_LIBS) gdi32.lib user32.lib opengl32.lib
main.obj: main.cpp
$(CC) /MT /Iinclude/ /std:c++20 /c main.cpp
clean:
del main.exe main.obj
.PHONY: clean
Compiling
You should be able to invoke make and run the executable successfully:
$ make
$ ./main
Success!
On Windows, you'll need to open up a Visual Studio Native Tools Command Prompt
and invoke nmake instead of make.
Using the Material with Filament
For simplicity, this demo doesn't do anything useful with the built material package. To use the material with Filament, pass the material package's data into a Filament Material builder:
Package package = builder.build();
filament::Material* myMaterial = Material::Builder()
.package(package.getData(), package.getSize())
.build(*engine);
Note that this will require linking against Filament's libraries in addition to Filamat's.
Filamat Lite
The filamat_lite library is interchangeable with filamat, with a few caveats:
- Material compilation is only supported for the OpenGL backend.
- No shader-level optimization is performed.
- GLSL correctness is not checked.
In addition, filamat_lite only performs a simple text match to determine which properties on the
MaterialInputs structure are set. The material input variable must also always be refered to by
the name material.
void anotherFunction(inout MaterialInputs m) {
// Incorrect! The MaterialInputs is being referred to by the name "m".
m.metallic = 0.0;
}
void aFunction(inout MaterialInputs material) {
// Works, but only because the variable name "material" is used.
material.reflectance = 0.5;
}
// The MaterialInputs variable must be named material.
void material(inout MaterialInputs material) {
prepareMaterial(material);
// Good.
material.roughness = materialParams.roughness;
material.baseColor.rgb = vec3(1.0, 0.0, 1.0);
aFunction(material);
anotherFunction(material);
}