* Optimize Shader LineDictionary with Variable-Length 3-Streams
This commit completely reorganizes the string dictionary compression
pipeline used by compiled Material Text Chunks and improves matinfo
dictionary output.
1. Multi-Base Variable-Length Scaling:
We replaced the static 16-bit indices overhead with a
bounded payload. Now, indices scale dynamically:
- 0 to 239 evaluate in 1 byte.
- 240 to 3584 leverage 0xF0-0xFD escapes evaluating in 2 bytes.
- 3584+ are locked behind a 0xFF marker to 3 bytes.
This natively eradicated the massive monolithic lengths and zero-padding
issues previously dominating shader packages.
2. Variable-Length 3-Stream Decoding:
To solve the Zstandard/Zlib entropy fragmentation that conventionally
plagues interleaved variable byte lengths (which previously inflated
our `filament.aar` boundary constraint by +2KB), we segregated the encoded
payloads.
By grouping high-entropy string boundaries into a `Base Stream`
and isolating offset digits inside an `Extension Stream`,
predictive LZ77 ZIP sliding-windows perfectly map
over both arrays independently without disruption.
3. Optimize Numeric Stream using LEB128
Prior to this change, numerical suffixes split from shader
variables (e.g., `param_1024` -> `param_` + `1024`) were fed back
into the localized String Dictionary. Because high-frequency numbers were
assigned disjointed localized IDs per shader variant, LZ77
failed to cross-reference their repetitive structures across
shipped `.aar` archives, fracturing compression sequences.
This patch implements a unified 3-Stream topology. It
extracts numerical primitives (< 32768) away from the baseline
String Dictionary, writing them into an isolated, contiguous
LEB128 array.
By using a dedicated `[254]` Escape Token within the primary stream, numerical
variables maintain exact 1-byte (`< 128`) or 2-byte (`>= 128`)
geometric layouts across all permutations.
The resulting deterministic alignment guarantees that Zlib sliding windows
can deduplicate highly repetitive variables across the
entire application binary block.
4. We use the ShaderStage information to create distinct index ranges, which
further help use 1-byte indices.
Verification Metrics:
`filament-android.aar`: -7,938 B
`gltfio-android.aar`: -290,939 B
`libfilament.a`: -18,464 B
* Optimize shader dictionary by decoding '_' for numeric literals
Most numbers extracted from the shader text are preceded by an
underscore (e.g., from `_`, `hp_copy_`), which previously caused
standalone `_` strings to heavily pollute the LineDictionary.
This change removes the standalone `_` from the dictionary index:
- `MaterialChunk` rehydrates the `_` prefix when decoding these numeric
literals.
This frees up dictionary indices, yielding massive byte savings across
uncompressed binaries (e.g., -28.4 KB for volume_masked.filamat).
* Optimize ShaderMinifier to strip explicit spacing
Spirv-cross outputs GLSL with explicit spacing around generic
operators (e.g., ` = `, `, `, ` ) * `). This padding consumes a
significant amount of uncompressed bytes across large ubershaders.
By applying targeted string replacements at the end of the `ShaderMinifier`
pass, we strip this extraneous padding down to its raw tokens
(e.g., `a=b`, `a,b`, `a*b`).
This optimization preserves isolating spaces where valuable, ensuring
line-dictionary tokens (such as raw `=` or `,`) remain deduplicated
instead of fusing into unpredictable variables.
Impact:
This saves roughly ~9.1 KB in `libfilament.a` and ~3.2 KB in
`volume_masked.filamat` uncompressed, with proportional gains across
the downstream LZ4 compressed archives.
Since the struct refers to descriptors as opposed to specifically
bindings (although they do contain binding information as well), this
name is more appropriate.
The Glslang SPIR-V remapper was removed, and replaced by the CanonicalizeIds
pass in spirv-opt. See https://github.com/KhronosGroup/SPIRV-Tools/pull/6174
Filament's GLSL post-optimization algorithm already runs CanonicalizeIds.
Bug: b:450891564
The workaround option is now set to none by default. Use
```
matc -Wall
```
if you want to restore the previous behavior.
Current material workarounds:
- remove MergeReturnPass (except on Metal)
- remove SimplificationPass (except on Metal)
There are reason to believe these workarounds are no longer needed,
due to changes in spirv-opt.
It's possible that older devices might still need them.
These workaround affect *all* devices, this is why it is important
to disable them if not needed.
Currently the only values possible are 'none' and 'all'. 'all' is the
default. This option will be used to control code generation
workarounds individually. Currently 'all' disable the
MergeReturn and Simplification passes, which have causes issues in
the past on some older Android devices.
Dawn complains when consuming these lines with
WebGPU device error: ErrorType::Validation Error while parsing WGSL: :24:4 error: expected attribute
Did you mean 'size'?
Possible values: 'align', 'binding', 'blend_src', 'builtin', 'color', 'compute', 'diagnostic', 'fragment', 'group', 'id', 'input_attachment_index', 'interpolate', 'invariant', 'location', 'must_use', 'size', 'vertex', 'workgroup_size'
@stride(16) @internal(disable_validation__ignore_stride)
And we don't see a better workaround at this time.
* add sampler type in DescriptorSetLayout
[**New Material Version**]
* select the proper descriptor set layout in the rendering loop
Now that the per-view layout can be different based on the type of
shadow, we need to make sure we set the proper layout in the pipeline
and use the corresponding descriptor set.
There are now 16 "per-view" layouts and their corresponding descriptor
set. There are two sets of 8, the current set is determined by the
shadow type on the view.
Then during rendering, when we set the pipeline we need to select the
corresponding layout; each material now offers both versions of the
layout and we just pick the right one.
DescriptorSet::setBuffer and setSampler now need the layout in order
to do some validation that the correct type of descriptor is used.
This will help catch problems earlier.
* DescriptorType now includes the "dimension" of the texture
DescriptorType now also encodes wether the descriptor expects a 2D, 3D,
Cubemap and/or Array.
This is needed by both Metal and WebGPU.
[**New Material Version**]
* Fix typos.
* fix fog sampler descriptor type
the main aim of this PR is to consolidate how we access the
"per view" descriptor set to a single place.
some validation code is moved into DescriptorSets.cpp, so we get a
more centralized idea of what we do with the descriptors.
also factorize in one place the filtering of sampler list by layouts.
- Add option to build.sh to build for paritcular stereo
techniques (default to NONE). Only applies to samples.
- Consoldiate viewer checkbox for debugging stereo rendering
- Add DriverConfig flag for stereoscopic type so that it can
be used to determine availability of the feature and
(to be completed) enable corresponding GPU features.
Co-authored-by: Mathias Agopian <mathias@google.com>
The CL introducing the ESSL 1.0 chunk in materials inadvertently disabled
optimizations for said code. This commit reintroduces those optimizations and
fixes associated bugs which manifested. In particular, spirv-cross was
generating uints for bools; this has been fixed with a hack. Additionally,
spirv-cross is now compiled with exceptions enabled so that matc can gracefully
fail and show the code which failed to compile rather than abruptly aborting.
Since #7358 is blocked by an upstream spirv-cross issue, we can at least do a
bit of preprocessor optimization for ESSL 1.0 code in the meantime and introduce
the FILAMENT_EFFECTIVE_VERSION preprocessor definitions.
these materials would not generate proper structure or shadow buffer,
because they used a special variant that in most case removed the
user code.
now when the user code writes the depth or calls discard, the user
shader is kept.
The crashes are triggered by spirv-opt's MergeReturnPass, so we
just disable it. This pass also caused issues with AMD drivers on macOS.
fixes b/291140208
This PR sets up the ability for shaders to use `gl_ClipDistance`, which will be needed in the future. Desktop GL supports this natively. OpenGL ES requires the EXT_clip_cull_distance extension.
Unfortunately glslang does not support this extension, so we have to employ a workaround for mobile when going through glslang. We instead write to `filament_gl_ClipDistance`, and then modify the SPIR-V to decorate this as `gl_ClipDistance`. See the comment in SpirvFixup.h.
Note this PR does not actually use `gl_ClipDistance` yet, so there should be no change to shaders.
Mathias Agopian
Anish Goyal
Doris Wu
David Neto
Juan Caldas
Powei Feng
bridgewaterrobbie
Ben Doherty
Andy Hovingh
Juan David Caldas
Eliza Velasquez