* materials: introduce MaterialCache
Presently, Filament Materials are instantiated by first parsing a bunch of
read-only data from a material file, then applying a bunch of options from its
Builder before settling on a final, immutable Material object. If two different
Material instances need to be parameterized differently, e.g. setting their spec
constants independently, each has to do all of these steps independently for
each variation.
This change introduces two new concepts: MaterialDefinition, representing the
deserialized, read-only state of a material file, and MaterialCache, a
reference-counted system responsible for managing the lifetimes of
MaterialDefinitions. Now, each Material asks the cache if a MaterialDefinition
exists for the particular UUID of the data it's trying to read; if not,
MaterialCache creates a new entry transparently. If a hundred different
Materials all try to load the same material data, only one
MaterialDefinition (and its associated GPU resources) will be created.
This first PR is the least possible invasive implementation of this feature.
There are a lot of room for improvements (and more planned). For example, each
Material still manages its own compiled shader program cache, but we can easily
move this to the MaterialCache in future PRs, further enabling the planned
mutable spec constants feature.
Additionally, there's room here to add a Material::toBuilder() method, which
could take an extant material and create a Builder object from it already
parameterized with all of the same options, a la the prototype design pattern.
* material cache: key on crc32
* material cache: make materialParser private
* move RefCountedMap to utils and add unit tests
* material cache: make create functions private
* material cache: fix broken tests on iOS/web
* material cache: address more comments
We can significantly improve the effectiveness of the line dictionary
by splitting lines strategically. spirv optimization tends to create
new variable names (e.g. _1234) that are not always stable between
variants, hereby preventing lines from being duplicated. By simply
splitting lines around these patterns, we improve the effectiveness
of the compression.
The decompression is almost unchanged. To make the code easier, we
now keep the '\n' at the end of lines in the dictionary. That's the
only change to the decompression side.
On certain materials like aiDefault.mat the compression ratio is
improved from 2.8x to 15x.
This also fixes a small but where the line dictionary relied on the
last line of the input to be a newline.
This change introduces a new chunk type to material files for precompiled Metal libraries. Previously, SPIR-V was the only binary type, so there's also a couple of refactor commits present here. Nothing is changed in Filament or matc yet.
BUGS=[333547148]
Previously, when a material was expilicitly built for feature level 0, it was
necessary to write ESSL 1.0 code which was incompatible with the OpenGL feature
level 1 implementation in Filament. Rather than adjust the Filament
implementation so that feature level 1 uses the same workarounds as feature
level 0 to emulate GLES 3.0+ features, it's more runtime-efficient to have matc
embed ESSL 1.0 and 3.0 shaders as separate chunks and load the corresponding
one based on the active feature level.
Feature level 0 material shaders must still be written in what is effectively
ESSL 1.0. To assist with this, a small set of compatibility definitions are
introduced when building the ESSL 3.0 variant. These are virtually all
textureXXX() functions, and end up either optimized out or inlined by glslang.
One final effect of this change is that external and 3D samplers are now
properly supported in feature level 0 materials.
It supports KHR_parallel_shader_compile as well as a
thread pool of GL contexts.
- we have a new 2-priorities queue for shader compilation
- use this feature in gltfio in the ubershader case
Some shaders can be shared across all materials (e.g. the depth
shaders). We use the filament default material as the "source" of
the cache, but until now we relied on an a priori knowledge of which
variants were present in the default material.
With this change, we now query once the list of variants (of interest)
in the default material and reuse that list for caching these variants
later.
This is better because the cached variants are now entirely driven by
the default material (which they depend on anyways). This is also faster
because we don't need to query which variant we need each time we create
a material.
This happened because the code iterated over the keys of a hashmap,
which obviously were not guaranteed to be in the same order as those
entries where added to the hashmap.
We fix this by adding a "visitor" to the materialchunk, so we can
iterate through it in order and retried the info we need.
* minimal backend support for compute
- added api to dispatch a compute shader
- added api to create and bind a ssbo
- added api to read back a buffer
Only implemented in the gl backend
* Add a backend compute test suite
* basic support for compute shaders in matc
this is still very much work-in-progress.
We're not supporting images nor ssbo for now.
* rename UniformInterfaceBlock to BufferInterfaceBlock
* augment BufferInterfaceBlock to support ssbo features
- add support for std430
- add support for ssbo
- add support for variable-size array
- add support for memory qualifiers
* reformat MaterialBuilder
* material format: move subpasses outside of parameters
subpasses now are their own json property instead of being a
"parameter".
* refactor parameter() methods to match Buffer/SamplerInterfaceBlock
We're just shuffling the arguments.
* add support for buffers in .mat files
* filamat now generates buffer blocks (ssbo)
* take feature level into consideration when optimizing shaders
* don't store the 'uniform binding' chunk for level 2 materials
this includes some refactoring/cleanups of MaterialParser
* matinfo: fixes for compute
- separate subpasses from parameters
- don't attempt to print material properties
This was too tricky, it highlights that we definitely need refactor this
to share code with the flattener / unflattener pipeline. I'll look at
that next.
MaterialInstance: generate versions of setParameter() methods that
are templated on the parameter sizeof(), which allows us to
coalesce implementations, e.g. vec4<float> and vec4<int> are now
the same. We make the compiler generate non-inline version of all
these methods, the typed version just call through, taking
advantage of the tail-call optimization in clang.
MaterialParser: improve code to avoid calling multiple methods that
all did more or less the same thing and were inlined.
PostProcessManager: use a loop to initialize the materials
ToneMapping: de-dup ACES implementation
And other more minor optimizations.
Fix alignment issues with setting booleans in UBOs
Program::shader() was taking a string before which didn't make sense
for spirv. Now it's just a blob, in the case of GL/Metal, the blob
must be a null terminated c-string, and the size must include the
terminating null character.
This fixes an out-of-bound access in ShaderBuilder::getShader() (which
doesn't exist anymore), because it was creating a CString passing
a size that included the null terminating char, which is not was CString
expects. CString can now assert() in that case.
driver::Program now uses a std::vector<> for storage, which we should
fix at some point (b/c it's a public header). CString was not suited to
store binary blobs.
filaflat only had on header dependency on filabridge (DriverEnums.h)
and only needed two small enum types.
In fact, I don't think it was right for filaflat to assume any
particular value for these fields -- this is the responsibility of the
callers.
With this change, filaflat looses almost all its dependencies on
filabridge (only needs DriverEnums.h) and becomes only a parsing
library that doesn't know anything about materials.
It still knows about "shaders" as blobs of text or data (spirv).
Filaflat itself is used by matinfo and filament.
We now expose things like
BlobDictionary: dictionary content
MaterialChunk: construct a material from the BlobDictionary
SpirvDictionaryReader: reads a spirv dictionary
TextDictionaryReader: reads an ascii dictionary
All these classes are generic enough and don't have any outside
dependencies.
The only remaining class which has outside dependencies is
MaterialParser, which will be eliminated soon.
Material archives already contain two version chunks (post process and
normal) but the renderer was ignoring these. The change makes it so that
matc writes a new MaterialEnums value into these chunks, and the engine
panics when receiving a material version that it does not expect.
This also adds a --version option to matc. No changes were necessary
to matinfo because it already prints out these values.
Fixes#796.
This removes a lot of heap allocations/deallocations, and reduces
code size. Most improvements come from using CString and
StaticString instead of std::string and better using move
semantics.
This fixes the "shader is not ASCII" error seen with GL backend,
introduced when we changed blob ownership semantics to accommodate
shader compression. It was due to missing null terminators.
We actually never bothered including the trailing null in blob length,
which was wrong but happened to work because the BlobDictionary held a
weak reference to chunk data. SInce it now holds an actual copy, the
lack of null caused our strings to contain garbage memory.
- move UBO declarations in UibGenerator, i.e. next
to their Uniform Interface Block definition.
This should help forgetting to update one but
not the other.
- move UniformInterfaceBlock.h and SamplerInferfaceBlock.h under
private/filament/.
- do the same things for samplers
