wip: color space stuff

First, this commit introduces some very simple bugfixes regarding ES2
compatibility related to postprocessing.

Second, this commit adds support for creating textures specified as R8, SRGB8,
and SRGB8_A8 in ES2. R8 is trivial: just use GL_LUMINANCE instead. The sRGB
formats, however, are maybe a bit more controversial. As implemented, they
instead just use the equivalent non-sRGB formats. This is of course technically
incorrect. There are a few approaches to how to add sRGB compatibility for ES2
that I can think of.

1. Do a bunch of complex shader nonsense in matc. Maybe even traversing the AST
and ensuring any texture lookup of a texture flagged as sRGB uses some
compatibility function. This would require static analysis to track if samplers
are reassigned to another variable, for example. This of course also breaks down
if you don't know at compile time if the shader will receive an RGB or an sRGB
sampler, or if the shader should be able to support both RGB or sRGB samplers.
Really only worth mentioning here for the sake of completion.

2. You could also generate simple compatibility functions to look up each
sampler, which would only apply to FL0 materials.

First, we would have to extend the material format to be able to explicitly
"color" a sampler as sRGB or not, like:

```
parameters : [
    {
        type : sampler2d,
        name : albedo,
        precision : medium,
        colorSpace : srgb,
    },
    {
        type : sampler2d,
        name : normal,
        precision : medium,
        colorSpace : linear,
    }
],
```

Then, the following GLSL code would be generated.

```glsl
\#if __VERSION__ == 100
vec4 texture_albedo(vec2 position) {
  return sRGBtoLinear(texture2D(materialParams_albedo, position));
}
vec4 texture_normal(vec2 position) {
  return texture2D(materialParams_albedo, position);
}
\#else
vec4 texture_albedo(vec2 position) {
  return texture(materialParams_albedo, position);
}
vec4 texture_normal(vec2 normal) {
  return texture(materialParams_normal, position);
}
\#endif
```

Finally, at runtime, if a sampler is "colored" one way or the other, we would
verify that only the appropriate kinds of samplers are bound.

I'm actually very partial to this solution. Since sRGB compatibility is only a
concern on ES2, we can generate this code only for FL0 shaders, which already
require GLSL shader authors to care about ESSL 1.0 compatibility by calling the
appropriate `textureXX` functions. Additionally, it provides a layer of
high-level validation that texture lookups are correct, even if a real ES2
context is not available on the device being tested.

3. Leave it entirely up to the client. (What this commit does.) This leaves
client code ripe for making mistakes, but luckily, we can go back and do
solution 2 whenever. If specifying a color space for a sampler remains optional,
then if this feature is retrofitted in the future, client code will continue to
compile.

.dir-locals.el

@@ -0,0 +1,7 @@
+;;; Directory Local Variables            -*- no-byte-compile: t -*-
+;;; For more information see (info "(emacs) Directory Variables")
+
+((c++-mode . ((c-file-style . "filament")
+              (apheleia-inhibit . t)))
+ (c-mode . ((c-file-style . "filament")
+            (apheleia-inhibit . t))))

.github/workflows/android-continuous.yml

@@ -29,10 +29,6 @@ jobs:
         with:
           name: filamat-android-full
           path: out/filamat-android-release.aar
-      - uses: actions/upload-artifact@v1.0.0
-        with:
-          name: filamat-android-lite
-          path: out/filamat-android-lite-release.aar
       - uses: actions/upload-artifact@v1.0.0
         with:
           name: gltfio-android-release

.github/workflows/release.yml

@@ -124,7 +124,6 @@ jobs:
           cd ../..
           mv out/filament-android-release.aar out/filament-${TAG}-android.aar
           mv out/filamat-android-release.aar out/filamat-${TAG}-android.aar
-          mv out/filamat-android-lite-release.aar out/filamat-${TAG}-lite-android.aar
           mv out/gltfio-android-release.aar out/gltfio-${TAG}-android.aar
           mv out/filament-utils-android-release.aar out/filament-utils-${TAG}-android.aar
       - name: Sign sample-gltf-viewer

.gitignore

@@ -16,3 +16,5 @@ settings.json
 test*.png
 test*.json
 results
+/compile_commands.json
+/.cache

BUILDING.md

@@ -56,9 +56,11 @@ To trigger both incremental debug and release builds:
 ./build.sh debug release
 ```
 
+If build fails for some reasons, it may leave the `out/` directory in a broken state. You can
+force a clean build by adding the `-c` flag in that case.
+
 To install the libraries and executables in `out/debug/` and `out/release/`, add the `-i` flag.
-You can force a clean build by adding the `-c` flag. The script offers more features described
-by executing `build.sh -h`.
+The script offers more features described by executing `build.sh -h`.
 
 ### Filament-specific CMake Options
 
@@ -172,12 +174,12 @@ See [ios/samples/README.md](./ios/samples/README.md) for more information.
 
 ### Windows
 
-#### Building on Windows with Visual Studio 2019
+#### Building on Windows with Visual Studio 2019 or later
 
 Install the following components:
 
-- [Visual Studio 2019](https://www.visualstudio.com/downloads)
-- [Windows 10 SDK](https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk)
+- [Visual Studio 2019 or later](https://www.visualstudio.com/downloads)
+- [Windows SDK](https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/)
 - [Python 3.7](https://www.python.org/ftp/python/3.7.0/python-3.7.0.exe)
 - [CMake 3.14 or later](https://github.com/Kitware/CMake/releases/download/v3.14.7/cmake-3.14.7-win64-x64.msi)
 

NEW_RELEASE_NOTES.md

@@ -7,3 +7,12 @@ for next branch cut* header.
 appropriate header in [RELEASE_NOTES.md](./RELEASE_NOTES.md).
 
 ## Release notes for next branch cut
+
+- engine: Allow instantiating Engine at a given feature level via `Engine::Builder::featureLevel`
+- matc: Enable `GL_OES_standard_derivatives` extension in ESSL 1.0 shaders
+- matc: Fix code generation of double sided and masked materials in ESSL 1.0 shaders
+- filagui: Add support for feature level 0
+- matc: Add support for post-process materials in feature level 0
+- engine: Add `Material::getFeatureLevel()`
+- engine: Add missing `Material::getReflectionMode()` method in Java
+- engine: Support basic usage of post-processing materials on feature level 0

README.md

@@ -31,7 +31,7 @@ repositories {
 }
 
 dependencies {
-    implementation 'com.google.android.filament:filament-android:1.44.0'
+    implementation 'com.google.android.filament:filament-android:1.45.0'
 }
 ```
 
@@ -51,7 +51,7 @@ Here are all the libraries available in the group `com.google.android.filament`:
 iOS projects can use CocoaPods to install the latest release:
 
 ```shell
-pod 'Filament', '~> 1.44.0'
+pod 'Filament', '~> 1.45.0'
 ```
 
 ### Snapshots

RELEASE_GUIDE.md

@@ -128,3 +128,15 @@ Navigate to [Filament's release
 workflow](https://github.com/google/filament/actions/workflows/release.yml). Hit the _Run workflow_
 dropdown. Modify _Platform to build_ and _Release tag to build_, then hit _Run workflow_. This will
 initiate a new release run.
+
+## 11. Kick off the npm and CocoaPods release jobs
+
+Navigate to [Filament's npm deploy
+workflow](https://github.com/google/filament/actions/workflows/npm-deploy.yml).
+Hit the _Run workflow_ dropdown. Modify _Release tag to deploy_ to the tag corresponding to this
+release (for example, v1.42.2).
+
+Navigate to [Filament's CocoaPods deploy
+workflow](https://github.com/google/filament/actions/workflows/cocopods-deploy.yml).
+Hit the _Run workflow_ dropdown. Modify _Release tag to deploy_ to the tag corresponding to this
+release (for example, v1.42.2).

RELEASE_NOTES.md

@@ -7,6 +7,16 @@ A new header is inserted each time a *tag* is created.
 Instead, if you are authoring a PR for the main branch, add your release note to
 [NEW_RELEASE_NOTES.md](./NEW_RELEASE_NOTES.md).
 
+## v1.45.1
+
+- engine: Added parameter for configuring JobSystem thread count
+- engine: In Java, introduce Engine.Builder
+- gltfio: fix ubershader index for transmission&volume material
+- engine: New tone mapper: `AgXTonemapper`.
+- matinfo: Add support for viewing ESSL 1.0 shaders
+- engine: Add `Renderer::getClearOptions()` [b/243846268]
+- engine: Fix stable shadows (again) when an IBL rotation is used
+
 ## v1.45.0
 
 - materials: fix alpha masked materials when MSAA is turned on [⚠️ **Recompile materials**]

android/filament-android/src/main/cpp/Engine.cpp

@@ -25,15 +25,8 @@
 using namespace filament;
 using namespace utils;
 
-extern "C" JNIEXPORT jlong JNICALL
-Java_com_google_android_filament_Engine_nCreateEngine(JNIEnv*, jclass, jlong backend,
-        jlong sharedContext) {
-    return (jlong) Engine::create((Engine::Backend) backend, nullptr, (void*) sharedContext);
-}
-
 extern "C" JNIEXPORT void JNICALL
-Java_com_google_android_filament_Engine_nDestroyEngine(JNIEnv*, jclass,
-        jlong nativeEngine) {
+Java_com_google_android_filament_Engine_nDestroyEngine(JNIEnv*, jclass, jlong nativeEngine) {
     Engine* engine = (Engine*) nativeEngine;
     Engine::destroy(&engine);
 }
@@ -454,4 +447,56 @@ Java_com_google_android_filament_Engine_nGetActiveFeatureLevel(JNIEnv *, jclass,
         jlong nativeEngine) {
     Engine* engine = (Engine*) nativeEngine;
     return (jint)engine->getActiveFeatureLevel();
-}
+}
+
+extern "C" JNIEXPORT jlong JNICALL Java_com_google_android_filament_Engine_nCreateBuilder(JNIEnv*,
+        jclass) {
+    Engine::Builder* builder = new Engine::Builder{};
+    return (jlong) builder;
+}
+
+extern "C" JNIEXPORT void JNICALL Java_com_google_android_filament_Engine_nDestroyBuilder(JNIEnv*,
+        jclass, jlong nativeBuilder) {
+    Engine::Builder* builder = (Engine::Builder*) nativeBuilder;
+    delete builder;
+}
+
+extern "C" JNIEXPORT void JNICALL Java_com_google_android_filament_Engine_nSetBuilderBackend(
+        JNIEnv*, jclass, jlong nativeBuilder, jlong backend) {
+    Engine::Builder* builder = (Engine::Builder*) nativeBuilder;
+    builder->backend((Engine::Backend) backend);
+}
+
+extern "C" JNIEXPORT void JNICALL Java_com_google_android_filament_Engine_nSetBuilderConfig(JNIEnv*,
+        jclass, jlong nativeBuilder, jlong commandBufferSizeMB, jlong perRenderPassArenaSizeMB,
+        jlong driverHandleArenaSizeMB, jlong minCommandBufferSizeMB, jlong perFrameCommandsSizeMB,
+        jlong jobSystemThreadCount) {
+    Engine::Builder* builder = (Engine::Builder*) nativeBuilder;
+    Engine::Config config = {
+            .commandBufferSizeMB = (uint32_t) commandBufferSizeMB,
+            .perRenderPassArenaSizeMB = (uint32_t) perRenderPassArenaSizeMB,
+            .driverHandleArenaSizeMB = (uint32_t) driverHandleArenaSizeMB,
+            .minCommandBufferSizeMB = (uint32_t) minCommandBufferSizeMB,
+            .perFrameCommandsSizeMB = (uint32_t) perFrameCommandsSizeMB,
+            .jobSystemThreadCount = (uint32_t) jobSystemThreadCount,
+    };
+    builder->config(&config);
+}
+
+extern "C" JNIEXPORT void JNICALL Java_com_google_android_filament_Engine_nSetBuilderFeatureLevel(
+        JNIEnv*, jclass, jlong nativeBuilder, jint ordinal) {
+    Engine::Builder* builder = (Engine::Builder*) nativeBuilder;
+    builder->featureLevel((Engine::FeatureLevel)ordinal);
+}
+
+extern "C" JNIEXPORT void JNICALL Java_com_google_android_filament_Engine_nSetBuilderSharedContext(
+        JNIEnv*, jclass, jlong nativeBuilder, jlong sharedContext) {
+    Engine::Builder* builder = (Engine::Builder*) nativeBuilder;
+    builder->sharedContext((void*) sharedContext);
+}
+
+extern "C" JNIEXPORT jlong JNICALL
+Java_com_google_android_filament_Engine_nBuilderBuild(JNIEnv*, jclass, jlong nativeBuilder) {
+    Engine::Builder* builder = (Engine::Builder*) nativeBuilder;
+    return (jlong) builder->build();
+}

android/filament-android/src/main/cpp/Material.cpp

@@ -105,6 +105,22 @@ Java_com_google_android_filament_Material_nGetRefractionType(JNIEnv*, jclass,
     return (jint) material->getRefractionType();
 }
 
+extern "C"
+JNIEXPORT jint JNICALL
+Java_com_google_android_filament_Material_nGetReflectionMode(JNIEnv*, jclass,
+        jlong nativeMaterial) {
+    Material* material = (Material*) nativeMaterial;
+    return (jint) material->getReflectionMode();
+}
+
+extern "C"
+JNIEXPORT jint JNICALL
+Java_com_google_android_filament_Material_nGetFeatureLevel(JNIEnv*, jclass,
+        jlong nativeMaterial) {
+    Material* material = (Material*) nativeMaterial;
+    return (jint) material->getFeatureLevel();
+}
+
 extern "C"
 JNIEXPORT jint JNICALL
 Java_com_google_android_filament_Material_nGetVertexDomain(JNIEnv*, jclass,

android/filament-android/src/main/cpp/Scene.cpp

@@ -71,6 +71,13 @@ Java_com_google_android_filament_Scene_nRemoveEntities(JNIEnv *env, jclass type,
     env->ReleaseIntArrayElements(entities, (jint*) nativeEntities, JNI_ABORT);
 }
 
+extern "C" JNIEXPORT jint JNICALL
+Java_com_google_android_filament_Scene_nGetEntityCount(JNIEnv *env, jclass type,
+        jlong nativeScene) {
+    Scene* scene = (Scene*) nativeScene;
+    return (jint) scene->getEntityCount();
+}
+
 extern "C" JNIEXPORT jint JNICALL
 Java_com_google_android_filament_Scene_nGetRenderableCount(JNIEnv *env, jclass type,
         jlong nativeScene) {

android/filament-android/src/main/cpp/ToneMapper.cpp

@@ -47,6 +47,11 @@ Java_com_google_android_filament_ToneMapper_nCreateFilmicToneMapper(JNIEnv*, jcl
     return (jlong) new FilmicToneMapper();
 }
 
+extern "C" JNIEXPORT jlong JNICALL
+Java_com_google_android_filament_ToneMapper_nCreateAgxToneMapper(JNIEnv*, jclass, jint look) {
+    return (jlong) new AgxToneMapper(AgxToneMapper::AgxLook(look));
+}
+
 extern "C" JNIEXPORT jlong JNICALL
 Java_com_google_android_filament_ToneMapper_nCreateGenericToneMapper(JNIEnv*, jclass,
         jfloat contrast, jfloat midGrayIn, jfloat midGrayOut, jfloat hdrMax) {

android/filament-android/src/main/java/com/google/android/filament/Engine.java

@@ -154,6 +154,195 @@ public class Engine {
         FEATURE_LEVEL_2
     };
 
+    /**
+     * Constructs <code>Engine</code> objects using a builder pattern.
+     */
+    public static class Builder {
+        @SuppressWarnings({"FieldCanBeLocal", "UnusedDeclaration"})
+        private final BuilderFinalizer mFinalizer;
+        private final long mNativeBuilder;
+
+        public Builder() {
+            mNativeBuilder = nCreateBuilder();
+            mFinalizer = new BuilderFinalizer(mNativeBuilder);
+        }
+
+        /**
+         * Sets the {@link Backend} for the Engine.
+         *
+         * @param backend Driver backend to use
+         * @return A reference to this Builder for chaining calls.
+         */
+        public Builder backend(Backend backend) {
+            nSetBuilderBackend(mNativeBuilder, backend.ordinal());
+            return this;
+        }
+
+        /**
+         * Sets a sharedContext for the Engine.
+         *
+         * @param sharedContext  A platform-dependant OpenGL context used as a shared context
+         *                       when creating filament's internal context. On Android this parameter
+         *                       <b>must be</b> an instance of {@link android.opengl.EGLContext}.
+         * @return A reference to this Builder for chaining calls.
+         */
+        public Builder sharedContext(Object sharedContext) {
+            if (Platform.get().validateSharedContext(sharedContext)) {
+                nSetBuilderSharedContext(mNativeBuilder,
+                        Platform.get().getSharedContextNativeHandle(sharedContext));
+                return this;
+            }
+            throw new IllegalArgumentException("Invalid shared context " + sharedContext);
+        }
+
+        /**
+         * Configure the Engine with custom parameters.
+         *
+         * @param config A {@link Config} object
+         * @return A reference to this Builder for chaining calls.
+         */
+        public Builder config(Config config) {
+            nSetBuilderConfig(mNativeBuilder, config.commandBufferSizeMB,
+                    config.perRenderPassArenaSizeMB, config.driverHandleArenaSizeMB,
+                    config.minCommandBufferSizeMB, config.perFrameCommandsSizeMB,
+                    config.jobSystemThreadCount);
+            return this;
+        }
+
+        /**
+         * Sets the initial featureLevel for the Engine.
+         *
+         * @param featureLevel The feature level at which initialize Filament.
+         * @return A reference to this Builder for chaining calls.
+         */
+        public Builder featureLevel(FeatureLevel featureLevel) {
+            nSetBuilderFeatureLevel(mNativeBuilder, featureLevel.ordinal());
+            return this;
+        }
+
+        /**
+         * Creates an instance of Engine
+         *
+         * @return A newly created <code>Engine</code>, or <code>null</code> if the GPU driver couldn't
+         *         be initialized, for instance if it doesn't support the right version of OpenGL or
+         *         OpenGL ES.
+         *
+         * @exception IllegalStateException can be thrown if there isn't enough memory to
+         * allocate the command buffer.
+         */
+        public Engine build() {
+            long nativeEngine = nBuilderBuild(mNativeBuilder);
+            if (nativeEngine == 0) throw new IllegalStateException("Couldn't create Engine");
+            return new Engine(nativeEngine);
+        }
+
+        private static class BuilderFinalizer {
+            private final long mNativeObject;
+
+            BuilderFinalizer(long nativeObject) {
+                mNativeObject = nativeObject;
+            }
+
+            @Override
+            public void finalize() {
+                try {
+                    super.finalize();
+                } catch (Throwable t) { // Ignore
+                } finally {
+                    nDestroyBuilder(mNativeObject);
+                }
+            }
+        }
+    }
+
+    /**
+     * Parameters for customizing the initialization of {@link Engine}.
+     */
+    public static class Config {
+
+        // #defines in Engine.h
+        private static final long FILAMENT_PER_RENDER_PASS_ARENA_SIZE_IN_MB = 3;
+        private static final long FILAMENT_PER_FRAME_COMMANDS_SIZE_IN_MB = 2;
+        private static final long FILAMENT_MIN_COMMAND_BUFFERS_SIZE_IN_MB = 1;
+        private static final long FILAMENT_COMMAND_BUFFER_SIZE_IN_MB =
+                FILAMENT_MIN_COMMAND_BUFFERS_SIZE_IN_MB * 3;
+
+        /**
+         * Size in MiB of the low-level command buffer arena.
+         *
+         * Each new command buffer is allocated from here. If this buffer is too small the program
+         * might terminate or rendering errors might occur.
+         *
+         * This is typically set to minCommandBufferSizeMB * 3, so that up to 3 frames can be
+         * batched-up at once.
+         *
+         * This value affects the application's memory usage.
+         */
+        public long commandBufferSizeMB = FILAMENT_COMMAND_BUFFER_SIZE_IN_MB;
+
+        /**
+         * Size in MiB of the per-frame data arena.
+         *
+         * This is the main arena used for allocations when preparing a frame.
+         * e.g.: Froxel data and high-level commands are allocated from this arena.
+         *
+         * If this size is too small, the program will abort on debug builds and have undefined
+         * behavior otherwise.
+         *
+         * This value affects the application's memory usage.
+         */
+        public long perRenderPassArenaSizeMB = FILAMENT_PER_RENDER_PASS_ARENA_SIZE_IN_MB;
+
+        /**
+         * Size in MiB of the backend's handle arena.
+         *
+         * Backends will fallback to slower heap-based allocations when running out of space and
+         * log this condition.
+         *
+         * If 0, then the default value for the given platform is used
+         *
+         * This value affects the application's memory usage.
+         */
+        public long driverHandleArenaSizeMB = 0;
+
+        /**
+         * Minimum size in MiB of a low-level command buffer.
+         *
+         * This is how much space is guaranteed to be available for low-level commands when a new
+         * buffer is allocated. If this is too small, the engine might have to stall to wait for
+         * more space to become available, this situation is logged.
+         *
+         * This value does not affect the application's memory usage.
+         */
+        public long minCommandBufferSizeMB = FILAMENT_MIN_COMMAND_BUFFERS_SIZE_IN_MB;
+
+        /**
+         * Size in MiB of the per-frame high level command buffer.
+         *
+         * This buffer is related to the number of draw calls achievable within a frame, if it is
+         * too small, the program will abort on debug builds and have undefined behavior otherwise.
+         *
+         * It is allocated from the 'per-render-pass arena' above. Make sure that at least 1 MiB is
+         * left in the per-render-pass arena when deciding the size of this buffer.
+         *
+         * This value does not affect the application's memory usage.
+         */
+        public long perFrameCommandsSizeMB = FILAMENT_PER_FRAME_COMMANDS_SIZE_IN_MB;
+
+        /**
+         * Number of threads to use in Engine's JobSystem.
+         *
+         * Engine uses a utils::JobSystem to carry out paralleization of Engine workloads. This
+         * value sets the number of threads allocated for JobSystem. Configuring this value can be
+         * helpful in CPU-constrained environments where too many threads can cause contention of
+         * CPU and reduce performance.
+         *
+         * The default value is 0, which implies that the Engine will use a heuristic to determine
+         * the number of threads to use.
+         */
+        public long jobSystemThreadCount = 0;
+    }
+
     private Engine(long nativeEngine) {
         mNativeObject = nativeEngine;
         mTransformManager = new TransformManager(nGetTransformManager(nativeEngine));
@@ -177,9 +366,7 @@ public class Engine {
      */
     @NonNull
     public static Engine create() {
-        long nativeEngine = nCreateEngine(0, 0);
-        if (nativeEngine == 0) throw new IllegalStateException("Couldn't create Engine");
-        return new Engine(nativeEngine);
+        return new Builder().build();
     }
 
     /**
@@ -199,9 +386,9 @@ public class Engine {
      */
     @NonNull
     public static Engine create(@NonNull Backend backend) {
-        long nativeEngine = nCreateEngine(backend.ordinal(), 0);
-        if (nativeEngine == 0) throw new IllegalStateException("Couldn't create Engine");
-        return new Engine(nativeEngine);
+        return new Builder()
+            .backend(backend)
+            .build();
     }
 
     /**
@@ -223,13 +410,9 @@ public class Engine {
      */
     @NonNull
     public static Engine create(@NonNull Object sharedContext) {
-        if (Platform.get().validateSharedContext(sharedContext)) {
-            long nativeEngine = nCreateEngine(0,
-                    Platform.get().getSharedContextNativeHandle(sharedContext));
-            if (nativeEngine == 0) throw new IllegalStateException("Couldn't create Engine");
-            return new Engine(nativeEngine);
-        }
-        throw new IllegalArgumentException("Invalid shared context " + sharedContext);
+        return new Builder()
+            .sharedContext(sharedContext)
+            .build();
     }
 
     /**
@@ -296,17 +479,23 @@ public class Engine {
     }
 
     /**
-     * Activate all features of a given feature level. By default FeatureLevel::FEATURE_LEVEL_1 is
-     * active. The selected feature level must not be higher than the value returned by
-     * getActiveFeatureLevel() and it's not possible lower the active feature level.
+     * Activate all features of a given feature level. If an explicit feature level is not specified
+     * at Engine initialization time via {@link Builder#featureLevel}, the default feature level is
+     * {@link FeatureLevel#FEATURE_LEVEL_0} on devices not compatible with GLES 3.0; otherwise, the
+     * default is {@link FeatureLevel::FEATURE_LEVEL_1}. The selected feature level must not be
+     * higher than the value returned by {@link #getActiveFeatureLevel} and it's not possible lower
+     * the active feature level. Additionally, it is not possible to modify the feature level at all
+     * if the Engine was initialized at {@link FeatureLevel#FEATURE_LEVEL_0}.
      *
-     * @param featureLevel the feature level to activate. If featureLevel is lower than
-     *                     getActiveFeatureLevel(), the current (higher) feature level is kept.
-     *                     If featureLevel is higher than getSupportedFeatureLevel(), an exception
-     *                     is thrown, or the program is terminated if exceptions are disabled.
+     * @param featureLevel the feature level to activate. If featureLevel is lower than {@link
+     *                     #getActiveFeatureLevel}, the current (higher) feature level is kept. If
+     *                     featureLevel is higher than {@link #getSupportedFeatureLevel}, or if the
+     *                     engine was initialized at feature level 0, an exception is thrown, or the
+     *                     program is terminated if exceptions are disabled.
      *
      * @return the active feature level.
      *
+     * @see Builder#featureLevel
      * @see #getSupportedFeatureLevel
      * @see #getActiveFeatureLevel
      */
@@ -914,7 +1103,6 @@ public class Engine {
         }
     }
 
-    private static native long nCreateEngine(long backend, long sharedContext);
     private static native void nDestroyEngine(long nativeEngine);
     private static native long nGetBackend(long nativeEngine);
     private static native long nCreateSwapChain(long nativeEngine, Object nativeWindow, long flags);
@@ -971,4 +1159,14 @@ public class Engine {
     private static native int nGetSupportedFeatureLevel(long nativeEngine);
     private static native int nSetActiveFeatureLevel(long nativeEngine, int ordinal);
     private static native int nGetActiveFeatureLevel(long nativeEngine);
+
+    private static native long nCreateBuilder();
+    private static native void nDestroyBuilder(long nativeBuilder);
+    private static native void nSetBuilderBackend(long nativeBuilder, long backend);
+    private static native void nSetBuilderConfig(long nativeBuilder, long commandBufferSizeMB,
+            long perRenderPassArenaSizeMB, long driverHandleArenaSizeMB,
+            long minCommandBufferSizeMB, long perFrameCommandsSizeMB, long jobSystemThreadCount);
+    private static native void nSetBuilderFeatureLevel(long nativeBuilder, int ordinal);
+    private static native void nSetBuilderSharedContext(long nativeBuilder, long sharedContext);
+    private static native long nBuilderBuild(long nativeBuilder);
 }

android/filament-android/src/main/java/com/google/android/filament/Material.java

@@ -21,6 +21,7 @@ import androidx.annotation.NonNull;
 import androidx.annotation.Size;
 
 import com.google.android.filament.proguard.UsedByNative;
+import com.google.android.filament.Engine.FeatureLevel;
 
 import java.nio.Buffer;
 import java.util.ArrayList;
@@ -46,6 +47,8 @@ public class Material {
         static final BlendingMode[] sBlendingModeValues = BlendingMode.values();
         static final RefractionMode[] sRefractionModeValues = RefractionMode.values();
         static final RefractionType[] sRefractionTypeValues = RefractionType.values();
+        static final ReflectionMode[] sReflectionModeValues = ReflectionMode.values();
+        static final FeatureLevel[] sFeatureLevelValues = FeatureLevel.values();
         static final VertexDomain[] sVertexDomainValues = VertexDomain.values();
         static final CullingMode[] sCullingModeValues = CullingMode.values();
         static final VertexBuffer.VertexAttribute[] sVertexAttributeValues =
@@ -181,6 +184,18 @@ public class Material {
         THIN
     }
 
+    /**
+     * Supported reflection modes
+     *
+     * @see
+     * <a href="https://google.github.io/filament/Materials.html#materialdefinitions/materialblock/lighting:reflections">
+     * Lighting: reflections</a>
+     */
+    public enum ReflectionMode {
+        DEFAULT,
+        SCREEN_SPACE
+    }
+
     /**
      * Supported types of vertex domains
      *
@@ -437,6 +452,28 @@ public class Material {
         return EnumCache.sRefractionTypeValues[nGetRefractionType(getNativeObject())];
     }
 
+    /**
+     * Returns the reflection mode of this material.
+     *
+     * @see
+     * <a href="https://google.github.io/filament/Materials.html#materialdefinitions/materialblock/lighting:reflections">
+     * Lighting: reflections</a>
+     */
+    public ReflectionMode getReflectionMode() {
+        return EnumCache.sReflectionModeValues[nGetReflectionMode(getNativeObject())];
+    }
+
+    /**
+     * Returns the minimum required feature level for this material.
+     *
+     * @see
+     * <a href="https://google.github.io/filament/Materials.html#materialdefinitions/materialblock/general:featurelevel">
+     * General: featureLevel</a>
+     */
+    public FeatureLevel getFeatureLevel() {
+        return EnumCache.sFeatureLevelValues[nGetFeatureLevel(getNativeObject())];
+    }
+
     /**
      * Returns the vertex domain of this material.
      *
@@ -932,6 +969,8 @@ public class Material {
     private static native float nGetSpecularAntiAliasingThreshold(long nativeMaterial);
     private static native int nGetRefractionMode(long nativeMaterial);
     private static native int nGetRefractionType(long nativeMaterial);
+    private static native int nGetReflectionMode(long nativeMaterial);
+    private static native int nGetFeatureLevel(long nativeMaterial);
 
 
     private static native int nGetParameterCount(long nativeMaterial);

android/filament-android/src/main/java/com/google/android/filament/Scene.java

@@ -146,18 +146,29 @@ public class Scene {
     }
 
     /**
-     * Returns the number of {@link RenderableManager} components in the <code>Scene</code>.
+     * Returns the total number of Entities in the <code>Scene</code>, whether alive or not.
      *
-     * @return number of {@link RenderableManager} components in the <code>Scene</code>..
+     * @return the total number of Entities in the <code>Scene</code>.
+     */
+    public int getEntityCount() {
+        return nGetEntityCount(getNativeObject());
+    }
+
+    /**
+     * Returns the number of active (alive) {@link RenderableManager} components in the
+     * <code>Scene</code>.
+     *
+     * @return number of {@link RenderableManager} components in the <code>Scene</code>.
      */
     public int getRenderableCount() {
         return nGetRenderableCount(getNativeObject());
     }
 
     /**
-     * Returns the number of {@link LightManager} components in the <code>Scene</code>.
+     * Returns the number of active (alive) {@link LightManager} components in the
+     * <code>Scene</code>.
      *
-     * @return number of {@link LightManager} components in the <code>Scene</code>..
+     * @return number of {@link LightManager} components in the <code>Scene</code>.
      */
     public int getLightCount() {
         return nGetLightCount(getNativeObject());
@@ -189,6 +200,7 @@ public class Scene {
     private static native void nAddEntities(long nativeScene, int[] entities);
     private static native void nRemove(long nativeScene, int entity);
     private static native void nRemoveEntities(long nativeScene, int[] entities);
+    private static native int nGetEntityCount(long nativeScene);
     private static native int nGetRenderableCount(long nativeScene);
     private static native int nGetLightCount(long nativeScene);
     private static native boolean nHasEntity(long nativeScene, int entity);

android/filament-android/src/main/java/com/google/android/filament/SwapChain.java

@@ -103,6 +103,31 @@ public class SwapChain {
      */
     public static final long CONFIG_SRGB_COLORSPACE = 0x10;
 
+    /**
+     * Indicates that this SwapChain should allocate a stencil buffer in addition to a depth buffer.
+     *
+     * This flag is necessary when using View::setStencilBufferEnabled and rendering directly into
+     * the SwapChain (when post-processing is disabled).
+     *
+     * The specific format of the stencil buffer depends on platform support. The following pixel
+     * formats are tried, in order of preference:
+     *
+     * Depth only (without CONFIG_HAS_STENCIL_BUFFER):
+     * - DEPTH32F
+     * - DEPTH24
+     *
+     * Depth + stencil (with CONFIG_HAS_STENCIL_BUFFER):
+     * - DEPTH32F_STENCIL8
+     * - DEPTH24F_STENCIL8
+     *
+     * Note that enabling the stencil buffer may hinder depth precision and should only be used if
+     * necessary.
+     *
+     * @see View#setStencilBufferEnabled
+     * @see View#setPostProcessingEnabled
+     */
+    public static final long CONFIG_HAS_STENCIL_BUFFER = 0x20;
+
     SwapChain(long nativeSwapChain, Object surface) {
         mNativeObject = nativeSwapChain;
         mSurface = surface;

android/filament-android/src/main/java/com/google/android/filament/ToneMapper.java

@@ -100,6 +100,45 @@ public class ToneMapper {
         }
     }
 
+    /**
+     * AgX tone mapping operator.
+     */
+    public static class Agx extends ToneMapper {
+
+        public enum AgxLook {
+            /**
+             * Base contrast with no look applied
+             */
+            NONE,
+
+            /**
+             * A punchy and more chroma laden look for sRGB displays
+             */
+            PUNCHY,
+
+            /**
+             * A golden tinted, slightly washed look for BT.1886 displays
+             */
+            GOLDEN
+        }
+
+        /**
+         * Builds a new AgX tone mapper with no look applied.
+         */
+        public Agx() {
+            this(AgxLook.NONE);
+        }
+
+        /**
+         * Builds a new AgX tone mapper.
+         *
+         * @param look: an optional creative adjustment to contrast and saturation
+         */
+        public Agx(AgxLook look) {
+            super(nCreateAgxToneMapper(look.ordinal()));
+        }
+    }
+
     /**
      * Generic tone mapping operator that gives control over the tone mapping
      * curve. This operator can be used to control the aesthetics of the final
@@ -194,6 +233,7 @@ public class ToneMapper {
     private static native long nCreateACESToneMapper();
     private static native long nCreateACESLegacyToneMapper();
     private static native long nCreateFilmicToneMapper();
+    private static native long nCreateAgxToneMapper(int look);
     private static native long nCreateGenericToneMapper(
             float contrast, float midGrayIn, float midGrayOut, float hdrMax);
 

android/filament-android/src/main/java/com/google/android/filament/View.java

@@ -1032,7 +1032,8 @@ public class View {
      * </p>
      *
      * <p>
-     * Post-processing must be enabled in order to use the stencil buffer.
+     * If post-processing is disabled, then the SwapChain must have the CONFIG_HAS_STENCIL_BUFFER
+     * flag set in order to use the stencil buffer.
      * </p>
      *
      * <p>
@@ -1929,6 +1930,7 @@ public class View {
          * PCF with soft shadows and contact hardening
          */
         PCSS,
+        PCFd,
     }
 
     /**

android/filament-android/src/main/java/com/google/android/filament/android/UiHelper.java

@@ -227,8 +227,10 @@ public class UiHelper {
             if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.ICE_CREAM_SANDWICH_MR1) {
                 mTextureView.getSurfaceTexture().setDefaultBufferSize(width, height);
             }
-            // the call above won't cause TextureView.onSurfaceTextureSizeChanged()
-            mRenderCallback.onResized(width, height);
+            if (mRenderCallback != null) {
+                // the call above won't cause TextureView.onSurfaceTextureSizeChanged()
+                mRenderCallback.onResized(width, height);
+            }
         }
 
         @Override
@@ -298,7 +300,6 @@ public class UiHelper {
 
     /**
      * Checks whether we are ready to render into the attached surface.
-     *
      * Using OpenGL ES when this returns true, will result in drawing commands being lost,
      * HOWEVER, GLES state will be preserved. This is useful to initialize the engine.
      *
@@ -343,7 +344,6 @@ public class UiHelper {
     /**
      * Controls whether the render target (SurfaceView or TextureView) is opaque or not.
      * The render target is considered opaque by default.
-     *
      * Must be called before calling {@link #attachTo(SurfaceView)}, {@link #attachTo(TextureView)},
      * or {@link #attachTo(SurfaceHolder)}.
      *
@@ -366,10 +366,8 @@ public class UiHelper {
      * positioned above other surfaces but below the activity's surface. This property
      * only has an effect when used in combination with {@link #setOpaque(boolean) setOpaque(false)}
      * and does not affect TextureView targets.
-     *
      * Must be called before calling {@link #attachTo(SurfaceView)}
      * or {@link #attachTo(TextureView)}.
-     *
      * Has no effect when using {@link #attachTo(SurfaceHolder)}.
      *
      * @param overlay Indicates whether the render target should be rendered below the activity's
@@ -390,7 +388,6 @@ public class UiHelper {
 
     /**
      * Associate UiHelper with a SurfaceView.
-     *
      * As soon as SurfaceView is ready (i.e. has a Surface), we'll create the
      * EGL resources needed, and call user callbacks if needed.
      */
@@ -412,21 +409,23 @@ public class UiHelper {
 
             final SurfaceHolder.Callback callback = new SurfaceHolder.Callback() {
                 @Override
-                public void surfaceCreated(SurfaceHolder holder) {
+                public void surfaceCreated(@NonNull SurfaceHolder holder) {
                     if (LOGGING) Log.d(LOG_TAG, "surfaceCreated()");
                     createSwapChain(holder.getSurface());
                 }
 
                 @Override
                 public void surfaceChanged(
-                        SurfaceHolder holder, int format, int width, int height) {
+                    @NonNull SurfaceHolder holder, int format, int width, int height) {
                     // Note: this is always called at least once after surfaceCreated()
                     if (LOGGING) Log.d(LOG_TAG, "surfaceChanged(" + width + ", " + height + ")");
-                    mRenderCallback.onResized(width, height);
+                    if (mRenderCallback != null) {
+                        mRenderCallback.onResized(width, height);
+                    }
                 }
 
                 @Override
-                public void surfaceDestroyed(SurfaceHolder holder) {
+                public void surfaceDestroyed(@NonNull SurfaceHolder holder) {
                     if (LOGGING) Log.d(LOG_TAG, "surfaceDestroyed()");
                     destroySwapChain();
                 }
@@ -450,7 +449,6 @@ public class UiHelper {
 
     /**
      * Associate UiHelper with a TextureView.
-     *
      * As soon as TextureView is ready (i.e. has a buffer), we'll create the
      * EGL resources needed, and call user callbacks if needed.
      */
@@ -463,7 +461,7 @@ public class UiHelper {
             TextureView.SurfaceTextureListener listener = new TextureView.SurfaceTextureListener() {
                 @Override
                 public void onSurfaceTextureAvailable(
-                        SurfaceTexture surfaceTexture, int width, int height) {
+                    @NonNull SurfaceTexture surfaceTexture, int width, int height) {
                     if (LOGGING) Log.d(LOG_TAG, "onSurfaceTextureAvailable()");
 
                     if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.ICE_CREAM_SANDWICH_MR1) {
@@ -478,40 +476,44 @@ public class UiHelper {
 
                     createSwapChain(surface);
 
-                    // Call this the first time because onSurfaceTextureSizeChanged()
-                    // isn't called at initialization time
-                    mRenderCallback.onResized(width, height);
+                    if (mRenderCallback != null) {
+                        // Call this the first time because onSurfaceTextureSizeChanged()
+                        // isn't called at initialization time
+                        mRenderCallback.onResized(width, height);
+                    }
                 }
 
                 @Override
                 public void onSurfaceTextureSizeChanged(
-                        SurfaceTexture surfaceTexture, int width, int height) {
+                    @NonNull SurfaceTexture surfaceTexture, int width, int height) {
                     if (LOGGING) Log.d(LOG_TAG, "onSurfaceTextureSizeChanged()");
-                    if (mDesiredWidth > 0 && mDesiredHeight > 0) {
-                        surfaceTexture.setDefaultBufferSize(mDesiredWidth, mDesiredHeight);
-                        mRenderCallback.onResized(mDesiredWidth, mDesiredHeight);
-                    } else {
-                        mRenderCallback.onResized(width, height);
+                    if (mRenderCallback != null) {
+                        if (mDesiredWidth > 0 && mDesiredHeight > 0) {
+                            surfaceTexture.setDefaultBufferSize(mDesiredWidth, mDesiredHeight);
+                            mRenderCallback.onResized(mDesiredWidth, mDesiredHeight);
+                        } else {
+                            mRenderCallback.onResized(width, height);
+                        }
+                        // We must recreate the SwapChain to guarantee that it sees the new size.
+                        // More precisely, for an EGL client, the EGLSurface must be recreated. For
+                        // a Vulkan client, the SwapChain must be recreated. Calling
+                        // onNativeWindowChanged() will accomplish that.
+                        // This requirement comes from SurfaceTexture.setDefaultBufferSize()
+                        // documentation.
+                        TextureViewHandler textureViewHandler = (TextureViewHandler) mRenderSurface;
+                        mRenderCallback.onNativeWindowChanged(textureViewHandler.getSurface());
                     }
-                    // We must recreate the SwapChain to guarantee that it sees the new size.
-                    // More precisely, for an EGL client, the EGLSurface must be recreated. For
-                    // a Vulkan client, the SwapChain must be recreated. Calling
-                    // onNativeWindowChanged() will accomplish that.
-                    // This requirement comes from SurfaceTexture.setDefaultBufferSize()
-                    // documentation.
-                    TextureViewHandler textureViewHandler = (TextureViewHandler) mRenderSurface;
-                    mRenderCallback.onNativeWindowChanged(textureViewHandler.getSurface());
                 }
 
                 @Override
-                public boolean onSurfaceTextureDestroyed(SurfaceTexture surfaceTexture) {
+                public boolean onSurfaceTextureDestroyed(@NonNull SurfaceTexture surfaceTexture) {
                     if (LOGGING) Log.d(LOG_TAG, "onSurfaceTextureDestroyed()");
                     destroySwapChain();
                     return true;
                 }
 
                 @Override
-                public void onSurfaceTextureUpdated(SurfaceTexture surface) { }
+                public void onSurfaceTextureUpdated(@NonNull SurfaceTexture surface) { }
             };
 
             view.setSurfaceTextureListener(listener);
@@ -519,14 +521,15 @@ public class UiHelper {
             // in case the View's SurfaceTexture already existed
             if (view.isAvailable()) {
                 SurfaceTexture surfaceTexture = view.getSurfaceTexture();
-                listener.onSurfaceTextureAvailable(surfaceTexture, mDesiredWidth, mDesiredHeight);
+                if (surfaceTexture != null) {
+                    listener.onSurfaceTextureAvailable(surfaceTexture, mDesiredWidth, mDesiredHeight);
+                }
             }
         }
     }
 
     /**
      * Associate UiHelper with a SurfaceHolder.
-     *
      * As soon as a Surface is created, we'll create the
      * EGL resources needed, and call user callbacks if needed.
      */
@@ -539,20 +542,22 @@ public class UiHelper {
 
             final SurfaceHolder.Callback callback = new SurfaceHolder.Callback() {
                 @Override
-                public void surfaceCreated(SurfaceHolder surfaceHolder) {
+                public void surfaceCreated(@NonNull SurfaceHolder surfaceHolder) {
                     if (LOGGING) Log.d(LOG_TAG, "surfaceCreated()");
                     createSwapChain(holder.getSurface());
                 }
 
                 @Override
-                public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
+                public void surfaceChanged(@NonNull SurfaceHolder holder, int format, int width, int height) {
                     // Note: this is always called at least once after surfaceCreated()
                     if (LOGGING) Log.d(LOG_TAG, "surfaceChanged(" + width + ", " + height + ")");
-                    mRenderCallback.onResized(width, height);
+                    if (mRenderCallback != null) {
+                        mRenderCallback.onResized(width, height);
+                    }
                 }
 
                 @Override
-                public void surfaceDestroyed(SurfaceHolder surfaceHolder) {
+                public void surfaceDestroyed(@NonNull SurfaceHolder surfaceHolder) {
                     if (LOGGING) Log.d(LOG_TAG, "surfaceDestroyed()");
                     destroySwapChain();
                 }
@@ -587,7 +592,9 @@ public class UiHelper {
     }
 
     private void createSwapChain(@NonNull Surface surface) {
-        mRenderCallback.onNativeWindowChanged(surface);
+        if (mRenderCallback != null) {
+            mRenderCallback.onNativeWindowChanged(surface);
+        }
         mHasSwapChain = true;
     }
 
@@ -595,7 +602,9 @@ public class UiHelper {
         if (mRenderSurface != null) {
             mRenderSurface.detach();
         }
-        mRenderCallback.onDetachedFromSurface();
+        if (mRenderCallback != null) {
+            mRenderCallback.onDetachedFromSurface();
+        }
         mHasSwapChain = false;
     }
 }

android/gradle.properties

@@ -1,5 +1,5 @@
 GROUP=com.google.android.filament
-VERSION_NAME=1.44.0
+VERSION_NAME=1.45.0
 
 POM_DESCRIPTION=Real-time physically based rendering engine for Android.
 

android/samples/sample-hello-triangle/src/main/java/com/google/android/filament/hellotriangle/MainActivity.kt

@@ -110,7 +110,7 @@ class MainActivity : Activity() {
     }
 
     private fun setupFilament() {
-        engine = Engine.create()
+        engine = Engine.Builder().featureLevel(Engine.FeatureLevel.FEATURE_LEVEL_0).build()
         renderer = engine.createRenderer()
         scene = engine.createScene()
         view = engine.createView()
@@ -120,13 +120,8 @@ class MainActivity : Activity() {
     private fun setupView() {
         scene.skybox = Skybox.Builder().color(0.035f, 0.035f, 0.035f, 1.0f).build(engine)
 
-        if (engine.activeFeatureLevel == Engine.FeatureLevel.FEATURE_LEVEL_0) {
-            // post-processing is not supported at feature level 0
-            view.isPostProcessingEnabled = false
-        } else {
-            // NOTE: Try to disable post-processing (tone-mapping, etc.) to see the difference
-            // view.isPostProcessingEnabled = false
-        }
+        // post-processing is not supported at feature level 0
+        view.isPostProcessingEnabled = false
 
         // Tell the view which camera we want to use
         view.camera = camera

build.sh

@@ -191,6 +191,7 @@ function build_clean {
     rm -Rf android/filamat-android/build android/filamat-android/.externalNativeBuild android/filamat-android/.cxx
     rm -Rf android/gltfio-android/build android/gltfio-android/.externalNativeBuild android/gltfio-android/.cxx
     rm -Rf android/filament-utils-android/build android/filament-utils-android/.externalNativeBuild android/filament-utils-android/.cxx
+    rm -f compile_commands.json
 }
 
 function build_clean_aggressive {
@@ -232,6 +233,8 @@ function build_desktop_target {
             ${ASAN_UBSAN_OPTION} \
             ${architectures} \
             ../..
+        ln -sf "out/cmake-${lc_target}/compile_commands.json" \
+           ../../compile_commands.json
     fi
     ${BUILD_COMMAND} ${build_targets}
 
@@ -287,6 +290,8 @@ function build_webgl_with_target {
             -DCMAKE_INSTALL_PREFIX="../webgl-${lc_target}/filament" \
             -DWEBGL=1 \
             ../..
+        ln -sf "out/cmake-webgl-${lc_target}/compile_commands.json" \
+           ../../compile_commands.json
         ${BUILD_COMMAND} ${BUILD_TARGETS}
         )
     fi
@@ -359,6 +364,8 @@ function build_android_target {
             ${MATOPT_OPTION} \
             ${VULKAN_ANDROID_OPTION} \
             ../..
+        ln -sf "out/cmake-android-${lc_target}-${arch}/compile_commands.json" \
+           ../../compile_commands.json
     fi
 
     # We must always install Android libraries to build the AAR
@@ -495,13 +502,13 @@ function build_android {
 
         if [[ "${INSTALL_COMMAND}" ]]; then
             echo "Installing out/filamat-android-debug.aar..."
-            cp filamat-android/build/outputs/aar/filamat-android-full-debug.aar ../out/filamat-android-debug.aar
+            cp filamat-android/build/outputs/aar/filamat-android-debug.aar ../out/filamat-android-debug.aar
 
             echo "Installing out/filament-android-debug.aar..."
             cp filament-android/build/outputs/aar/filament-android-debug.aar ../out/
 
             echo "Installing out/gltfio-android-debug.aar..."
-            cp gltfio-android/build/outputs/aar/gltfio-android-full-debug.aar ../out/gltfio-android-debug.aar
+            cp gltfio-android/build/outputs/aar/gltfio-android-debug.aar ../out/gltfio-android-debug.aar
 
             echo "Installing out/filament-utils-android-debug.aar..."
             cp filament-utils-android/build/outputs/aar/filament-utils-android-debug.aar ../out/filament-utils-android-debug.aar
@@ -544,13 +551,13 @@ function build_android {
 
         if [[ "${INSTALL_COMMAND}" ]]; then
             echo "Installing out/filamat-android-release.aar..."
-            cp filamat-android/build/outputs/aar/filamat-android-full-release.aar ../out/filamat-android-release.aar
+            cp filamat-android/build/outputs/aar/filamat-android-release.aar ../out/filamat-android-release.aar
 
             echo "Installing out/filament-android-release.aar..."
             cp filament-android/build/outputs/aar/filament-android-release.aar ../out/
 
             echo "Installing out/gltfio-android-release.aar..."
-            cp gltfio-android/build/outputs/aar/gltfio-android-full-release.aar ../out/gltfio-android-release.aar
+            cp gltfio-android/build/outputs/aar/gltfio-android-release.aar ../out/gltfio-android-release.aar
 
             echo "Installing out/filament-utils-android-release.aar..."
             cp filament-utils-android/build/outputs/aar/filament-utils-android-release.aar ../out/filament-utils-android-release.aar
@@ -591,6 +598,8 @@ function build_ios_target {
             ${MATDBG_OPTION} \
             ${MATOPT_OPTION} \
             ../..
+        ln -sf "out/cmake-ios-${lc_target}-${arch}/compile_commands.json" \
+           ../../compile_commands.json
     fi
 
     ${BUILD_COMMAND}

build/common/test_list.txt

@@ -5,7 +5,6 @@ libs/math/test_math
 libs/image/test_image compare libs/image/tests/reference/
 libs/utils/test_utils
 libs/filamat/test_filamat
-libs/filamat/test_filamat_lite
 tools/matc/test_matc
 tools/cmgen/test_cmgen compare
 tools/glslminifier/test_glslminifier

docs/viewer/filament-viewer.js

@@ -15,7 +15,7 @@
 */
 
 // If you are bundling this with rollup, webpack, or esbuild, the following URL should be trimmed.
-import { LitElement, html, css } from "https://unpkg.com/lit?module";
+import { LitElement, html, css } from "https://unpkg.com/lit@2.8.0?module";
 
 // This little utility checks if the Filament module is ready for action.
 // If so, it immediately calls the given function. If not, it asks the Filament
@@ -287,12 +287,12 @@ class FilamentViewer extends LitElement {
         // Dropping a glb file is simple because there are no external resources.
         if (this.srcBlob && this.srcBlob.name.endsWith(".glb")) {
             this.srcBlob.arrayBuffer().then(buffer => {
-                this.asset = this.loader.createAssetFromBinary(new Uint8Array(buffer));
+                this.asset = this.loader.createAsset(new Uint8Array(buffer));
                 const aabb = this.asset.getBoundingBox();
                 this.assetRoot = this.asset.getRoot();
                 this.unitCubeTransform = Filament.fitIntoUnitCube(aabb, zoffset);
                 this.asset.loadResources();
-                this.animator = this.asset.getAnimator();
+                this.animator = this.asset.getInstance().getAnimator();
                 this.animationStartTime = Date.now();
                 this._updateOverlay();
             });
@@ -304,8 +304,6 @@ class FilamentViewer extends LitElement {
 
             const config = {
                 normalizeSkinningWeights: true,
-                recomputeBoundingBoxes: false,
-                ignoreBindTransform: false,
                 asyncInterval: 30
             };
 
@@ -320,22 +318,20 @@ class FilamentViewer extends LitElement {
                         resourceLoader.delete();
                         stbProvider.delete();
                         ktx2Provider.delete();
-                        this.animator = this.asset.getAnimator();
+                        this.animator = this.asset.getInstance().getAnimator();
                         this.animationStartTime = Date.now();
                     }
                 }, config.asyncInterval);
             };
 
             this.srcBlob.arrayBuffer().then(buffer => {
-                this.asset = this.loader.createAssetFromJson(new Uint8Array(buffer));
+                this.asset = this.loader.createAsset(new Uint8Array(buffer));
                 const aabb = this.asset.getBoundingBox();
                 this.assetRoot = this.asset.getRoot();
                 this.unitCubeTransform = Filament.fitIntoUnitCube(aabb, zoffset);
 
                 const resourceLoader = new Filament.gltfio$ResourceLoader(this.engine,
-                    config.normalizeSkinningWeights,
-                    config.recomputeBoundingBoxes,
-                    config.ignoreBindTransform);
+                    config.normalizeSkinningWeights);
 
                 const stbProvider = new Filament.gltfio$StbProvider(this.engine);
                 const ktx2Provider = new Filament.gltfio$Ktx2Provider(this.engine);
@@ -367,12 +363,7 @@ class FilamentViewer extends LitElement {
             return response.arrayBuffer();
         }).then(arrayBuffer => {
             const modelData = new Uint8Array(arrayBuffer);
-            if (this.src.endsWith(".glb")) {
-                this.asset = this.loader.createAssetFromBinary(modelData);
-            } else {
-                this.asset = this.loader.createAssetFromJson(modelData);
-            }
-
+            this.asset = this.loader.createAsset(modelData);
             const aabb = this.asset.getBoundingBox();
             this.assetRoot = this.asset.getRoot();
             this.unitCubeTransform = Filament.fitIntoUnitCube(aabb, zoffset);
@@ -380,7 +371,7 @@ class FilamentViewer extends LitElement {
             const basePath = '' + new URL(this.src, document.location);
 
             this.asset.loadResources(() => {
-                this.animator = this.asset.getAnimator();
+                this.animator = this.asset.getInstance().getAnimator();
                 this.animationStartTime = Date.now();
                 this._applyMaterialVariant();
             }, null, basePath);
@@ -441,14 +432,15 @@ class FilamentViewer extends LitElement {
         if (!this.hasAttribute("materialVariant")) {
             return;
         }
-        const names = this.asset.getMaterialVariantNames();
+        const instance = this.asset.getInstance();
+        const names = instance.getMaterialVariantNames();
         const index = this.materialVariant;
         if (index < 0 || index >= names.length) {
             console.error(`Material variant ${index} does not exist in this asset.`);
             return;
         }
         console.info(this.src, `Applying material variant: ${names[index]}`);
-        this.asset.applyMaterialVariant(index);
+        instance.applyMaterialVariant(index);
     }
 }
 

docs/viewer/index.html

@@ -43,7 +43,7 @@ filament-viewer::part(canvas) {
 </p>
 
 </main>
-<script src="https://unpkg.com/filament@1.25.3/filament.js"></script>
+<script src="https://unpkg.com/filament@1.44.0/filament.js"></script>
 <script src="https://unpkg.com/gltumble"></script>
 <script src="filament-viewer.js" type="module"></script>
 </body>

filament/CMakeLists.txt

@@ -218,6 +218,7 @@ set(MATERIAL_SRCS
         src/materials/colorGrading/customResolveAsSubpass.mat
         src/materials/debugShadowCascades.mat
         src/materials/defaultMaterial.mat
+        src/materials/defaultMaterial0.mat
         src/materials/dof/dof.mat
         src/materials/dof/dofCoc.mat
         src/materials/dof/dofDownsample.mat
@@ -237,6 +238,7 @@ set(MATERIAL_SRCS
         src/materials/ssao/bilateralBlurBentNormals.mat
         src/materials/ssao/mipmapDepth.mat
         src/materials/skybox.mat
+        src/materials/skybox0.mat
         src/materials/ssao/sao.mat
         src/materials/ssao/saoBentNormals.mat
         src/materials/separableGaussianBlur.mat
@@ -245,11 +247,6 @@ set(MATERIAL_SRCS
         src/materials/vsmMipmap.mat
 )
 
-set(MATERIAL_ES2_SRCS
-        src/materials/defaultMaterial0.mat
-        src/materials/skybox0.mat
-)
-
 # Embed the binary resource blob for materials.
 get_resgen_vars(${RESOURCE_DIR} materials)
 list(APPEND PRIVATE_HDRS ${RESGEN_HEADER})
@@ -315,23 +312,6 @@ foreach (mat_src ${MATERIAL_SRCS})
     list(APPEND MATERIAL_BINS ${output_path})
 endforeach()
 
-if (IS_MOBILE_TARGET AND FILAMENT_SUPPORTS_OPENGL)
-    foreach (mat_src ${MATERIAL_ES2_SRCS})
-        get_filename_component(localname "${mat_src}" NAME_WE)
-        get_filename_component(fullname "${mat_src}" ABSOLUTE)
-        set(output_path "${MATERIAL_DIR}/${localname}.filamat")
-
-        add_custom_command(
-                OUTPUT ${output_path}
-                COMMAND matc -a opengl -p ${MATC_TARGET} ${MATC_OPT_FLAGS} -o ${output_path} ${fullname}
-                MAIN_DEPENDENCY ${fullname}
-                DEPENDS matc
-                COMMENT "Compiling material ${mat_src} to ${output_path}"
-        )
-        list(APPEND MATERIAL_BINS ${output_path})
-    endforeach ()
-endif ()
-
 # Additional dependencies on included files for materials
 
 add_custom_command(

filament/backend/CMakeLists.txt

@@ -133,6 +133,7 @@ if (FILAMENT_SUPPORTS_METAL)
             src/metal/MetalExternalImage.mm
             src/metal/MetalHandles.mm
             src/metal/MetalPlatform.mm
+            src/metal/MetalShaderCompiler.mm
             src/metal/MetalState.mm
             src/metal/MetalTimerQuery.mm
             src/metal/MetalUtils.mm

filament/backend/include/backend/DriverEnums.h

@@ -77,6 +77,11 @@ static constexpr uint64_t SWAP_CHAIN_CONFIG_APPLE_CVPIXELBUFFER = 0x8;
  */
 static constexpr uint64_t SWAP_CHAIN_CONFIG_SRGB_COLORSPACE     = 0x10;
 
+/**
+ * Indicates that the SwapChain should also contain a stencil component.
+ */
+static constexpr uint64_t SWAP_CHAIN_HAS_STENCIL_BUFFER         = 0x20;
+
 
 static constexpr size_t MAX_VERTEX_ATTRIBUTE_COUNT  = 16;   // This is guaranteed by OpenGL ES.
 static constexpr size_t MAX_SAMPLER_COUNT           = 62;   // Maximum needed at feature level 3.
@@ -149,6 +154,19 @@ enum class ShaderLanguage {
     MSL = 3,
 };
 
+static constexpr const char* shaderLanguageToString(ShaderLanguage shaderLanguage) {
+    switch (shaderLanguage) {
+        case ShaderLanguage::ESSL1:
+            return "ESSL 1.0";
+        case ShaderLanguage::ESSL3:
+            return "ESSL 3.0";
+        case ShaderLanguage::SPIRV:
+            return "SPIR-V";
+        case ShaderLanguage::MSL:
+            return "MSL";
+    }
+}
+
 /**
  * Bitmask for selecting render buffers
  */
@@ -338,6 +356,13 @@ enum class SamplerFormat : uint8_t {
     SHADOW = 3      //!< shadow sampler (PCF)
 };
 
+//! Texture color space
+enum class SamplerTransferFunction : uint8_t {
+    UNDEFINED = 0,      //!< don't care
+    IDENTITY = 1,       //!< sampled as-is
+    SRGB_TO_LINEAR = 2, //!< sampled as sRGB and converted to linear
+};
+
 /**
  * Supported element types
  */

filament/backend/include/backend/Platform.h

@@ -46,6 +46,13 @@ public:
          * Driver clamps to valid values.
          */
         size_t handleArenaSize = 0;
+
+        /*
+         * this number of most-recently destroyed textures will be tracked for use-after-free.
+         * Throws an exception when a texture is freed but still bound to a SamplerGroup and used in
+         * a draw call. 0 disables completely. Currently only respected by the Metal backend.
+         */
+        size_t textureUseAfterFreePoolSize = 0;
     };
 
     Platform() noexcept;
@@ -107,6 +114,7 @@ public:
      * Platform.  The <insert> and <retrieve> Invocables may be called at any time and
      * from any thread from the time at which setBlobFunc is called until the time that Platform
      * is destroyed. Concurrent calls to these functions from different threads is also allowed.
+     * Either function can be null.
      *
      * @param insertBlob    an Invocable that inserts a new value into the cache and associates
      *                      it with the given key
@@ -116,9 +124,21 @@ public:
     void setBlobFunc(InsertBlobFunc&& insertBlob, RetrieveBlobFunc&& retrieveBlob) noexcept;
 
     /**
-     * @return true if setBlobFunc was called.
+     * @return true if insertBlob is valid.
      */
-    bool hasBlobFunc() const noexcept;
+    bool hasInsertBlobFunc() const noexcept;
+
+    /**
+     * @return true if retrieveBlob is valid.
+     */
+    bool hasRetrieveBlobFunc() const noexcept;
+
+    /**
+     * @return true if either of insertBlob or retrieveBlob are valid.
+     */
+    bool hasBlobFunc() const noexcept {
+        return hasInsertBlobFunc() || hasRetrieveBlobFunc();
+    }
 
     /**
      * To insert a new binary value into the cache and associate it with a given

filament/backend/include/backend/platforms/PlatformEGL.h

@@ -42,6 +42,9 @@ public:
     void createContext(bool shared) override;
     void releaseContext() noexcept override;
 
+    // Return true if we're on an OpenGL platform (as opposed to OpenGL ES). false by default.
+    virtual bool isOpenGL() const noexcept;
+
 protected:
 
     // --------------------------------------------------------------------------------------------
@@ -126,6 +129,7 @@ protected:
     EGLSurface mCurrentDrawSurface = EGL_NO_SURFACE;
     EGLSurface mCurrentReadSurface = EGL_NO_SURFACE;
     EGLSurface mEGLDummySurface = EGL_NO_SURFACE;
+    // mEGLConfig is valid only if ext.egl.KHR_no_config_context is false
     EGLConfig mEGLConfig = EGL_NO_CONFIG_KHR;
     Config mContextAttribs;
     std::vector<EGLContext> mAdditionalContexts;
@@ -146,8 +150,8 @@ protected:
 
     void initializeGlExtensions() noexcept;
 
-private:
-    EGLConfig findSwapChainConfig(uint64_t flags) const;
+protected:
+    EGLConfig findSwapChainConfig(uint64_t flags, bool window, bool pbuffer) const;
 };
 
 } // namespace filament::backend

filament/backend/include/backend/platforms/PlatformEGLHeadless.h

@@ -30,6 +30,9 @@ public:
 
     Driver* createDriver(void* sharedContext,
             const Platform::DriverConfig& driverConfig) noexcept override;
+
+protected:
+    bool isOpenGL() const noexcept override;
 };
 
 } // namespace filament

filament/backend/include/backend/platforms/VulkanPlatform.h

@@ -20,6 +20,7 @@
 #include <backend/Platform.h>
 
 #include <bluevk/BlueVK.h>
+#include <utils/CString.h>
 #include <utils/FixedCapacityVector.h>
 #include <utils/PrivateImplementation.h>
 
@@ -89,36 +90,44 @@ public:
 
     // ----------------------------------------------------
     // ---------- Platform Customization options ----------
-    /**
-     * The client preference can be stored within the struct.  We allow for two specification of
-     * preference:
-     *     1) A substring to match against `VkPhysicalDeviceProperties.deviceName`.
-     *     2) Index of the device in the list as returned by vkEnumeratePhysicalDevices.
-     */
-    struct GPUPreference {
-        std::string deviceName;
-        int8_t index = -1;
+    struct Customization {
+        /**
+         * The client can specify the GPU (i.e. VkDevice) for the platform. We allow the
+         * following preferences:
+         *     1) A substring to match against `VkPhysicalDeviceProperties.deviceName`. Empty string
+         *        by default.
+         *     2) Index of the device in the list as returned by
+         *        `vkEnumeratePhysicalDevices`. -1 by default to indicate no preference.
+         */
+        struct GPUPreference {
+            utils::CString deviceName;
+            int8_t index = -1;
+        } gpu;
+
+        /**
+         * Whether the platform supports sRGB swapchain. Default is true.
+         */
+        bool isSRGBSwapChainSupported = true;
+
+        /**
+         * When the platform window is resized, we will flush and wait on the command queues
+         * before recreating the swapchain. Default is true.
+         */
+        bool flushAndWaitOnWindowResize = true;
     };
 
     /**
-     * Client can provide a preference over the GPU to use in the vulkan instance
-     * @return            `GPUPreference` struct that indicates the client's preference
+     * Client can override to indicate customized behavior or parameter for their platform.
+     * @return            `Customization` struct that indicates the client's platform
+     *                    customizations.
      */
-    virtual GPUPreference getPreferredGPU() noexcept {
+    virtual Customization getCustomization() const noexcept {
         return {};
     }
+
     // -------- End platform customization options --------
     // ----------------------------------------------------
 
-    /**
-     * Returns whether the platform supports sRGB swapchain. This is true by default, and the client
-     * needs to override this method to specify otherwise.
-     * @return            Whether the platform supports sRGB swapchain.
-     */
-    virtual bool isSRGBSwapChainSupported() const {
-        return true;
-    }
-
     /**
      * Get the images handles and format of the memory backing the swapchain. This should be called
      * after createSwapChain() or after recreateIfResized().

filament/backend/include/private/backend/Driver.h

@@ -24,6 +24,7 @@
 #include <backend/PipelineState.h>
 #include <backend/TargetBufferInfo.h>
 
+#include <utils/CString.h>
 #include <utils/compiler.h>
 
 #include <functional>

filament/backend/include/private/backend/DriverAPI.inc

@@ -219,7 +219,7 @@ DECL_DRIVER_API_R_N(backend::TextureHandle, importTexture,
         backend::TextureUsage, usage)
 
 DECL_DRIVER_API_R_N(backend::SamplerGroupHandle, createSamplerGroup,
-        uint32_t, size)
+        uint32_t, size, utils::FixedSizeString<32>, debugName)
 
 DECL_DRIVER_API_R_N(backend::RenderPrimitiveHandle, createRenderPrimitive,
         backend::VertexBufferHandle, vbh,

filament/backend/include/private/backend/HandleAllocator.h

@@ -33,7 +33,7 @@
 
 #define HandleAllocatorGL  HandleAllocator<16, 64, 208>
 #define HandleAllocatorVK  HandleAllocator<16, 64, 880>
-#define HandleAllocatorMTL HandleAllocator<16, 64, 576>
+#define HandleAllocatorMTL HandleAllocator<16, 64, 584>
 
 namespace filament::backend {
 
@@ -239,14 +239,16 @@ private:
         }
     };
 
-
+// FIXME: We should be using a Spinlock here, at least on platforms where mutexes are not
+//        efficient (i.e. non-Linux). However, we've seen some hangs on that spinlock, which
+//        we don't understand well (b/308029108).
 #ifndef NDEBUG
     using HandleArena = utils::Arena<Allocator,
-            utils::LockingPolicy::SpinLock,
+            utils::LockingPolicy::Mutex,
             utils::TrackingPolicy::DebugAndHighWatermark>;
 #else
     using HandleArena = utils::Arena<Allocator,
-            utils::LockingPolicy::SpinLock>;
+            utils::LockingPolicy::Mutex>;
 #endif
 
     // allocateHandle()/deallocateHandle() selects the pool to use at compile-time based on the
@@ -256,6 +258,7 @@ private:
     HandleBase::HandleId allocateHandle() noexcept {
         if constexpr (SIZE <= P0) { return allocateHandleInPool<P0>(); }
         if constexpr (SIZE <= P1) { return allocateHandleInPool<P1>(); }
+        static_assert(SIZE <= P2);
         return allocateHandleInPool<P2>();
     }
 
@@ -266,6 +269,7 @@ private:
         } else if constexpr (SIZE <= P1) {
             deallocateHandleFromPool<P1>(id);
         } else {
+            static_assert(SIZE <= P2);
             deallocateHandleFromPool<P2>(id);
         }
     }

filament/backend/src/Platform.cpp

@@ -28,14 +28,16 @@ bool Platform::pumpEvents() noexcept {
 }
 
 void Platform::setBlobFunc(InsertBlobFunc&& insertBlob, RetrieveBlobFunc&& retrieveBlob) noexcept {
-    if (!mInsertBlob && !mRetrieveBlob) {
-        mInsertBlob = std::move(insertBlob);
-        mRetrieveBlob = std::move(retrieveBlob);
-    }
+    mInsertBlob = std::move(insertBlob);
+    mRetrieveBlob = std::move(retrieveBlob);
 }
 
-bool Platform::hasBlobFunc() const noexcept {
-    return mInsertBlob && mRetrieveBlob;
+bool Platform::hasInsertBlobFunc() const noexcept {
+    return bool(mInsertBlob);
+}
+
+bool Platform::hasRetrieveBlobFunc() const noexcept {
+    return bool(mRetrieveBlob);
 }
 
 void Platform::insertBlob(void const* key, size_t keySize, void const* value, size_t valueSize) {

filament/backend/src/metal/MetalContext.h

@@ -18,12 +18,15 @@
 #define TNT_METALCONTEXT_H
 
 #include "MetalResourceTracker.h"
+#include "MetalShaderCompiler.h"
 #include "MetalState.h"
 
 #include <CoreVideo/CVMetalTextureCache.h>
 #include <Metal/Metal.h>
 #include <QuartzCore/QuartzCore.h>
 
+#include <utils/FixedCircularBuffer.h>
+
 #include <array>
 #include <atomic>
 #include <stack>
@@ -53,6 +56,9 @@ struct MetalVertexBuffer;
 constexpr static uint8_t MAX_SAMPLE_COUNT = 8;  // Metal devices support at most 8 MSAA samples
 
 struct MetalContext {
+    explicit MetalContext(size_t metalFreedTextureListSize)
+        : texturesToDestroy(metalFreedTextureListSize) {}
+
     MetalDriver* driver;
     id<MTLDevice> device = nullptr;
     id<MTLCommandQueue> commandQueue = nullptr;
@@ -111,6 +117,14 @@ struct MetalContext {
     tsl::robin_set<MetalSamplerGroup*> samplerGroups;
     tsl::robin_set<MetalTexture*> textures;
 
+    // This circular buffer implements delayed destruction for Metal texture handles. It keeps a
+    // handle to a fixed number of the most recently destroyed texture handles. When we're asked to
+    // destroy a texture handle, we free its texture memory, but keep the MetalTexture object alive,
+    // marking it as "terminated". If we later are asked to use that texture, we can check its
+    // terminated status and throw an Objective-C error instead of crashing, which is helpful for
+    // debugging use-after-free issues in release builds.
+    utils::FixedCircularBuffer<Handle<HwTexture>> texturesToDestroy;
+
     MetalBufferPool* bufferPool;
 
     MetalSwapChain* currentDrawSwapChain = nil;
@@ -136,6 +150,8 @@ struct MetalContext {
 
     MTLViewport currentViewport;
 
+    MetalShaderCompiler* shaderCompiler = nullptr;
+
 #if defined(FILAMENT_METAL_PROFILING)
     // Logging and profiling.
     os_log_t log;

filament/backend/src/metal/MetalDriver.h

@@ -34,11 +34,11 @@ namespace backend {
 class MetalPlatform;
 
 class MetalBuffer;
+class MetalProgram;
 class MetalSamplerGroup;
 class MetalTexture;
 struct MetalUniformBuffer;
 struct MetalContext;
-struct MetalProgram;
 struct BufferState;
 
 #ifndef FILAMENT_METAL_HANDLE_ARENA_SIZE_IN_MB

filament/backend/src/metal/MetalDriver.mm

@@ -50,7 +50,8 @@ UTILS_NOINLINE
 Driver* MetalDriver::create(MetalPlatform* const platform, const Platform::DriverConfig& driverConfig) {
     assert_invariant(platform);
     size_t defaultSize = FILAMENT_METAL_HANDLE_ARENA_SIZE_IN_MB * 1024U * 1024U;
-    Platform::DriverConfig validConfig { .handleArenaSize = std::max(driverConfig.handleArenaSize, defaultSize) };
+    Platform::DriverConfig validConfig {driverConfig};
+    validConfig.handleArenaSize = std::max(driverConfig.handleArenaSize, defaultSize);
     return new MetalDriver(platform, validConfig);
 }
 
@@ -60,7 +61,7 @@ Dispatcher MetalDriver::getDispatcher() const noexcept {
 
 MetalDriver::MetalDriver(MetalPlatform* platform, const Platform::DriverConfig& driverConfig) noexcept
         : mPlatform(*platform),
-          mContext(new MetalContext),
+          mContext(new MetalContext(driverConfig.textureUseAfterFreePoolSize)),
           mHandleAllocator("Handles", driverConfig.handleArenaSize) {
     mContext->driver = this;
 
@@ -143,6 +144,9 @@ MetalDriver::MetalDriver(MetalPlatform* platform, const Platform::DriverConfig&
         mContext->eventListener = [[MTLSharedEventListener alloc] initWithDispatchQueue:queue];
     }
 
+    mContext->shaderCompiler = new MetalShaderCompiler(mContext->device, *this);
+    mContext->shaderCompiler->init();
+
 #if defined(FILAMENT_METAL_PROFILING)
     mContext->log = os_log_create("com.google.filament", "Metal");
     mContext->signpostId = os_signpost_id_generate(mContext->log);
@@ -157,6 +161,7 @@ MetalDriver::~MetalDriver() noexcept {
     delete mContext->bufferPool;
     delete mContext->blitter;
     delete mContext->timerQueryImpl;
+    delete mContext->shaderCompiler;
     delete mContext;
 }
 
@@ -303,8 +308,9 @@ void MetalDriver::importTextureR(Handle<HwTexture> th, intptr_t i,
         target, levels, format, samples, width, height, depth, usage, metalTexture));
 }
 
-void MetalDriver::createSamplerGroupR(Handle<HwSamplerGroup> sbh, uint32_t size) {
-    mContext->samplerGroups.insert(construct_handle<MetalSamplerGroup>(sbh, size));
+void MetalDriver::createSamplerGroupR(
+        Handle<HwSamplerGroup> sbh, uint32_t size, utils::FixedSizeString<32> debugName) {
+    mContext->samplerGroups.insert(construct_handle<MetalSamplerGroup>(sbh, size, debugName));
 }
 
 void MetalDriver::createRenderPrimitiveR(Handle<HwRenderPrimitive> rph,
@@ -317,7 +323,7 @@ void MetalDriver::createRenderPrimitiveR(Handle<HwRenderPrimitive> rph,
 }
 
 void MetalDriver::createProgramR(Handle<HwProgram> rph, Program&& program) {
-    construct_handle<MetalProgram>(rph, mContext->device, program);
+    construct_handle<MetalProgram>(rph, *mContext, std::move(program));
 }
 
 void MetalDriver::createDefaultRenderTargetR(Handle<HwRenderTarget> rth, int dummy) {
@@ -530,8 +536,18 @@ void MetalDriver::destroyTexture(Handle<HwTexture> th) {
         return;
     }
 
-    mContext->textures.erase(handle_cast<MetalTexture>(th));
-    destruct_handle<MetalTexture>(th);
+    auto* metalTexture = handle_cast<MetalTexture>(th);
+    mContext->textures.erase(metalTexture);
+
+    // Free memory from the texture and mark it as freed.
+    metalTexture->terminate();
+
+    // Add this texture handle to our texturesToDestroy queue to be destroyed later.
+    if (auto handleToFree = mContext->texturesToDestroy.push(th)) {
+        // If texturesToDestroy is full, then .push evicts the oldest texture handle in the
+        // queue (or simply th, if use-after-free detection is disabled).
+        destruct_handle<MetalTexture>(handleToFree.value());
+    }
 }
 
 void MetalDriver::destroyRenderTarget(Handle<HwRenderTarget> rth) {
@@ -557,6 +573,12 @@ void MetalDriver::destroyTimerQuery(Handle<HwTimerQuery> tqh) {
 }
 
 void MetalDriver::terminate() {
+    // Terminate any outstanding MetalTextures.
+    while (!mContext->texturesToDestroy.empty()) {
+        Handle<HwTexture> toDestroy = mContext->texturesToDestroy.pop();
+        destruct_handle<MetalTexture>(toDestroy);
+    }
+
     // finish() will flush the pending command buffer and will ensure all GPU work has finished.
     // This must be done before calling bufferPool->reset() to ensure no buffers are in flight.
     finish();
@@ -566,6 +588,7 @@ void MetalDriver::terminate() {
 
     MetalExternalImage::shutdown(*mContext);
     mContext->blitter->shutdown();
+    mContext->shaderCompiler->terminate();
 }
 
 ShaderModel MetalDriver::getShaderModel() const noexcept {
@@ -701,7 +724,7 @@ bool MetalDriver::isStereoSupported() {
 }
 
 bool MetalDriver::isParallelShaderCompileSupported() {
-    return false;
+    return true;
 }
 
 bool MetalDriver::isWorkaroundNeeded(Workaround workaround) {
@@ -854,13 +877,17 @@ void MetalDriver::updateSamplerGroup(Handle<HwSamplerGroup> sbh, BufferDescripto
     assert_invariant(sb->size == data.size / sizeof(SamplerDescriptor));
     auto const* const samplers = (SamplerDescriptor const*) data.buffer;
 
-#ifndef NDEBUG
-    // In debug builds, verify that all the textures in the sampler group are still alive.
+    // Verify that all the textures in the sampler group are still alive.
     // These bugs lead to memory corruption and can be difficult to track down.
     for (size_t s = 0; s < data.size / sizeof(SamplerDescriptor); s++) {
         if (!samplers[s].t) {
             continue;
         }
+        // The difference between this check and the one below is that in release, we do this for
+        // only a set number of recently freed textures, while the debug check is exhaustive.
+        auto* metalTexture = handle_cast<MetalTexture>(samplers[s].t);
+        metalTexture->checkUseAfterFree(sb->debugName.c_str(), s);
+#ifndef NDEBUG
         auto iter = mContext->textures.find(handle_cast<MetalTexture>(samplers[s].t));
         if (iter == mContext->textures.end()) {
             utils::slog.e << "updateSamplerGroup: texture #"
@@ -868,8 +895,8 @@ void MetalDriver::updateSamplerGroup(Handle<HwSamplerGroup> sbh, BufferDescripto
                           << samplers[s].t << utils::io::endl;
         }
         assert_invariant(iter != mContext->textures.end());
-    }
 #endif
+    }
 
     // Create a MTLArgumentEncoder for these textures.
     // Ideally, we would create this encoder at createSamplerGroup time, but we need to know the
@@ -935,7 +962,7 @@ void MetalDriver::updateSamplerGroup(Handle<HwSamplerGroup> sbh, BufferDescripto
 void MetalDriver::compilePrograms(CompilerPriorityQueue priority,
         CallbackHandler* handler, CallbackHandler::Callback callback, void* user) {
     if (callback) {
-        scheduleCallback(handler, user, callback);
+        mContext->shaderCompiler->notifyWhenAllProgramsAreReady(handler, callback, user);
     }
 }
 
@@ -1357,23 +1384,27 @@ void MetalDriver::finalizeSamplerGroup(MetalSamplerGroup* samplerGroup) {
 
     id<MTLCommandBuffer> cmdBuffer = getPendingCommandBuffer(mContext);
 
-#ifndef NDEBUG
-    // In debug builds, verify that all the textures in the sampler group are still alive.
+    // Verify that all the textures in the sampler group are still alive.
     // These bugs lead to memory corruption and can be difficult to track down.
     const auto& handles = samplerGroup->getTextureHandles();
     for (size_t s = 0; s < handles.size(); s++) {
         if (!handles[s]) {
             continue;
         }
-        auto iter = mContext->textures.find(handle_cast<MetalTexture>(handles[s]));
+        // The difference between this check and the one below is that in release, we do this for
+        // only a set number of recently freed textures, while the debug check is exhaustive.
+        auto* metalTexture = handle_cast<MetalTexture>(handles[s]);
+        metalTexture->checkUseAfterFree(samplerGroup->debugName.c_str(), s);
+#ifndef NDEBUG
+        auto iter = mContext->textures.find(metalTexture);
         if (iter == mContext->textures.end()) {
             utils::slog.e << "finalizeSamplerGroup: texture #"
                           << (int) s << " is dead, texture handle = "
                           << handles[s] << utils::io::endl;
         }
         assert_invariant(iter != mContext->textures.end());
-    }
 #endif
+    }
 
     utils::FixedCapacityVector<id<MTLTexture>> newTextures(samplerGroup->size, nil);
     for (size_t binding = 0; binding < samplerGroup->size; binding++) {
@@ -1447,14 +1478,19 @@ void MetalDriver::draw(PipelineState ps, Handle<HwRenderPrimitive> rph, uint32_t
     auto program = handle_cast<MetalProgram>(ps.program);
     const auto& rs = ps.rasterState;
 
+    // This might block until the shader compilation has finished.
+    auto functions = program->getFunctions();
+
     // If the material debugger is enabled, avoid fatal (or cascading) errors and that can occur
     // during the draw call when the program is invalid. The shader compile error has already been
     // dumped to the console at this point, so it's fine to simply return early.
-    if (FILAMENT_ENABLE_MATDBG && UTILS_UNLIKELY(!program->isValid)) {
+    if (FILAMENT_ENABLE_MATDBG && UTILS_UNLIKELY(!functions)) {
         return;
     }
 
-    ASSERT_PRECONDITION(program->isValid, "Attempting to draw with an invalid Metal program.");
+    ASSERT_PRECONDITION(bool(functions), "Attempting to draw with an invalid Metal program.");
+
+    auto [fragment, vertex] = functions.getRasterFunctions();
 
     // Pipeline state
     MTLPixelFormat colorPixelFormat[MRT::MAX_SUPPORTED_RENDER_TARGET_COUNT] = { MTLPixelFormatInvalid };
@@ -1477,8 +1513,8 @@ void MetalDriver::draw(PipelineState ps, Handle<HwRenderPrimitive> rph, uint32_t
         assert_invariant(isMetalFormatStencil(stencilPixelFormat));
     }
     MetalPipelineState pipelineState {
-        .vertexFunction = program->vertexFunction,
-        .fragmentFunction = program->fragmentFunction,
+        .vertexFunction = vertex,
+        .fragmentFunction = fragment,
         .vertexDescription = primitive->vertexDescription,
         .colorAttachmentPixelFormat = {
             colorPixelFormat[0],
@@ -1623,7 +1659,7 @@ void MetalDriver::draw(PipelineState ps, Handle<HwRenderPrimitive> rph, uint32_t
         if (!samplerGroup) {
             continue;
         }
-        const auto& stageFlags = program->samplerGroupInfo[s].stageFlags;
+        const auto& stageFlags = program->getSamplerGroupInfo()[s].stageFlags;
         if (stageFlags == ShaderStageFlags::NONE) {
             continue;
         }
@@ -1650,26 +1686,23 @@ void MetalDriver::draw(PipelineState ps, Handle<HwRenderPrimitive> rph, uint32_t
 
     // Bind the user vertex buffers.
 
-    MetalBuffer* buffers[MAX_VERTEX_BUFFER_COUNT] = {};
+    MetalBuffer* vertexBuffers[MAX_VERTEX_BUFFER_COUNT] = {};
     size_t vertexBufferOffsets[MAX_VERTEX_BUFFER_COUNT] = {};
-    size_t bufferIndex = 0;
+    size_t maxBufferIndex = 0;
 
     auto vb = primitive->vertexBuffer;
-    for (uint32_t attributeIndex = 0; attributeIndex < vb->attributes.size(); attributeIndex++) {
-        const auto& attribute = vb->attributes[attributeIndex];
-        if (attribute.buffer == Attribute::BUFFER_UNUSED) {
-            continue;
-        }
-
-        assert_invariant(vb->buffers[attribute.buffer]);
-        buffers[bufferIndex] = vb->buffers[attribute.buffer];
-        vertexBufferOffsets[bufferIndex] = attribute.offset;
-        bufferIndex++;
+    for (auto m : primitive->bufferMapping) {
+        assert_invariant(
+                m.bufferArgumentIndex >= USER_VERTEX_BUFFER_BINDING_START &&
+                m.bufferArgumentIndex < USER_VERTEX_BUFFER_BINDING_START + MAX_VERTEX_BUFFER_COUNT);
+        size_t vertexBufferIndex = m.bufferArgumentIndex - USER_VERTEX_BUFFER_BINDING_START;
+        vertexBuffers[vertexBufferIndex] = vb->buffers[m.sourceBufferIndex];
+        maxBufferIndex = std::max(maxBufferIndex, vertexBufferIndex);
     }
 
-    const auto bufferCount = bufferIndex;
+    const auto bufferCount = maxBufferIndex + 1;
     MetalBuffer::bindBuffers(getPendingCommandBuffer(mContext), mContext->currentRenderPassEncoder,
-            USER_VERTEX_BUFFER_BINDING_START, MetalBuffer::Stage::VERTEX, buffers,
+            USER_VERTEX_BUFFER_BINDING_START, MetalBuffer::Stage::VERTEX, vertexBuffers,
             vertexBufferOffsets, bufferCount);
 
     // Bind the zero buffer, used for missing vertex attributes.
@@ -1696,21 +1729,26 @@ void MetalDriver::dispatchCompute(Handle<HwProgram> program, math::uint3 workGro
 
     auto mtlProgram = handle_cast<MetalProgram>(program);
 
+    // This might block until the shader compilation has finished.
+    auto functions = mtlProgram->getFunctions();
+
     // If the material debugger is enabled, avoid fatal (or cascading) errors and that can occur
     // during the draw call when the program is invalid. The shader compile error has already been
     // dumped to the console at this point, so it's fine to simply return early.
-    if (FILAMENT_ENABLE_MATDBG && UTILS_UNLIKELY(!mtlProgram->isValid)) {
+    if (FILAMENT_ENABLE_MATDBG && UTILS_UNLIKELY(!functions)) {
         return;
     }
 
-    assert_invariant(mtlProgram->isValid && mtlProgram->computeFunction);
+    auto compute = functions.getComputeFunction();
+
+    assert_invariant(bool(functions) && compute);
 
     id<MTLComputeCommandEncoder> computeEncoder =
             [getPendingCommandBuffer(mContext) computeCommandEncoder];
 
     NSError* error = nil;
     id<MTLComputePipelineState> computePipelineState =
-            [mContext->device newComputePipelineStateWithFunction:mtlProgram->computeFunction
+            [mContext->device newComputePipelineStateWithFunction:compute
                                                             error:&error];
     if (error) {
         auto description = [error.localizedDescription cStringUsingEncoding:NSUTF8StringEncoding];

filament/backend/src/metal/MetalHandles.h

@@ -32,6 +32,7 @@
 #include "private/backend/SamplerGroup.h"
 
 #include <utils/bitset.h>
+#include <utils/CString.h>
 #include <utils/FixedCapacityVector.h>
 #include <utils/Panic.h>
 
@@ -66,6 +67,7 @@ public:
     id<MTLTexture> acquireDrawable();
 
     id<MTLTexture> acquireDepthTexture();
+    id<MTLTexture> acquireStencilTexture();
 
     void releaseDrawable();
 
@@ -94,12 +96,16 @@ private:
     void scheduleFrameScheduledCallback();
     void scheduleFrameCompletedCallback();
 
+    static MTLPixelFormat decideDepthStencilFormat(uint64_t flags);
+    void ensureDepthStencilTexture();
+
     MetalContext& context;
     id<CAMetalDrawable> drawable = nil;
-    id<MTLTexture> depthTexture = nil;
+    id<MTLTexture> depthStencilTexture = nil;
     id<MTLTexture> headlessDrawable = nil;
-    NSUInteger headlessWidth;
-    NSUInteger headlessHeight;
+    MTLPixelFormat depthStencilFormat = MTLPixelFormatInvalid;
+    NSUInteger headlessWidth = 0;
+    NSUInteger headlessHeight = 0;
     CAMetalLayer* layer = nullptr;
     MetalExternalImage externalImage;
     SwapChainType type;
@@ -154,6 +160,7 @@ struct MetalIndexBuffer : public HwIndexBuffer {
 };
 
 struct MetalRenderPrimitive : public HwRenderPrimitive {
+    MetalRenderPrimitive();
     void setBuffers(MetalVertexBuffer* vertexBuffer, MetalIndexBuffer* indexBuffer);
     // The pointers to MetalVertexBuffer and MetalIndexBuffer are "weak".
     // The MetalVertexBuffer and MetalIndexBuffer must outlive the MetalRenderPrimitive.
@@ -163,18 +170,35 @@ struct MetalRenderPrimitive : public HwRenderPrimitive {
 
     // This struct is used to create the pipeline description to describe vertex assembly.
     VertexDescription vertexDescription = {};
+
+    struct Entry {
+        uint8_t sourceBufferIndex = 0;
+        uint8_t stride = 0;
+        // maps to ->
+        uint8_t bufferArgumentIndex = 0;
+
+        Entry(uint8_t sourceBufferIndex, uint8_t stride, uint8_t bufferArgumentIndex)
+                : sourceBufferIndex(sourceBufferIndex),
+                  stride(stride),
+                  bufferArgumentIndex(bufferArgumentIndex) {}
+    };
+    utils::FixedCapacityVector<Entry> bufferMapping;
 };
 
-struct MetalProgram : public HwProgram {
-    MetalProgram(id<MTLDevice> device, const Program& program) noexcept;
+class MetalProgram : public HwProgram {
+public:
+    MetalProgram(MetalContext& context, Program&& program) noexcept;
 
-    id<MTLFunction> vertexFunction;
-    id<MTLFunction> fragmentFunction;
-    id<MTLFunction> computeFunction;
+    const MetalShaderCompiler::MetalFunctionBundle& getFunctions();
+    const Program::SamplerGroupInfo& getSamplerGroupInfo() { return samplerGroupInfo; }
+
+private:
+    void initialize();
 
     Program::SamplerGroupInfo samplerGroupInfo;
-
-    bool isValid = false;
+    MetalContext& mContext;
+    MetalShaderCompiler::MetalFunctionBundle mFunctionBundle;
+    MetalShaderCompiler::program_token_t mToken;
 };
 
 struct PixelBufferShape {
@@ -223,8 +247,8 @@ public:
     // - using the texture as a render target attachment
     // - calling setMinMaxLevels
     // A texture's available mips are consistent throughout a render pass.
-    void setLodRange(uint32_t minLevel, uint32_t maxLevel);
-    void extendLodRangeTo(uint32_t level);
+    void setLodRange(uint16_t minLevel, uint16_t maxLevel);
+    void extendLodRangeTo(uint16_t level);
 
     static MTLPixelFormat decidePixelFormat(MetalContext* context, TextureFormat format);
 
@@ -238,6 +262,26 @@ public:
 
     MTLPixelFormat devicePixelFormat;
 
+    // Frees memory associated with this texture and marks it as "terminated".
+    // Used to track "use after free" scenario.
+    void terminate() noexcept;
+    bool isTerminated() const noexcept { return terminated; }
+    inline void checkUseAfterFree(const char* samplerGroupDebugName, size_t textureIndex) const {
+        if (UTILS_LIKELY(!isTerminated())) {
+            return;
+        }
+        NSString* reason =
+                [NSString stringWithFormat:
+                                  @"Filament Metal texture use after free, sampler group = "
+                                  @"%s, texture index = %zu",
+                          samplerGroupDebugName, textureIndex];
+        NSException* useAfterFreeException =
+                [NSException exceptionWithName:@"MetalTextureUseAfterFree"
+                                        reason:reason
+                                      userInfo:nil];
+        [useAfterFreeException raise];
+    }
+
 private:
     void loadSlice(uint32_t level, MTLRegion region, uint32_t byteOffset, uint32_t slice,
             PixelBufferDescriptor const& data) noexcept;
@@ -254,14 +298,17 @@ private:
     id<MTLTexture> swizzledTextureView = nil;
     id<MTLTexture> lodTextureView = nil;
 
-    uint32_t minLod = UINT_MAX;
-    uint32_t maxLod = 0;
+    uint16_t minLod = std::numeric_limits<uint16_t>::max();
+    uint16_t maxLod = 0;
+
+    bool terminated = false;
 };
 
 class MetalSamplerGroup : public HwSamplerGroup {
 public:
-    explicit MetalSamplerGroup(size_t size) noexcept
+    explicit MetalSamplerGroup(size_t size, utils::FixedSizeString<32> name) noexcept
         : size(size),
+          debugName(name),
           textureHandles(size, Handle<HwTexture>()),
           textures(size, nil),
           samplers(size, nil) {}
@@ -271,12 +318,10 @@ public:
         textureHandles[index] = th;
     }
 
-#ifndef NDEBUG
     // This method is only used for debugging, to ensure all texture handles are alive.
     const auto& getTextureHandles() const {
         return textureHandles;
     }
-#endif
 
     // Encode a MTLTexture into this SamplerGroup at the given index.
     inline void setFinalizedTexture(size_t index, id<MTLTexture> t) {
@@ -322,6 +367,7 @@ public:
     void useResources(id<MTLRenderCommandEncoder> renderPassEncoder);
 
     size_t size;
+    utils::FixedSizeString<32> debugName;
 
 public:
 

filament/backend/src/metal/MetalHandles.mm

@@ -57,8 +57,11 @@ static inline MTLTextureUsage getMetalTextureUsage(TextureUsage usage) {
 }
 
 MetalSwapChain::MetalSwapChain(MetalContext& context, CAMetalLayer* nativeWindow, uint64_t flags)
-        : context(context), layer(nativeWindow), externalImage(context),
-        type(SwapChainType::CAMETALLAYER) {
+    : context(context),
+      depthStencilFormat(decideDepthStencilFormat(flags)),
+      layer(nativeWindow),
+      externalImage(context),
+      type(SwapChainType::CAMETALLAYER) {
 
     if (!(flags & SwapChain::CONFIG_TRANSPARENT) && !nativeWindow.opaque) {
         utils::slog.w << "Warning: Filament SwapChain has no CONFIG_TRANSPARENT flag, "
@@ -79,17 +82,30 @@ MetalSwapChain::MetalSwapChain(MetalContext& context, CAMetalLayer* nativeWindow
 }
 
 MetalSwapChain::MetalSwapChain(MetalContext& context, int32_t width, int32_t height, uint64_t flags)
-        : context(context), headlessWidth(width), headlessHeight(height), externalImage(context),
-        type(SwapChainType::HEADLESS) { }
+    : context(context),
+      depthStencilFormat(decideDepthStencilFormat(flags)),
+      headlessWidth(width),
+      headlessHeight(height),
+      externalImage(context),
+      type(SwapChainType::HEADLESS) {}
 
 MetalSwapChain::MetalSwapChain(MetalContext& context, CVPixelBufferRef pixelBuffer, uint64_t flags)
-        : context(context), externalImage(context), type(SwapChainType::CVPIXELBUFFERREF) {
+    : context(context),
+      depthStencilFormat(decideDepthStencilFormat(flags)),
+      externalImage(context),
+      type(SwapChainType::CVPIXELBUFFERREF) {
     assert_invariant(flags & SWAP_CHAIN_CONFIG_APPLE_CVPIXELBUFFER);
     MetalExternalImage::assertWritableImage(pixelBuffer);
     externalImage.set(pixelBuffer);
     assert_invariant(externalImage.isValid());
 }
 
+MTLPixelFormat MetalSwapChain::decideDepthStencilFormat(uint64_t flags) {
+    // These formats are supported on all devices, both iOS and macOS.
+    return flags & SwapChain::CONFIG_HAS_STENCIL_BUFFER ? MTLPixelFormatDepth32Float_Stencil8
+                                                        : MTLPixelFormatDepth32Float;
+}
+
 MetalSwapChain::~MetalSwapChain() {
     externalImage.set(nullptr);
 }
@@ -156,37 +172,40 @@ void MetalSwapChain::releaseDrawable() {
 }
 
 id<MTLTexture> MetalSwapChain::acquireDepthTexture() {
-    if (depthTexture) {
-        // If the surface size has changed, we'll need to allocate a new depth texture.
-        if (depthTexture.width != getSurfaceWidth() ||
-            depthTexture.height != getSurfaceHeight()) {
-            depthTexture = nil;
+    ensureDepthStencilTexture();
+    assert_invariant(depthStencilTexture);
+    return depthStencilTexture;
+}
+
+id<MTLTexture> MetalSwapChain::acquireStencilTexture() {
+    if (!isMetalFormatStencil(depthStencilFormat)) {
+        return nil;
+    }
+    ensureDepthStencilTexture();
+    assert_invariant(depthStencilTexture);
+    return depthStencilTexture;
+}
+
+void MetalSwapChain::ensureDepthStencilTexture() {
+    NSUInteger width = getSurfaceWidth();
+    NSUInteger height = getSurfaceHeight();
+    if (UTILS_LIKELY(depthStencilTexture)) {
+        // If the surface size has changed, we'll need to allocate a new depth/stencil texture.
+        if (UTILS_UNLIKELY(
+                    depthStencilTexture.width != width || depthStencilTexture.height != height)) {
+            depthStencilTexture = nil;
         } else {
-            return depthTexture;
+            return;
         }
     }
-
-    const MTLPixelFormat depthFormat =
-#if defined(IOS)
-            MTLPixelFormatDepth32Float;
-#else
-    context.device.depth24Stencil8PixelFormatSupported ?
-            MTLPixelFormatDepth24Unorm_Stencil8 : MTLPixelFormatDepth32Float;
-#endif
-
-    const NSUInteger width = getSurfaceWidth();
-    const NSUInteger height = getSurfaceHeight();
     MTLTextureDescriptor* descriptor =
-            [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:depthFormat
+            [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:depthStencilFormat
                                                                width:width
                                                               height:height
                                                            mipmapped:NO];
     descriptor.usage = MTLTextureUsageRenderTarget;
     descriptor.resourceOptions = MTLResourceStorageModePrivate;
-
-    depthTexture = [context.device newTextureWithDescriptor:descriptor];
-
-    return depthTexture;
+    depthStencilTexture = [context.device newTextureWithDescriptor:descriptor];
 }
 
 void MetalSwapChain::setFrameScheduledCallback(FrameScheduledCallback callback, void* user) {
@@ -280,6 +299,9 @@ MetalIndexBuffer::MetalIndexBuffer(MetalContext& context, BufferUsage usage, uin
         uint32_t indexCount) : HwIndexBuffer(elementSize, indexCount),
         buffer(context, BufferObjectBinding::VERTEX, usage, elementSize * indexCount, true) { }
 
+MetalRenderPrimitive::MetalRenderPrimitive()
+    : bufferMapping(utils::FixedCapacityVector<Entry>::with_capacity(MAX_VERTEX_BUFFER_COUNT)) {}
+
 void MetalRenderPrimitive::setBuffers(MetalVertexBuffer* vertexBuffer, MetalIndexBuffer*
         indexBuffer) {
     this->vertexBuffer = vertexBuffer;
@@ -287,118 +309,94 @@ void MetalRenderPrimitive::setBuffers(MetalVertexBuffer* vertexBuffer, MetalInde
 
     const size_t attributeCount = vertexBuffer->attributes.size();
 
+    auto& mapping = bufferMapping;
+    mapping.clear();
     vertexDescription = {};
 
-    // Each attribute gets its own vertex buffer, starting at logical buffer 1.
-    uint32_t bufferIndex = 1;
+    // Set the layout for the zero buffer, which unused attributes are mapped to.
+    vertexDescription.layouts[ZERO_VERTEX_BUFFER_LOGICAL_INDEX] = {
+            .step = MTLVertexStepFunctionConstant, .stride = 16
+    };
+
+    // Here we map each source buffer to a Metal buffer argument.
+    // Each attribute has a source buffer, offset, and stride.
+    // Two source buffers with the same index and stride can share the same Metal buffer argument
+    // index.
+    //
+    // The source buffer is the buffer index that the Filament client sets.
+    //                                       * source buffer
+    // .attribute(VertexAttribute::POSITION, 0, VertexBuffer::AttributeType::FLOAT2, 0, 12)
+    // .attribute(VertexAttribute::UV,       0, VertexBuffer::AttributeType::HALF2,  8, 12)
+    // .attribute(VertexAttribute::COLOR,    1, VertexBuffer::AttributeType::UBYTE4, 0,  4)
+
+    auto allocateOrGetBufferArgumentIndex =
+            [&mapping, currentBufferArgumentIndex = USER_VERTEX_BUFFER_BINDING_START, this](
+                    auto sourceBuffer, auto sourceBufferStride) mutable -> uint8_t {
+        auto match = [&](const auto& e) {
+            return e.sourceBufferIndex == sourceBuffer && e.stride == sourceBufferStride;
+        };
+        if (auto it = std::find_if(mapping.begin(), mapping.end(), match); it != mapping.end()) {
+            return it->bufferArgumentIndex;
+        } else {
+            auto bufferArgumentIndex = currentBufferArgumentIndex++;
+            mapping.emplace_back(sourceBuffer, sourceBufferStride, bufferArgumentIndex);
+            vertexDescription.layouts[bufferArgumentIndex] = {
+                    .step = MTLVertexStepFunctionPerVertex, .stride = sourceBufferStride
+            };
+            return bufferArgumentIndex;
+        }
+    };
+
     for (uint32_t attributeIndex = 0; attributeIndex < attributeCount; attributeIndex++) {
         const auto& attribute = vertexBuffer->attributes[attributeIndex];
-        if (attribute.buffer == Attribute::BUFFER_UNUSED) {
-            const uint8_t flags = attribute.flags;
-            const MTLVertexFormat format = (flags & Attribute::FLAG_INTEGER_TARGET) ?
-                    MTLVertexFormatUInt4 : MTLVertexFormatFloat4;
 
-            // If the attribute is not enabled, bind it to the zero buffer. It's a Metal error for a
-            // shader to read from missing vertex attributes.
+        // If the attribute is unused, bind it to the zero buffer. It's a Metal error for a shader
+        // to read from missing vertex attributes.
+        if (attribute.buffer == Attribute::BUFFER_UNUSED) {
+            const MTLVertexFormat format = (attribute.flags & Attribute::FLAG_INTEGER_TARGET)
+                    ? MTLVertexFormatUInt4
+                    : MTLVertexFormatFloat4;
             vertexDescription.attributes[attributeIndex] = {
-                    .format = format,
-                    .buffer = ZERO_VERTEX_BUFFER_LOGICAL_INDEX,
-                    .offset = 0
-            };
-            vertexDescription.layouts[ZERO_VERTEX_BUFFER_LOGICAL_INDEX] = {
-                    .step = MTLVertexStepFunctionConstant,
-                    .stride = 16
+                    .format = format, .buffer = ZERO_VERTEX_BUFFER_LOGICAL_INDEX, .offset = 0
             };
             continue;
         }
 
+        // Map the source buffer and stride of this attribute to a Metal buffer argument.
+        auto bufferArgumentIndex =
+                allocateOrGetBufferArgumentIndex(attribute.buffer, attribute.stride);
+
         vertexDescription.attributes[attributeIndex] = {
-                .format = getMetalFormat(attribute.type,
-                                         attribute.flags & Attribute::FLAG_NORMALIZED),
-                .buffer = bufferIndex,
-                .offset = 0
+                .format = getMetalFormat(
+                        attribute.type, attribute.flags & Attribute::FLAG_NORMALIZED),
+                .buffer = uint32_t(bufferArgumentIndex),
+                .offset = attribute.offset
         };
-        vertexDescription.layouts[bufferIndex] = {
-                .step = MTLVertexStepFunctionPerVertex,
-                .stride = attribute.stride
-        };
-
-        bufferIndex++;
-    };
+    }
 }
 
-MetalProgram::MetalProgram(id<MTLDevice> device, const Program& program) noexcept
-    : HwProgram(program.getName()), vertexFunction(nil), fragmentFunction(nil),
-            computeFunction(nil), isValid(false) {
-
-    using MetalFunctionPtr = __strong id<MTLFunction>*;
-
-    static_assert(Program::SHADER_TYPE_COUNT == 3, "Only vertex, fragment, and/or compute shaders expected.");
-    MetalFunctionPtr shaderFunctions[3] = { &vertexFunction, &fragmentFunction, &computeFunction };
-
-    const auto& sources = program.getShadersSource();
-    for (size_t i = 0; i < Program::SHADER_TYPE_COUNT; i++) {
-        const auto& source = sources[i];
-        // It's okay for some shaders to be empty, they shouldn't be used in any draw calls.
-        if (source.empty()) {
-            continue;
-        }
-
-        assert_invariant( source[source.size() - 1] == '\0' );
-
-        // the shader string is null terminated and the length includes the null character
-        NSString* objcSource = [[NSString alloc] initWithBytes:source.data()
-                                                        length:source.size() - 1
-                                                      encoding:NSUTF8StringEncoding];
-        NSError* error = nil;
-        // When options is nil, Metal uses the most recent language version available.
-        id<MTLLibrary> library = [device newLibraryWithSource:objcSource
-                                                      options:nil
-                                                        error:&error];
-        if (library == nil) {
-            if (error) {
-                auto description =
-                        [error.localizedDescription cStringUsingEncoding:NSUTF8StringEncoding];
-                utils::slog.w << description << utils::io::endl;
-            }
-            PANIC_LOG("Failed to compile Metal program.");
-            return;
-        }
-
-        MTLFunctionConstantValues* constants = [MTLFunctionConstantValues new];
-        auto const& specializationConstants = program.getSpecializationConstants();
-        for (auto const& sc : specializationConstants) {
-            const std::array<MTLDataType, 3> types{
-                MTLDataTypeInt, MTLDataTypeFloat, MTLDataTypeBool };
-            std::visit([&sc, constants, type = types[sc.value.index()]](auto&& arg) {
-                [constants setConstantValue:&arg
-                                       type:type
-                                    atIndex:sc.id];
-            }, sc.value);
-        }
-
-        id<MTLFunction> function = [library newFunctionWithName:@"main0"
-                                                 constantValues:constants
-                                                          error:&error];
-        if (!program.getName().empty()) {
-            function.label = @(program.getName().c_str());
-        }
-        assert_invariant(function);
-        *shaderFunctions[i] = function;
-    }
-
-    UTILS_UNUSED_IN_RELEASE const bool isRasterizationProgram =
-            vertexFunction != nil && fragmentFunction != nil;
-    UTILS_UNUSED_IN_RELEASE const bool isComputeProgram = computeFunction != nil;
-    // The program must be either a rasterization program XOR a compute program.
-    assert_invariant(isRasterizationProgram != isComputeProgram);
-
-    // All stages of the program have compiled successfully, this is a valid program.
-    isValid = true;
+MetalProgram::MetalProgram(MetalContext& context, Program&& program) noexcept
+    : HwProgram(program.getName()), mContext(context) {
 
     // Save this program's SamplerGroupInfo, it's used during draw calls to bind sampler groups to
     // the appropriate stage(s).
     samplerGroupInfo = program.getSamplerGroupInfo();
+
+    mToken = context.shaderCompiler->createProgram(program.getName(), std::move(program));
+    assert_invariant(mToken);
+}
+
+const MetalShaderCompiler::MetalFunctionBundle& MetalProgram::getFunctions() {
+    initialize();
+    return mFunctionBundle;
+}
+
+void MetalProgram::initialize() {
+    if (!mToken) {
+        return;
+    }
+    mFunctionBundle = mContext.shaderCompiler->getProgram(mToken);
+    assert_invariant(!mToken);
 }
 
 MetalTexture::MetalTexture(MetalContext& context, SamplerType target, uint8_t levels,
@@ -516,6 +514,15 @@ MetalTexture::MetalTexture(MetalContext& context, SamplerType target, uint8_t le
     setLodRange(0, levels - 1);
 }
 
+void MetalTexture::terminate() noexcept {
+    texture = nil;
+    swizzledTextureView = nil;
+    lodTextureView = nil;
+    msaaSidecar = nil;
+    externalImage.set(nullptr);
+    terminated = true;
+}
+
 MetalTexture::~MetalTexture() {
     externalImage.set(nullptr);
 }
@@ -788,14 +795,14 @@ void MetalTexture::loadWithBlit(uint32_t level, uint32_t slice, MTLRegion region
     context.blitter->blit(getPendingCommandBuffer(&context), args, "Texture upload blit");
 }
 
-void MetalTexture::extendLodRangeTo(uint32_t level) {
+void MetalTexture::extendLodRangeTo(uint16_t level) {
     assert_invariant(!isInRenderPass(&context));
     minLod = std::min(minLod, level);
     maxLod = std::max(maxLod, level);
     lodTextureView = nil;
 }
 
-void MetalTexture::setLodRange(uint32_t min, uint32_t max) {
+void MetalTexture::setLodRange(uint16_t min, uint16_t max) {
     assert_invariant(!isInRenderPass(&context));
     assert_invariant(min <= max);
     minLod = min;
@@ -1123,7 +1130,7 @@ MetalRenderTarget::Attachment MetalRenderTarget::getDepthAttachment() {
 MetalRenderTarget::Attachment MetalRenderTarget::getStencilAttachment() {
     Attachment result = stencil;
     if (defaultRenderTarget) {
-        // TODO: do we want the default SwapChain to have a default stencil buffer?
+        result.texture = context->currentDrawSwapChain->acquireStencilTexture();
     }
     return result;
 }

filament/backend/src/metal/MetalShaderCompiler.h

@@ -0,0 +1,110 @@
+/*
+ * Copyright (C) 2023 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef TNT_FILAMENT_BACKEND_METAL_METALSHADERCOMPILER_H
+#define TNT_FILAMENT_BACKEND_METAL_METALSHADERCOMPILER_H
+
+#include "CompilerThreadPool.h"
+
+#include "CallbackManager.h"
+
+#include <backend/CallbackHandler.h>
+#include <backend/Program.h>
+
+#include <utils/CString.h>
+
+#include <Metal/Metal.h>
+
+#include <array>
+#include <memory>
+
+namespace filament::backend {
+
+class MetalDriver;
+
+class MetalShaderCompiler {
+    struct MetalProgramToken;
+
+public:
+    class MetalFunctionBundle {
+    public:
+        MetalFunctionBundle() = default;
+        MetalFunctionBundle(id<MTLFunction> fragment, id<MTLFunction> vertex)
+            : functions{fragment, vertex} {
+            assert_invariant(fragment && vertex);
+            assert_invariant(fragment.functionType == MTLFunctionTypeFragment);
+            assert_invariant(vertex.functionType == MTLFunctionTypeVertex);
+        }
+        explicit MetalFunctionBundle(id<MTLFunction> compute) : functions{compute, nil} {
+            assert_invariant(compute);
+            assert_invariant(compute.functionType == MTLFunctionTypeKernel);
+        }
+
+        std::pair<id<MTLFunction>, id<MTLFunction>> getRasterFunctions() const noexcept {
+            assert_invariant(functions[0].functionType == MTLFunctionTypeFragment);
+            assert_invariant(functions[1].functionType == MTLFunctionTypeVertex);
+            return {functions[0], functions[1]};
+        }
+
+        id<MTLFunction> getComputeFunction() const noexcept {
+            assert_invariant(functions[0].functionType == MTLFunctionTypeKernel);
+            return functions[0];
+        }
+
+        explicit operator bool() const { return functions[0] != nil; }
+
+    private:
+        // Can hold two functions, either:
+        // - fragment and vertex (for rasterization pipelines)
+        // - compute (for compute pipelines)
+        id<MTLFunction> functions[2] = {nil, nil};
+    };
+
+    using program_token_t = std::shared_ptr<MetalProgramToken>;
+
+    explicit MetalShaderCompiler(id<MTLDevice> device, MetalDriver& driver);
+
+    MetalShaderCompiler(MetalShaderCompiler const& rhs) = delete;
+    MetalShaderCompiler(MetalShaderCompiler&& rhs) = delete;
+    MetalShaderCompiler& operator=(MetalShaderCompiler const& rhs) = delete;
+    MetalShaderCompiler& operator=(MetalShaderCompiler&& rhs) = delete;
+
+    void init() noexcept;
+    void terminate() noexcept;
+
+    // Creates a program asynchronously
+    program_token_t createProgram(utils::CString const& name, Program&& program);
+
+    // Returns the functions, blocking if necessary. The Token is destroyed and becomes invalid.
+    MetalFunctionBundle getProgram(program_token_t& token);
+
+    // Destroys a valid token and all associated resources. Used to "cancel" a program compilation.
+    static void terminate(program_token_t& token);
+
+    void notifyWhenAllProgramsAreReady(
+            CallbackHandler* handler, CallbackHandler::Callback callback, void* user);
+
+private:
+    static MetalFunctionBundle compileProgram(const Program& program, id<MTLDevice> device);
+
+    CompilerThreadPool mCompilerThreadPool;
+    id<MTLDevice> mDevice;
+    CallbackManager mCallbackManager;
+};
+
+} // namespace filament::backend
+
+#endif  // TNT_FILAMENT_BACKEND_METAL_METALSHADERCOMPILER_H

filament/backend/src/metal/MetalShaderCompiler.mm

@@ -0,0 +1,222 @@
+/*
+ * Copyright (C) 2023 The Android Open Source Project
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "MetalShaderCompiler.h"
+
+#include "MetalDriver.h"
+
+#include <backend/Program.h>
+
+#include <utils/JobSystem.h>
+#include <utils/Mutex.h>
+
+#include <chrono>
+
+namespace filament::backend {
+
+using namespace utils;
+
+struct MetalShaderCompiler::MetalProgramToken : ProgramToken {
+
+    MetalProgramToken(MetalShaderCompiler& compiler) noexcept
+            : compiler(compiler) {
+    }
+    ~MetalProgramToken() override;
+
+    void set(MetalFunctionBundle p) noexcept {
+        std::unique_lock const l(lock);
+        std::swap(program, p);
+        signaled = true;
+        cond.notify_one();
+    }
+
+    MetalFunctionBundle get() const noexcept {
+        std::unique_lock l(lock);
+        cond.wait(l, [this](){ return signaled; });
+        return program;
+    }
+
+    void wait() const noexcept {
+        std::unique_lock l(lock);
+        cond.wait(l, [this]() { return signaled; });
+    }
+
+    bool isReady() const noexcept {
+        std::unique_lock l(lock);
+        using namespace std::chrono_literals;
+        return cond.wait_for(l, 0s, [this]() { return signaled; });
+    }
+
+    MetalShaderCompiler& compiler;
+    CallbackManager::Handle handle{};
+    MetalFunctionBundle program{};
+    mutable utils::Mutex lock;
+    mutable utils::Condition cond;
+    bool signaled = false;
+};
+
+MetalShaderCompiler::MetalProgramToken::~MetalProgramToken() = default;
+
+MetalShaderCompiler::MetalShaderCompiler(id<MTLDevice> device, MetalDriver& driver)
+        : mDevice(device),
+          mCallbackManager(driver) {
+
+}
+
+void MetalShaderCompiler::init() noexcept {
+    const uint32_t poolSize = 2;
+    mCompilerThreadPool.init(poolSize, []() {}, []() {});
+}
+
+void MetalShaderCompiler::terminate() noexcept {
+    mCompilerThreadPool.terminate();
+    mCallbackManager.terminate();
+}
+
+/* static */ MetalShaderCompiler::MetalFunctionBundle MetalShaderCompiler::compileProgram(
+        const Program& program, id<MTLDevice> device) {
+    std::array<id<MTLFunction>, Program::SHADER_TYPE_COUNT> functions = { nil };
+    const auto& sources = program.getShadersSource();
+    for (size_t i = 0; i < Program::SHADER_TYPE_COUNT; i++) {
+        const auto& source = sources[i];
+        // It's okay for some shaders to be empty, they shouldn't be used in any draw calls.
+        if (source.empty()) {
+            continue;
+        }
+
+        assert_invariant(source[source.size() - 1] == '\0');
+
+        // the shader string is null terminated and the length includes the null character
+        NSString* objcSource = [[NSString alloc] initWithBytes:source.data()
+                                                        length:source.size() - 1
+                                                      encoding:NSUTF8StringEncoding];
+        NSError* error = nil;
+        // When options is nil, Metal uses the most recent language version available.
+        id<MTLLibrary> library = [device newLibraryWithSource:objcSource
+                                                      options:nil
+                                                        error:&error];
+        if (library == nil) {
+            if (error) {
+                auto description =
+                        [error.localizedDescription cStringUsingEncoding:NSUTF8StringEncoding];
+                utils::slog.w << description << utils::io::endl;
+            }
+            PANIC_LOG("Failed to compile Metal program.");
+            return {};
+        }
+
+        MTLFunctionConstantValues* constants = [MTLFunctionConstantValues new];
+        auto const& specializationConstants = program.getSpecializationConstants();
+        for (auto const& sc : specializationConstants) {
+            const std::array<MTLDataType, 3> types{
+                    MTLDataTypeInt, MTLDataTypeFloat, MTLDataTypeBool };
+            std::visit([&sc, constants, type = types[sc.value.index()]](auto&& arg) {
+                [constants setConstantValue:&arg
+                                       type:type
+                                    atIndex:sc.id];
+            }, sc.value);
+        }
+
+        id<MTLFunction> function = [library newFunctionWithName:@"main0"
+                                                 constantValues:constants
+                                                          error:&error];
+        if (!program.getName().empty()) {
+            function.label = @(program.getName().c_str());
+        }
+        assert_invariant(function);
+        functions[i] = function;
+    }
+
+    static_assert(Program::SHADER_TYPE_COUNT == 3,
+            "Only vertex, fragment, and/or compute shaders expected.");
+    id<MTLFunction> vertexFunction = functions[0];
+    id<MTLFunction> fragmentFunction = functions[1];
+    id<MTLFunction> computeFunction = functions[2];
+    const bool isRasterizationProgram = vertexFunction != nil && fragmentFunction != nil;
+    const bool isComputeProgram = computeFunction != nil;
+    // The program must be either a rasterization program XOR a compute program.
+    assert_invariant(isRasterizationProgram != isComputeProgram);
+
+    if (isRasterizationProgram) {
+        return {fragmentFunction, vertexFunction};
+    }
+
+    if (isComputeProgram) {
+        return MetalFunctionBundle{computeFunction};
+    }
+
+    return {};
+}
+
+MetalShaderCompiler::program_token_t MetalShaderCompiler::createProgram(
+        CString const& name, Program&& program) {
+    auto token = std::make_shared<MetalProgramToken>(*this);
+
+    token->handle = mCallbackManager.get();
+
+    CompilerPriorityQueue const priorityQueue = program.getPriorityQueue();
+    mCompilerThreadPool.queue(priorityQueue, token,
+            [this, name, device = mDevice, program = std::move(program), token]() {
+                int sleepTime = atoi(name.c_str());
+                sleep(sleepTime);
+
+                MetalFunctionBundle compiledProgram = compileProgram(program, device);
+
+                token->set(compiledProgram);
+                mCallbackManager.put(token->handle);
+            });
+
+    return token;
+}
+
+MetalShaderCompiler::MetalFunctionBundle MetalShaderCompiler::getProgram(program_token_t& token) {
+    assert_invariant(token);
+
+    if (!token->isReady()) {
+        auto job = mCompilerThreadPool.dequeue(token);
+        if (job) {
+            job();
+        }
+    }
+
+    MetalShaderCompiler::MetalFunctionBundle program = token->get();
+
+    token = nullptr;
+
+    return program;
+}
+
+/* static */ void MetalShaderCompiler::terminate(program_token_t& token) {
+    assert_invariant(token);
+
+    auto job = token->compiler.mCompilerThreadPool.dequeue(token);
+    if (!job) {
+        // The job is being executed right now (or has already executed).
+        token->wait();
+    } else {
+        // The job has not executed yet.
+        token->compiler.mCallbackManager.put(token->handle);
+    }
+
+    token.reset();
+}
+
+void MetalShaderCompiler::notifyWhenAllProgramsAreReady(
+        CallbackHandler* handler, CallbackHandler::Callback callback, void* user) {
+    mCallbackManager.setCallback(handler, callback, user);
+}
+
+} // namespace filament::backend

filament/backend/src/opengl/GLUtils.h

@@ -373,16 +373,19 @@ constexpr inline GLenum getCullingMode(CullingMode mode) noexcept {
 constexpr inline std::pair<GLenum, GLenum> textureFormatToFormatAndType(
         TextureFormat format) noexcept {
     switch (format) {
-        case TextureFormat::RGB8:       return { GL_RGB,                GL_UNSIGNED_BYTE };
-        case TextureFormat::RGBA8:      return { GL_RGBA,               GL_UNSIGNED_BYTE };
-        case TextureFormat::RGB565:     return { GL_RGB,                GL_UNSIGNED_SHORT_5_6_5 };
-        case TextureFormat::RGB5_A1:    return { GL_RGBA,               GL_UNSIGNED_SHORT_5_5_5_1 };
-        case TextureFormat::RGBA4:      return { GL_RGBA,               GL_UNSIGNED_SHORT_4_4_4_4 };
-        case TextureFormat::DEPTH16:    return { GL_DEPTH_COMPONENT,    GL_UNSIGNED_SHORT };
-        case TextureFormat::DEPTH24:    return { GL_DEPTH_COMPONENT,    GL_UNSIGNED_INT };
+        case TextureFormat::R8:         return { 0x1909 /*GL_LUMINANCE*/, GL_UNSIGNED_BYTE };
+        case TextureFormat::RGB8:       return { GL_RGB,                  GL_UNSIGNED_BYTE };
+        case TextureFormat::SRGB8:      return { GL_RGB,                  GL_UNSIGNED_BYTE };
+        case TextureFormat::RGBA8:      return { GL_RGBA,                 GL_UNSIGNED_BYTE };
+        case TextureFormat::SRGB8_A8:   return { GL_RGBA,                 GL_UNSIGNED_BYTE };
+        case TextureFormat::RGB565:     return { GL_RGB,                  GL_UNSIGNED_SHORT_5_6_5 };
+        case TextureFormat::RGB5_A1:    return { GL_RGBA,                 GL_UNSIGNED_SHORT_5_5_5_1 };
+        case TextureFormat::RGBA4:      return { GL_RGBA,                 GL_UNSIGNED_SHORT_4_4_4_4 };
+        case TextureFormat::DEPTH16:    return { GL_DEPTH_COMPONENT,      GL_UNSIGNED_SHORT };
+        case TextureFormat::DEPTH24:    return { GL_DEPTH_COMPONENT,      GL_UNSIGNED_INT };
         case TextureFormat::DEPTH24_STENCIL8:
-                                        return { GL_DEPTH24_STENCIL8,   GL_UNSIGNED_INT_24_8 };
-        default:                        return { GL_NONE,               GL_NONE };
+                                        return { GL_DEPTH24_STENCIL8,     GL_UNSIGNED_INT_24_8 };
+        default:                        return { GL_NONE,                 GL_NONE };
     }
 }
 

filament/backend/src/opengl/OpenGLBlobCache.cpp

@@ -16,6 +16,8 @@
 
 #include "OpenGLBlobCache.h"
 
+#include "OpenGLContext.h"
+
 #include <backend/Platform.h>
 #include <backend/Program.h>
 
@@ -28,17 +30,18 @@ struct OpenGLBlobCache::Blob {
     char data[];
 };
 
-GLuint OpenGLBlobCache::retrieve(BlobCacheKey* outKey, Platform& platform,
-        Program const& program) noexcept {
-    SYSTRACE_CALL();
+OpenGLBlobCache::OpenGLBlobCache(OpenGLContext& gl) noexcept
+    : mCachingSupported(gl.gets.num_program_binary_formats >= 1) {
+}
 
-    if (!platform.hasBlobFunc()) {
+GLuint OpenGLBlobCache::retrieve(BlobCacheKey* outKey, Platform& platform,
+        Program const& program) const noexcept {
+    SYSTRACE_CALL();
+    if (!mCachingSupported || !platform.hasRetrieveBlobFunc()) {
         // the key is never updated in that case
         return 0;
     }
 
-    SYSTRACE_CONTEXT();
-
     GLuint programId = 0;
 
 #ifndef FILAMENT_SILENCE_NOT_SUPPORTED_BY_ES2
@@ -64,8 +67,10 @@ GLuint OpenGLBlobCache::retrieve(BlobCacheKey* outKey, Platform& platform,
 
         programId = glCreateProgram();
 
-        SYSTRACE_NAME("glProgramBinary");
-        glProgramBinary(programId, blob->format, blob->data, programBinarySize);
+        { // scope for systrace
+            SYSTRACE_NAME("glProgramBinary");
+            glProgramBinary(programId, blob->format, blob->data, programBinarySize);
+        }
 
         if (UTILS_UNLIKELY(glGetError() != GL_NO_ERROR)) {
             // glProgramBinary can fail if for instance the driver has been updated
@@ -85,19 +90,28 @@ GLuint OpenGLBlobCache::retrieve(BlobCacheKey* outKey, Platform& platform,
 
 void OpenGLBlobCache::insert(Platform& platform,
         BlobCacheKey const& key, GLuint program) noexcept {
-#ifndef FILAMENT_SILENCE_NOT_SUPPORTED_BY_ES2
     SYSTRACE_CALL();
-    if (platform.hasBlobFunc()) {
-        SYSTRACE_CONTEXT();
-        GLenum format;
-        GLint programBinarySize;
+    if (!mCachingSupported || !platform.hasInsertBlobFunc()) {
+        // the key is never updated in that case
+        return;
+    }
+
+#ifndef FILAMENT_SILENCE_NOT_SUPPORTED_BY_ES2
+    GLenum format;
+    GLint programBinarySize = 0;
+    { // scope for systrace
         SYSTRACE_NAME("glGetProgramiv");
         glGetProgramiv(program, GL_PROGRAM_BINARY_LENGTH, &programBinarySize);
-        if (programBinarySize) {
-            size_t const size = sizeof(Blob) + programBinarySize;
-            std::unique_ptr<Blob, decltype(&::free)> blob{ (Blob*)malloc(size), &::free };
-            SYSTRACE_NAME("glGetProgramBinary");
-            glGetProgramBinary(program, programBinarySize, &programBinarySize, &format, blob->data);
+    }
+    if (programBinarySize) {
+        size_t const size = sizeof(Blob) + programBinarySize;
+        std::unique_ptr<Blob, decltype(&::free)> blob{ (Blob*)malloc(size), &::free };
+        if (UTILS_LIKELY(blob)) {
+            { // scope for systrace
+                SYSTRACE_NAME("glGetProgramBinary");
+                glGetProgramBinary(program, programBinarySize,
+                        &programBinarySize, &format, blob->data);
+            }
             GLenum const error = glGetError();
             if (error == GL_NO_ERROR) {
                 blob->format = format;
@@ -108,18 +122,4 @@ void OpenGLBlobCache::insert(Platform& platform,
 #endif
 }
 
-void OpenGLBlobCache::insert(Platform& platform, BlobCacheKey const& key,
-        GLenum format, void* data, GLsizei programBinarySize) noexcept {
-    SYSTRACE_CALL();
-    if (platform.hasBlobFunc()) {
-        if (programBinarySize) {
-            size_t const size = sizeof(Blob) + programBinarySize;
-            std::unique_ptr<Blob, decltype(&::free)> blob{ (Blob*)malloc(size), &::free };
-            blob->format = format;
-            memcpy(blob->data, data, programBinarySize);
-            platform.insertBlob(key.data(), key.size(), blob.get(), size);
-        }
-    }
-}
-
 } // namespace filament::backend

filament/backend/src/opengl/OpenGLBlobCache.h

@@ -25,20 +25,21 @@ namespace filament::backend {
 
 class Platform;
 class Program;
+class OpenGLContext;
 
 class OpenGLBlobCache {
 public:
-    static GLuint retrieve(BlobCacheKey* key, Platform& platform,
-            Program const& program) noexcept;
+    explicit OpenGLBlobCache(OpenGLContext& gl) noexcept;
 
-    static void insert(Platform& platform,
+    GLuint retrieve(BlobCacheKey* key, Platform& platform,
+            Program const& program) const noexcept;
+
+    void insert(Platform& platform,
             BlobCacheKey const& key, GLuint program) noexcept;
 
-    static void insert(Platform& platform, BlobCacheKey const& key,
-            GLenum format, void* data, GLsizei programBinarySize) noexcept;
-
 private:
     struct Blob;
+    bool mCachingSupported = false;
 };
 
 } // namespace filament::backend

filament/backend/src/opengl/OpenGLContext.cpp

@@ -99,38 +99,41 @@ OpenGLContext::OpenGLContext() noexcept {
 
     if (mFeatureLevel >= FeatureLevel::FEATURE_LEVEL_1) {
 #ifndef FILAMENT_SILENCE_NOT_SUPPORTED_BY_ES2
-        glGetIntegerv(GL_MAX_UNIFORM_BLOCK_SIZE,
-                &gets.max_uniform_block_size);
-        glGetIntegerv(GL_MAX_UNIFORM_BUFFER_BINDINGS,
-                &gets.max_uniform_buffer_bindings);
-        glGetIntegerv(GL_UNIFORM_BUFFER_OFFSET_ALIGNMENT,
-                &gets.uniform_buffer_offset_alignment);
-        glGetIntegerv(GL_MAX_SAMPLES,
-                &gets.max_samples);
-        glGetIntegerv(GL_MAX_DRAW_BUFFERS,
-                &gets.max_draw_buffers);
-        glGetIntegerv(GL_MAX_TRANSFORM_FEEDBACK_SEPARATE_ATTRIBS,
-                &gets.max_transform_feedback_separate_attribs);
 #ifdef GL_EXT_texture_filter_anisotropic
         if (ext.EXT_texture_filter_anisotropic) {
             glGetFloatv(GL_MAX_TEXTURE_MAX_ANISOTROPY_EXT, &gets.max_anisotropy);
         }
 #endif
+        glGetIntegerv(GL_MAX_DRAW_BUFFERS,
+                &gets.max_draw_buffers);
+        glGetIntegerv(GL_MAX_SAMPLES,
+                &gets.max_samples);
+        glGetIntegerv(GL_MAX_TRANSFORM_FEEDBACK_SEPARATE_ATTRIBS,
+                &gets.max_transform_feedback_separate_attribs);
+        glGetIntegerv(GL_MAX_UNIFORM_BLOCK_SIZE,
+                &gets.max_uniform_block_size);
+        glGetIntegerv(GL_MAX_UNIFORM_BUFFER_BINDINGS,
+                &gets.max_uniform_buffer_bindings);
+        glGetIntegerv(GL_NUM_PROGRAM_BINARY_FORMATS,
+                &gets.num_program_binary_formats);
+        glGetIntegerv(GL_UNIFORM_BUFFER_OFFSET_ALIGNMENT,
+                &gets.uniform_buffer_offset_alignment);
 #endif
     }
+
 #ifdef BACKEND_OPENGL_VERSION_GLES
     else {
+        gets.max_anisotropy = 1;
+        gets.max_draw_buffers = 1;
+        gets.max_samples = 1;
+        gets.max_transform_feedback_separate_attribs = 0;
         gets.max_uniform_block_size = 0;
         gets.max_uniform_buffer_bindings = 0;
+        gets.num_program_binary_formats = 0;
         gets.uniform_buffer_offset_alignment = 0;
-        gets.max_samples = 1;
-        gets.max_draw_buffers = 1;
-        gets.max_transform_feedback_separate_attribs = 0;
-        gets.max_anisotropy = 1;
     }
 #endif
 
-
     slog.v << "Feature level: " << +mFeatureLevel << '\n';
     slog.v << "Active workarounds: " << '\n';
     UTILS_NOUNROLL
@@ -143,13 +146,29 @@ OpenGLContext::OpenGLContext() noexcept {
 
 #ifndef NDEBUG
     // this is useful for development
-    slog.v  << "GL_MAX_DRAW_BUFFERS = " << gets.max_draw_buffers << '\n'
-            << "GL_MAX_RENDERBUFFER_SIZE = " << gets.max_renderbuffer_size << '\n'
-            << "GL_MAX_SAMPLES = " << gets.max_samples << '\n'
-            << "GL_MAX_TEXTURE_MAX_ANISOTROPY_EXT = " << gets.max_anisotropy << '\n'
-            << "GL_MAX_UNIFORM_BLOCK_SIZE = " << gets.max_uniform_block_size << '\n'
-            << "GL_MAX_TEXTURE_IMAGE_UNITS = " << gets.max_texture_image_units << '\n'
-            << "GL_UNIFORM_BUFFER_OFFSET_ALIGNMENT = " << gets.uniform_buffer_offset_alignment << '\n'
+    slog.v
+            << "GL_MAX_TEXTURE_MAX_ANISOTROPY_EXT = "
+                    << gets.max_anisotropy << '\n'
+            << "GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS = "
+                    << gets.max_combined_texture_image_units << '\n'
+            << "GL_MAX_DRAW_BUFFERS = "
+                    << gets.max_draw_buffers << '\n'
+            << "GL_MAX_RENDERBUFFER_SIZE = "
+                    << gets.max_renderbuffer_size << '\n'
+            << "GL_MAX_SAMPLES = "
+                    << gets.max_samples << '\n'
+            << "GL_MAX_TEXTURE_IMAGE_UNITS = "
+                    << gets.max_texture_image_units << '\n'
+            << "GL_MAX_TRANSFORM_FEEDBACK_SEPARATE_ATTRIBS = "
+                    << gets.max_transform_feedback_separate_attribs << '\n'
+            << "GL_MAX_UNIFORM_BLOCK_SIZE = "
+                    << gets.max_uniform_block_size << '\n'
+            << "GL_MAX_UNIFORM_BUFFER_BINDINGS = "
+                    << gets.max_uniform_buffer_bindings << '\n'
+            << "GL_NUM_PROGRAM_BINARY_FORMATS = "
+                    << gets.num_program_binary_formats << '\n'
+            << "GL_UNIFORM_BUFFER_OFFSET_ALIGNMENT = "
+                    << gets.uniform_buffer_offset_alignment << '\n'
             ;
     flush(slog.v);
 #endif

filament/backend/src/opengl/OpenGLContext.h

@@ -153,14 +153,15 @@ public:
     // glGet*() values
     struct Gets {
         GLfloat max_anisotropy;
+        GLint max_combined_texture_image_units;
         GLint max_draw_buffers;
         GLint max_renderbuffer_size;
         GLint max_samples;
-        GLint max_uniform_block_size;
         GLint max_texture_image_units;
-        GLint max_combined_texture_image_units;
         GLint max_transform_feedback_separate_attribs;
-        GLint max_uniform_buffer_bindings; 
+        GLint max_uniform_block_size;
+        GLint max_uniform_buffer_bindings;
+        GLint num_program_binary_formats;
         GLint uniform_buffer_offset_alignment;
     } gets = {};
 

filament/backend/src/opengl/OpenGLDriver.cpp

@@ -147,8 +147,8 @@ Driver* OpenGLDriver::create(OpenGLPlatform* const platform,
 #endif
 
     size_t const defaultSize = FILAMENT_OPENGL_HANDLE_ARENA_SIZE_IN_MB * 1024U * 1024U;
-    Platform::DriverConfig const validConfig {
-        .handleArenaSize = std::max(driverConfig.handleArenaSize, defaultSize) };
+    Platform::DriverConfig validConfig {driverConfig};
+    validConfig.handleArenaSize = std::max(driverConfig.handleArenaSize, defaultSize);
     OpenGLDriver* const driver = new OpenGLDriver(ec, validConfig);
     return driver;
 }
@@ -555,7 +555,8 @@ void OpenGLDriver::createProgramR(Handle<HwProgram> ph, Program&& program) {
     CHECK_GL_ERROR(utils::slog.e)
 }
 
-void OpenGLDriver::createSamplerGroupR(Handle<HwSamplerGroup> sbh, uint32_t size) {
+void OpenGLDriver::createSamplerGroupR(Handle<HwSamplerGroup> sbh, uint32_t size,
+        utils::FixedSizeString<32> debugName) {
     DEBUG_MARKER()
 
     construct<GLSamplerGroup>(sbh, size);
@@ -1289,7 +1290,9 @@ void OpenGLDriver::createRenderTargetR(Handle<HwRenderTarget> rth,
                 checkDimensions(rt->gl.color[i], color[i].level);
             }
         }
-        glDrawBuffers((GLsizei)maxDrawBuffers, bufs);
+        if (UTILS_LIKELY(!getContext().isES2())) {
+            glDrawBuffers((GLsizei)maxDrawBuffers, bufs);
+        }
         CHECK_GL_ERROR(utils::slog.e)
     }
 #endif
@@ -1450,6 +1453,11 @@ void OpenGLDriver::destroySamplerGroup(Handle<HwSamplerGroup> sbh) {
     DEBUG_MARKER()
     if (sbh) {
         GLSamplerGroup* sb = handle_cast<GLSamplerGroup*>(sbh);
+        for (auto& binding : mSamplerBindings) {
+            if (binding == sb) {
+                binding = nullptr;
+            }
+        }
         destruct(sbh, sb);
     }
 }
@@ -1458,9 +1466,9 @@ void OpenGLDriver::destroyTexture(Handle<HwTexture> th) {
     DEBUG_MARKER()
 
     if (th) {
+        auto& gl = mContext;
         GLTexture* t = handle_cast<GLTexture*>(th);
         if (UTILS_LIKELY(!t->gl.imported)) {
-            auto& gl = mContext;
             if (UTILS_LIKELY(t->usage & TextureUsage::SAMPLEABLE)) {
                 gl.unbindTexture(t->gl.target, t->gl.id);
                 if (UTILS_UNLIKELY(t->hwStream)) {
@@ -1478,6 +1486,8 @@ void OpenGLDriver::destroyTexture(Handle<HwTexture> th) {
             if (t->gl.sidecarRenderBufferMS) {
                 glDeleteRenderbuffers(1, &t->gl.sidecarRenderBufferMS);
             }
+        } else {
+            gl.unbindTexture(t->gl.target, t->gl.id);
         }
         destruct(th, t);
     }
@@ -1774,6 +1784,10 @@ bool OpenGLDriver::isRenderTargetFormatSupported(TextureFormat format) {
     // support more formats, but it requires querying GL_INTERNALFORMAT_SUPPORTED which is not
     // available in OpenGL ES.
     auto& gl = mContext;
+    if (UTILS_UNLIKELY(gl.isES2())) {
+        auto [es2format, type] = textureFormatToFormatAndType(format);
+        return es2format != GL_NONE && type != GL_NONE;
+    }
     switch (format) {
         // Core formats.
         case TextureFormat::R8:
@@ -2268,8 +2282,16 @@ void OpenGLDriver::setTextureData(GLTexture* t, uint32_t level,
         return;
     }
 
-    GLenum const glFormat = getFormat(p.format);
-    GLenum const glType = getType(p.type);
+    GLenum glFormat;
+    GLenum glType;
+    if (mContext.isES2()) {
+        auto formatAndType = textureFormatToFormatAndType(t->format);
+        glFormat = formatAndType.first;
+        glType = formatAndType.second;
+    } else {
+        glFormat = getFormat(p.format);
+        glType = getType(p.type);
+    }
 
 #ifndef FILAMENT_SILENCE_NOT_SUPPORTED_BY_ES2
     if (!gl.isES2()) {

filament/backend/src/opengl/OpenGLProgram.cpp

@@ -212,6 +212,7 @@ void OpenGLProgram::updateSamplers(OpenGLDriver* const gld) const noexcept {
         assert_invariant(binding < Program::SAMPLER_BINDING_COUNT);
         auto const * const sb = samplerBindings[binding];
         assert_invariant(sb);
+        if (!sb) continue; // should never happen, this would be a user error.
         for (uint8_t j = 0, m = sb->textureUnitEntries.size(); j < m; ++j, ++tmu) { // "<=" on purpose here
             const GLTexture* const t = sb->textureUnitEntries[j].texture;
             if (t) { // program may not use all samplers of sampler group

filament/backend/src/opengl/ShaderCompilerService.cpp

@@ -140,6 +140,7 @@ void* ShaderCompilerService::getUserData(const program_token_t& token) noexcept
 
 ShaderCompilerService::ShaderCompilerService(OpenGLDriver& driver)
         : mDriver(driver),
+          mBlobCache(driver.getContext()),
           mCallbackManager(driver),
           KHR_parallel_shader_compile(driver.getContext().ext.KHR_parallel_shader_compile) {
 }
@@ -219,7 +220,7 @@ ShaderCompilerService::program_token_t ShaderCompilerService::createProgram(
         token->attributes = std::move(program.getAttributes());
     }
 
-    token->gl.program = OpenGLBlobCache::retrieve(&token->key, mDriver.mPlatform, program);
+    token->gl.program = mBlobCache.retrieve(&token->key, mDriver.mPlatform, program);
     if (token->gl.program) {
         return token;
     }
@@ -249,7 +250,7 @@ ShaderCompilerService::program_token_t ShaderCompilerService::createProgram(
                     // We need to query the link status here to guarantee that the
                     // program is compiled and linked now (we don't want this to be
                     // deferred to later). We don't care about the result at this point.
-                    GLint status;
+                    GLint status = GL_FALSE;
                     glGetProgramiv(glProgram, GL_LINK_STATUS, &status);
                     programData.program = glProgram;
 
@@ -262,9 +263,9 @@ ShaderCompilerService::program_token_t ShaderCompilerService::createProgram(
                     mCallbackManager.put(token->handle);
 
                     // caching must be the last thing we do
-                    if (token->key) {
+                    if (token->key && status == GL_TRUE) {
                         // Attempt to cache. This calls glGetProgramBinary.
-                        OpenGLBlobCache::insert(mDriver.mPlatform, token->key, glProgram);
+                        mBlobCache.insert(mDriver.mPlatform, token->key, glProgram);
                     }
                 });
 
@@ -317,7 +318,7 @@ ShaderCompilerService::program_token_t ShaderCompilerService::createProgram(
                 //       do this later, maybe depending on CPU usage?
                 // attempt to cache if we don't have a thread pool (otherwise it's done
                 // by the pool).
-                OpenGLBlobCache::insert(mDriver.mPlatform, token->key, token->gl.program);
+                mBlobCache.insert(mDriver.mPlatform, token->key, token->gl.program);
             }
 
             return true;
@@ -431,7 +432,7 @@ GLuint ShaderCompilerService::initialize(program_token_t& token) noexcept {
             mCallbackManager.put(token->handle);
 
             if (token->key) {
-                OpenGLBlobCache::insert(mDriver.mPlatform, token->key, token->gl.program);
+                mBlobCache.insert(mDriver.mPlatform, token->key, token->gl.program);
             }
         } else {
             // if we don't have a program yet, block until we get it.

filament/backend/src/opengl/ShaderCompilerService.h

@@ -21,6 +21,7 @@
 
 #include "CallbackManager.h"
 #include "CompilerThreadPool.h"
+#include "OpenGLBlobCache.h"
 
 #include <backend/CallbackHandler.h>
 #include <backend/Program.h>
@@ -95,6 +96,7 @@ public:
 
 private:
     OpenGLDriver& mDriver;
+    OpenGLBlobCache mBlobCache;
     CallbackManager mCallbackManager;
     CompilerThreadPool mCompilerThreadPool;
 

filament/backend/src/opengl/platforms/PlatformEGL.cpp

@@ -97,13 +97,32 @@ int PlatformEGL::getOSVersion() const noexcept {
     return 0;
 }
 
+bool PlatformEGL::isOpenGL() const noexcept {
+    return false;
+}
 
 Driver* PlatformEGL::createDriver(void* sharedContext, const Platform::DriverConfig& driverConfig) noexcept {
     mEGLDisplay = eglGetDisplay(EGL_DEFAULT_DISPLAY);
     assert_invariant(mEGLDisplay != EGL_NO_DISPLAY);
 
     EGLint major, minor;
-    EGLBoolean const initialized = eglInitialize(mEGLDisplay, &major, &minor);
+    EGLBoolean initialized = eglInitialize(mEGLDisplay, &major, &minor);
+
+    if (!initialized) {
+        EGLDeviceEXT eglDevice;
+        EGLint numDevices;
+        PFNEGLQUERYDEVICESEXTPROC eglQueryDevicesEXT =
+                (PFNEGLQUERYDEVICESEXTPROC)eglGetProcAddress("eglQueryDevicesEXT");
+        if (eglQueryDevicesEXT != nullptr) {
+            eglQueryDevicesEXT(1, &eglDevice, &numDevices);
+            if(auto* getPlatformDisplay = reinterpret_cast<PFNEGLGETPLATFORMDISPLAYEXTPROC>(
+                    eglGetProcAddress("eglGetPlatformDisplay"))) {
+                mEGLDisplay = getPlatformDisplay(EGL_PLATFORM_DEVICE_EXT, eglDevice, 0);
+                initialized = eglInitialize(mEGLDisplay, &major, &minor);
+            }
+        }
+    }
+
     if (UTILS_UNLIKELY(!initialized)) {
         slog.e << "eglInitialize failed" << io::endl;
         return nullptr;
@@ -148,10 +167,16 @@ Driver* PlatformEGL::createDriver(void* sharedContext, const Platform::DriverCon
     constexpr bool requestES2Context = false;
 #endif
 
-    // Request a ES2 context, devices that support ES3 will return an ES3 context
-    Config contextAttribs = {
-            { EGL_CONTEXT_CLIENT_VERSION, 2 },
-    };
+    Config contextAttribs;
+
+    if (isOpenGL()) {
+        // Request a OpenGL 4.1 context
+        contextAttribs[EGL_CONTEXT_MAJOR_VERSION] = 4;
+        contextAttribs[EGL_CONTEXT_MINOR_VERSION] = 1;
+    } else {
+        // Request a ES2 context, devices that support ES3 will return an ES3 context
+        contextAttribs[EGL_CONTEXT_CLIENT_VERSION] = 2;
+    }
 
     // FOR TESTING ONLY, enforce the ES version we're asking for.
     // FIXME: we should check EGL_ANGLE_create_context_backwards_compatible, however, at least
@@ -172,14 +197,17 @@ Driver* PlatformEGL::createDriver(void* sharedContext, const Platform::DriverCon
     // config use for creating the context
     EGLConfig eglConfig = EGL_NO_CONFIG_KHR;
 
-    // find a config we can use if we don't have "EGL_KHR_no_config_context" and that we can use
-    // for the dummy pbuffer surface.
-    mEGLConfig = findSwapChainConfig(0);
-    if (UTILS_UNLIKELY(mEGLConfig == EGL_NO_CONFIG_KHR)) {
-        goto error; // error already logged
-    }
 
     if (UTILS_UNLIKELY(!ext.egl.KHR_no_config_context)) {
+        // find a config we can use if we don't have "EGL_KHR_no_config_context" and that we can use
+        // for the dummy pbuffer surface.
+        mEGLConfig = findSwapChainConfig(
+                SWAP_CHAIN_CONFIG_TRANSPARENT |
+                SWAP_CHAIN_HAS_STENCIL_BUFFER,
+                true, true);
+        if (UTILS_UNLIKELY(mEGLConfig == EGL_NO_CONFIG_KHR)) {
+            goto error; // error already logged
+        }
         // if we don't have the EGL_KHR_no_config_context the context must be created with
         // the same config as the swapchain, so we have no choice but to create a
         // transparent config.
@@ -333,22 +361,36 @@ void PlatformEGL::terminate() noexcept {
     eglReleaseThread();
 }
 
-EGLConfig PlatformEGL::findSwapChainConfig(uint64_t flags) const {
+EGLConfig PlatformEGL::findSwapChainConfig(uint64_t flags, bool window, bool pbuffer) const {
     // Find config that support ES3.
     EGLConfig config = EGL_NO_CONFIG_KHR;
     EGLint configsCount;
     Config configAttribs = {
-            { EGL_RENDERABLE_TYPE,  EGL_OPENGL_ES2_BIT },
             { EGL_RED_SIZE,         8 },
             { EGL_GREEN_SIZE,       8 },
             { EGL_BLUE_SIZE,        8 },
             { EGL_ALPHA_SIZE,      (flags & SWAP_CHAIN_CONFIG_TRANSPARENT) ? 8 : 0 },
             { EGL_DEPTH_SIZE,      24 },
+            { EGL_STENCIL_SIZE,    (flags & SWAP_CHAIN_HAS_STENCIL_BUFFER) ? 8 : 0 }
     };
 
+    if (!ext.egl.KHR_no_config_context) {
+        if (isOpenGL()) {
+            configAttribs[EGL_RENDERABLE_TYPE] = EGL_OPENGL_BIT;
+        } else {
+            configAttribs[EGL_RENDERABLE_TYPE] = EGL_OPENGL_ES2_BIT;
+            if (ext.egl.KHR_create_context) {
+                configAttribs[EGL_RENDERABLE_TYPE] |= EGL_OPENGL_ES3_BIT_KHR;
+            }
+        }
+    }
 
-    if (ext.egl.KHR_create_context) {
-        configAttribs[EGL_RECORDABLE_ANDROID] |= EGL_OPENGL_ES3_BIT_KHR;
+    if (window) {
+        configAttribs[EGL_SURFACE_TYPE] |= EGL_WINDOW_BIT;
+    }
+
+    if (pbuffer) {
+        configAttribs[EGL_SURFACE_TYPE] |= EGL_PBUFFER_BIT;
     }
 
     if (ext.egl.ANDROID_recordable) {
@@ -391,7 +433,7 @@ Platform::SwapChain* PlatformEGL::createSwapChain(
 
     EGLConfig config = EGL_NO_CONFIG_KHR;
     if (UTILS_LIKELY(ext.egl.KHR_no_config_context)) {
-        config = findSwapChainConfig(flags);
+        config = findSwapChainConfig(flags, true, false);
     } else {
         config = mEGLConfig;
     }
@@ -427,7 +469,7 @@ Platform::SwapChain* PlatformEGL::createSwapChain(
 
     EGLConfig config = EGL_NO_CONFIG_KHR;
     if (UTILS_LIKELY(ext.egl.KHR_no_config_context)) {
-        config = findSwapChainConfig(flags);
+        config = findSwapChainConfig(flags, false, true);
     } else {
         config = mEGLConfig;
     }
@@ -569,7 +611,7 @@ EGLint& PlatformEGL::Config::operator[](EGLint name) {
     auto pos = std::find_if(mConfig.begin(), mConfig.end(),
             [name](auto&& v) { return v.first == name; });
     if (pos == mConfig.end()) {
-        mConfig.insert(pos - 1, { name, EGL_NONE });
+        mConfig.insert(pos - 1, { name, 0 });
         pos = mConfig.end() - 2;
     }
     return pos->second;

filament/backend/src/opengl/platforms/PlatformEGLAndroid.cpp

@@ -185,7 +185,7 @@ AcquiredImage PlatformEGLAndroid::transformAcquiredImage(AcquiredImage source) n
         AcquiredImage acquiredImage;
         EGLDisplay display;
     };
-    Closure* closure = new Closure(source, mEGLDisplay);
+    Closure* closure = new(std::nothrow) Closure(source, mEGLDisplay);
     auto patchedCallback = [](void* image, void* userdata) {
         Closure* closure = (Closure*)userdata;
         if (eglDestroyImageKHR(closure->display, (EGLImageKHR) image) == EGL_FALSE) {

filament/backend/src/opengl/platforms/PlatformEGLHeadless.cpp

@@ -30,21 +30,14 @@ using namespace utils;
 namespace filament {
 using namespace backend;
 
-namespace glext {
-UTILS_PRIVATE PFNEGLCREATESYNCKHRPROC eglCreateSyncKHR = {};
-UTILS_PRIVATE PFNEGLDESTROYSYNCKHRPROC eglDestroySyncKHR = {};
-UTILS_PRIVATE PFNEGLCLIENTWAITSYNCKHRPROC eglClientWaitSyncKHR = {};
-UTILS_PRIVATE PFNEGLCREATEIMAGEKHRPROC eglCreateImageKHR = {};
-UTILS_PRIVATE PFNEGLDESTROYIMAGEKHRPROC eglDestroyImageKHR = {};
-}
-using namespace glext;
-
-// ---------------------------------------------------------------------------------------------
-
 PlatformEGLHeadless::PlatformEGLHeadless() noexcept
         : PlatformEGL() {
 }
 
+bool PlatformEGLHeadless::isOpenGL() const noexcept {
+    return  true;
+}
+
 backend::Driver* PlatformEGLHeadless::createDriver(void* sharedContext,
         const Platform::DriverConfig& driverConfig) noexcept {
     EGLBoolean bindAPI = eglBindAPI(EGL_OPENGL_API);
@@ -58,166 +51,7 @@ backend::Driver* PlatformEGLHeadless::createDriver(void* sharedContext,
         return nullptr;
     }
 
-    // Copied from the base class and modified slightly. Should be cleaned up/improved later.
-    mEGLDisplay = eglGetDisplay(EGL_DEFAULT_DISPLAY);
-    assert_invariant(mEGLDisplay != EGL_NO_DISPLAY);
-
-    EGLint major, minor;
-    EGLBoolean initialized = eglInitialize(mEGLDisplay, &major, &minor);
-
-    if (!initialized) {
-      EGLDeviceEXT eglDevice;
-      EGLint numDevices;
-
-      PFNEGLQUERYDEVICESEXTPROC eglQueryDevicesEXT =
-              (PFNEGLQUERYDEVICESEXTPROC)eglGetProcAddress("eglQueryDevicesEXT");
-      if (eglQueryDevicesEXT != NULL) {
-          eglQueryDevicesEXT(1, &eglDevice, &numDevices);
-          if(auto* getPlatformDisplay = reinterpret_cast<PFNEGLGETPLATFORMDISPLAYEXTPROC>(
-              eglGetProcAddress("eglGetPlatformDisplay"))) {
-            mEGLDisplay = getPlatformDisplay(EGL_PLATFORM_DEVICE_EXT, eglDevice, 0);
-            initialized = eglInitialize(mEGLDisplay, &major, &minor);
-          }
-      }
-    }
-
-    if (UTILS_UNLIKELY(!initialized)) {
-        slog.e << "eglInitialize failed" << io::endl;
-        return nullptr;
-    }
-
-    auto extensions = GLUtils::split(eglQueryString(mEGLDisplay, EGL_EXTENSIONS));
-
-    eglCreateSyncKHR = (PFNEGLCREATESYNCKHRPROC) eglGetProcAddress("eglCreateSyncKHR");
-    eglDestroySyncKHR = (PFNEGLDESTROYSYNCKHRPROC) eglGetProcAddress("eglDestroySyncKHR");
-    eglClientWaitSyncKHR = (PFNEGLCLIENTWAITSYNCKHRPROC) eglGetProcAddress("eglClientWaitSyncKHR");
-
-    eglCreateImageKHR = (PFNEGLCREATEIMAGEKHRPROC) eglGetProcAddress("eglCreateImageKHR");
-    eglDestroyImageKHR = (PFNEGLDESTROYIMAGEKHRPROC) eglGetProcAddress("eglDestroyImageKHR");
-
-    EGLint configsCount;
-
-    EGLint configAttribs[] = {
-           EGL_SURFACE_TYPE, EGL_PBUFFER_BIT,
-           EGL_RED_SIZE, 8,
-           EGL_GREEN_SIZE, 8,
-           EGL_BLUE_SIZE, 8,
-           EGL_ALPHA_SIZE,  0,
-           EGL_DEPTH_SIZE, 32,
-           EGL_RENDERABLE_TYPE, EGL_OPENGL_BIT,
-           EGL_NONE
-    };
-
-    EGLint contextAttribs[] = {
-            EGL_CONTEXT_CLIENT_VERSION, 3,
-            EGL_NONE, EGL_NONE, // reserved for EGL_CONTEXT_OPENGL_NO_ERROR_KHR below
-            EGL_NONE
-    };
-
-    EGLint pbufferAttribs[] = {
-            EGL_WIDTH,  1,
-            EGL_HEIGHT, 1,
-            EGL_NONE
-    };
-
-#ifdef NDEBUG
-    // When we don't have a shared context and we're in release mode, we always activate the
-    // EGL_KHR_create_context_no_error extension.
-    if (!sharedContext && extensions.has("EGL_KHR_create_context_no_error")) {
-        contextAttribs[2] = EGL_CONTEXT_OPENGL_NO_ERROR_KHR;
-        contextAttribs[3] = EGL_TRUE;
-    }
-#endif
-
-    EGLConfig eglConfig = nullptr;
-
-    // find an opaque config
-    if (!eglChooseConfig(mEGLDisplay, configAttribs, &mEGLConfig, 1, &configsCount)) {
-        logEglError("eglChooseConfig");
-        goto error;
-    }
-
-    // fallback to a 24-bit depth buffer
-    if (configsCount == 0) {
-        configAttribs[10] = EGL_DEPTH_SIZE;
-        configAttribs[11] = 24;
-
-        if (!eglChooseConfig(mEGLDisplay, configAttribs, &mEGLConfig, 1, &configsCount)) {
-            logEglError("eglChooseConfig");
-            goto error;
-        }
-    }
-
-    // find a transparent config
-    configAttribs[8] = EGL_ALPHA_SIZE;
-    configAttribs[9] = 8;
-    if (!eglChooseConfig(mEGLDisplay, configAttribs, &mEGLTransparentConfig, 1, &configsCount) ||
-            (configAttribs[13] == EGL_DONT_CARE && configsCount == 0)) {
-        logEglError("eglChooseConfig");
-        goto error;
-    }
-
-    if (!extensions.has("EGL_KHR_no_config_context")) {
-        // if we have the EGL_KHR_no_config_context, we don't need to worry about the config
-        // when creating the context, otherwise, we must always pick a transparent config.
-        eglConfig = mEGLConfig = mEGLTransparentConfig;
-    }
-
-    // the pbuffer dummy surface is always created with a transparent surface because
-    // either we have EGL_KHR_no_config_context and it doesn't matter, or we don't and
-    // we must use a transparent surface
-    mEGLDummySurface = eglCreatePbufferSurface(mEGLDisplay, mEGLTransparentConfig, pbufferAttribs);
-    if (mEGLDummySurface == EGL_NO_SURFACE) {
-        logEglError("eglCreatePbufferSurface");
-        goto error;
-    }
-
-    mEGLContext = eglCreateContext(mEGLDisplay, eglConfig, (EGLContext)sharedContext, contextAttribs);
-    if (mEGLContext == EGL_NO_CONTEXT && sharedContext &&
-        extensions.has("EGL_KHR_create_context_no_error")) {
-        // context creation could fail because of EGL_CONTEXT_OPENGL_NO_ERROR_KHR
-        // not matching the sharedContext. Try with it.
-        contextAttribs[2] = EGL_CONTEXT_OPENGL_NO_ERROR_KHR;
-        contextAttribs[3] = EGL_TRUE;
-        mEGLContext = eglCreateContext(mEGLDisplay, eglConfig, (EGLContext)sharedContext, contextAttribs);
-    }
-    if (UTILS_UNLIKELY(mEGLContext == EGL_NO_CONTEXT)) {
-        // eglCreateContext failed
-        logEglError("eglCreateContext");
-        goto error;
-    }
-
-    if (!makeCurrent(mEGLDummySurface, mEGLDummySurface)) {
-        // eglMakeCurrent failed
-        logEglError("eglMakeCurrent");
-        goto error;
-    }
-
-    initializeGlExtensions();
-
-    clearGlError();
-
-    // success!!
-    return OpenGLPlatform::createDefaultDriver(this, sharedContext, driverConfig);
-
-error:
-    // if we're here, we've failed
-    if (mEGLDummySurface) {
-        eglDestroySurface(mEGLDisplay, mEGLDummySurface);
-    }
-    if (mEGLContext) {
-        eglDestroyContext(mEGLDisplay, mEGLContext);
-    }
-
-    mEGLDummySurface = EGL_NO_SURFACE;
-    mEGLContext = EGL_NO_CONTEXT;
-
-    eglTerminate(mEGLDisplay);
-    eglReleaseThread();
-
-    return nullptr;
+    return PlatformEGL::createDriver(sharedContext, driverConfig);
 }
 
 } // namespace filament
-
-// ---------------------------------------------------------------------------------------------

filament/backend/src/opengl/platforms/PlatformWGL.cpp

@@ -168,7 +168,7 @@ error:
 }
 
 bool PlatformWGL::isExtraContextSupported() const noexcept {
-    return true;
+    return false;
 }
 
 void PlatformWGL::createContext(bool shared) {

filament/backend/src/vulkan/VulkanBlitter.cpp

@@ -237,11 +237,12 @@ void VulkanBlitter::lazyInit() noexcept {
 
     VkShaderModule vertexShader = decode(VKSHADERS_BLITDEPTHVS_DATA, VKSHADERS_BLITDEPTHVS_SIZE);
     VkShaderModule fragmentShader = decode(VKSHADERS_BLITDEPTHFS_DATA, VKSHADERS_BLITDEPTHFS_SIZE);
-    mDepthResolveProgram = new VulkanProgram(mDevice, vertexShader, fragmentShader);
 
     // Allocate one anonymous sampler at slot 0.
-    mDepthResolveProgram->samplerGroupInfo[0].samplers.reserve(1);
-    mDepthResolveProgram->samplerGroupInfo[0].samplers.resize(1);
+    VulkanProgram::CustomSamplerInfoList samplers = {
+        {0, 0, ShaderStageFlags::FRAGMENT},
+    };
+    mDepthResolveProgram = new VulkanProgram(mDevice, vertexShader, fragmentShader, samplers);
 
 #if FVK_ENABLED(FVK_DEBUG_BLITTER)
     utils::slog.d << "Created Shader Module for VulkanBlitter "
@@ -359,7 +360,7 @@ void VulkanBlitter::blitSlowDepth(VkFilter filter, const VkExtent2D srcExtent, V
     // DRAW THE TRIANGLE
     // -----------------
 
-    mPipelineCache.bindProgram(*mDepthResolveProgram);
+    mPipelineCache.bindProgram(mDepthResolveProgram);
     mPipelineCache.bindPrimitiveTopology(VK_PRIMITIVE_TOPOLOGY_TRIANGLE_STRIP);
 
     auto vkraster = mPipelineCache.getCurrentRasterState();

filament/backend/src/vulkan/VulkanContext.h

@@ -20,10 +20,12 @@
 #include "VulkanConstants.h"
 #include "VulkanImageUtility.h"
 #include "VulkanPipelineCache.h"
+#include "VulkanUtility.h"
 
+#include <utils/bitset.h>
+#include <utils/FixedCapacityVector.h>
 #include <utils/Mutex.h>
 #include <utils/Slice.h>
-#include <utils/bitset.h>
 
 #include <memory>
 
@@ -100,8 +102,8 @@ public:
         return (uint32_t) VK_MAX_MEMORY_TYPES;
     }
 
-    inline VkFormat getDepthFormat() const {
-        return mDepthFormat;
+    inline VkFormatList const& getAttachmentDepthFormats() const {
+        return mDepthFormats;
     }
 
     inline VkPhysicalDeviceLimits const& getPhysicalDeviceLimits() const noexcept {
@@ -130,7 +132,7 @@ private:
     bool mDebugMarkersSupported = false;
     bool mDebugUtilsSupported = false;
 
-    VkFormat mDepthFormat;
+    VkFormatList mDepthFormats;
 
     // For convenience so that VulkanPlatform can initialize the private fields.
     friend class VulkanPlatform;

filament/backend/src/vulkan/VulkanDriver.cpp

@@ -156,7 +156,8 @@ VulkanDriver::VulkanDriver(VulkanPlatform* platform, VulkanContext const& contex
       mThreadSafeResourceManager(&mResourceAllocator),
       mPipelineCache(&mResourceAllocator),
       mBlitter(mStagePool, mPipelineCache, mFramebufferCache, mSamplerCache),
-      mReadPixels(mPlatform->getDevice()) {
+      mReadPixels(mPlatform->getDevice()),
+      mIsSRGBSwapChainSupported(mPlatform->getCustomization().isSRGBSwapChainSupported) {
 
 #if FVK_ENABLED(FVK_DEBUG_VALIDATION)
     UTILS_UNUSED const PFN_vkCreateDebugReportCallbackEXT createDebugReportCallback
@@ -214,8 +215,8 @@ Driver* VulkanDriver::create(VulkanPlatform* platform, VulkanContext const& cont
          Platform::DriverConfig const& driverConfig) noexcept {
     assert_invariant(platform);
     size_t defaultSize = FVK_HANDLE_ARENA_SIZE_IN_MB * 1024U * 1024U;
-    Platform::DriverConfig validConfig{
-            .handleArenaSize = std::max(driverConfig.handleArenaSize, defaultSize)};
+    Platform::DriverConfig validConfig {driverConfig};
+    validConfig.handleArenaSize = std::max(driverConfig.handleArenaSize, defaultSize);
     return new VulkanDriver(platform, context, validConfig);
 }
 
@@ -315,7 +316,8 @@ void VulkanDriver::finish(int dummy) {
     FVK_SYSTRACE_END();
 }
 
-void VulkanDriver::createSamplerGroupR(Handle<HwSamplerGroup> sbh, uint32_t count) {
+void VulkanDriver::createSamplerGroupR(Handle<HwSamplerGroup> sbh, uint32_t count,
+        utils::FixedSizeString<32> debugName) {
     auto sg = mResourceAllocator.construct<VulkanSamplerGroup>(sbh, count);
     mResourceManager.acquire(sg);
 }
@@ -705,10 +707,6 @@ FenceStatus VulkanDriver::getFenceStatus(Handle<HwFence> fh) {
 // the GPU supports the given texture format with non-zero optimal tiling features.
 bool VulkanDriver::isTextureFormatSupported(TextureFormat format) {
     VkFormat vkformat = getVkFormat(format);
-    // We automatically use an alternative format when the client requests DEPTH24.
-    if (format == TextureFormat::DEPTH24) {
-        vkformat = mContext.getDepthFormat();
-    }
     if (vkformat == VK_FORMAT_UNDEFINED) {
         return false;
     }
@@ -736,10 +734,6 @@ bool VulkanDriver::isTextureFormatMipmappable(TextureFormat format) {
 
 bool VulkanDriver::isRenderTargetFormatSupported(TextureFormat format) {
     VkFormat vkformat = getVkFormat(format);
-    // We automatically use an alternative format when the client requests DEPTH24.
-    if (format == TextureFormat::DEPTH24) {
-        vkformat = mContext.getDepthFormat();
-    }
     if (vkformat == VK_FORMAT_UNDEFINED) {
         return false;
     }
@@ -765,7 +759,7 @@ bool VulkanDriver::isAutoDepthResolveSupported() {
 }
 
 bool VulkanDriver::isSRGBSwapChainSupported() {
-    return mPlatform->isSRGBSwapChainSupported();
+    return mIsSRGBSwapChainSupported;
 }
 
 bool VulkanDriver::isStereoSupported() {
@@ -1548,7 +1542,7 @@ void VulkanDriver::draw(PipelineState pipelineState, Handle<HwRenderPrimitive> r
     VkDeviceSize const* offsets = prim.vertexBuffer->getOffsets();
 
     // Push state changes to the VulkanPipelineCache instance. This is fast and does not make VK calls.
-    mPipelineCache.bindProgram(*program);
+    mPipelineCache.bindProgram(program);
     mPipelineCache.bindRasterState(mPipelineCache.getCurrentRasterState());
     mPipelineCache.bindPrimitiveTopology(prim.primitiveTopology);
     mPipelineCache.bindVertexArray(attribDesc, bufferDesc, bufferCount);
@@ -1560,68 +1554,69 @@ void VulkanDriver::draw(PipelineState pipelineState, Handle<HwRenderPrimitive> r
     VkDescriptorImageInfo samplerInfo[VulkanPipelineCache::SAMPLER_BINDING_COUNT] = {};
     VulkanTexture* samplerTextures[VulkanPipelineCache::SAMPLER_BINDING_COUNT] = {nullptr};
 
-    VulkanPipelineCache::UsageFlags usage;
+    auto const& bindingToSamplerIndex = program->getBindingToSamplerIndex();
+    VulkanPipelineCache::UsageFlags  usage = program->getUsage();
 
     UTILS_NOUNROLL
-    for (uint8_t samplerGroupIdx = 0; samplerGroupIdx < Program::SAMPLER_BINDING_COUNT; samplerGroupIdx++) {
-        const auto& samplerGroup = program->samplerGroupInfo[samplerGroupIdx];
-        const auto& samplers = samplerGroup.samplers;
-        if (samplers.empty()) {
+    for (uint8_t binding = 0; binding < VulkanPipelineCache::SAMPLER_BINDING_COUNT; binding++) {
+        uint16_t const indexPair = bindingToSamplerIndex[binding];
+
+        if (indexPair == 0xffff) {
+            usage = VulkanPipelineCache::disableUsageFlags(binding, usage);
             continue;
         }
-        VulkanSamplerGroup* vksb = mSamplerBindings[samplerGroupIdx];
+
+        uint16_t const samplerGroupInd = (indexPair >> 8) & 0xff;
+        uint16_t const samplerInd = (indexPair & 0xff);
+
+        VulkanSamplerGroup* vksb = mSamplerBindings[samplerGroupInd];
         if (!vksb) {
+            usage = VulkanPipelineCache::disableUsageFlags(binding, usage);
             continue;
         }
-        SamplerGroup* sb = vksb->sb.get();
-        assert_invariant(sb->getSize() == samplers.size());
-        size_t samplerIdx = 0;
-        for (auto& sampler : samplers) {
-            const SamplerDescriptor* boundSampler = sb->data() + samplerIdx;
-            samplerIdx++;
+        SamplerDescriptor const* boundSampler = ((SamplerDescriptor*) vksb->sb->data()) + samplerInd;
 
-            if (UTILS_LIKELY(boundSampler->t)) {
-                VulkanTexture* texture = mResourceAllocator.handle_cast<VulkanTexture*>(boundSampler->t);
-                VkImageViewType const expectedType = texture->getViewType();
-
-                // TODO: can this uninitialized check be checked in a higher layer?
-                // This fallback path is very flaky because the dummy texture might not have
-                // matching characteristics. (e.g. if the missing texture is a 3D texture)
-                if (UTILS_UNLIKELY(texture->getPrimaryImageLayout() == VulkanLayout::UNDEFINED)) {
-#if FVK_ENABLED(FVK_DEBUG_TEXTURE)
-                    utils::slog.w << "Uninitialized texture bound to '" << sampler.name.c_str() << "'";
-                    utils::slog.w << " in material '" << program->name.c_str() << "'";
-                    utils::slog.w << " at binding point " << +sampler.binding << utils::io::endl;
-#endif
-                    texture = mEmptyTexture.get();
-                }
-
-                const SamplerParams& samplerParams = boundSampler->s;
-                VkSampler vksampler = mSamplerCache.getSampler(samplerParams);
-
-                usage = VulkanPipelineCache::getUsageFlags(sampler.binding, samplerGroup.stageFlags, usage);
-
-                VkImageView imageView = VK_NULL_HANDLE;
-                VkImageSubresourceRange const range = texture->getPrimaryViewRange();
-                if (any(texture->usage & TextureUsage::DEPTH_ATTACHMENT)
-                        && expectedType == VK_IMAGE_VIEW_TYPE_2D) {
-                    // If the sampler is part of a mipmapped depth texture, where one of the level
-                    // *can* be an attachment, then the sampler for this texture has the same view
-                    // properties as a view for an attachment. Therefore, we can use
-                    // getAttachmentView to get a corresponding VkImageView.
-                    imageView = texture->getAttachmentView(range);
-                } else {
-                    imageView = texture->getViewForType(range, expectedType);
-                }
-
-                samplerInfo[sampler.binding] = {
-                    .sampler = vksampler,
-                    .imageView = imageView,
-                    .imageLayout = ImgUtil::getVkLayout(texture->getPrimaryImageLayout())
-                };
-                samplerTextures[sampler.binding] = texture;
-            }
+        if (UTILS_UNLIKELY(!boundSampler->t)) {
+            usage = VulkanPipelineCache::disableUsageFlags(binding, usage);
+            continue;
         }
+
+        VulkanTexture* texture = mResourceAllocator.handle_cast<VulkanTexture*>(boundSampler->t);
+        VkImageViewType const expectedType = texture->getViewType();
+
+        // TODO: can this uninitialized check be checked in a higher layer?
+        // This fallback path is very flaky because the dummy texture might not have
+        // matching characteristics. (e.g. if the missing texture is a 3D texture)
+        if (UTILS_UNLIKELY(texture->getPrimaryImageLayout() == VulkanLayout::UNDEFINED)) {
+#if FVK_ENABLED(FVK_DEBUG_TEXTURE)
+            utils::slog.w << "Uninitialized texture bound to '" << sampler.name.c_str() << "'";
+            utils::slog.w << " in material '" << program->name.c_str() << "'";
+            utils::slog.w << " at binding point " << +sampler.binding << utils::io::endl;
+#endif
+            texture = mEmptyTexture.get();
+        }
+
+        SamplerParams const& samplerParams = boundSampler->s;
+        VkSampler const vksampler = mSamplerCache.getSampler(samplerParams);
+        VkImageView imageView = VK_NULL_HANDLE;
+        VkImageSubresourceRange const range = texture->getPrimaryViewRange();
+        if (any(texture->usage & TextureUsage::DEPTH_ATTACHMENT) &&
+                expectedType == VK_IMAGE_VIEW_TYPE_2D) {
+            // If the sampler is part of a mipmapped depth texture, where one of the level *can* be
+            // an attachment, then the sampler for this texture has the same view properties as a
+            // view for an attachment. Therefore, we can use getAttachmentView to get a
+            // corresponding VkImageView.
+            imageView = texture->getAttachmentView(range);
+        } else {
+            imageView = texture->getViewForType(range, expectedType);
+        }
+
+        samplerInfo[binding] = {
+            .sampler = vksampler,
+            .imageView = imageView,
+            .imageLayout = ImgUtil::getVkLayout(texture->getPrimaryImageLayout())
+        };
+        samplerTextures[binding] = texture;
     }
 
     mPipelineCache.bindSamplers(samplerInfo, samplerTextures, usage);

filament/backend/src/vulkan/VulkanDriver.h

@@ -112,6 +112,8 @@ private:
     VulkanBlitter mBlitter;
     VulkanSamplerGroup* mSamplerBindings[VulkanPipelineCache::SAMPLER_BINDING_COUNT] = {};
     VulkanReadPixels mReadPixels;
+
+    bool const mIsSRGBSwapChainSupported;
 };
 
 } // namespace filament::backend

filament/backend/src/vulkan/VulkanHandles.cpp

@@ -49,88 +49,100 @@ static void clampToFramebuffer(VkRect2D* rect, uint32_t fbWidth, uint32_t fbHeig
 VulkanProgram::VulkanProgram(VkDevice device, const Program& builder) noexcept
     : HwProgram(builder.getName()),
       VulkanResource(VulkanResourceType::PROGRAM),
+      mInfo(new PipelineInfo(builder.getSpecializationConstants().size())),
       mDevice(device) {
-    auto const& blobs = builder.getShadersSource();
-    VkShaderModule* modules[2] = {&bundle.vertex, &bundle.fragment};
-    // TODO: handle compute shaders.
-    for (size_t i = 0; i < 2; i++) {
+    auto& blobs = builder.getShadersSource();
+    auto& modules = mInfo->shaders;
+    for (size_t i = 0; i < MAX_SHADER_MODULES; i++) {
         const auto& blob = blobs[i];
-        VkShaderModule* module = modules[i];
-        VkShaderModuleCreateInfo moduleInfo = {};
-        moduleInfo.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO;
-        moduleInfo.codeSize = blob.size();
-        moduleInfo.pCode = (uint32_t*) blob.data();
-        VkResult result = vkCreateShaderModule(mDevice, &moduleInfo, VKALLOC, module);
+        uint32_t* data = (uint32_t*)blob.data();
+        VkShaderModule& module = modules[i];
+        VkShaderModuleCreateInfo moduleInfo = {
+            .sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO,
+            .codeSize = blob.size(),
+            .pCode = data,
+        };
+        VkResult result = vkCreateShaderModule(mDevice, &moduleInfo, VKALLOC, &module);
         ASSERT_POSTCONDITION(result == VK_SUCCESS, "Unable to create shader module.");
     }
 
+    // Note that bools are 4-bytes in Vulkan
+    // https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkBool32.html
+    constexpr uint32_t const CONSTANT_SIZE = 4;
+
     // populate the specialization constants requirements right now
     auto const& specializationConstants = builder.getSpecializationConstants();
-    if (!specializationConstants.empty()) {
-        // Allocate a single heap block to store all the specialization constants structures
-        // our supported types are int32, float and bool, so we use 4 bytes per data. bool will
-        // just use the first byte.
-        char* pStorage = (char*)malloc(
-                sizeof(VkSpecializationInfo) +
-                specializationConstants.size() * sizeof(VkSpecializationMapEntry) +
-                specializationConstants.size() * 4);
-
-        VkSpecializationInfo* const pInfo = (VkSpecializationInfo*)pStorage;
-        VkSpecializationMapEntry* const pEntries =
-                (VkSpecializationMapEntry*)(pStorage + sizeof(VkSpecializationInfo));
-        void* pData = pStorage + sizeof(VkSpecializationInfo) +
-                      specializationConstants.size() * sizeof(VkSpecializationMapEntry);
-
-        *pInfo = {
-                .mapEntryCount = specializationConstants.size(),
-                .pMapEntries = pEntries,
-                .dataSize = specializationConstants.size() * 4,
-                .pData = pData,
+    uint32_t const specConstCount = static_cast<uint32_t>(specializationConstants.size());
+    char* specData = mInfo->specConstData.get();
+    if (specConstCount > 0) {
+        mInfo->specializationInfo = {
+            .mapEntryCount = specConstCount,
+            .pMapEntries = mInfo->specConsts.data(),
+            .dataSize = specConstCount * CONSTANT_SIZE,
+            .pData = specData,
         };
-
-        for (size_t i = 0; i < specializationConstants.size(); i++) {
-            uint32_t const offset = uint32_t(i) * 4;
-            pEntries[i] = {
-                .constantID = specializationConstants[i].id,
-                .offset = offset,
-                // Note that bools are 4-bytes in Vulkan
-                // https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkBool32.html
-                .size = 4,
-            };
-
-            using SpecConstant = Program::SpecializationConstant::Type;
-            char const* addr = (char*)pData + offset;
-            SpecConstant const& arg = specializationConstants[i].value;
-            if (std::holds_alternative<bool>(arg)) {
-                *((VkBool32*)addr) = std::get<bool>(arg) ? VK_TRUE : VK_FALSE;
-            } else if (std::holds_alternative<float>(arg)) {
-                *((float*)addr) = std::get<float>(arg);
-            } else {
-                *((int32_t*)addr) = std::get<int32_t>(arg);
-            }
+    }
+    for (uint32_t i = 0; i < specConstCount; ++i) {
+        uint32_t const offset = i * CONSTANT_SIZE;
+        mInfo->specConsts[i] = {
+            .constantID = specializationConstants[i].id,
+            .offset = offset,
+            .size = CONSTANT_SIZE,
+        };
+        using SpecConstant = Program::SpecializationConstant::Type;
+        char const* addr = (char*)specData + offset;
+        SpecConstant const& arg = specializationConstants[i].value;
+        if (std::holds_alternative<bool>(arg)) {
+            *((VkBool32*)addr) = std::get<bool>(arg) ? VK_TRUE : VK_FALSE;
+        } else if (std::holds_alternative<float>(arg)) {
+            *((float*)addr) = std::get<float>(arg);
+        } else {
+            *((int32_t*)addr) = std::get<int32_t>(arg);
+        }
+    }
+
+    auto& groupInfo = builder.getSamplerGroupInfo();
+    auto& bindingToSamplerIndex = mInfo->bindingToSamplerIndex;
+    auto& usage = mInfo->usage;
+    for (uint8_t groupInd = 0; groupInd < Program::SAMPLER_BINDING_COUNT; groupInd++) {
+        auto const& group = groupInfo[groupInd];
+        auto const& samplers = group.samplers;
+        for (size_t i = 0; i < samplers.size(); ++i) {
+            uint32_t const binding = samplers[i].binding;
+            bindingToSamplerIndex[binding] = (groupInd << 8) | (0xff & i);
+            usage = VulkanPipelineCache::getUsageFlags(binding, group.stageFlags, usage);
         }
-        bundle.specializationInfos = pInfo;
     }
 
-    // Make a copy of the binding map
-    samplerGroupInfo = builder.getSamplerGroupInfo();
     #if FVK_ENABLED(FVK_DEBUG_SHADER_MODULE)
         utils::slog.d << "Created VulkanProgram " << builder << ", shaders = (" << bundle.vertex
                       << ", " << bundle.fragment << ")" << utils::io::endl;
     #endif
 }
 
-VulkanProgram::VulkanProgram(VkDevice device, VkShaderModule vs, VkShaderModule fs) noexcept
+VulkanProgram::VulkanProgram(VkDevice device, VkShaderModule vs, VkShaderModule fs,
+        CustomSamplerInfoList const& samplerInfo) noexcept
     : VulkanResource(VulkanResourceType::PROGRAM),
+      mInfo(new PipelineInfo(0)),
       mDevice(device) {
-    bundle.vertex = vs;
-    bundle.fragment = fs;
+    mInfo->shaders[0] = vs;
+    mInfo->shaders[1] = fs;
+    auto& bindingToSamplerIndex = mInfo->bindingToSamplerIndex;
+    auto& usage = mInfo->usage;
+    bindingToSamplerIndex.resize(samplerInfo.size());
+    for (uint16_t binding = 0; binding < samplerInfo.size(); ++binding) {
+        auto const& sampler = samplerInfo[binding];
+        bindingToSamplerIndex[binding]
+                = (sampler.groupIndex << 8) | (0xff & sampler.samplerIndex);
+        usage = VulkanPipelineCache::getUsageFlags(binding, sampler.flags, usage);
+    }
 }
 
 VulkanProgram::~VulkanProgram() {
-    vkDestroyShaderModule(mDevice, bundle.vertex, VKALLOC);
-    vkDestroyShaderModule(mDevice, bundle.fragment, VKALLOC);
-    free(bundle.specializationInfos);
+    for (auto shader: mInfo->shaders) {
+        vkDestroyShaderModule(mDevice, shader, VKALLOC);
+    }
+    delete mInfo;
 }
 
 // Creates a special "default" render target (i.e. associated with the swap chain)

filament/backend/src/vulkan/VulkanHandles.h

@@ -37,14 +37,64 @@ namespace filament::backend {
 class VulkanTimestamps;
 
 struct VulkanProgram : public HwProgram, VulkanResource {
+
     VulkanProgram(VkDevice device, const Program& builder) noexcept;
-    VulkanProgram(VkDevice device, VkShaderModule vs, VkShaderModule fs) noexcept;
+
+    struct CustomSamplerInfo {
+        uint8_t groupIndex;
+        uint8_t samplerIndex;
+        ShaderStageFlags flags;
+    };
+    using CustomSamplerInfoList = utils::FixedCapacityVector<CustomSamplerInfo>;
+
+    // We allow custom descriptor of the samplers within shaders.  This is needed if we want to use
+    // a program that exists only in the backend - for example, for shader-based bliting.
+    VulkanProgram(VkDevice device, VkShaderModule vs, VkShaderModule fs,
+            CustomSamplerInfoList const& samplerInfo) noexcept;
     ~VulkanProgram();
-    VulkanPipelineCache::ProgramBundle bundle;
-    Program::SamplerGroupInfo samplerGroupInfo;
+
+    inline VkShaderModule getVertexShader() const {
+        return mInfo->shaders[0];
+    }
+
+    inline VkShaderModule getFragmentShader() const { return mInfo->shaders[1]; }
+
+    inline VulkanPipelineCache::UsageFlags getUsage() const { return mInfo->usage; }
+
+    inline utils::FixedCapacityVector<uint16_t> const& getBindingToSamplerIndex() const {
+        return mInfo->bindingToSamplerIndex;
+    }
+
+    inline VkSpecializationInfo const& getSpecConstInfo() const {
+        return mInfo->specializationInfo;
+    }
 
 private:
-    VkDevice mDevice;
+    // TODO: handle compute shaders.
+    // The expected order of shaders - from frontend to backend - is vertex, fragment, compute.
+    static constexpr uint8_t MAX_SHADER_MODULES = 2;
+
+    struct PipelineInfo {
+        PipelineInfo(size_t specConstsCount) :
+            bindingToSamplerIndex(MAX_SAMPLER_COUNT, 0xffff),
+            specConsts(specConstsCount, VkSpecializationMapEntry{}),
+            specConstData(new char[specConstsCount * 4])
+        {}
+
+        // This bitset maps to each of the sampler in the sampler groups associated with this
+        // program, and whether each sampler is used in which shader (i.e. vert, frag, compute).
+        VulkanPipelineCache::UsageFlags usage;
+
+        // We store the samplerGroupIndex as the top 8-bit and the index within each group as the lower 8-bit.
+        utils::FixedCapacityVector<uint16_t> bindingToSamplerIndex;
+        VkShaderModule shaders[MAX_SHADER_MODULES] = {VK_NULL_HANDLE};
+        VkSpecializationInfo specializationInfo = {};
+        utils::FixedCapacityVector<VkSpecializationMapEntry> specConsts;
+        std::unique_ptr<char[]> specConstData;
+    };
+
+    PipelineInfo* mInfo;
+    VkDevice mDevice = VK_NULL_HANDLE;
 };
 
 // The render target bundles together a set of attachments, each of which can have one of the

filament/backend/src/vulkan/VulkanPipelineCache.cpp

@@ -65,6 +65,13 @@ VulkanPipelineCache::getUsageFlags(uint16_t binding, ShaderStageFlags flags, Usa
     return src;
 }
 
+VulkanPipelineCache::UsageFlags VulkanPipelineCache::disableUsageFlags(uint16_t binding,
+        UsageFlags src) {
+    src.unset(binding);
+    src.unset(MAX_SAMPLER_COUNT + binding);
+    return src;
+}
+
 VulkanPipelineCache::VulkanPipelineCache(VulkanResourceAllocator* allocator)
     : mCurrentRasterState(createDefaultRasterState()),
       mResourceAllocator(allocator),
@@ -558,12 +565,10 @@ VulkanPipelineCache::PipelineLayoutCacheEntry* VulkanPipelineCache::getOrCreateP
     return &mPipelineLayouts.emplace(mPipelineRequirements.layout, cacheEntry).first.value();
 }
 
-void VulkanPipelineCache::bindProgram(const VulkanProgram& program) noexcept {
-    const VkShaderModule shaders[2] = { program.bundle.vertex, program.bundle.fragment };
-    for (uint32_t ssi = 0; ssi < SHADER_MODULE_COUNT; ssi++) {
-        mPipelineRequirements.shaders[ssi] = shaders[ssi];
-    }
-    mSpecializationRequirements = program.bundle.specializationInfos;
+void VulkanPipelineCache::bindProgram(VulkanProgram* program) noexcept {
+    mPipelineRequirements.shaders[0] = program->getVertexShader();
+    mPipelineRequirements.shaders[1] = program->getFragmentShader();
+    mSpecializationRequirements = &program->getSpecConstInfo();
 }
 
 void VulkanPipelineCache::bindRasterState(const RasterState& rasterState) noexcept {

filament/backend/src/vulkan/VulkanPipelineCache.h

@@ -91,6 +91,7 @@ public:
 
     using UsageFlags = utils::bitset128;
     static UsageFlags getUsageFlags(uint16_t binding, ShaderStageFlags stages, UsageFlags src = {});
+    static UsageFlags disableUsageFlags(uint16_t binding, UsageFlags src);
 
     #pragma clang diagnostic push
     #pragma clang diagnostic warning "-Wpadded"
@@ -150,7 +151,7 @@ public:
     void bindScissor(VkCommandBuffer cmdbuffer, VkRect2D scissor) noexcept;
 
     // Each of the following methods are fast and do not make Vulkan calls.
-    void bindProgram(const VulkanProgram& program) noexcept;
+    void bindProgram(VulkanProgram* program) noexcept;
     void bindRasterState(const RasterState& rasterState) noexcept;
     void bindRenderPass(VkRenderPass renderPass, int subpassIndex) noexcept;
     void bindPrimitiveTopology(VkPrimitiveTopology topology) noexcept;
@@ -415,7 +416,7 @@ private:
     RasterState mCurrentRasterState;
     PipelineKey mPipelineRequirements = {};
     DescriptorKey mDescriptorRequirements = {};
-    VkSpecializationInfo* mSpecializationRequirements = {};
+    VkSpecializationInfo const* mSpecializationRequirements = nullptr;
 
     // Current bindings for the pipeline and descriptor sets.
     PipelineKey mBoundPipeline = {};

filament/backend/src/vulkan/VulkanSwapChain.cpp

@@ -34,6 +34,8 @@ VulkanSwapChain::VulkanSwapChain(VulkanPlatform* platform, VulkanContext const&
       mAllocator(allocator),
       mStagePool(stagePool),
       mHeadless(extent.width != 0 && extent.height != 0 && !nativeWindow),
+      mFlushAndWaitOnResize(platform->getCustomization().flushAndWaitOnWindowResize),
+      mImageReady(VK_NULL_HANDLE),
       mAcquired(false),
       mIsFirstRenderPass(true) {
     swapChain = mPlatform->createSwapChain(nativeWindow, flags, extent);
@@ -42,8 +44,13 @@ VulkanSwapChain::VulkanSwapChain(VulkanPlatform* platform, VulkanContext const&
     VkSemaphoreCreateInfo const createInfo = {
             .sType = VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO,
     };
-    VkResult result = vkCreateSemaphore(mPlatform->getDevice(), &createInfo, nullptr, &mImageReady);
-    ASSERT_POSTCONDITION(result == VK_SUCCESS, "Failed to create semaphore");
+
+    // No need to wait on this semaphore before drawing when in Headless mode.
+    if (!mHeadless) {
+        VkResult result =
+                vkCreateSemaphore(mPlatform->getDevice(), &createInfo, nullptr, &mImageReady);
+        ASSERT_POSTCONDITION(result == VK_SUCCESS, "Failed to create semaphore");
+    }
 
     update();
 }
@@ -55,7 +62,9 @@ VulkanSwapChain::~VulkanSwapChain() {
     mCommands->wait();
 
     mPlatform->destroy(swapChain);
-    vkDestroySemaphore(mPlatform->getDevice(), mImageReady, VKALLOC);
+    if (mImageReady != VK_NULL_HANDLE) {
+        vkDestroySemaphore(mPlatform->getDevice(), mImageReady, VKALLOC);
+    }
 }
 
 void VulkanSwapChain::update() {
@@ -109,8 +118,10 @@ void VulkanSwapChain::acquire(bool& resized) {
 
     // Check if the swapchain should be resized.
     if ((resized = mPlatform->hasResized(swapChain))) {
-        mCommands->flush();
-        mCommands->wait();
+        if (mFlushAndWaitOnResize) {
+            mCommands->flush();
+            mCommands->wait();
+        }
         mPlatform->recreate(swapChain);
         update();
     }
@@ -118,7 +129,9 @@ void VulkanSwapChain::acquire(bool& resized) {
     VkResult const result = mPlatform->acquire(swapChain, mImageReady, &mCurrentSwapIndex);
     ASSERT_POSTCONDITION(result == VK_SUCCESS || result == VK_SUBOPTIMAL_KHR,
             "Cannot acquire in swapchain.");
-    mCommands->injectDependency(mImageReady);
+    if (mImageReady != VK_NULL_HANDLE) {
+        mCommands->injectDependency(mImageReady);
+    }
     mAcquired = true;
 }
 

filament/backend/src/vulkan/VulkanSwapChain.h

@@ -76,6 +76,7 @@ private:
     VmaAllocator mAllocator;
     VulkanStagePool& mStagePool;
     bool const mHeadless;
+    bool const mFlushAndWaitOnResize;
 
     // We create VulkanTextures based on VkImages. VulkanTexture has facilities for doing layout
     // transitions, which are useful here.

filament/backend/src/vulkan/VulkanTexture.cpp

@@ -61,9 +61,7 @@ VulkanTexture::VulkanTexture(VkDevice device, VkPhysicalDevice physicalDevice,
     : HwTexture(target, levels, samples, w, h, depth, tformat, tusage),
       VulkanResource(
               heapAllocated ? VulkanResourceType::HEAP_ALLOCATED : VulkanResourceType::TEXTURE),
-      // Vulkan does not support 24-bit depth, use the official fallback format.
-      mVkFormat(tformat == TextureFormat::DEPTH24 ? context.getDepthFormat()
-                                                  : backend::getVkFormat(tformat)),
+      mVkFormat(backend::getVkFormat(tformat)),
       mViewType(ImgUtil::getViewType(target)),
       mSwizzle(swizzle),
       mStagePool(stagePool),
@@ -146,7 +144,7 @@ VulkanTexture::VulkanTexture(VkDevice device, VkPhysicalDevice physicalDevice,
     // any kind of attachment (color or depth).
     const auto& limits = context.getPhysicalDeviceLimits();
     if (imageInfo.usage & VK_IMAGE_USAGE_SAMPLED_BIT) {
-        samples = reduceSampleCount(samples, isDepthFormat(mVkFormat)
+        samples = reduceSampleCount(samples, isVkDepthFormat(mVkFormat)
                                                      ? limits.sampledImageDepthSampleCounts
                                                      : limits.sampledImageColorSampleCounts);
     }
@@ -274,6 +272,8 @@ void VulkanTexture::updateImage(const PixelBufferDescriptor& data, uint32_t widt
         return;
     }
 
+    assert_invariant(hostData->size > 0 && "Data is empty");
+
     // Otherwise, use vkCmdCopyBufferToImage.
     void* mapped = nullptr;
     VulkanStage const* stage = mStagePool.acquireStage(hostData->size);
@@ -454,7 +454,7 @@ void VulkanTexture::transitionLayout(VkCommandBuffer cmdbuf, const VkImageSubres
                   << "," << range.levelCount << ")"
                   << " from=" << oldLayout << " to=" << newLayout
                   << " format=" << mVkFormat
-                  << " depth=" << isDepthFormat(mVkFormat)
+                  << " depth=" << isVkDepthFormat(mVkFormat)
                   << " slice-by-slice=" << transitionSliceBySlice
                   << utils::io::endl;
 #endif

filament/backend/src/vulkan/VulkanUtility.cpp

@@ -121,8 +121,8 @@ VkFormat getVkFormat(TextureFormat format) {
         case TextureFormat::RGB8UI:            return VK_FORMAT_R8G8B8A8_UINT;
         case TextureFormat::RGB8I:             return VK_FORMAT_R8G8B8A8_SINT;
 
-        case TextureFormat::DEPTH24:
-            return VK_FORMAT_UNDEFINED;
+        // A 32-bit format but 8 bits are unused.
+        case TextureFormat::DEPTH24:           return VK_FORMAT_X8_D24_UNORM_PACK32;
 
         // 32 bits per element.
         case TextureFormat::R32F:              return VK_FORMAT_R32_SFLOAT;
@@ -638,18 +638,12 @@ VkImageAspectFlags getImageAspect(VkFormat format) {
     }
 }
 
-bool isDepthFormat(VkFormat format) {
-    switch (format) {
-        case VK_FORMAT_D16_UNORM:
-        case VK_FORMAT_X8_D24_UNORM_PACK32:
-        case VK_FORMAT_D16_UNORM_S8_UINT:
-        case VK_FORMAT_D24_UNORM_S8_UINT:
-        case VK_FORMAT_D32_SFLOAT:
-        case VK_FORMAT_D32_SFLOAT_S8_UINT:
-            return true;
-        default:
-            return false;
-    }
+bool isVkDepthFormat(VkFormat format) {
+    return (getImageAspect(format) & VK_IMAGE_ASPECT_DEPTH_BIT) != 0;
+}
+
+bool isVkStencilFormat(VkFormat format) {
+    return (getImageAspect(format) & VK_IMAGE_ASPECT_STENCIL_BIT) != 0;
 }
 
 static uint32_t mostSignificantBit(uint32_t x) { return 1ul << (31ul - utils::clz(x)); }

filament/backend/src/vulkan/VulkanUtility.h

@@ -41,7 +41,9 @@ VkShaderStageFlags getShaderStageFlags(ShaderStageFlags stageFlags);
 
 bool equivalent(const VkRect2D& a, const VkRect2D& b);
 bool equivalent(const VkExtent2D& a, const VkExtent2D& b);
-bool isDepthFormat(VkFormat format);
+bool isVkDepthFormat(VkFormat format);
+bool isVkStencilFormat(VkFormat format);
+
 VkImageAspectFlags getImageAspect(VkFormat format);
 uint8_t reduceSampleCount(uint8_t sampleCount, VkSampleCountFlags mask);
 
@@ -86,6 +88,10 @@ utils::FixedCapacityVector<OutType> enumerate(
 #undef EXPAND_ENUM_NO_ARGS
 #undef EXPAND_ENUM_ARGS
 
+
+// Useful shorthands
+using VkFormatList = utils::FixedCapacityVector<VkFormat>;
+
 } // namespace filament::backend
 
 #endif // TNT_FILAMENT_BACKEND_VULKANUTILITY_H

filament/backend/src/vulkan/platform/VulkanPlatform.cpp

@@ -420,9 +420,9 @@ inline int deviceTypeOrder(VkPhysicalDeviceType deviceType) {
 }
 
 VkPhysicalDevice selectPhysicalDevice(VkInstance instance,
-        VulkanPlatform::GPUPreference const& gpuPreference) {
-    FixedCapacityVector<VkPhysicalDevice> const physicalDevices
-            = filament::backend::enumerate(vkEnumeratePhysicalDevices, instance);
+        VulkanPlatform::Customization::GPUPreference const& gpuPreference) {
+    FixedCapacityVector<VkPhysicalDevice> const physicalDevices =
+            filament::backend::enumerate(vkEnumeratePhysicalDevices, instance);
     struct DeviceInfo {
         VkPhysicalDevice device = VK_NULL_HANDLE;
         VkPhysicalDeviceType deviceType = VK_PHYSICAL_DEVICE_TYPE_OTHER;
@@ -488,10 +488,10 @@ VkPhysicalDevice selectPhysicalDevice(VkInstance instance,
                     return true;
                 }
                 if (!pref.deviceName.empty()) {
-                    if (a.name.find(pref.deviceName) != a.name.npos) {
+                    if (a.name.find(pref.deviceName.c_str()) != a.name.npos) {
                         return false;
                     }
-                    if (b.name.find(pref.deviceName) != b.name.npos) {
+                    if (b.name.find(pref.deviceName.c_str()) != b.name.npos) {
                         return true;
                     }
                 }
@@ -508,17 +508,28 @@ VkPhysicalDevice selectPhysicalDevice(VkInstance instance,
     return device;
 }
 
-VkFormat findSupportedFormat(VkPhysicalDevice device) {
+VkFormatList findAttachmentDepthFormats(VkPhysicalDevice device) {
     VkFormatFeatureFlags const features = VK_FORMAT_FEATURE_DEPTH_STENCIL_ATTACHMENT_BIT;
-    VkFormat const formats[] = {VK_FORMAT_D32_SFLOAT, VK_FORMAT_X8_D24_UNORM_PACK32};
+
+    // The ordering here indicates the preference of choosing depth+stencil format.
+    VkFormat const formats[] = {
+        VK_FORMAT_D32_SFLOAT,
+        VK_FORMAT_X8_D24_UNORM_PACK32,
+
+        VK_FORMAT_D32_SFLOAT_S8_UINT,
+        VK_FORMAT_D24_UNORM_S8_UINT,
+    };
+    std::vector<VkFormat> selectedFormats;
     for (VkFormat format: formats) {
         VkFormatProperties props;
         vkGetPhysicalDeviceFormatProperties(device, format, &props);
         if ((props.optimalTilingFeatures & features) == features) {
-            return format;
+            selectedFormats.push_back(format);
         }
     }
-    return VK_FORMAT_UNDEFINED;
+    VkFormatList ret(selectedFormats.size());
+    std::copy(selectedFormats.begin(), selectedFormats.end(), ret.begin());
+    return ret;
 }
 
 }// anonymous namespace
@@ -603,7 +614,7 @@ Driver* VulkanPlatform::createDriver(void* sharedContext,
 
     bluevk::bindInstance(mImpl->mInstance);
 
-    VulkanPlatform::GPUPreference const pref = getPreferredGPU();
+    VulkanPlatform::Customization::GPUPreference const pref = getCustomization().gpu;
     bool const hasGPUPreference = pref.index >= 0 || !pref.deviceName.empty();
     ASSERT_PRECONDITION(!(hasGPUPreference && sharedContext),
             "Cannot both share context and indicate GPU preference");
@@ -660,9 +671,9 @@ Driver* VulkanPlatform::createDriver(void* sharedContext,
     context.mDebugMarkersSupported
             = deviceExts.find(VK_EXT_DEBUG_MARKER_EXTENSION_NAME) != deviceExts.end();
 
-    // Choose a depth format that meets our requirements. Take care not to include stencil formats
-    // just yet, since that would require a corollary change to the "aspect" flags for the VkImage.
-    context.mDepthFormat = findSupportedFormat(mImpl->mPhysicalDevice);
+    context.mDepthFormats = findAttachmentDepthFormats(mImpl->mPhysicalDevice);
+
+    assert_invariant(context.mDepthFormats.size() > 0);
 
 #if FVK_ENABLED(FVK_DEBUG_VALIDATION)
     printDepthFormats(mImpl->mPhysicalDevice);

filament/backend/src/vulkan/platform/VulkanPlatformAndroidLinuxWindows.cpp

@@ -30,8 +30,6 @@
 // Platform specific includes and defines
 #if defined(__ANDROID__)
     #include <android/native_window.h>
-#elif defined(__linux__) && defined(FILAMENT_SUPPORTS_GGP)
-    #include <ggp_c/ggp.h>
 #elif defined(__linux__) && defined(FILAMENT_SUPPORTS_WAYLAND)
     #include <dlfcn.h>
     namespace {
@@ -86,8 +84,6 @@ VulkanPlatform::ExtensionSet VulkanPlatform::getRequiredInstanceExtensions() {
     VulkanPlatform::ExtensionSet ret;
     #if defined(__ANDROID__)
         ret.insert("VK_KHR_android_surface");
-    #elif defined(__linux__) && defined(FILAMENT_SUPPORTS_GGP)
-        ret.insert(VK_GGP_STREAM_DESCRIPTOR_SURFACE_EXTENSION_NAME);
     #elif defined(__linux__) && defined(FILAMENT_SUPPORTS_WAYLAND)
         ret.insert("VK_KHR_wayland_surface");
     #elif LINUX_OR_FREEBSD && defined(FILAMENT_SUPPORTS_X11)
@@ -121,20 +117,6 @@ VulkanPlatform::SurfaceBundle VulkanPlatform::createVkSurfaceKHR(void* nativeWin
         VkResult const result = vkCreateAndroidSurfaceKHR(instance, &createInfo, VKALLOC,
                 (VkSurfaceKHR*) &surface);
         ASSERT_POSTCONDITION(result == VK_SUCCESS, "vkCreateAndroidSurfaceKHR error.");
-    #elif defined(__linux__) && defined(FILAMENT_SUPPORTS_GGP)
-        VkStreamDescriptorSurfaceCreateInfoGGP const surface_create_info = {
-                .sType = VK_STRUCTURE_TYPE_STREAM_DESCRIPTOR_SURFACE_CREATE_INFO_GGP,
-                .streamDescriptor = kGgpPrimaryStreamDescriptor,
-        };
-        PFN_vkCreateStreamDescriptorSurfaceGGP fpCreateStreamDescriptorSurfaceGGP
-                = reinterpret_cast<PFN_vkCreateStreamDescriptorSurfaceGGP>(
-                        vkGetInstanceProcAddr(instance, "vkCreateStreamDescriptorSurfaceGGP"));
-        ASSERT_PRECONDITION(fpCreateStreamDescriptorSurfaceGGP != nullptr,
-                "Error getting VkInstance "
-                "function vkCreateStreamDescriptorSurfaceGGP");
-        VkResult const result = fpCreateStreamDescriptorSurfaceGGP(instance, &surface_create_info,
-                nullptr, (VkSurfaceKHR*) &surface);
-        ASSERT_POSTCONDITION(result == VK_SUCCESS, "vkCreateStreamDescriptorSurfaceGGP error.");
     #elif defined(__linux__) && defined(FILAMENT_SUPPORTS_WAYLAND)
         wl* ptrval = reinterpret_cast<wl*>(nativeWindow);
         extent.width = ptrval->width;

filament/backend/src/vulkan/platform/VulkanPlatformSwapChainImpl.cpp

@@ -30,7 +30,7 @@ namespace {
 
 std::tuple<VkImage, VkDeviceMemory> createImageAndMemory(VulkanContext const& context,
         VkDevice device, VkExtent2D extent, VkFormat format) {
-    bool const isDepth = format == context.getDepthFormat();
+    bool const isDepth = isVkDepthFormat(format);
     // Filament expects blit() to work with any texture, so we almost always set these usage flags.
     // TODO: investigate performance implications of setting these flags.
     VkImageUsageFlags const blittable
@@ -76,6 +76,14 @@ std::tuple<VkImage, VkDeviceMemory> createImageAndMemory(VulkanContext const& co
     return std::tuple(image, imageMemory);
 }
 
+VkFormat selectDepthFormat(VkFormatList const& depthFormats, bool hasStencil) {
+    auto const formatItr = std::find_if(depthFormats.begin(), depthFormats.end(),
+            hasStencil ? isVkStencilFormat : isVkDepthFormat);
+    assert_invariant(
+            formatItr != depthFormats.end() && "Cannot find suitable swapchain depth format");
+    return *formatItr;
+}
+
 }// anonymous namespace
 
 VulkanPlatformSwapChainImpl::VulkanPlatformSwapChainImpl(VulkanContext const& context,
@@ -116,7 +124,8 @@ VulkanPlatformSurfaceSwapChain::VulkanPlatformSurfaceSwapChain(VulkanContext con
       mPhysicalDevice(physicalDevice),
       mSurface(surface),
       mFallbackExtent(fallbackExtent),
-      mUsesRGB((flags & backend::SWAP_CHAIN_CONFIG_SRGB_COLORSPACE) != 0) {
+      mUsesRGB((flags & backend::SWAP_CHAIN_CONFIG_SRGB_COLORSPACE) != 0),
+      mHasStencil((flags & backend::SWAP_CHAIN_HAS_STENCIL_BUFFER) != 0) {
     assert_invariant(surface);
     create();
 }
@@ -152,10 +161,15 @@ VkResult VulkanPlatformSurfaceSwapChain::create() {
     // Find a suitable surface format.
     FixedCapacityVector<VkSurfaceFormatKHR> const surfaceFormats
             = enumerate(vkGetPhysicalDeviceSurfaceFormatsKHR, mPhysicalDevice, mSurface);
-    FixedCapacityVector<VkFormat> expectedFormats
-            = {VK_FORMAT_R8G8B8A8_UNORM, VK_FORMAT_B8G8R8A8_UNORM};
+    std::array<VkFormat, 2> expectedFormats = {
+            VK_FORMAT_R8G8B8A8_UNORM,
+            VK_FORMAT_B8G8R8A8_UNORM,
+    };
     if (mUsesRGB) {
-        expectedFormats = {VK_FORMAT_R8G8B8A8_SRGB, VK_FORMAT_B8G8R8A8_SRGB};
+        expectedFormats = {
+                VK_FORMAT_R8G8B8A8_SRGB,
+                VK_FORMAT_B8G8R8A8_SRGB,
+        };
     }
     for (VkSurfaceFormatKHR const& format: surfaceFormats) {
         if (std::any_of(expectedFormats.begin(), expectedFormats.end(),
@@ -228,13 +242,18 @@ VkResult VulkanPlatformSurfaceSwapChain::create() {
 
     mSwapChainBundle.colors = enumerate(vkGetSwapchainImagesKHR, mDevice, mSwapchain);
     mSwapChainBundle.colorFormat = surfaceFormat.format;
+    mSwapChainBundle.depthFormat =
+            selectDepthFormat(mContext.getAttachmentDepthFormats(), mHasStencil);
+    mSwapChainBundle.depth = createImage(mSwapChainBundle.extent, mSwapChainBundle.depthFormat);
+
     slog.i << "vkCreateSwapchain"
            << ": " << mSwapChainBundle.extent.width << "x" << mSwapChainBundle.extent.height << ", "
            << surfaceFormat.format << ", " << surfaceFormat.colorSpace << ", "
-           << mSwapChainBundle.colors.size() << ", " << caps.currentTransform << io::endl;
+           << "swapchain-size=" << mSwapChainBundle.colors.size() << ", "
+           << "identity-transform=" << (caps.currentTransform == 1) << ", "
+           << "depth=" << mSwapChainBundle.depthFormat
+           << io::endl;
 
-    mSwapChainBundle.depthFormat = mContext.getDepthFormat();
-    mSwapChainBundle.depth = createImage(mSwapChainBundle.extent, mSwapChainBundle.depthFormat);
     return result;
 }
 
@@ -309,7 +328,9 @@ VulkanPlatformHeadlessSwapChain::VulkanPlatformHeadlessSwapChain(VulkanContext c
         images[i] = createImage(extent, mSwapChainBundle.colorFormat);
     }
 
-    mSwapChainBundle.depthFormat = context.getDepthFormat();
+    bool const hasStencil = (flags & backend::SWAP_CHAIN_HAS_STENCIL_BUFFER) != 0;
+    mSwapChainBundle.depthFormat =
+            selectDepthFormat(mContext.getAttachmentDepthFormats(), hasStencil);
     mSwapChainBundle.depth = createImage(extent, mSwapChainBundle.depthFormat);
 }
 
@@ -325,20 +346,7 @@ VkResult VulkanPlatformHeadlessSwapChain::present(uint32_t index, VkSemaphore fi
 VkResult VulkanPlatformHeadlessSwapChain::acquire(VkSemaphore clientSignal, uint32_t* index) {
     *index = mCurrentIndex;
     mCurrentIndex = (mCurrentIndex + 1) % HEADLESS_SWAPCHAIN_SIZE;
-    VkSemaphore const localSignal = clientSignal;
-    VkSubmitInfo const submitInfo{
-            .sType = VK_STRUCTURE_TYPE_SUBMIT_INFO,
-            .waitSemaphoreCount = 0,
-            .pWaitSemaphores = nullptr,
-            .pWaitDstStageMask = nullptr,
-            .commandBufferCount = 0,
-            .pCommandBuffers = nullptr,
-            .signalSemaphoreCount = 1u,
-            .pSignalSemaphores = &localSignal,
-    };
-    UTILS_UNUSED_IN_RELEASE VkResult result = vkQueueSubmit(mQueue, 1, &submitInfo, VK_NULL_HANDLE);
-    assert_invariant(result == VK_SUCCESS);
-    return result;
+    return VK_SUCCESS;
 }
 
 void VulkanPlatformHeadlessSwapChain::destroy() {

filament/backend/src/vulkan/platform/VulkanPlatformSwapChainImpl.h

@@ -108,6 +108,7 @@ private:
     VkExtent2D const mFallbackExtent;
 
     bool mUsesRGB = false;
+    bool mHasStencil = false;
     bool mSuboptimal;
 };
 

filament/backend/test/test_FeedbackLoops.cpp

@@ -193,7 +193,8 @@ TEST_F(BackendTest, FeedbackLoops) {
             sparams.filterMag = SamplerMagFilter::LINEAR;
             sparams.filterMin = SamplerMinFilter::LINEAR_MIPMAP_NEAREST;
             samplers.setSampler(0, { texture, sparams });
-            auto sgroup = api.createSamplerGroup(samplers.getSize());
+            auto sgroup =
+                    api.createSamplerGroup(samplers.getSize(), utils::FixedSizeString<32>("Test"));
             api.updateSamplerGroup(sgroup, samplers.toBufferDescriptor(api));
             auto ubuffer = api.createBufferObject(sizeof(MaterialParams),
                     BufferObjectBinding::UNIFORM, BufferUsage::STATIC);

filament/backend/test/test_LoadImage.cpp

@@ -303,7 +303,7 @@ TEST_F(BackendTest, UpdateImage2D) {
         sparams.filterMag = SamplerMagFilter::LINEAR;
         sparams.filterMin = SamplerMinFilter::LINEAR_MIPMAP_NEAREST;
         samplers.setSampler(0, { texture, sparams });
-        auto sgroup = api.createSamplerGroup(samplers.getSize());
+        auto sgroup = api.createSamplerGroup(samplers.getSize(), utils::FixedSizeString<32>("Test"));
         api.updateSamplerGroup(sgroup, samplers.toBufferDescriptor(api));
 
         api.bindSamplers(0, sgroup);
@@ -394,7 +394,7 @@ TEST_F(BackendTest, UpdateImageSRGB) {
     sparams.filterMag = SamplerMagFilter::LINEAR;
     sparams.filterMin = SamplerMinFilter::LINEAR_MIPMAP_NEAREST;
     samplers.setSampler(0, { texture, sparams });
-    auto sgroup = api.createSamplerGroup(samplers.getSize());
+    auto sgroup = api.createSamplerGroup(samplers.getSize(), utils::FixedSizeString<32>("Test"));
     api.updateSamplerGroup(sgroup, samplers.toBufferDescriptor(api));
 
     api.bindSamplers(0, sgroup);
@@ -469,7 +469,7 @@ TEST_F(BackendTest, UpdateImageMipLevel) {
     sparams.filterMag = SamplerMagFilter::LINEAR;
     sparams.filterMin = SamplerMinFilter::LINEAR_MIPMAP_NEAREST;
     samplers.setSampler(0, { texture, sparams });
-    auto sgroup = api.createSamplerGroup(samplers.getSize());
+    auto sgroup = api.createSamplerGroup(samplers.getSize(), utils::FixedSizeString<32>("Test"));
     api.updateSamplerGroup(sgroup, samplers.toBufferDescriptor(api));
 
     api.bindSamplers(0, sgroup);
@@ -556,7 +556,7 @@ TEST_F(BackendTest, UpdateImage3D) {
     sparams.filterMag = SamplerMagFilter::LINEAR;
     sparams.filterMin = SamplerMinFilter::LINEAR_MIPMAP_NEAREST;
     samplers.setSampler(0, { texture, sparams});
-    auto sgroup = api.createSamplerGroup(samplers.getSize());
+    auto sgroup = api.createSamplerGroup(samplers.getSize(), utils::FixedSizeString<32>("Test"));
     api.updateSamplerGroup(sgroup, samplers.toBufferDescriptor(api));
 
     api.bindSamplers(0, sgroup);

filament/backend/test/test_MipLevels.cpp

@@ -194,7 +194,8 @@ TEST_F(BackendTest, SetMinMaxLevel) {
         samplerParams.filterMag = SamplerMagFilter::NEAREST;
         samplerParams.filterMin = SamplerMinFilter::NEAREST_MIPMAP_NEAREST;
         samplers.setSampler(0, { texture, samplerParams });
-        backend::Handle<HwSamplerGroup> samplerGroup = api.createSamplerGroup(1);
+        backend::Handle<HwSamplerGroup> samplerGroup =
+                api.createSamplerGroup(1, utils::FixedSizeString<32>("Test"));
         api.updateSamplerGroup(samplerGroup, samplers.toBufferDescriptor(api));
         api.bindSamplers(0, samplerGroup);
 
@@ -242,4 +243,4 @@ TEST_F(BackendTest, SetMinMaxLevel) {
     getDriver().purge();
 }
 
-} // namespace test
+} // namespace test

filament/backend/test/test_RenderExternalImage.cpp

@@ -113,7 +113,8 @@ TEST_F(BackendTest, RenderExternalImageWithoutSet) {
 
     SamplerGroup samplers(1);
     samplers.setSampler(0, { texture, {} });
-    backend::Handle<HwSamplerGroup> samplerGroup = getDriverApi().createSamplerGroup(1);
+    backend::Handle<HwSamplerGroup> samplerGroup =
+            getDriverApi().createSamplerGroup(1, utils::FixedSizeString<32>("Test"));
     getDriverApi().updateSamplerGroup(samplerGroup, samplers.toBufferDescriptor(getDriverApi()));
     getDriverApi().bindSamplers(0, samplerGroup);
 
@@ -234,7 +235,8 @@ TEST_F(BackendTest, RenderExternalImage) {
 
     SamplerGroup samplers(1);
     samplers.setSampler(0, { texture, {} });
-    backend::Handle<HwSamplerGroup> samplerGroup = getDriverApi().createSamplerGroup(1);
+    backend::Handle<HwSamplerGroup> samplerGroup =
+            getDriverApi().createSamplerGroup(1, utils::FixedSizeString<32>("Test"));
     getDriverApi().updateSamplerGroup(samplerGroup, samplers.toBufferDescriptor(getDriverApi()));
     getDriverApi().bindSamplers(0, samplerGroup);
 

filament/benchmark/README.md

@@ -6,18 +6,36 @@
 
 ## Running the benchmark
 
-`adb shell /data/local/tmp/benchmark_filament`
+`adb shell /data/local/tmp/benchmark_filament --benchmark_counters_tabular=true`
 
 
 ## Benchmark results
 
+### Macbook Pro M1 Pro
+```
+--------------------------------------------------------------------------------------
+Benchmark                              Time           CPU Iterations items_per_second
+--------------------------------------------------------------------------------------
+FilamentCullingFixture/boxCulling           702 ns        702 ns     819874       729.274M/s
+FilamentCullingFixture/sphereCulling        485 ns        485 ns    1430396       1054.82M/s
+```
+
+### Pixel 8 Pro
+```
+----------------------------------------------------------------------------------------------------------------------------------
+Benchmark                              Time           CPU Iterations        BPU          C        CPI          I items_per_second
+----------------------------------------------------------------------------------------------------------------------------------
+FilamentCullingFixture/boxCulling          1212 ns       1208 ns     578797          0    6.87234   0.328354    20.9297       423.884M/s
+FilamentCullingFixture/sphereCulling        748 ns        745 ns     938377          0    4.24185    0.39125    10.8418       686.839M/s
+```
+
 ### Galaxy S20+
 ```
 ----------------------------------------------------------------------------------------------------------------------------------
 Benchmark                              Time           CPU Iterations        BPU          C        CPI          I items_per_second
 ----------------------------------------------------------------------------------------------------------------------------------
-FilamentFixture/boxCulling          1695 ns       1688 ns     414849          0    9.35888   0.422963     22.127       303.397M/s
-FilamentFixture/sphereCulling       1160 ns       1147 ns     610602          0    6.35746   0.526617    12.0723       446.543M/s
+FilamentCullingFixture/boxCulling          1695 ns       1688 ns     414849          0    9.35888   0.422963     22.127       303.397M/s
+FilamentCullingFixture/sphereCulling       1160 ns       1147 ns     610602          0    6.35746   0.526617    12.0723       446.543M/s
 ```
 
 ### Pixel 4
@@ -25,6 +43,7 @@ FilamentFixture/sphereCulling       1160 ns       1147 ns     610602          0
 ----------------------------------------------------------------------------------------------------------------------------------
 Benchmark                              Time           CPU Iterations        BPU          C        CPI          I items_per_second
 ----------------------------------------------------------------------------------------------------------------------------------
-FilamentFixture/boxCulling          2114 ns       2106 ns     332395          0    9.93665   0.449074     22.127       243.169M/s
-FilamentFixture/sphereCulling       1407 ns       1402 ns     497755          0    6.61423   0.547886    12.0723         365.3M/s
+FilamentCullingFixture/boxCulling          2114 ns       2106 ns     332395          0    9.93665   0.449074     22.127       243.169M/s
+FilamentCullingFixture/sphereCulling       1407 ns       1402 ns     497755          0    6.61423   0.547886    12.0723         365.3M/s
 ```
+

filament/benchmark/benchmark_filament.cpp

@@ -33,7 +33,7 @@ using namespace filament::math;
 using namespace utils;
 
 
-class FilamentFixture : public benchmark::Fixture {
+class FilamentCullingFixture : public benchmark::Fixture {
 protected:
     static constexpr size_t BATCH_SIZE = 512;
 
@@ -45,7 +45,7 @@ protected:
 
 
 public:
-    FilamentFixture() {
+    FilamentCullingFixture() {
 
         std::default_random_engine gen; // NOLINT
         std::uniform_real_distribution<float> rand(-100.0f, 100.0f);
@@ -75,12 +75,12 @@ public:
         visibles = (Culler::result_type*)utils::aligned_alloc(batch * sizeof(*visibles), 32);
     }
 
-    ~FilamentFixture() override {
+    ~FilamentCullingFixture() override {
         utils::aligned_free(visibles);
     }
 };
 
-BENCHMARK_F(FilamentFixture, boxCulling)(benchmark::State& state) {
+BENCHMARK_F(FilamentCullingFixture, boxCulling)(benchmark::State& state) {
     {
         PerformanceCounters pc(state);
         for (auto _ : state) {
@@ -92,7 +92,7 @@ BENCHMARK_F(FilamentFixture, boxCulling)(benchmark::State& state) {
     }
 }
 
-BENCHMARK_F(FilamentFixture, sphereCulling)(benchmark::State& state) {
+BENCHMARK_F(FilamentCullingFixture, sphereCulling)(benchmark::State& state) {
     {
         PerformanceCounters pc(state);
         for (auto _ : state) {

filament/include/filament/BufferObject.h

@@ -110,6 +110,10 @@ public:
      * @return The maximum capacity of the BufferObject.
      */
     size_t getByteCount() const noexcept;
+
+protected:
+    // prevent heap allocation
+    ~BufferObject() = default;
 };
 
 } // namespace filament

filament/include/filament/Camera.h

@@ -26,6 +26,7 @@
 #include <math/mathfwd.h>
 #include <math/vec2.h>
 #include <math/vec4.h>
+#include <math/mat4.h>
 
 namespace utils {
 class Entity;
@@ -172,6 +173,30 @@ public:
         HORIZONTAL      //!< the field-of-view angle is defined on the horizontal axis
     };
 
+    /** Returns the projection matrix from the field-of-view.
+     *
+     * @param fovInDegrees full field-of-view in degrees. 0 < \p fov < 180.
+     * @param aspect       aspect ratio \f$ \frac{width}{height} \f$. \p aspect > 0.
+     * @param near         distance in world units from the camera to the near plane. \p near > 0.
+     * @param far          distance in world units from the camera to the far plane. \p far > \p near.
+     * @param direction    direction of the \p fovInDegrees parameter.
+     *
+     * @see Fov.
+     */
+    static math::mat4 projection(Fov direction, double fovInDegrees,
+            double aspect, double near, double far = std::numeric_limits<double>::infinity());
+
+    /** Returns the projection matrix from the focal length.
+     *
+     * @param focalLengthInMillimeters lens's focal length in millimeters. \p focalLength > 0.
+     * @param aspect      aspect ratio \f$ \frac{width}{height} \f$. \p aspect > 0.
+     * @param near        distance in world units from the camera to the near plane. \p near > 0.
+     * @param far         distance in world units from the camera to the far plane. \p far > \p near.
+     */
+    static math::mat4 projection(double focalLengthInMillimeters,
+            double aspect, double near, double far = std::numeric_limits<double>::infinity());
+
+
     /** Sets the projection matrix from a frustum defined by six planes.
      *
      * @param projection    type of #Projection to use.
@@ -209,7 +234,8 @@ public:
             double bottom, double top,
             double near, double far);
 
-    /** Sets the projection matrix from the field-of-view.
+
+    /** Utility to set the projection matrix from the field-of-view.
      *
      * @param fovInDegrees full field-of-view in degrees. 0 < \p fov < 180.
      * @param aspect       aspect ratio \f$ \frac{width}{height} \f$. \p aspect > 0.
@@ -222,7 +248,7 @@ public:
     void setProjection(double fovInDegrees, double aspect, double near, double far,
                        Fov direction = Fov::VERTICAL);
 
-    /** Sets the projection matrix from the focal length.
+    /** Utility to set the projection matrix from the focal length.
      *
      * @param focalLengthInMillimeters lens's focal length in millimeters. \p focalLength > 0.
      * @param aspect      aspect ratio \f$ \frac{width}{height} \f$. \p aspect > 0.
@@ -232,13 +258,8 @@ public:
     void setLensProjection(double focalLengthInMillimeters,
             double aspect, double near, double far);
 
+
     /** Sets a custom projection matrix.
-     *
-     * The projection matrix must be of one of the following form:
-     *       a 0 tx 0              a 0 0 tx
-     *       0 b ty 0              0 b 0 ty
-     *       0 0 tz c              0 0 c tz
-     *       0 0 -1 0              0 0 0 1
      *
      * The projection matrix must define an NDC system that must match the OpenGL convention,
      * that is all 3 axis are mapped to [-1, 1].
@@ -249,6 +270,19 @@ public:
      */
     void setCustomProjection(math::mat4 const& projection, double near, double far) noexcept;
 
+    /** Sets the projection matrix.
+     *
+     * The projection matrices must define an NDC system that must match the OpenGL convention,
+     * that is all 3 axis are mapped to [-1, 1].
+     *
+     * @param projection  custom projection matrix used for rendering
+     * @param projectionForCulling  custom projection matrix used for culling
+     * @param near        distance in world units from the camera to the near plane. \p near > 0.
+     * @param far         distance in world units from the camera to the far plane. \p far > \p near.
+     */
+    void setCustomProjection(math::mat4 const& projection, math::mat4 const& projectionForCulling,
+            double near, double far) noexcept;
+
     /** Sets a custom projection matrix for each eye.
      *
      * The projectionForCulling, near, and far parameters establish a "culling frustum" which must
@@ -267,25 +301,6 @@ public:
     void setCustomEyeProjection(math::mat4 const* projection, size_t count,
             math::mat4 const& projectionForCulling, double near, double far);
 
-    /** Sets the projection matrix.
-     *
-     * The projection matrices must be of one of the following form:
-     *       a 0 tx 0              a 0 0 tx
-     *       0 b ty 0              0 b 0 ty
-     *       0 0 tz c              0 0 c tz
-     *       0 0 -1 0              0 0 0 1
-     *
-     * The projection matrices must define an NDC system that must match the OpenGL convention,
-     * that is all 3 axis are mapped to [-1, 1].
-     *
-     * @param projection  custom projection matrix used for rendering
-     * @param projectionForCulling  custom projection matrix used for culling
-     * @param near        distance in world units from the camera to the near plane. \p near > 0.
-     * @param far         distance in world units from the camera to the far plane. \p far > \p near.
-     */
-    void setCustomProjection(math::mat4 const& projection, math::mat4 const& projectionForCulling,
-            double near, double far) noexcept;
-
     /** Sets an additional matrix that scales the projection matrix.
      *
      * This is useful to adjust the aspect ratio of the camera independent from its projection.
@@ -524,33 +539,6 @@ public:
      *
      * \param p the projection matrix to inverse
      * \returns the inverse of the projection matrix \p p
-     *
-     * \warning the projection matrix to invert must have one of the form below:
-     * - perspective projection
-     *
-     *      \f$
-     *      \left(
-     *      \begin{array}{cccc}
-     *      a & 0 & tx & 0 \\
-     *      0 & b & ty & 0 \\
-     *      0 & 0 & tz & c \\
-     *      0 & 0 & -1 & 0 \\
-     *      \end{array}
-     *      \right)
-     *      \f$
-     *
-     * - orthographic projection
-     *
-     *      \f$
-     *      \left(
-     *      \begin{array}{cccc}
-     *      a & 0 & 0 & tx \\
-     *      0 & b & 0 & ty \\
-     *      0 & 0 & c & tz \\
-     *      0 & 0 & 0 & 1  \\
-     *      \end{array}
-     *      \right)
-     *      \f$
      */
     static math::mat4 inverseProjection(const math::mat4& p) noexcept;
 
@@ -577,6 +565,10 @@ public:
      * @return                  effective full field of view in degrees
      */
     static double computeEffectiveFov(double fovInDegrees, double focusDistance) noexcept;
+
+protected:
+    // prevent heap allocation
+    ~Camera() = default;
 };
 
 } // namespace filament

filament/include/filament/ColorGrading.h

@@ -478,6 +478,10 @@ public:
     private:
         friend class FColorGrading;
     };
+
+protected:
+    // prevent heap allocation
+    ~ColorGrading() = default;
 };
 
 } // namespace filament

filament/include/filament/DebugRegistry.h

@@ -67,15 +67,28 @@ public:
      * @return Address of the data of the \p name property
      * @{
      */
-    void* getPropertyAddress(const char* name) noexcept;
+    void* getPropertyAddress(const char* name);
+
+    void const* getPropertyAddress(const char* name) const noexcept;
 
     template<typename T>
-    inline T* getPropertyAddress(const char* name) noexcept {
+    inline T* getPropertyAddress(const char* name) {
         return static_cast<T*>(getPropertyAddress(name));
     }
 
     template<typename T>
-    inline bool getPropertyAddress(const char* name, T** p) noexcept {
+    inline T const* getPropertyAddress(const char* name) const noexcept {
+        return static_cast<T*>(getPropertyAddress(name));
+    }
+
+    template<typename T>
+    inline bool getPropertyAddress(const char* name, T** p) {
+        *p = getPropertyAddress<T>(name);
+        return *p != nullptr;
+    }
+
+    template<typename T>
+    inline bool getPropertyAddress(const char* name, T* const* p) const noexcept {
         *p = getPropertyAddress<T>(name);
         return *p != nullptr;
     }
@@ -129,6 +142,10 @@ public:
         float pid_i = 0.0f;
         float pid_d = 0.0f;
     };
+
+protected:
+    // prevent heap allocation
+    ~DebugRegistry() = default;
 };
 
 

filament/include/filament/Engine.h

@@ -172,10 +172,11 @@ public:
     using Platform = backend::Platform;
     using Backend = backend::Backend;
     using DriverConfig = backend::Platform::DriverConfig;
+    using FeatureLevel = backend::FeatureLevel;
 
     /**
      * Config is used to define the memory footprint used by the engine, such as the
-     * command buffer size. Config can be used to customize engine requirements based 
+     * command buffer size. Config can be used to customize engine requirements based
      * on the applications needs.
      *
      *    .perRenderPassArenaSizeMB (default: 3 MiB)
@@ -267,6 +268,29 @@ public:
          * This value does not affect the application's memory usage.
          */
         uint32_t perFrameCommandsSizeMB = FILAMENT_PER_FRAME_COMMANDS_SIZE_IN_MB;
+
+        /**
+         * Number of threads to use in Engine's JobSystem.
+         *
+         * Engine uses a utils::JobSystem to carry out paralleization of Engine workloads. This
+         * value sets the number of threads allocated for JobSystem. Configuring this value can be
+         * helpful in CPU-constrained environments where too many threads can cause contention of
+         * CPU and reduce performance.
+         *
+         * The default value is 0, which implies that the Engine will use a heuristic to determine
+         * the number of threads to use.
+         */
+        uint32_t jobSystemThreadCount = 0;
+
+        /*
+         * Number of most-recently destroyed textures to track for use-after-free.
+         *
+         * This will cause the backend to throw an exception when a texture is freed but still bound
+         * to a SamplerGroup and used in a draw call. 0 disables completely.
+         *
+         * Currently only respected by the Metal backend.
+         */
+        size_t textureUseAfterFreePoolSize = 0;
     };
 
 
@@ -328,6 +352,12 @@ public:
          */
         Builder& sharedContext(void* sharedContext) noexcept;
 
+        /**
+         * @param featureLevel The feature level at which initialize Filament.
+         * @return A reference to this Builder for chaining calls.
+         */
+        Builder& featureLevel(FeatureLevel featureLevel) noexcept;
+
 #if UTILS_HAS_THREADING
         /**
          * Creates the filament Engine asynchronously.
@@ -459,9 +489,6 @@ public:
      */
     static void destroy(Engine* engine);
 
-    using FeatureLevel = backend::FeatureLevel;
-
-
     /**
      * Query the feature level supported by the selected backend.
      *
@@ -473,17 +500,23 @@ public:
     FeatureLevel getSupportedFeatureLevel() const noexcept;
 
     /**
-     * Activate all features of a given feature level. By default FeatureLevel::FEATURE_LEVEL_1 is
-     * active. The selected feature level must not be higher than the value returned by
-     * getActiveFeatureLevel() and it's not possible lower the active feature level.
+     * Activate all features of a given feature level. If an explicit feature level is not specified
+     * at Engine initialization time via Builder::featureLevel, the default feature level is
+     * FeatureLevel::FEATURE_LEVEL_0 on devices not compatible with GLES 3.0; otherwise, the default
+     * is FeatureLevel::FEATURE_LEVEL_1. The selected feature level must not be higher than the
+     * value returned by getActiveFeatureLevel() and it's not possible lower the active feature
+     * level. Additionally, it is not possible to modify the feature level at all if the Engine was
+     * initialized at FeatureLevel::FEATURE_LEVEL_0.
      *
      * @param featureLevel the feature level to activate. If featureLevel is lower than
-     *                     getActiveFeatureLevel(), the current (higher) feature level is kept.
-     *                     If featureLevel is higher than getSupportedFeatureLevel(), an exception
-     *                     is thrown, or the program is terminated if exceptions are disabled.
+     *                     getActiveFeatureLevel(), the current (higher) feature level is kept. If
+     *                     featureLevel is higher than getSupportedFeatureLevel(), or if the engine
+     *                     was initialized at feature level 0, an exception is thrown, or the
+     *                     program is terminated if exceptions are disabled.
      *
      * @return the active feature level.
      *
+     * @see Builder::featureLevel
      * @see getSupportedFeatureLevel
      * @see getActiveFeatureLevel
      */
@@ -804,14 +837,14 @@ public:
 #if defined(__EMSCRIPTEN__)
     /**
       * WebGL only: Tells the driver to reset any internal state tracking if necessary.
-      * 
-      * This is only useful when integrating an external renderer into Filament on platforms 
+      *
+      * This is only useful when integrating an external renderer into Filament on platforms
       * like WebGL, where share contexts do not exist. Filament keeps track of the GL
       * state it has set (like which texture is bound), and does not re-set that state if
       * it does not think it needs to. However, if an external renderer has set different
       * state in the mean time, Filament will use that new state unknowingly.
-      * 
-      * If you are in this situation, call this function - ideally only once per frame, 
+      *
+      * If you are in this situation, call this function - ideally only once per frame,
       * immediately after calling Engine::execute().
       */
     void resetBackendState() noexcept;

filament/include/filament/Fence.h

@@ -75,6 +75,10 @@ public:
      *          FenceStatus::ERROR otherwise.
      */
     static FenceStatus waitAndDestroy(Fence* fence, Mode mode = Mode::FLUSH);
+
+protected:
+    // prevent heap allocation
+    ~Fence() = default;
 };
 
 } // namespace filament

filament/include/filament/FilamentAPI.h

@@ -49,8 +49,6 @@ public:
     // prevent heap allocation
     static void *operator new     (size_t) = delete;
     static void *operator new[]   (size_t) = delete;
-    static void  operator delete  (void*)  = delete;
-    static void  operator delete[](void*)  = delete;
 };
 
 template<typename T>

filament/include/filament/IndexBuffer.h

@@ -118,6 +118,10 @@ public:
      * @return The number of indices the IndexBuffer holds.
      */
     size_t getIndexCount() const noexcept;
+
+protected:
+    // prevent heap allocation
+    ~IndexBuffer() = default;
 };
 
 } // namespace filament

filament/include/filament/IndirectLight.h

@@ -342,6 +342,10 @@ public:
     /** @deprecated use static versions instead */
     UTILS_DEPRECATED
     math::float4 getColorEstimate(math::float3 direction) const noexcept;
+
+protected:
+    // prevent heap allocation
+    ~IndirectLight() = default;
 };
 
 } // namespace filament

filament/include/filament/InstanceBuffer.h

@@ -91,6 +91,10 @@ public:
      * @param offset index of the first instance to set local transforms
      */
     void setLocalTransforms(math::mat4f const* localTransforms, size_t count, size_t offset = 0);
+
+protected:
+    // prevent heap allocation
+    ~InstanceBuffer() = default;
 };
 
 } // namespace filament

filament/include/filament/LightManager.h

@@ -143,20 +143,13 @@ public:
     using Instance = utils::EntityInstance<LightManager>;
 
     /**
-     * Returns the number of component in the LightManager, not that component are not
+     * Returns the number of component in the LightManager, note that component are not
      * guaranteed to be active. Use the EntityManager::isAlive() before use if needed.
      *
      * @return number of component in the LightManager
      */
     size_t getComponentCount() const noexcept;
 
-    /**
-     * Returns the list of Entity for all components. Use getComponentCount() to know the size
-     * of the list.
-     * @return a pointer to Entity
-     */
-    utils::Entity const* getEntities() const noexcept;
-
     /**
      * Returns whether a particular Entity is associated with a component of this LightManager
      * @param e An Entity.
@@ -164,6 +157,24 @@ public:
      */
     bool hasComponent(utils::Entity e) const noexcept;
 
+    /**
+     * @return true if the this manager has no components
+     */
+    bool empty() const noexcept;
+
+    /**
+     * Retrieve the `Entity` of the component from its `Instance`.
+     * @param i Instance of the component obtained from getInstance()
+     * @return
+     */
+    utils::Entity getEntity(Instance i) const noexcept;
+
+    /**
+     * Retrieve the Entities of all the components of this manager.
+     * @return A list, in no particular order, of all the entities managed by this manager.
+     */
+    utils::Entity const* getEntities() const noexcept;
+
     /**
      * Gets an Instance representing the Light component associated with the given Entity.
      * @param e An Entity.
@@ -953,19 +964,9 @@ public:
      */
     bool isShadowCaster(Instance i) const noexcept;
 
-    /**
-     * Helper to process all components with a given function
-     * @tparam F    a void(Entity entity, Instance instance)
-     * @param func  a function of type F
-     */
-    template<typename F>
-    void forEachComponent(F func) noexcept {
-        utils::Entity const* const pEntity = getEntities();
-        for (size_t i = 0, c = getComponentCount(); i < c; i++) {
-            // Instance 0 is the invalid instance
-            func(pEntity[i], Instance(i + 1));
-        }
-    }
+protected:
+    // prevent heap allocation
+    ~LightManager() = default;
 };
 
 } // namespace filament

filament/include/filament/Material.h

@@ -294,6 +294,9 @@ public:
     //! Returns the reflection mode used by this material.
     ReflectionMode getReflectionMode() const noexcept;
 
+    //! Returns the minimum required feature level for this material.
+    backend::FeatureLevel getFeatureLevel() const noexcept;
+
     /**
      * Returns the number of parameters declared by this material.
      * The returned value can be 0.
@@ -375,6 +378,10 @@ public:
 
     //! Returns this material's default instance.
     MaterialInstance const* getDefaultInstance() const noexcept;
+
+protected:
+    // prevent heap allocation
+    ~Material() = default;
 };
 
 } // namespace filament

filament/include/filament/MaterialInstance.h

@@ -479,6 +479,10 @@ public:
      */
     void setStencilWriteMask(uint8_t writeMask,
             StencilFace face = StencilFace::FRONT_AND_BACK) noexcept;
+
+protected:
+    // prevent heap allocation
+    ~MaterialInstance() = default;
 };
 
 } // namespace filament

filament/include/filament/MorphTargetBuffer.h

@@ -136,6 +136,10 @@ public:
      * @return The number of targets the MorphTargetBuffer holds.
      */
     size_t getCount() const noexcept;
+
+protected:
+    // prevent heap allocation
+    ~MorphTargetBuffer() = default;
 };
 
 } // namespace filament

filament/include/filament/Options.h

@@ -479,7 +479,8 @@ enum class ShadowType : uint8_t {
     PCF,        //!< percentage-closer filtered shadows (default)
     VSM,        //!< variance shadows
     DPCF,       //!< PCF with contact hardening simulation
-    PCSS        //!< PCF with soft shadows and contact hardening
+    PCSS,       //!< PCF with soft shadows and contact hardening
+    PCFd,       // for debugging only, don't use.
 };
 
 /**

filament/include/filament/RenderTarget.h

@@ -180,6 +180,10 @@ public:
      * @return Number of color attachments usable in a render target.
      */
     uint8_t getSupportedColorAttachmentsCount() const noexcept;
+
+protected:
+    // prevent heap allocation
+    ~RenderTarget() = default;
 };
 
 } // namespace filament

filament/include/filament/RenderableManager.h

@@ -102,6 +102,29 @@ public:
      */
     Instance getInstance(utils::Entity e) const noexcept;
 
+    /**
+     * @return the number of Components
+     */
+    size_t getComponentCount() const noexcept;
+
+    /**
+     * @return true if the this manager has no components
+     */
+    bool empty() const noexcept;
+
+    /**
+     * Retrieve the `Entity` of the component from its `Instance`.
+     * @param i Instance of the component obtained from getInstance()
+     * @return
+     */
+    utils::Entity getEntity(Instance i) const noexcept;
+
+    /**
+     * Retrieve the Entities of all the components of this manager.
+     * @return A list, in no particular order, of all the entities managed by this manager.
+     */
+    utils::Entity const* getEntities() const noexcept;
+
     /**
      * The transformation associated with a skinning joint.
      *
@@ -829,6 +852,10 @@ public:
             typename = typename is_supported_index_type<INDEX>::type>
     static Box computeAABB(VECTOR const* vertices, INDEX const* indices, size_t count,
             size_t stride = sizeof(VECTOR)) noexcept;
+
+protected:
+    // prevent heap allocation
+    ~RenderableManager() = default;
 };
 
 RenderableManager::Builder& RenderableManager::Builder::morphing(uint8_t level, size_t primitiveIndex,

filament/include/filament/Renderer.h

@@ -173,6 +173,12 @@ public:
      */
     void setClearOptions(const ClearOptions& options);
 
+    /**
+     * Returns the ClearOptions currently set.
+     * @return A reference to a ClearOptions structure.
+     */
+    ClearOptions const& getClearOptions() const noexcept;
+
     /**
      * Get the Engine that created this Renderer.
      *
@@ -573,6 +579,10 @@ public:
      * getUserTime()
      */
     void resetUserTime();
+
+protected:
+    // prevent heap allocation
+    ~Renderer() = default;
 };
 
 } // namespace filament

filament/include/filament/Scene.h

@@ -140,16 +140,22 @@ public:
     void removeEntities(const utils::Entity* entities, size_t count);
 
     /**
-     * Returns the number of Renderable objects in the Scene.
+     * Returns the total number of Entities in the Scene, whether alive or not.
+     * @return Total number of Entities in the Scene.
+     */
+    size_t getEntityCount() const noexcept;
+
+    /**
+     * Returns the number of active (alive) Renderable objects in the Scene.
      *
-     * @return number of Renderable objects in the Scene.
+     * @return The number of active (alive) Renderable objects in the Scene.
      */
     size_t getRenderableCount() const noexcept;
 
     /**
-     * Returns the total number of Light objects in the Scene.
+     * Returns the number of active (alive) Light objects in the Scene.
      *
-     * @return The total number of Light objects in the Scene.
+     * @return The number of active (alive) Light objects in the Scene.
      */
     size_t getLightCount() const noexcept;
 
@@ -168,6 +174,10 @@ public:
      * @param functor User provided functor called for each entity in the scene
      */
     void forEach(utils::Invocable<void(utils::Entity entity)>&& functor) const noexcept;
+
+protected:
+    // prevent heap allocation
+    ~Scene() = default;
 };
 
 } // namespace filament

filament/include/filament/SkinningBuffer.h

@@ -115,6 +115,10 @@ public:
      * @return The number of bones the SkinningBuffer holds.
      */
     size_t getBoneCount() const noexcept;
+
+protected:
+    // prevent heap allocation
+    ~SkinningBuffer() = default;
 };
 
 } // namespace filament