add a material option to use linear fog calculations (#9030)

The new `linearFog` material property, when set to `true` enables a
simplified fog calculation. The fog equation becomes linear which is
unrealistic, but more efficient to compute. In some situations with
a shallow fog range, it doesn't make a huge difference visually.

In this mode, height falloff and in-scattering are ignored.

The linear equation slope is calculated from the regular parameters to
match the slope of the real equation at a camera height. If 
`heightFalloff` is disabled, set to 0, the `density` parameter 
exactly corresponds to the slope of the equation in [1/m] units.

NEW_RELEASE_NOTES.md

@@ -8,4 +8,5 @@ appropriate header in [RELEASE_NOTES.md](./RELEASE_NOTES.md).
 
 ## Release notes for next branch cut
 
+- engine: add a `linearFog` material parameter. [⚠️ **New Material Version**]
 - opengl: When `Material::compile()` is called on a platform which doesn't support parallel compilation, shaders are automatically compiled over a number of frames

android/filament-android/src/main/java/com/google/android/filament/View.java

@@ -1561,7 +1561,9 @@ public class View {
     }
 
     /**
-     * Options to control large-scale fog in the scene
+     * Options to control large-scale fog in the scene. Materials can enable the `linearFog` property,
+     * which uses a simplified, linear equation for fog calculation; in this mode, the heightFalloff
+     * is ignored as well as the mipmap selection in IBL or skyColor mode.
      */
     public static class FogOptions {
         /**
@@ -1578,7 +1580,7 @@ public class View {
          */
         public float cutOffDistance = Float.POSITIVE_INFINITY;
         /**
-         * fog's maximum opacity between 0 and 1
+         * fog's maximum opacity between 0 and 1. Ignored in `linearFog` mode.
          */
         public float maximumOpacity = 1.0f;
         /**
@@ -1586,12 +1588,15 @@ public class View {
          */
         public float height = 0.0f;
         /**
-         * How fast the fog dissipates with altitude. heightFalloff has a unit of [1/m].
+         * How fast the fog dissipates with the altitude. heightFalloff has a unit of [1/m].
          * It can be expressed as 1/H, where H is the altitude change in world units [m] that causes a
          * factor 2.78 (e) change in fog density.
          *
          * A falloff of 0 means the fog density is constant everywhere and may result is slightly
          * faster computations.
+         *
+         * In `linearFog` mode, only use to compute the slope of the linear equation. Completely
+         * ignored if set to 0.
          */
         public float heightFalloff = 1.0f;
         /**
@@ -1612,7 +1617,7 @@ public class View {
         @NonNull @Size(min = 3)
         public float[] color = {1.0f, 1.0f, 1.0f};
         /**
-         * Extinction factor in [1/m] at altitude 'height'. The extinction factor controls how much
+         * Extinction factor in [1/m] at an altitude 'height'. The extinction factor controls how much
          * light is absorbed and out-scattered per unit of distance. Each unit of extinction reduces
          * the incoming light to 37% of its original value.
          *
@@ -1621,10 +1626,15 @@ public class View {
          * the composition of the fog/atmosphere.
          *
          * For historical reason this parameter is called `density`.
+         *
+         * In `linearFog` mode this is the slope of the linear equation if heightFalloff is set to 0.
+         * Otherwise, heightFalloff affects the slope calculation such that it matches the slope of
+         * the standard equation at the camera height.
          */
         public float density = 0.1f;
         /**
          * Distance in world units [m] from the camera where the Sun in-scattering starts.
+         * Ignored in `linearFog` mode.
          */
         public float inScatteringStart = 0.0f;
         /**
@@ -1632,6 +1642,7 @@ public class View {
          * is scattered (by the fog) towards the camera.
          * Size of the Sun in-scattering (>0 to activate). Good values are >> 1 (e.g. ~10 - 100).
          * Smaller values result is a larger scattering size.
+         * Ignored in `linearFog` mode.
          */
         public float inScatteringSize = -1.0f;
         /**
@@ -1657,6 +1668,8 @@ public class View {
          *
          * `fogColorFromIbl` is ignored when skyTexture is specified.
          *
+         * In `linearFog` mode mipmap level 0 is always used.
+         *
          * @see Texture
          * @see fogColorFromIbl
          */
@@ -1671,7 +1684,7 @@ public class View {
     /**
      * Options to control Depth of Field (DoF) effect in the scene.
      *
-     * cocScale can be used to set the depth of field blur independently from the camera
+     * cocScale can be used to set the depth of field blur independently of the camera
      * aperture, e.g. for artistic reasons. This can be achieved by setting:
      *      cocScale = cameraAperture / desiredDoFAperture
      *

docs_src/src_markdeep/Materials.md.html

@@ -1199,6 +1199,24 @@ material {
 }
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
+### General: linearFog
+
+Type
+:    `boolean`
+
+Value
+:     `true` or `false`. Defaults to `false`.
+
+Description
+:    When set to `true`, a simplified fog equation is used for large-scale fog calculations. In this mode,
+     in-scattering is ignored as well as height falloff.
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ JSON
+material {
+    linearFog : true
+}
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 ### General: quality
 
 Type

filament/include/filament/Options.h

@@ -164,7 +164,9 @@ struct BloomOptions {
 };
 
 /**
- * Options to control large-scale fog in the scene
+ * Options to control large-scale fog in the scene. Materials can enable the `linearFog` property,
+ * which uses a simplified, linear equation for fog calculation; in this mode, the heightFalloff
+ * is ignored as well as the mipmap selection in IBL or skyColor mode.
  */
 struct FogOptions {
     /**
@@ -183,7 +185,7 @@ struct FogOptions {
     float cutOffDistance = INFINITY;
 
     /**
-     * fog's maximum opacity between 0 and 1
+     * fog's maximum opacity between 0 and 1. Ignored in `linearFog` mode.
      */
     float maximumOpacity = 1.0f;
 
@@ -193,12 +195,15 @@ struct FogOptions {
     float height = 0.0f;
 
     /**
-     * How fast the fog dissipates with altitude. heightFalloff has a unit of [1/m].
+     * How fast the fog dissipates with the altitude. heightFalloff has a unit of [1/m].
      * It can be expressed as 1/H, where H is the altitude change in world units [m] that causes a
      * factor 2.78 (e) change in fog density.
      *
      * A falloff of 0 means the fog density is constant everywhere and may result is slightly
      * faster computations.
+     *
+     * In `linearFog` mode, only use to compute the slope of the linear equation. Completely
+     * ignored if set to 0.
      */
     float heightFalloff = 1.0f;
 
@@ -220,7 +225,7 @@ struct FogOptions {
     LinearColor color = { 1.0f, 1.0f, 1.0f };
 
     /**
-     * Extinction factor in [1/m] at altitude 'height'. The extinction factor controls how much
+     * Extinction factor in [1/m] at an altitude 'height'. The extinction factor controls how much
      * light is absorbed and out-scattered per unit of distance. Each unit of extinction reduces
      * the incoming light to 37% of its original value.
      *
@@ -229,11 +234,16 @@ struct FogOptions {
      * the composition of the fog/atmosphere.
      *
      * For historical reason this parameter is called `density`.
+     *
+     * In `linearFog` mode this is the slope of the linear equation if heightFalloff is set to 0.
+     * Otherwise, heightFalloff affects the slope calculation such that it matches the slope of
+     * the standard equation at the camera height.
      */
     float density = 0.1f;
 
     /**
      * Distance in world units [m] from the camera where the Sun in-scattering starts.
+     * Ignored in `linearFog` mode.
      */
     float inScatteringStart = 0.0f;
 
@@ -242,6 +252,7 @@ struct FogOptions {
      * is scattered (by the fog) towards the camera.
      * Size of the Sun in-scattering (>0 to activate). Good values are >> 1 (e.g. ~10 - 100).
      * Smaller values result is a larger scattering size.
+     * Ignored in `linearFog` mode.
      */
     float inScatteringSize = -1.0f;
 
@@ -269,6 +280,8 @@ struct FogOptions {
      *
      * `fogColorFromIbl` is ignored when skyTexture is specified.
      *
+     * In `linearFog` mode mipmap level 0 is always used.
+     *
      * @see Texture
      * @see fogColorFromIbl
      */
@@ -283,7 +296,7 @@ struct FogOptions {
 /**
  * Options to control Depth of Field (DoF) effect in the scene.
  *
- * cocScale can be used to set the depth of field blur independently from the camera
+ * cocScale can be used to set the depth of field blur independently of the camera
  * aperture, e.g. for artistic reasons. This can be achieved by setting:
  *      cocScale = cameraAperture / desiredDoFAperture
  *

filament/src/ds/ColorPassDescriptorSet.cpp

@@ -19,8 +19,6 @@
 
 #include "Froxelizer.h"
 #include "PerViewDescriptorSetUtils.h"
-#include "HwDescriptorSetLayoutFactory.h"
-#include "ShadowMapManager.h"
 #include "TypedUniformBuffer.h"
 
 #include "components/LightManager.h"
@@ -30,10 +28,8 @@
 #include "details/IndirectLight.h"
 #include "details/Texture.h"
 
-#include <filament/Engine.h>
 #include <filament/Exposure.h>
 #include <filament/Options.h>
-#include <filament/TextureSampler.h>
 #include <filament/MaterialEnums.h>
 #include <filament/Viewport.h>
 
@@ -56,9 +52,11 @@
 #include <algorithm>
 #include <array>
 #include <cmath>
+#include <cstring>
+#include <cstddef>
+#include <limits>
 #include <random>
 
-#include <stddef.h>
 #include <stdint.h>
 
 namespace filament {
@@ -243,6 +241,22 @@ void ColorPassDescriptorSet::prepareFog(FEngine& engine, const CameraInfo& camer
                     .filterMin = SamplerMinFilter::LINEAR_MIPMAP_LINEAR
             });
 
+    // Fog calculation details:
+    // Optical path: (
+    //   f = heightFalloff
+    //   Te(y, z) = z * density * (exp(-f * eye_y) - exp(-f * eye_y - f * y)) / (f * y)
+    // Transmittance:
+    //   t(y , z) = exp(-Te(y, z))
+
+    // In Linear Mode, formally the slope of the linear equation is: dt(y,z)/dz(0, eye_y)
+    // (the derivative of the transmittance at distance 0 and camera height). When the height
+    // falloff is disabled, the density parameter exactly represents this value.
+    constexpr double EPSILON = std::numeric_limits<float>::epsilon();
+    double const f = heightFalloff;
+    double const eye = userCameraPosition.y - options.height;
+    double const dt = options.density * (f <= EPSILON ? 1.0 : (std::exp(-f * eye) - std::exp(-2.0 * f * eye)) / (f * eye));
+    float const fogEndLinear = float(1.0 / dt);
+
     s.fogStart             = options.distance;
     s.fogMaxOpacity        = options.maximumOpacity;
     s.fogHeightFalloff     = heightFalloff;
@@ -253,6 +267,8 @@ void ColorPassDescriptorSet::prepareFog(FEngine& engine, const CameraInfo& camer
     s.fogInscatteringSize  = options.inScatteringSize;
     s.fogColorFromIbl      = fogColorTextureHandle ? 1.0f : 0.0f;
     s.fogFromWorldMatrix   = mat3f{ cof(fogFromWorld) };
+    s.fogLinearParams       = { 1.0f / (fogEndLinear - options.distance),
+            -options.distance / (fogEndLinear - options.distance) };
 }
 
 void ColorPassDescriptorSet::prepareSSAO(Handle<HwTexture> ssao,
@@ -332,7 +348,7 @@ void ColorPassDescriptorSet::prepareDirectionalLight(FEngine& engine,
     }
 }
 
-void ColorPassDescriptorSet::prepareAmbientLight(FEngine& engine, FIndirectLight const& ibl,
+void ColorPassDescriptorSet::prepareAmbientLight(FEngine const& engine, FIndirectLight const& ibl,
         float const intensity, float const exposure) noexcept {
     auto& s = mUniforms.edit();
 

filament/src/ds/ColorPassDescriptorSet.h

@@ -115,7 +115,7 @@ public:
     void prepareDirectionalLight(FEngine& engine, float exposure,
             math::float3 const& sceneSpaceDirection, LightManagerInstance instance) noexcept;
 
-    void prepareAmbientLight(FEngine& engine,
+    void prepareAmbientLight(FEngine const& engine,
             FIndirectLight const& ibl, float intensity, float exposure) noexcept;
 
     void prepareDynamicLights(Froxelizer& froxelizer) noexcept;

libs/filabridge/include/private/filament/UibStructs.h

@@ -177,6 +177,8 @@ struct PerViewUib { // NOLINT(cppcoreguidelines-pro-type-member-init)
     float fogOneOverFarMinusNear;
     float fogNearOverFarMinusNear;
     std140::mat33 fogFromWorldMatrix;
+    math::float2 fogLinearParams; // { 1/(end-start), -start/(end-start) }
+    math::float2 fogReserved[1];
 
     // --------------------------------------------------------------------------------------------
     // Screen-space reflections [variant: SSR (i.e.: VSM | SRE)]
@@ -202,7 +204,7 @@ struct PerViewUib { // NOLINT(cppcoreguidelines-pro-type-member-init)
     float es2Reserved2;
 
     // bring PerViewUib to 2 KiB
-    math::float4 reserved[40];
+    math::float4 reserved[39];
 };
 
 // 2 KiB == 128 float4s

libs/filamat/include/filamat/MaterialBuilder.h

@@ -532,6 +532,9 @@ public:
     //! Enable / disable flipping of the Y coordinate of UV attributes, enabled by default.
     MaterialBuilder& flipUV(bool flipUV) noexcept;
 
+    //! Enable / disable the cheapest linear fog, disabled by default.
+    MaterialBuilder& linearFog(bool enabled) noexcept;
+
     //! Enable / disable multi-bounce ambient occlusion, disabled by default on mobile.
     MaterialBuilder& multiBounceAmbientOcclusion(bool multiBounceAO) noexcept;
 
@@ -948,6 +951,7 @@ private:
     bool mClearCoatIorChange = true;
 
     bool mFlipUV = true;
+    bool mLinearFog = false;
 
     bool mMultiBounceAO = false;
     bool mMultiBounceAOSet = false;

libs/filamat/src/MaterialBuilder.cpp

@@ -539,6 +539,11 @@ MaterialBuilder& MaterialBuilder::flipUV(bool const flipUV) noexcept {
     return *this;
 }
 
+MaterialBuilder& MaterialBuilder::linearFog(bool const enabled) noexcept {
+    mLinearFog = enabled;
+    return *this;
+}
+
 MaterialBuilder& MaterialBuilder::customSurfaceShading(bool const customSurfaceShading) noexcept {
     mCustomSurfaceShading = customSurfaceShading;
     return *this;
@@ -708,6 +713,7 @@ void MaterialBuilder::prepareToBuild(MaterialInfo& info) noexcept {
     info.specularAntiAliasing = mSpecularAntiAliasing;
     info.clearCoatIorChange = mClearCoatIorChange;
     info.flipUV = mFlipUV;
+    info.linearFog = mLinearFog;
     info.requiredAttributes = mRequiredAttributes;
     info.blendingMode = mBlendingMode;
     info.postLightingBlendingMode = mPostLightingBlendingMode;

libs/filamat/src/shaders/CodeGenerator.cpp

@@ -250,6 +250,8 @@ utils::io::sstream& CodeGenerator::generateCommonProlog(utils::io::sstream& out,
         CodeGenerator::generateDefine(out, "LEGACY_MORPHING", material.useLegacyMorphing);
     }
     if (stage == ShaderStage::FRAGMENT) {
+        CodeGenerator::generateDefine(out, "FILAMENT_LINEAR_FOG",
+                material.linearFog);
         CodeGenerator::generateDefine(out, "MATERIAL_HAS_CUSTOM_DEPTH",
                 material.userMaterialHasCustomDepth);
     }

libs/filamat/src/shaders/MaterialInfo.h

@@ -44,6 +44,7 @@ struct UTILS_PUBLIC MaterialInfo {
     bool specularAntiAliasing;
     bool clearCoatIorChange;
     bool flipUV;
+    bool linearFog;
     bool multiBounceAO;
     bool multiBounceAOSet;
     bool specularAOSet;

libs/filamat/src/shaders/UibGenerator.cpp

@@ -189,6 +189,8 @@ BufferInterfaceBlock const& UibGenerator::getPerViewUib() noexcept  {
             { "fogOneOverFarMinusNear",  0, Type::FLOAT, Precision::HIGH },
             { "fogNearOverFarMinusNear", 0, Type::FLOAT, Precision::HIGH },
             { "fogFromWorldMatrix",      0, Type::MAT3, Precision::HIGH, FeatureLevel::FEATURE_LEVEL_0 },
+            { "fogLinearParams",         0, Type::FLOAT2, Precision::HIGH, FeatureLevel::FEATURE_LEVEL_0 },
+            { "fogReserved",             1, Type::FLOAT2, Precision::HIGH },
 
             // ------------------------------------------------------------------------------------
             // Screen-space reflections [variant: SSR (i.e.: VSM | SRE)]

libs/filament-matp/src/ParametersProcessor.cpp

@@ -1185,6 +1185,11 @@ static bool processFlipUV(MaterialBuilder& builder, const JsonishValue& value) {
     return true;
 }
 
+static bool processLinearFog(MaterialBuilder& builder, const JsonishValue& value) {
+    builder.linearFog(value.toJsonBool()->getBool());
+    return true;
+}
+
 static bool processMultiBounceAO(MaterialBuilder& builder, const JsonishValue& value) {
     builder.multiBounceAmbientOcclusion(value.toJsonBool()->getBool());
     return true;
@@ -1397,6 +1402,7 @@ ParametersProcessor::ParametersProcessor() {
     mParameters["featureLevel"]                  = { &processFeatureLevel, Type::NUMBER };
     mParameters["groupSize"]                     = { &processGroupSizes, Type::ARRAY };
     mParameters["stereoscopicType"]              = { &processStereoscopicType, Type::STRING };
+    mParameters["linearFog"]                     = { &processLinearFog, Type::BOOL };
 }
 
 bool ParametersProcessor::process(MaterialBuilder& builder, const JsonishObject& jsonObject) {

shaders/src/surface_fog.fs

@@ -96,3 +96,57 @@ vec4 fog(vec4 color, highp vec3 view) {
 
     return color;
 }
+
+// A linear approximation of the fog function
+vec4 fogLinear(vec4 color, highp vec3 view) {
+    // note: d can be +inf with the skybox
+    highp float d = length(view);
+
+    // early exit for object "in front" of the fog
+    if (d < frameUniforms.fogStart) {
+        return color;
+    }
+
+    // fogCutOffDistance is set to +inf to disable the cutoff distance
+    if (d > frameUniforms.fogCutOffDistance) {
+        return color;
+    }
+
+    // compute fog color
+    highp float A = frameUniforms.fogLinearParams.x;
+    highp float B = frameUniforms.fogLinearParams.y;
+    float fogOpacity = saturate(A * d + B);
+
+    vec3 fogColor = frameUniforms.fogColor;
+
+#if MATERIAL_FEATURE_LEVEL > 0
+    if (frameUniforms.fogColorFromIbl > 0.0) {
+        lowp vec2 minMaxMip = unpackHalf2x16(frameUniforms.fogMinMaxMip);
+        // when sampling the IBL we need to take into account the IBL transform. We know it's a
+        // a rigid transform, so we can take the transpose instead of the inverse, and for the
+        // same reason we can use it directly instead of taking the cof() to transform a vector.
+        highp mat3 worldFromUserWorldMatrix = transpose(mat3(frameUniforms.userWorldFromWorldMatrix));
+        fogColor *= textureLod(sampler0_fog, worldFromUserWorldMatrix * view, minMaxMip.x).rgb;
+    }
+#endif
+
+    fogColor *= frameUniforms.iblLuminance * fogOpacity;
+
+#if   defined(BLEND_MODE_OPAQUE)
+    // nothing to do here
+#elif defined(BLEND_MODE_TRANSPARENT)
+    fogColor *= color.a;
+#elif defined(BLEND_MODE_ADD)
+    fogColor = vec3(0.0);
+#elif defined(BLEND_MODE_MASKED)
+    // nothing to do here
+#elif defined(BLEND_MODE_MULTIPLY)
+    // FIXME: unclear what to do here
+#elif defined(BLEND_MODE_SCREEN)
+    // FIXME: unclear what to do here
+#endif
+
+    color.rgb = color.rgb * (1.0 - fogOpacity) + fogColor;
+
+    return color;
+}

shaders/src/surface_main.fs

@@ -65,9 +65,15 @@ void main() {
 #if defined(VARIANT_HAS_FOG)
     highp vec3 view = getWorldPosition() - getWorldCameraPosition();
     view = frameUniforms.fogFromWorldMatrix * view;
+
+#if defined (FILAMENT_LINEAR_FOG)
+    fragColor = fogLinear(fragColor, view);
+#else
     fragColor = fog(fragColor, view);
 #endif
 
+#endif
+
 #if defined(VARIANT_HAS_SHADOWING) && defined(VARIANT_HAS_DIRECTIONAL_LIGHTING)
     if (CONFIG_DEBUG_DIRECTIONAL_SHADOWMAP) {
         float a = fragColor.a;

web/filament-js/filament.d.ts

@@ -1301,7 +1301,9 @@ export interface View$BloomOptions {
 }
 
 /**
- * Options to control large-scale fog in the scene
+ * Options to control large-scale fog in the scene. Materials can enable the `linearFog` property,
+ * which uses a simplified, linear equation for fog calculation; in this mode, the heightFalloff
+ * is ignored as well as the mipmap selection in IBL or skyColor mode.
  */
 export interface View$FogOptions {
     /**
@@ -1318,7 +1320,7 @@ export interface View$FogOptions {
      */
     cutOffDistance?: number;
     /**
-     * fog's maximum opacity between 0 and 1
+     * fog's maximum opacity between 0 and 1. Ignored in `linearFog` mode.
      */
     maximumOpacity?: number;
     /**
@@ -1326,12 +1328,15 @@ export interface View$FogOptions {
      */
     height?: number;
     /**
-     * How fast the fog dissipates with altitude. heightFalloff has a unit of [1/m].
+     * How fast the fog dissipates with the altitude. heightFalloff has a unit of [1/m].
      * It can be expressed as 1/H, where H is the altitude change in world units [m] that causes a
      * factor 2.78 (e) change in fog density.
      *
      * A falloff of 0 means the fog density is constant everywhere and may result is slightly
      * faster computations.
+     *
+     * In `linearFog` mode, only use to compute the slope of the linear equation. Completely
+     * ignored if set to 0.
      */
     heightFalloff?: number;
     /**
@@ -1351,7 +1356,7 @@ export interface View$FogOptions {
      */
     color?: float3;
     /**
-     * Extinction factor in [1/m] at altitude 'height'. The extinction factor controls how much
+     * Extinction factor in [1/m] at an altitude 'height'. The extinction factor controls how much
      * light is absorbed and out-scattered per unit of distance. Each unit of extinction reduces
      * the incoming light to 37% of its original value.
      *
@@ -1360,10 +1365,15 @@ export interface View$FogOptions {
      * the composition of the fog/atmosphere.
      *
      * For historical reason this parameter is called `density`.
+     *
+     * In `linearFog` mode this is the slope of the linear equation if heightFalloff is set to 0.
+     * Otherwise, heightFalloff affects the slope calculation such that it matches the slope of
+     * the standard equation at the camera height.
      */
     density?: number;
     /**
      * Distance in world units [m] from the camera where the Sun in-scattering starts.
+     * Ignored in `linearFog` mode.
      */
     inScatteringStart?: number;
     /**
@@ -1371,6 +1381,7 @@ export interface View$FogOptions {
      * is scattered (by the fog) towards the camera.
      * Size of the Sun in-scattering (>0 to activate). Good values are >> 1 (e.g. ~10 - 100).
      * Smaller values result is a larger scattering size.
+     * Ignored in `linearFog` mode.
      */
     inScatteringSize?: number;
     /**
@@ -1400,7 +1411,7 @@ export enum View$DepthOfFieldOptions$Filter {
 /**
  * Options to control Depth of Field (DoF) effect in the scene.
  *
- * cocScale can be used to set the depth of field blur independently from the camera
+ * cocScale can be used to set the depth of field blur independently of the camera
  * aperture, e.g. for artistic reasons. This can be achieved by setting:
  *      cocScale = cameraAperture / desiredDoFAperture
  *