Scarfski/filament
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add way to retrieve the user world-space in materials (#6607)

Browse Source 
* Add way to retrieve the user world-space in materials

added `getUserWorldFromWorldMatrix()` and `getUserWorldPosition()` to
retrieve the API-level (user) world position in materials.
Deprecated `getWorldOffset()`

`getWorldOffset` didn't work when an IBL rotation was applied.

* fix large scenes with an ibl rotation

Rotate the IBL around the camera instead of the world so that the camera 
is always at the origin regardless of the rotation.
This commit is contained in:
Mathias Agopian
2023-03-03 21:34:17 -08:00
committed by GitHub
parent 9d30233a38
commit e2e5f7cee9
12 changed files with 59 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
NEW_RELEASE_NOTES.md
View File

@@ -8,6 +8,8 @@ appropriate header in [RELEASE_NOTES.md](./RELEASE_NOTES.md).
## Release notes for next branch cut
fog: fixed fog height falloff and computation precision on mobile [⚠️ **Recompile Materials**]
materials: new alphaToCoverage property can be used to control alpha to coverage behavior
engine: fix precision issue with `shading_view` in large scenes
- fog: fixed fog height falloff and computation precision on mobile [⚠️ **Recompile Materials**]
- materials: new alphaToCoverage property can be used to control alpha to coverage behavior
- materials: added `getUserWorldFromWorldMatrix()` and `getUserWorldPosition()` to retrieve the
API-level (user) world position in materials. Deprecated `getWorldOffset()`. [⚠️ **Recompile Materials**]
- engine: fix precision issue with `shading_view` in large scenes

20
docs/Materials.md.html
View File

@@ -1946,8 +1946,9 @@ struct MaterialVertexInputs {
!!! TIP: worldPosition
To achieve good precision, the `worldPosition` coordinate in the vertex shader is shifted by the
camera position. To get the true world-space position, users can add this to
`getWorldOffset()`.
camera position. To get the true world-space position, users can use
`getUserWorldPosition()`, however be aware that the true world-position might not
be able to fit in a `float` or might be represented with severely reduced precision.
!!! TIP: UV attributes
By default the vertex shader of a material will flip the Y coordinate of the UV attributes
@@ -2300,8 +2301,9 @@ type aliases:
Name | Type | Description
:-----------------------------------|:--------:|:------------------------------------
**getResolution()** | float4 | Dimensions of the view's effective (physical) viewport in pixels: `width`, `height`, `1 / width`, `1 / height`. This might be different from `View::getViewport()` for instance because of added rendering guard-bands.
**getWorldCameraPosition()** | float3 | Position of the camera/eye in world space
**getWorldOffset()** | float3 | The shift required to obtain API-level world space
**getWorldCameraPosition()** | float3 | Position of the camera/eye in world space (see note below)
**getWorldOffset()** | float3 | [deprecated] The shift required to obtain API-level world space. Use getUserWorldPosition() instead
**getUserWorldFromWorldMatrix()** | float4x4 | Matrix that converts from world space to API-level (user) world space.
**getTime()** | float | Current time as a remainder of 1 second. Yields a value between 0 and 1
**getUserTime()** | float4 | Current time in seconds: `time`, `(double)time - time`, `0`, `0`
**getUserTimeMode(float m)** | float | Current time modulo m in seconds
@@ -2311,7 +2313,7 @@ type aliases:
!!! TIP: world space
To achieve good precision, the "world space" in Filament's shading system does not necessarily
match the API-level world space. To obtain the position of the API-level camera, custom
materials can add `getWorldOffset()` to `getWorldCameraPosition()`.
materials can use `getUserWorldFromWorldMatrix()` to transform `getWorldCameraPosition()`.
### Vertex only
@@ -2333,6 +2335,7 @@ The following APIs are only available from the fragment block:
:---------------------------------------|:--------:|:------------------------------------
**getWorldTangentFrame()** | float3x3 | Matrix containing in each column the `tangent` (`frame[0]`), `bi-tangent` (`frame[1]`) and `normal` (`frame[2]`) of the vertex in world space. If the material does not compute a tangent space normal for bump mapping or if the shading is not anisotropic, only the `normal` is valid in this matrix.
**getWorldPosition()** | float3 | Position of the fragment in world space (see note below about world-space)
**getUserWorldPosition()** | float3 | Position of the fragment in API-level (user) world-space (see note below about world-space)
**getWorldViewVector()** | float3 | Normalized vector in world space from the fragment position to the eye
**getWorldNormalVector()** | float3 | Normalized normal in world space, after bump mapping (must be used after `prepareMaterial()`)
**getWorldGeometricNormalVector()** | float3 | Normalized normal in world space, before bump mapping (can be used before `prepareMaterial()`)
@@ -2349,9 +2352,10 @@ The following APIs are only available from the fragment block:
**ycbcrToRgb(float, float2)** | float3 | Converts a luminance and CbCr pair to a sRGB color
**uvToRenderTargetUV(float2)** | float2 | Transforms a UV coordinate to allow sampling from a `RenderTarget` attachment
!!! TIP: world space
To obtain API-level world space coordinates, custom materials should add `getWorldOffset()` to
`getWorldPosition()` (et al).
!!! TIP: world-space
To obtain API-level world-space coordinates, custom materials should use `getUserWorldPosition()`
or use `getUserWorldFromWorldMatrix()`. Note that API-level world-space coordinates should
never or rarely be used because they may not fit in a float3 or have severely reduced precision.
!!! TIP: sampling from render targets
When sampling from a `filament::Texture` that is attached to a `filament::RenderTarget` for

3
filament/src/PerShadowMapUniforms.cpp
View File

@@ -60,9 +60,8 @@ void PerShadowMapUniforms::prepareCamera(Transaction const& transaction,
s.viewFromClipMatrix = viewFromClip; // 1/projection
s.clipFromWorldMatrix = clipFromWorld; // projection * view
s.worldFromClipMatrix = worldFromClip; // 1/(projection * view)
s.userWorldFromWorldMatrix = mat4f(inverse(camera.worldOrigin));
s.clipTransform = camera.clipTransfrom;
s.cameraPosition = float3{ camera.getPosition() };
s.worldOffset = camera.getWorldOffset();
s.cameraFar = camera.zf;
s.oneOverFarMinusNear = 1.0f / (camera.zf - camera.zn);
s.nearOverFarMinusNear = camera.zn / (camera.zf - camera.zn);

3
filament/src/PerViewUniforms.cpp
View File

@@ -76,9 +76,8 @@ void PerViewUniforms::prepareCamera(FEngine& engine, const CameraInfo& camera) n
s.viewFromClipMatrix = viewFromClip; // 1/projection
s.clipFromWorldMatrix = clipFromWorld; // projection * view
s.worldFromClipMatrix = worldFromClip; // 1/(projection * view)
s.userWorldFromWorldMatrix = mat4f(inverse(camera.worldOrigin));
s.clipTransform = camera.clipTransfrom;
s.cameraPosition = float3{ camera.getPosition() };
s.worldOffset = camera.getWorldOffset();
s.cameraFar = camera.zf;
s.oneOverFarMinusNear = 1.0f / (camera.zf - camera.zn);
s.nearOverFarMinusNear = camera.zn / (camera.zf - camera.zn);

2
filament/src/ShadowMap.cpp
View File

@@ -357,7 +357,7 @@ ShadowMap::ShaderParameters ShadowMap::updateDirectional(FEngine& engine,
if (params.options.stable) {
// Use the world origin as reference point, fixed w.r.t. the camera
snapLightFrustum(s, o, Mv, -camera.getWorldOffset(),
snapLightFrustum(s, o, Mv, camera.worldOrigin[3].xyz,
1.0f / float(shadowMapInfo.shadowDimension));
}

1
filament/src/details/Camera.h
View File

@@ -219,7 +219,6 @@ struct CameraInfo {
float d{}; // focus distance [m]
math::float3 const& getPosition() const noexcept { return model[3].xyz; }
math::float3 getForwardVector() const noexcept { return normalize(-model[2].xyz); }
math::float3 getWorldOffset() const noexcept { return -worldOrigin[3].xyz; }
math::mat4 getUserViewMatrix() const noexcept { return view * worldOrigin; }
};

27
filament/src/details/View.cpp
View File

@@ -390,7 +390,7 @@ void FView::prepareLighting(FEngine& engine, FEngine::DriverApi& driver, ArenaSc
* Directional light (always at index 0)
*/
FLightManager::Instance directionalLight = lightData.elementAt<FScene::LIGHT_INSTANCE>(0);
FLightManager::Instance const directionalLight = lightData.elementAt<FScene::LIGHT_INSTANCE>(0);
const float3 sceneSpaceDirection = lightData.elementAt<FScene::DIRECTION>(0); // guaranteed normalized
mPerViewUniforms.prepareDirectionalLight(engine, exposure, sceneSpaceDirection, directionalLight);
mHasDirectionalLight = directionalLight.isValid();
@@ -401,32 +401,31 @@ CameraInfo FView::computeCameraInfo(FEngine& engine) const noexcept {
/*
* We apply a "world origin" to "everything" in order to implement the IBL rotation.
* The "world origin" could also be useful for other things, like keeping the origin
* close to the camera position to improve fp precision in the shader for large scenes.
* The "world origin" is also be used to kee the origin close to the camera position to
* improve fp precision in the shader for large scenes.
*/
mat4 worldOriginScene;
FIndirectLight const* const ibl = scene->getIndirectLight();
if (ibl) {
// the IBL transformation must be a rigid transform
mat3f rotation{ scene->getIndirectLight()->getRotation() };
// for a rigid-body transform, the inverse is the transpose
worldOriginScene = mat4{ transpose(rotation) };
}
mat4 translation;
mat4 rotation;
/*
* Calculate all camera parameters needed to render this View for this frame.
*/
FCamera const* const camera = mViewingCamera ? mViewingCamera : mCullingCamera;
if (engine.debug.view.camera_at_origin) {
// this moves the camera to the origin, effectively doing all shader computations in
// view-space, which improves floating point precision in the shader by staying around
// zero, where fp precision is highest. This also ensures that when the camera is placed
// very far from the origin, objects are still rendered and lit properly.
worldOriginScene[3].xyz -= camera->getPosition();
translation = mat4::translation( -camera->getPosition() );
}
return { *camera, worldOriginScene };
FIndirectLight const* const ibl = scene->getIndirectLight();
if (ibl) {
// the IBL transformation must be a rigid transform
rotation = mat4{ transpose(scene->getIndirectLight()->getRotation()) };
}
return { *camera, rotation * translation };
}
void FView::prepare(FEngine& engine, DriverApi& driver, ArenaScope& arena,

24
libs/filabridge/include/private/filament/UibStructs.h
View File

@@ -43,16 +43,17 @@ struct PerViewUib { // NOLINT(cppcoreguidelines-pro-type-member-init)
// Values that can be accessed in both surface and post-process materials
// --------------------------------------------------------------------------------------------
math::mat4f viewFromWorldMatrix;
math::mat4f worldFromViewMatrix;
math::mat4f clipFromViewMatrix;
math::mat4f viewFromClipMatrix;
math::mat4f clipFromWorldMatrix;
math::mat4f worldFromClipMatrix;
math::float4 clipTransform; // [sx, sy, tx, ty] only used by VERTEX_DOMAIN_DEVICE
math::mat4f viewFromWorldMatrix; // clip view <- world : view matrix
math::mat4f worldFromViewMatrix; // clip view -> world : model matrix
math::mat4f clipFromViewMatrix; // clip <- view world : projection matrix
math::mat4f viewFromClipMatrix; // clip -> view world : inverse projection matrix
math::mat4f clipFromWorldMatrix; // clip <- view <- world
math::mat4f worldFromClipMatrix; // clip -> view -> world
math::mat4f userWorldFromWorldMatrix; // userWorld <- world
math::float4 clipTransform; // [sx, sy, tx, ty] only used by VERTEX_DOMAIN_DEVICE
math::float2 clipControl; // clip control
float time; // time in seconds, with a 1 second period
float time; // time in seconds, with a 1-second period
float temporalNoise; // noise [0,1] when TAA is used, 0 otherwise
math::float4 userTime; // time(s), (double)time - (float)time, 0, 0
@@ -67,14 +68,9 @@ struct PerViewUib { // NOLINT(cppcoreguidelines-pro-type-member-init)
float lodBias; // load bias to apply to user materials
float refractionLodOffset;
float padding1;
float padding2;
// camera position in view space (when camera_at_origin is enabled), i.e. it's (0,0,0).
// Always add worldOffset in the shader to get the true world-space position of the camera.
math::float3 cameraPosition;
float oneOverFarMinusNear; // 1 / (f-n), always positive
math::float3 worldOffset; // this is (0,0,0) when camera_at_origin is disabled
float nearOverFarMinusNear; // n / (f-n), always positive
float cameraFar; // camera *culling* far-plane distance, always positive (projection far is at +inf)
float exposure;
@@ -164,7 +160,7 @@ struct PerViewUib { // NOLINT(cppcoreguidelines-pro-type-member-init)
float ssrStride; // ssr texel stride, >= 1.0
// bring PerViewUib to 2 KiB
math::float4 reserved[62];
math::float4 reserved[60];
};
// 2 KiB == 128 float4s

5
libs/filamat/src/shaders/UibGenerator.cpp
View File

@@ -41,6 +41,7 @@ BufferInterfaceBlock const& UibGenerator::getPerViewUib() noexcept {
{ "viewFromClipMatrix", 0, Type::MAT4, Precision::HIGH },
{ "clipFromWorldMatrix", 0, Type::MAT4, Precision::HIGH },
{ "worldFromClipMatrix", 0, Type::MAT4, Precision::HIGH },
{ "userWorldFromWorldMatrix",0,Type::MAT4, Precision::HIGH },
{ "clipTransform", 0, Type::FLOAT4, Precision::HIGH },
{ "clipControl", 0, Type::FLOAT2 },
@@ -58,12 +59,8 @@ BufferInterfaceBlock const& UibGenerator::getPerViewUib() noexcept {
{ "lodBias", 0, Type::FLOAT },
{ "refractionLodOffset", 0, Type::FLOAT },
{ "padding1", 0, Type::FLOAT },
{ "padding2", 0, Type::FLOAT },
{ "cameraPosition", 0, Type::FLOAT3, Precision::HIGH },
{ "oneOverFarMinusNear", 0, Type::FLOAT, Precision::HIGH },
{ "worldOffset", 0, Type::FLOAT3 },
{ "nearOverFarMinusNear", 0, Type::FLOAT, Precision::HIGH },
{ "cameraFar", 0, Type::FLOAT },
{ "exposure", 0, Type::FLOAT, Precision::HIGH }, // high precision to work around #3602 (qualcom),

11
shaders/src/common_getters.glsl
View File

@@ -32,6 +32,11 @@ highp mat4 getWorldFromClipMatrix() {
return frameUniforms.worldFromClipMatrix;
}
/** @public-api */
highp mat4 getUserWorldFromWorldMatrix() {
return frameUniforms.userWorldFromWorldMatrix;
}
/** @public-api */
float getTime() {
return frameUniforms.time;
@@ -81,12 +86,12 @@ highp vec4 getResolution() {
/** @public-api */
highp vec3 getWorldCameraPosition() {
return frameUniforms.cameraPosition;
return frameUniforms.worldFromViewMatrix[3].xyz;
}
/** @public-api */
/** @public-api, @deprecated use getUserWorldPosition() or getUserWorldFromWorldMatrix() instead */
highp vec3 getWorldOffset() {
return frameUniforms.worldOffset;
return getUserWorldFromWorldMatrix()[3].xyz;
}
/** @public-api */

5
shaders/src/getters.fs
View File

@@ -49,6 +49,11 @@ highp vec3 getWorldPosition() {
return shading_position;
}
/** @public-api */
highp vec3 getUserWorldPosition() {
return mulMat4x4Float3(getUserWorldFromWorldMatrix(), getWorldPosition()).xyz;
}
/** @public-api */
vec3 getWorldViewVector() {
return shading_view;

4
shaders/src/shading_parameters.fs
View File

@@ -37,8 +37,8 @@ void computeShadingParams() {
// With perspective camera, the view vector is cast from the fragment pos to the eye position,
// With ortho camera, however, the view vector is the same for all fragments:
highp vec3 sv = isPerspectiveProjection() ?
(frameUniforms.cameraPosition - shading_position) :
frameUniforms.worldFromViewMatrix[2].xyz; // ortho camera backward dir
(frameUniforms.worldFromViewMatrix[3].xyz - shading_position) :
frameUniforms.worldFromViewMatrix[2].xyz; // ortho camera backward dir
shading_view = normalize(sv);
// we do this so we avoid doing (matrix multiply), but we burn 4 varyings: