This is a pixel accurate implementation of picking. Picking queries
can be created on view, and upon completion a user provided callback
is called with the Entity of the renderable at the queried coordinates
in the viewport.
Picking queries typically have 1 or 2 frame of latency and may impact
performance on some drivers.
It is mostly intended for use by editors, or when latency is not a major
concern. This api should not be used for dragging/moving objects, it is
intended for initial picking only.
Picking is implemented in the structure pass.
The depth buffer value is retrieved and the fragment coordinate is
reconstructed and passed to PickingQueryResult. This can be used in
turn to calculate the view and/or world space position.
In low-light conditions, peak luminance sensitivity of the eye shifts
toward the blue end of the visible spectrum. This effect called the
Purkinje effect occurs during the transition from photopic (cone-based)
vision to scotopic (rod-based) vision. Because the rods and cones use the
same neural pathways, a color shift is introduced as the rods take over to
improve low-light perception.
This function aims to (somewhat) replicate this color shift and peak
luminance sensitivity increase to more faithfully reproduce scenes in
low-light conditions as they would be perceived by a human observer
(as opposed to an artificial observer such as a camera sensor).
The night adaptation can be controlled using a 0..1 factor for artistic
reasons: `ColorGrading::Builder::nightAdaptation()`.
* Use wide gamut color grading and gamut mapping
This change introduces a new API on ColorGrading to enable or disable gamut mapping at will. This feature is currently off by default and while useful and recommended, it isn't cheap and we need to figure out what we want our defaults to be especially on mobile targets.
Gamut mapping can be combined with luminance scaling to properly control hue skews.
* Small cleanup
* More precise Rec.2020 conversion
Augment the TransformManager so it can maintain the 4th column as
double precision; we accomplish this by storing an extra float3 per matrix.
It's calculated as `doubleTranslation - float3{doubleTranslation}`.
We also add methods to set and get a transform as mat4, internally only
the 4th column is kept as double precision.
Finally when we calculate the worldTransform, we take this extra data into
account, but only for calculating the new 4th column, so the extra work
is small.
Currently there are 8 light channels, both renderables and lights can
be associated to one or several light channel, lighting occurs when
at least one channel is active on both the light and the renderable.
Fixes#4275
We now have an option for evaluating the bent normals from SSAO.
This is used to improve specular AO.
Currently these bent normals are not used for diffuse lighting because
they're "flat" (face normals), which is too distracting.
The bent normal buffer is stored in the SSAO texture, in a separate
layer. This is the first post processing effect that uses 2D array
in order to limit our sampler usage.
This CL also changes how we handle SPECULAR_AO_BENT_NORMALS when
material bent normals are not available: we're now using the "Cones"
algorithm with the regular normal, instead of Lagarde's approximation.
This can have a dramatic effect on SSAO and needs
to be adjusted based on the scene's scale/units.
The default value used to be about 6cm, and has
been changed to 5mm.
* WIP New tone mapper API
* Implement tone mapper constructors and destructors
* Add new genertic tone mapper
* Make the generic tone mapper available in our sample UIs
* Fix warnings and crashes
* Fix generic tone mapper and graph mappers in the UI
* Add Java APIs for ToneMapper
* Implement copy/move operators for GenericToneMapper
The API now allows to create a standalone SkinningBuffer containing
Bones for skinning. This SkinningBuffer can even be very large and
contain bone data for several renderables.
New APIs on RenderableManager allow to switch to skinning buffer mode
and to set the SkinningBuffer (and window into it) to use.
Fixes#3238
* Generalize luminance scaling in color grading
Previously the luminance scaling step was tied to a specific tone
mapping operator, called "EVILS" in the API. This was however
somewhat misleading as EVILS really is a color handling system
that's independent of what we call tone mapping.
This change splits the luminance scaling step from the tone mapping
step, allowing the use of luminance scaling (the LICH part of EVILS)
with any tone mapping operator.
As a result, the operator known as EVILS is currently mapped to the
linear operator and renamed to "RESERVED" until it is replaced by
a proper — configurable — operator.
* Update release notes
* Fix web build
* Add new sample app: image_viewer
This app shows how to load and visualize images. It's also useful to work
on color grading and other post-processing effects by just loading an HDR
or EXR image as a reference.
This change also adds a new exposure setting to the color grading LUT.
* Fix comments and build
* Fix Metal
* Use high precision for the UV coordinates
* Fix refactoring issue
* Remove support for Java/desktop builds
These builds are never tested nor used on our end. We cannot
guarantee their proper support. It should also be possible
for an app to handle this itself.
* Remove Tungsten since it cannot be compiled anymore
This is still somewhat work-in-progress, but good enough to be useful.
Some caveats:
- large blurs don't work well with cascades
- the implementation uses more memory than it could
- use EVSM (exponential VSM), helps a lot with light bleeding
- don't apply shadow biases with VSM
- improve VSM user settings: added more controls, expose to java
- minor code cleanups:
- rename some structures
- remove unused parameters
- remove unused uniforms
deprecate Stream::stream(intptr_t).
This method was needed for ARCore back in the days, but there is now a
zero-copy way to achieve the same thing. This API shouldn't be used anymore.
New Api:
- Renderer::renderStandaloneView() is a new method that can be used
outside of beginFrame/endFrame on Views that have a RenderTarget
associated. This can be used as a poor man's compute API.
Improvements to Renderer:
- more aggressively skip lighting/shadowing code that won't be
needed by the current view
- Allow up to 64 KiB to be allocated from the render stream, up from
1 KiB
NOTE: after Metal and Vulkan support lands, we can potentially simplify
the Driver API by removing some of the old VertexBuffer entry points.
This introduces a new API-level object called `BufferObject`, and a new
backend-level object called `HwBufferObject`. This encapsulates a single
VBO. It's especially useful when clients need to share data between
multiple VertexBuffer objects. (e.g. for morphing).
In the future, buffer objects could perhaps be used for non-vertex data
(e.g. for compute).
The existing `VertexBuffer` class can now be configured to use buffer
objects by calling `enableBufferObjects` on the builder. By default its
API is unchanged. If buffer objects are enabled, then clients should use
`setBufferObjectAt` (which takes a buffer object) rather than
`setBufferAt` (which takes CPU data).
OpenGL is the trickiest backend for this, due to existence of VAOs.
The implementation in this PR uses a version-tracking scheme to
figure out when to update the VAO.
* move the focus distance from DofOptions to Camera
The DofOption parameter is now deprecated, but will still work if the
focus distance is not set on Camera.
* A few new helper APIs on Camera
- getFocalLength() which returns the focal length used internally for
DoF computations.
- computeEffectiveFov() and computeEffectiveFocalLength() which can be
used to better simulate a real camera's FOV changes with the
focus distance.
Modified gltf_viewer so it uses the effective fov/focallendth when
DoF is active.
* simplify setLensProjection
We can call setProjection directly instead of converting to a fov
first, which ends up performing atan(tan(x)).
Also improve parameters names.
- setImage() used the width of the texture for calculating the required
user buffer size, but this was too large when only updating a
portion of the texture. It would result in a BufferOverFlowException.
- setImage3D() had the opposite problem where it didn't take into
account the depth of the texture.
- the offset in the user buffer was called "bottom" on the C++ JNI side,
but it is called "top" on the java and filament side.
Fixes#3685
Currently the only NONE and MEDIAN are available options, at some
point we may add a "MAX" option which is cheaper.
There is no real practical uses for this option with these only two
choices, other than for debugging.
- Add a "shift" parameter to Camera. This has the effect of translating
the viewport, without changing its size.
This is an effect similar to using a shift lens.
- Camera::setScaling() now takes a double2 instead of double4, this is
because scaling the result of the projection in the Z direction
can lead to very confusing problems -- it will essentially move the
near/far planes, and we don't want to expose that as a public API.
- setCustomProjection() now allows to set a different projection for
rendering and culling (useful for e.g. for using an infinite far
rendering projection matrix).
* Stop using members as globals between methods
* Multi-thread shaders generation with JobSystem
* Pass JobSystem to MaterialBuilder::build()
* Fix MeshAssimp to use the new API
* Allow the Java API to pass a job system via Engine
* Update docs
* Apply suggestions from code review
Co-authored-by: Philip Rideout <philiprideout@gmail.com>
- Use lower depth LOD as the cone radius increases.
This is the same technique we use for SAO.
The better cache access significantly improve performance.
On a test on Pixel4 at 585MHz, SAO pass improves by 30%.
This also helps the algorithm scale with the shadow distance.
- Shadow direction/length no longer dependent on aspect-ratio and camera
orientation.
This is fixed by doing all the computations in screen-space instead of
normalized screen space.
- Shadow parameters are no longer dependent on the field of view.
- Also rename dominantLightShadowing.fs to ssct.fs
- remove zoom parameter, it wasn't very useful.
