Commit Graph

320 Commits

Author SHA1 Message Date
Mathias Agopian
d640ba853b rework how we size the HandleAllocator's pools
- update the pools sizes for metal and vulkan, which were very outdated.
- add debug code on all backends to print the size of each handle 
  (with a compile time switch)

The most important change is that now the 3 pools of HandleAllocator
are sized so that each can accommodate about the same amount of handles.
This makes it easier to reason about. The total amount of handles is
three times that, since there are 3 pools. 
We also try to allocate the buckets so that handles are evenly
distributed, however, that's very hand wavy.

With the current setup the number of handles per pool is as follows:
- GL : 3240 / pool / MiB
- VK : 1820 / pool / MiB
- MTL: 1310 / pool / MiB
2024-02-06 21:28:04 -08:00
Mathias Agopian
50d9d9f139 Improve memory allocations (#7540)
* Automatically flush CommandStream 

When generating commands, we now automatically flush the CommandStream,
so that we're guaranteed to not overrun the circular buffer.

* clenaup CircularBuffer implementation and API

Also fix a bug in DEBUG mode that could corrupt the CircularBuffer, it
was due to a wrong debugging code attempting to clear the unused
area of the buffer (this was wrong because in "ashmem" mode, there are
no guaranteed unused areas).

* Fix a couple threading vs. allocations

- prepareVisibleLights was run on a dedicated thread (via JobSystem), 
  but was using its own local ArenaScope. This is wrong because it
  could reset the root arena at any later point. This is fixed by
  just not using a local ArenaScope.

- related to the above, the root Arena (LinearAllocatorArena) didn't
  use a locked policy, which cause also cause problems since some
  allocations are done off the main thread. We now pre-allocate the one
  buffer we need.

This PR also renames some variable and types to improve readability.

* Rework RenderPass to improve allocations and API

RenderPass now is a fully immutable object that gets constructed with a
RenderPassBuilder. RenderPassBuilder can be passed around and doesn't
do any (major) allocations.

All RenderPass allocations and heavy lifting is done in 
RenderPassBuilder::Build().

Additionally, RenderPass cannot be copied anymore.

Where allocations happen is now much clearer.

* new LinearAllocatorWithFallback 

LinearAllocatorWithFallback is a linear allocator that can fall back
to the heap allocator. We use it for the high level command buffer to
avoid crashing when running out of memory.

FIXES=[277115740]

* Update filament/src/RenderPass.h

Co-authored-by: Powei Feng <powei@google.com>

* Update libs/utils/include/utils/Allocator.h

Co-authored-by: Powei Feng <powei@google.com>

---------

Co-authored-by: Powei Feng <powei@google.com>
2024-02-06 10:01:41 -08:00
Mathias Agopian
34f8b9aa20 add nullability attributes to filament public APIs 2024-02-01 10:36:42 -08:00
Mathias Agopian
59b59cf6be add missing includes in libutils public headers 2024-01-26 10:58:00 -08:00
Mathias Agopian
d5041fced7 support execinfo.h on Android 33 and above
Note: this currently requires to compile filament with API_LEVEL 33.
2024-01-25 14:50:29 -08:00
Mathias Agopian
6ee20a57aa remove all uses of our custom spinlock
This has caused issues and over time we have reduced the use of 
spinlocks, it was only used in few places and we still have evidence
that it's causing ANRs.

We use utils::Mutex instead which is a low overhead mutex implementation
on Linux systems.

FIXES=[321101014]
2024-01-19 12:05:07 -08:00
Powei Feng
5435a8ed3b Add missing includes (#7501)
BUG=320668410
2024-01-17 11:04:42 -08:00
Ben Doherty
6062b3c8c6 Fix ostream linking error when compiling Linux DSO (#7470) 2024-01-03 12:06:44 -08:00
Benjamin Doherty
2c9559f9b8 Add string.h header 2024-01-02 13:24:08 -08:00
Mathias Agopian
cabaa4323e add a way to set a log consumer
this is a private API to capture filament's logs.
2023-12-18 16:28:31 -08:00
Mathias Agopian
d6b2ec49b0 cleanup 2023-12-18 16:28:31 -08:00
Mathias Agopian
2a67152dda Dynamic "Material constants" support
Material constants (a.k.a: specialization constants) can only be set
during Material creation through Material::Builder.
This change somewhat relaxes that limitation by allowing constants to
be set at runtime on Material directly.

Currently this new API is still private and only supported on FMaterial.

This feature works by invalidating the HwProgram cache of the concerned
Material, causing a shader recompile per variant; so this API is costly
and should be used only for debugging or during app/game configuration.

The TAA material is modified to use constants instead of #defines for
various settings and those are exposed in TaaOptions as well is in 
ViewerGui. So with this change all aspects of the TAA material can
be changed at runtime.
2023-12-11 16:20:47 -08:00
Mathias Agopian
1a0b5ddc14 fix [Pixel]BufferDescriptor functor callbacks
and attempt to make functor<->callback code less confusing.
2023-11-10 13:15:29 -08:00
Powei Feng
0eb851ff8c Update debug.cpp (#7337) 2023-11-08 10:55:28 -08:00
Mathias Agopian
e674420e9c improvements to EntityManagers and Filament APIs (#7302)
* prevent public classes from being created on the stack

- we used to to this by deleting operator delete, but this prevented
  the internal "F" classes from being virtual; which can be useful
  when using EntityManger::Listener.
  now we just make the destructor protected in each class.

- EntityManger::Listener now has a virtual destructor so that
  objects could be correctly destroyed from Listener*

* improve EntityManger and Component managers

- all component managers now have the same "base" API
    - getComponentCount()
	- empty()
    - getEntity()
    - getEntities()

- Scene now has getEntityCount()

- EntityManager now has getEntityCount()

- all component manager implement gc() the same way, by calling destroy()

- SingleInstanceComponentManager::gc() that calls removeComponent() has
  been removed because it's dangerous. removeComponent() is often
  not enough, some additional cleanup might be needed.
2023-10-26 13:10:43 -07:00
Mathias Agopian
626621fb1c minor filament benchmarks cleanup 2023-10-23 10:02:26 -07:00
Ben Doherty
9d181a172a Create use-after-free detector for Metal textures (#7250) 2023-10-20 17:15:42 -04:00
Adrian Perez
3c77d2c3f5 StructureOfArrays can push_back move-only types 2023-10-19 12:13:28 -07:00
Mathias Agopian
97f20afdd7 remove the anonymous union in SamplerParams
- don't rely on it being 32-bits
- update the jni code to store SamplerParams in a long (64 bits)
  instead of a int. This gives us some future-proofing of the java side.
2023-08-24 21:36:42 -07:00
Mathias Agopian
f537f62adf Use 4 background threads for shader compiler on PowerVR
Since powervr supports parallel shader compilation well, we use 
4 background threads for shader compilation.
2023-08-11 15:22:19 -07:00
Y-way
3f77ff8815 Build error on msvc 2022 2023-07-21 10:00:51 -07:00
junsang.lee
420f4000c2 fix: 64-Bits Windows, Path::exists() fails for files over 2GB. (#6938) 2023-07-07 00:14:54 -07:00
Ben Doherty
9fa952d968 For web assembly, implement logging as emscripten_err and emscripten_out (#6913) 2023-06-22 11:18:23 +08:00
Mathias Agopian
74f5d8a066 the ShaderCompilerService handles async shader compilation (#6848)
It supports KHR_parallel_shader_compile as well as a
thread pool of GL contexts.

- we have a new 2-priorities queue for shader compilation
- use this feature in gltfio in the ubershader case
2023-06-14 10:51:02 -07:00
Romain Guy
64b15fafe2 Save ImGui settings in a user-specific directory (#6865)
* Save ImGui settings in a user-specific directory

Fixes #6862

* Fix Windows build

* Fix Web build

* Add missing file
2023-06-02 08:42:27 -07:00
Powei Feng
d891acb8c0 Fix missing include (#6858) 2023-06-01 15:15:34 -07:00
Mathias Agopian
25708e8082 don't hardcode the system page size to 4096
It's already 16K on macOS and other systems.
2023-05-24 21:35:38 -07:00
Mathias Agopian
aa1773d633 cache programs in gl backend 2023-05-12 10:46:30 -07:00
Mathias Agopian
0171452b02 fix stream unit-test 2023-03-24 17:30:00 -07:00
Mathias Agopian
9a2fd38f33 update libutils to output floats/double without loss 2023-03-23 15:23:18 -07:00
Mathias Agopian
3cef1991b2 Minor cleanup of GL backend 2023-03-08 12:49:22 -08:00
Mathias Agopian
2e7fc6229b Improve StructureOfArray
- StructureOfArray: don't initialize trivial ctors

We mimic the behavior of std::vector<> here, where a resize() won't
initialize the array if the type is trivially_default_constructible.
This can reveal existing bugs, where we depended on the initialization
to 0.

- StructureOfArray: add push_back(std::tuple<>)

This basically allows us to push_back() a struct of the SoA.

- Make PerRenderableData trivially constructible

this improves performance when we have tons of objects in the scene
because PerRenderableData is used in arrays.
2023-02-22 16:28:40 -08:00
Mathias Agopian
fc724575fa Make SYSTRACE_CALL work with SYSTRACE_CONTEXT
SYSTRACE_VALUE now requires a SYSTRACE_CONTEXT.
2023-02-22 16:28:40 -08:00
Mathias Agopian
4847b00f9e improve SoA a bit
Instead of storing the arrays into an array of void*, we use a 
tuple<> instead. This improves debugging because now the tuple<>
has pointer with the correct types.

It also improves most of the code except `push_back` which now
relies on a hack -- this is the only place where I'm not able to 
resolve the array strictly at compile time, even if in practice it is.
2023-02-08 16:22:00 -08:00
Powei Feng
51cbd74235 Add missing STL includes (#6496) 2023-01-26 16:07:58 -08:00
Ben Doherty
4c82f0259d Add systrace support on Apple platforms (#6480) 2023-01-23 14:11:47 -08:00
daemyung danny jang
6734dc6f43 Create a path from a string view (#6406) 2023-01-03 09:36:41 -08:00
Mathias Agopian
4215ba6476 code cleanup
fix new code quality warnings enabled by CLion
2022-12-09 11:05:26 -08:00
Mathias Agopian
8b96af8e3f AtlasAllocator improvements 2022-11-16 09:20:22 -08:00
Mathias Agopian
2ca218928f utils: get rid of GrowingSlice
it's just not used anywhere. change the implementation of Slice<> to
use two pointers.
2022-10-27 22:48:45 -07:00
Mathias Agopian
90f32e7277 A basic POT 2D allocator
This 2D allocator only supports power-of-two, square allocations. It
uses a quadtree to store the allocated regions.

The quadtree implementation currently has a fixed depth (templated).

The allocator does a best-fit match, but currently doesn't implement a
free method.
2022-10-26 11:02:17 -07:00
Ben Doherty
fe3d1713a4 Fix generic/Mutex.h not installed on Linux (#6117) 2022-09-28 16:36:38 -07:00
Mathias Agopian
c84834d666 bump max sampler count to 62
This is needed for feature level 2.
2022-09-13 10:49:21 -07:00
Mathias Agopian
b21d633c0c check that user materials don't exceed their allowed features (#6030)
* check that user materials don't exceed their allowed features

* backend sampler limits now take feature level into account


- in the backend, the constants are now in an array indexed by the 
  feature level

- samplerBindingMap now asserts only what it can.

- matc (filamat) now logs the user samplers when an error is detected.
2022-09-02 16:37:19 -07:00
Balazs Vegh
6a28d2b81c Implement setThreadName() on Windows (#5999) 2022-08-26 13:29:42 -04:00
gaborvalasek
39cb238065 Fix MSVC warnings (#5926)
* Fix warning C4146: unary minus operator applied to unsigned type, result still unsigned

* Fix warning C4068: unknown pragma 'nounroll'

* Fix warning C4068: unknown pragma 'unroll'

* Fix warning C4068: unknown pragma 'clang'

* Fix warning C4305: 'initializing': truncation from 'double' to 'float'

* Fix warning C4267: 'argument': conversion from 'size_t' to 'utils::FixedCapacityVector<filament::uberz::WritableArchive::Material,std::allocator<T>,true>::size_type', possible loss of data

* Fix warning C4267: 'argument': conversion from 'size_t' to 'uint32_t', possible loss of data

* Fix warning C4244: 'initializing': conversion from 'A' to 'T', possible loss of data

* Fix warning C4334: '<<': result of 32-bit shift implicitly converted to 64 bits (was 64-bit shift intended?)

* Fix warning C4293: '>>': shift count negative or too big, undefined behavior

* Fix diagnostic warning C4189: 'channels': local variable is initialized but not referenced

* Use [[maybe_unused]] where possible and revert aa79bd6fa8.

* Revert unary minus for non-MSVC compilers

* Add macro for enabling warnings temporarily

* Get rid of UTILS_HAS_CXX17

* Revisit warning related macros

Co-authored-by: Levente Koncz <levente.koncz@shapr3d.com>
2022-08-24 11:41:10 -07:00
Mathias Agopian
3ffe63c02c don't use std::is_pod which is soon to be deprecated
fixes #5980
2022-08-23 15:49:35 -07:00
Mathias Agopian
80e26ba4af Make BindingPoints an enum class 2022-08-19 22:08:59 -07:00
Mathias Agopian
f0eb1c59b1 don't hardcode uniform block bindings on filament side
GLES3.0 (and GL4.1) need informations about the uniform block bindings
because they don't allow to do that setup in the shaders. So we store
that information in the material blob and retrieve it on the filament
side in Material. This information is passed to Program so it can
create the bindings.

Prior to this change, this information was set by Material itself, but
this ment we had two places where these bindings were hardcoded.


The bulk of the change is to add a new material chunk which stores an
array of {name, binding}, retrieve it on the Material side and pass it
to Program.
2022-08-19 16:23:39 -07:00
Johnathon Selstad
1e3ddd612e Organize Subprojects into Folders in the IDE (#5934)
* Begin Sorting SubProjects into Folders

* Add more subprojects to folders

* Add even more subprojects to folders

* Add further subprojects to folders

* Move the last two projects

* Move Resources to a Resources subfolder

* Remove spaces to be stylistically coherent

* Revert Improper CMake Modifications

* Revert erroneous line removals

* Only specify sdl2's folder on WIN32

* Add the shader subprojects to a Generated folder

* Move shaders to Filament/Shaders
2022-08-17 12:42:21 -07:00