An assert checking invariants would sometime trigger, the problem
was a logic error that would be exposed by a race when running out
of space in the list.
The root of the problem is that in one place we were not remapping the
-1 offset to nullptr, storing a pointer violating our invariants.
Also added more asserts!!!
We're using timed condition variable in one place, but the STL version
pulls in a lot of code because it does clock calculations in
"long double" (!!!!). Since we already had an implementation
of condition_variable, we just add the timed version.
This saves several KiB of code.
Also don't use unique_lock() lock/unlock because it can throw exceptions.
This removes a lot of heap allocations/deallocations, and reduces
code size. Most improvements come from using CString and
StaticString instead of std::string and better using move
semantics.
* Clean-up EntityManager a bit
- use tsl::robin_set instead of std::set (which should have been unordered::set
anyways).
- getListeners() now returns a vector which avoids to traverse a set twice.
Turns out that copying the set wasn't as efficient as I thought.
* Improve jobsystem a bit
We recently added a job reference counting mechanism, but we were a bit
too aggressive about taking/release references.
Also make the API more complete by adding explicit retain/release,
which is needed to allow several threads to wait on the same job.
Also improve futex code by inlining it.
* Minor clean-ups
- fix a couple usage of std::function
- fix a couple usage of std::string
- remove ALIGN_LOOP, which didn't work
- fix a couple explicit/noexcept
- virtual -> override
* Fix spelling typos and other minor clang-tidy
We used to assume 32-bytes cache lines when running on ARM 32-bit mode,
however, that was wrong because 32-bit mode doesn't change the cpu's
cache line. On all modern platforms we support, the cache line
is 64 bytes.
Set Job storage to 48 bytes on all platforms.
* Fix a reuse after free in the job system
Jobs were destroyed and recycled while still in use
by wait() or run(). To fix this we introduce reference-counting of
jobs.
Jobs start with a ref-count of 1, which is decremented when a job
naturally finishes. Additionally, all user-facing methods acquire
a reference for the duration of the call.
* Fix an API inconsistency with JobSystem
JobSystem's API lets the user create jobs but not destroy them.
Jobs are destroyed automatically, without a way for the caller to
know when that happens.
We now explicitly enforce that jobs are no longer valid when
wait() returns. Multiple concurrent wait() are allowed however.
This is enforced by clearing the job pointer upon returning
from JobSystem::wait(Job* job).
* Rename linked-list put/get to push/pop
* Better fix for Job use after free
There was still a race condition where a run()'ed
job could be destroyed before wait() was called,
wait would then use a destroyed object.
The available APIs now are:
run() - runs and destroys a job
runAndWait() - run, then waits for and destroys a job
runAndRetain() - runs and keep a reference to the job
wait() - waits and destroys a job
wait() can only be used with a job obtained with runAndRetain().
* Get rid of unused code
This version of parallel_for has use-after-free issues anyways,
since we changed the semantics of run/wait/etc...
* Fix decRef() memory order
decRef() must ensure that all access to the
object have happened before destroying it.
* Fix memory order in atomic linked list's pop()
It needs acquire semantic, since we want to make
sure that no read/write are reordered before the
pop() -- which returns an object to the caller.
* Fix memory order on runningJobCount
we needed acquire semantic when about to destroy
the last job -- it's similar to decRef.
* Comment usages of std::memory_order_*
* Fix AtomicFreeList A-B-A bug
Turns out AtomicFreeList was not immune to the ABA bug. W're fixing
it here by using a 64-bits CAS, which is available on aarch64 and armv7.
This does not add any build stuff or new sample code yet, it just does
some prep to our C++ codebase:
- Similar to Android, WebAssembly will not be using BlueGL. We were
using a mixture of #if and #ifdef when checking for the existence of
certain GL prototypes, but the latter is what we want in order to
build robustly with any GL headers. (We still perform run-time checks
for extensions, this doesn't change that.)
- Add a trivial ContextManager, does a bit more than Dummy since it
needs to create an actual driver. This doesn't get compiled yet, it
just adds files to the tree.
- WebAssembly does not support mmap, execinfo, or asm volatile.
* Add single-threaded config to Filament.
This adds a tick method to Engine and disables a couple components
in Renderer (FrameSkipper and FrameInfoManager).
This will make it easier to support WebGL, and will allow us to remove
some of the command buffer debugging stuff that we added for Vulkan.
* tick => execute, and other review feedback
* Restore the ASSERT for FFence::wait.
* Try to clean up asset folders for sample apps.
This removes the build step where we copy a subset of assets, and makes
it so that FilamentApp hands out a "root path" for assets. For now this
is determined based on the location of the executable. This allows
developers to launch samples from any CWD.
Closes#11
* Restore asset copy to build.
computeLightTree() takes a list of light as
a bitfield and produces a depth-first
binary tree array that can be used to efficiently
check which lights volumes contain a given z
coordinate in screen space.
Currently computeLightTree() produces the array
locally on the stack, which isn’t useful, but
before we can make use of it, a lot of other
things have to happen.
* better froxel records compression
We used to compress froxel records by reusing the
previous record on the left if it was identical,
now we additionally check the record above if
that fails.
In practice this saves 10% to 30% space in the
froxel record buffer, which is a scarce resource
for us.
e.g. if a light occupied 3x3 froxels in a given
z-slice, without compression 9 records would be
used, with left-only compression, only 3,
and with left+above compression, only 1.
* use size_t for in and out parameters
this helps the compiler in many places, not
having to cast from 64 to 32 bits.
* implement bitset’s any() and all() operations with NEON
* empty froxels are a common case, handle it first
* JobSystem now automatically free Jobs
Until now Job allocation used a linear allocator
strategy which required to “reset” the JobSystem
periodically — typically once per frame in
filament.
This is no longer required. We use a pool allocator
now, which doesn’t add much overhead. It does
use a spin-lock for thread-safety though, since
we assume very little contention, this shouldn’t
be a problem.
* Thread Safe Object Pool Allocator
A lock-less, thread-safe object pool allocator,
now used for storing JobSystem’s jobs allocations.
This gets rid of the spin-lock introduced in the
previous cl.
We spawn MAX_LIGHTS/32 (currently 8) jobs and
multi-threading is active as soon as we have more
than one light. All jobs work on disjoint buffers
to avoid false sharing between threads.
It’s possible to adjust roughly the number of jobs
by changing the group size (currently 32). However
this also affects vectorization.
With the current parameters, with 256 lights max
we get 8 jobs, and 4 spot-light test / froxel.
Instead of building a list of froxel per light,
we now build a bitfield of lights per froxel.
To keep the code efficient we have to split
the bitfield array in 4, so that we can use
multiple threads without having them or’ing
each other’s cache-lines.
This data structure is also what we want in
froxelizeAssignRecords(), and the conversion from
one format to the other is much faster.
Our internal data structure is now 256 KiB
instead of 2 MiB. It would be 1 MiB for 1024 lights
(down from 16 MiB)
Additionally, because the code is simpler, we get
more vectorization, especially with spot-lights.
One draw-back of the current version is that
multi-threading happens only every 64 lights.
This will be fixed later.
