* CString can construct from a string_view
(string_views lack a null terminator)
* Update libs/utils/src/CString.cpp
Co-authored-by: Mathias Agopian <pixelflinger@gmail.com>
The hang was caused by a subtle race. When a job is completed, its
thread must signal all the threads that might be waiting on this job.
The signaling code was attempting to signal only the minimum number
of threads -- this was important especially in the case where no threads
were waiting, then the call to notify() could be avoided.
Unfortunately, for performance reasons we're not calling notify() with
the condition lock held, this meant that between the time the number of
waiting threads was latched and the time of the notify() call, more
threads could enter their condition variable wait(), and it would
then be possible for these threads to wake up, instead of the thread
we were trying to wake up (the one waiting on the job).
It would then get stuck forever.
This bug was introduced in 2df639133b
Also add some debugging code for this kind of failure (disabled)
assert() is now replaced by assert_invariant() which has the same
prototype and (currently) behaves the same than assert().
<assert.h> should not be used anymore, and is replaced by
<utils/debug.h>, which is where assert_invariant() is found.
The main motivation for this is to be able to set a breakpoint on the
assertion as lldb doesn't handle abort() very well, and doesn't
permit to inspect the stack trace.
A secondary motivation is to be able (at some point) to enable
assertions without necessarily doing a debug build.
The non-const getSkybox() method is especially useful to client code
that needs to change the clear color, but does not have direct
access to the Renderer object.
If hardware_concurrency() returned 2 while UTILS_HAS_HYPER_THREADING was
enabled, `mThreadCount` was resulting in zero, so jobs (such as texture
decoding) would simply never start. This problem was noticed with
GitHub Actions.
I tested this fix locally by replacing `hardware_concurrency()` with
fixed values like 1 or 2, and verified that the hang went away.
* debug option to track Entities
Set FILAMENT_UTILS_TRACK_ENTITIES to true when building libutils to
activate entity tracking. This adds two public methods:
getActiveEntities() and dumpActiveEntities() the later displays the
stack trace of where the remaining entities were allocated.
This is useful for tracking leaks.
* Update libs/utils/include/utils/EntityManager.h
Co-authored-by: Philip Rideout <philiprideout@gmail.com>
Co-authored-by: Philip Rideout <philiprideout@gmail.com>
There was actually no need to give special treatment to leading drive
designators since they effectively form the first path segment anyway.
To help prevent regressions, I added a few unit tests in a previous CL.
The motivation for the CL is to remove a dependency on `locale.cpp`,
which can result in shorter build times and reduced binary sizes.
TrackingPolicy::Debug didn't store the base pointer of the Area, and
instead relied on the first allocation to discover it, however, because
of alignment, the first allocation may not match the base pointer.
Because of that there could be an overflow in onRewind(), i.e. we
could rewind to a pointer before the (wrongly computed) base. This
overflow caused the debug memset to go awry and stomped on memory.
This is fixed by passing the base pointer to the constructor of the
TrackingPolicy. This base pointer could be nullptr with certain
allocators, but in that case, onReset/onRewind should never be called;
and this is enforced at compile time.
Also fixed a (luckily) harmless buffer overflow when preparing the
dynamic lights, if the number of lights wasn't a multiple of 4. This
was harmless because we use a linear allocator, so overflows are not
really overflows.
* Lower limit from API 21 to API 19
This was requested by an internal application. API 19 is when OpenGL
ES 3.0 support was added so there is no good reason for us to not
support this API level. The only trick is to avoid referring to the
glTexStorage2DMultisample symbol directly as it only exists in 3.1.
* Compile out code we never use
* Use reflection to handle shared EGL contexts pre-API 21.
* Remove comment
* More fixes required to run on API level 19
- dlym() fails for ashmem on API 19, so we only try on API 26+ instead.
- Older emulation for OpenGL ES 3.0 returns error states for valid API calls so we need to clear the GL error bit before we create the GL driver.
- Activity lifecycle changes since API 19 would cause animations to keep running and to reference destroyed objects.
- EGLContext.getNativeHandle() is new in API 21, we need to use reflection on API 19. The new code path uses the class loading trick to avoid a bytecode verification error on API 19.
* Filament now runs properly on API level 19
This commit adds a new api_level() API to libutils which can be used to
query the platform's API level. On Android it works as expected, other
platforms currently return 0.
This can be enabled by using the Tracking::Debug policy,
currently this just fills allocation on alloc() and free(),
which is useful to help detect access to uninitialized memory
and use after free.
Now enabled by default in libfilament allocators on debug builds.
This caught uninitialized access in the froxelizer.
* Add #include preprocessing to filamat and matc
* Update RELEASE_NOTES
* Fix RELEASE_NOTES
* Use final instead of virtual / override
* Clarify comments
* Use pure virtual for includer functions
* Use a callback instead of an interface
* Rename Includer.h to IncludeCallback.h
* Update comment
In the case where we have 2 cores, we would spawn only one thread in
the thread pool. If that thread got to try to steal() from another
thread before the main thread was adopted, it would end-up always
trying to steal from itself and enter an infinite loop.
This seems to happen during windows builds.
this is because the JobSystem's queue works as a LIFO, by creating
jobs in reverse (memory) order, we attempt to help streaming to
the d-cache on that threads -- until the point where
jobs are stolen.
we also execute the last job immediately instead of creating a job
for it -- since we're already in a job.
- parallel_for doesn't use recursion anymore to create the "leaf"
jobs, this is now done linearly on N thread (one thread per CPU).
This uses less stack space, and reduces miss-predicted branches.
- remove almost all SYSTRACE calls because they have a huge impact
on things like parallel_for() and are misleading. They can be
enabled again by setting HEAVY_SYSTRACE to true.
- we simplify the waiting code by using only a single
condition variable instead of two.
- wait() now behaves just like a looper, it will process jobs until
the one it's waiting for finishes -- before it could just sit there
(the idea was that the job would finish quickly, but that's not always
the case).
- we also make sure to never call notify_n() when it's not needed.
We track how many waiters we have and use that to decide if we need
to notify().
notify is pretty slow on all architectures, even on linux it's always
a syscall, so it's better to avoid it.
- don't use stand-alone fences, makes things ugly for no real benefit
- refactored the code a bit, hopefully it's more clear.
For jobs with that do very little work, the jobsystem can introduce
a lot of overhead, we mitigate this by:
- don't wake-up worker threads when scheduling several very small jobs,
like when scheduling the per-face jobs.
- don't wait for per-face jobs to finish -- we only did that to avoid
a copy of the job's data.
- don't use multi-threading at all if the job has too little work. We
evaluate the work using the scanline length and number of samples.
Even when hex modifier is used, 'char' should be printed as characters,
this is particularly relevant with 0.
e.g. out << (char)0, should write a nul terminator, not "0".
Instead of passing SamplerInterfaceBlocks and a SamplerBindingMap to
Program, we now set a 'sampler group' per binding point:
Program::addSamplerGroup(...)
A sampler group here consists of a list of N 'Sampler' and a 'Sampler'
is just a unique name (unifrom name in the shader) and binding point in
the shader.
That's all the driver layer needs.
With this change we get rid of the code that re-created the uniform
names in the driver -- this should never have been done there. And we
also remove the hash-map lookups in vulkan and metal drivers.
Program::shader() was taking a string before which didn't make sense
for spirv. Now it's just a blob, in the case of GL/Metal, the blob
must be a null terminated c-string, and the size must include the
terminating null character.
This fixes an out-of-bound access in ShaderBuilder::getShader() (which
doesn't exist anymore), because it was creating a CString passing
a size that included the null terminating char, which is not was CString
expects. CString can now assert() in that case.
driver::Program now uses a std::vector<> for storage, which we should
fix at some point (b/c it's a public header). CString was not suited to
store binary blobs.
This reverts a JobSystem optimization that attempted to avoid signaling
a condition when there was no waiters. Unfortunately, there was a
race that caused the the signaling thread to miss that the waiter flag
was set, thus not signaling.
- only signal waitAndRelease() when the corresponding job finishes and
only if there is waitAndRelease() active -- instead of signaling
every time a job ends.
- don't surrender time slice when attempting to steal a job and it fails
as long as some queue has jobs.
- check that we have to wait, because taking the lock
- add a benchmark
This change more than doubles the amount of jobs we can handle per
second (~965,000 jobs/s on Pixel3)
JobSystem::waitAndRelease used to spin to wait for the job to finish,
usually this wasn't a problem because the spinning thread was
able to handle other jobs. However, in cases where no job was
available it would actually spin in burn cpu cycles.
we now use a (separate) condition variable to handle that case.
