Commit Graph

110 Commits

Author SHA1 Message Date
Pixelflinger
79995349d8 use systrace when EXT_debug_marker is not supported
EXT_debug_marker is not supported GL debuggers won't show these
markers, so as an alternative, we dump them in systrace.
2020-05-13 19:36:03 -07:00
Mathias Agopian
ee3c3d0965 debug option to track Entities (#2526)
* debug option to track Entities

Set FILAMENT_UTILS_TRACK_ENTITIES to true when building libutils to
activate entity tracking. This adds two public methods:

getActiveEntities() and dumpActiveEntities() the later displays the
stack trace of where the remaining entities were allocated.

This is useful for tracking leaks.

* Update libs/utils/include/utils/EntityManager.h

Co-authored-by: Philip Rideout <philiprideout@gmail.com>

Co-authored-by: Philip Rideout <philiprideout@gmail.com>
2020-05-12 17:55:39 -07:00
Pixelflinger
fb0e89d969 fix a race-condition in JobSystem
mThreadMap needs to be protected as it's accessed from multiple threads.
2020-04-03 17:25:17 -07:00
Philip Rideout
c79f5a1a29 JobSystem: replace TLS with tid mapping. 2020-04-02 14:53:20 -07:00
Romain Guy
45df197bd6 Expose libutils APIs that should have been public (#2269)
* Expose libutils APIs that should have been public

* Don't make JobSystem public for now
2020-03-24 09:57:04 -07:00
Pixelflinger
4a5ce16322 Trivially make LinearAllocator 16 bytes instead of 24 2020-03-11 17:04:08 -07:00
Pixelflinger
8fb65a19b4 fix utils::ctz() when builtins are not available
Our fallback 64-bits ctz() was completely wrong. 
The unit tests were wrong too.
2020-03-10 15:41:43 -07:00
Pixelflinger
0cbf6f8d76 disable spinlocks on ARMv7
we've had issues with some devices running in arm mode, where they
would throw spurious SIGILL on the WFE instruction used in spin locks.
Since on Android Mutexes are very efficient, we just use that instead.

This might fix #2197
2020-03-07 13:54:58 -08:00
Mathias Agopian
4842b1ca13 rework stream operators for libmath
libmath itself doesn't expose any stream operators anymore. However,
libutils is able to automatically print libmath types into its
io::ostream -- however matrices are not formatted nicely.

Added a new optional library, libmathio, that provides std::ostream
operators for all libmath types. libmathio does a better job at
formating matrices.

Also removed apply() and map() from libmath because there were not used
anywhere and they forced us to depend on <functional> in public headers.
2020-03-05 22:50:49 -08:00
Philip Rideout
bd5aeb6b69 Allow non-WebGL builds to disable threads. 2020-01-23 11:06:17 -08:00
Mathias Agopian
e27158507e refactor Renderer to make it more flexible
This is in preparation to supporting screen-space effects.
There are two major changes:

- RenderPass is now copiable and intended to be passed by copy to
  the execute stage of frame graph passes

- The color pass is in its own function now


This actually simplify RenderPass api.
2020-01-08 16:10:09 -08:00
Philip Rideout
898ed70aa0 Remove <iostream> from math headers.
This speeds up our build time by using <iosfwd> and un-inlining the
vector print functions.
2020-01-06 15:09:06 -08:00
Tim Psiaki
b030f67eb7 Make EntityManager and Panic dynamic-linkable. 2020-01-06 10:24:17 -08:00
Jordan Rupprecht
311f6eacd7 Add missing <assert.h> include. (#1969)
An version of clang is smarter about generating warnings for template code. `assert` here is not included, so clang generates: `error: use of undeclared identifier 'assert'`
2019-12-13 09:26:02 -08:00
Mathias Agopian
30a07a49d0 add a way to import/export an Entity from/to an integer
This is intended to be used for interop with other programing languages.
2019-11-11 14:37:10 -08:00
Mathias Agopian
0a37cce5fe don't use our own sync primitives when TSAN is used
TSAN seems to have problems with homegrown sync primitives, such as
spinlocks, so we just don't use them when TSAN is enabled.
2019-11-11 14:36:49 -08:00
Mathias Agopian
53ff133b03 fix a possible memory corruption in debug builds
TrackingPolicy::Debug didn't store the base pointer of the Area, and
instead relied on the first allocation to discover it, however, because
of alignment, the first allocation may not match the base pointer.
Because of that there could be an overflow in onRewind(), i.e. we
could rewind to a pointer before the (wrongly computed) base. This
overflow caused the debug memset to go awry and stomped on memory.

This is fixed by passing the base pointer to the constructor of the
TrackingPolicy. This base pointer could be nullptr with certain
allocators, but in that case, onReset/onRewind should never be called;
and this is enforced at compile time.

Also fixed a (luckily) harmless buffer overflow when preparing the
dynamic lights, if the number of lights wasn't a multiple of 4. This
was harmless because we use a linear allocator, so overflows are not 
really overflows.
2019-11-07 10:03:41 -08:00
Philip Rideout
87a05f057a Fix out-of-bounds in an [unused] utility.
This was caught by ASAN.

Our algorithm header has many one-liners that compute the "next power of
two length / 2" but they all have the caveat that if the input is
already POT, then the "/ 2" part does not occur.

Usually we deal with this by testing the difference against zero.
However in `partition_point` we were skipping the test, thus causing a
potential out-of-bounds access.

I fixed `partition_point` and added a few more tests for non-POT cases.
2019-11-05 17:14:15 -08:00
Ben Doherty
20f4ec6576 Use platform-specific temporary directories for CircularBuffer (#1769) 2019-10-15 12:01:03 -07:00
Ben Doherty
ef34dd2461 Fix operator overrides for Clang / MSVC on Windows (#1715) 2019-09-30 18:39:38 -07:00
Gregory Popovitch
d5c0d11404 Final changes for building with msvc 2019 on Windows (#1681) 2019-09-30 14:18:06 -07:00
Romain Guy
a67d92c2ca Lower limit from API 21 to API 19 (#1701)
* Lower limit from API 21 to API 19

This was requested by an internal application. API 19 is when OpenGL
ES 3.0 support was added so there is no good reason for us to not
support this API level. The only trick is to avoid referring to the
glTexStorage2DMultisample symbol directly as it only exists in 3.1.

* Compile out code we never use

* Use reflection to handle shared EGL contexts pre-API 21.

* Remove comment

* More fixes required to run on API level 19

- dlym() fails for ashmem on API 19, so we only try on API 26+ instead.
- Older emulation for OpenGL ES 3.0 returns error states for valid API calls so we need to clear the GL error bit before we create the GL driver.
- Activity lifecycle changes since API 19 would cause animations to keep running and to reference destroyed objects.
- EGLContext.getNativeHandle() is new in API 21, we need to use reflection on API 19. The new code path uses the class loading trick to avoid a bytecode verification error on API 19.

* Filament now runs properly on API level 19

This commit adds a new api_level() API to libutils which can be used to
query the platform's API level. On Android it works as expected, other
platforms currently return 0.
2019-09-28 14:13:06 +03:00
Philip Rideout
21b208398b Renderable doxygen now uses \see etc. 2019-09-27 15:26:08 -07:00
Philip Rideout
65b3e956df Add doxygen for RenderableManager and NameComponentMananger. 2019-09-27 15:26:08 -07:00
Mathias Agopian
5ed07639af workaround an MSVC bug with brace initialization
the C++ standard says:

   if T is a class type with a default constructor that is neither
   user-provided nor deleted (that is, it may be a class with an
   implicitly-defined or defaulted default constructor), the object
   is zero-initialized and then it is default-initialized if it has a
   non-trivial default constructor

Unfortunately, MSVC always calls the default constructor, even if it
is trivial, which breaks constexpr-ness.
To workaround this, we're always zero-initializing TVecN<>

Also removed constexpr from default constructors, since they never can
be constexpr as they're not initializing the vector.
2019-09-26 19:05:33 -07:00
Mathias Agopian
087b12c802 Fix minor time calculation imprecision
use double for the shader's 'second' time calculation, since 1 billion cannot be stored
in a float accurately.
2019-09-18 15:25:51 -07:00
Mathias Agopian
7b200ec85e make TextureUsage and TargetBufferFlags enum class
This prevents values like NONE or COLOR which are very generic
names to collide with other classic enums. It also forces us to be
specific about what we're doing when converting to bitfields.

Also added a utility to easily enable any enum or enum class to
support binary operation (i.e. Bitmask contract), e.g.:

template<> struct utils::EnableBitMaskOperators<Foo> : public std::true_type{};

gives all binary operators to 'Foo'. Foo must be an enum.

Also added any() and none() convenience to convert an enum to a boolean.
2019-09-12 11:19:54 -07:00
Romain Guy
2bc9256a7a Fix compilation error when DEBUG_COMMAND_STREAM is true (#1588)
Fixes issue #1585
2019-09-03 10:00:11 -07:00
Mathias Agopian
844dde4e6d simple allocation debugger
This can be enabled by using the Tracking::Debug policy,
currently this just fills allocation on alloc() and free(),
which is useful to help detect access to uninitialized memory
and use after free.

Now enabled by default in libfilament allocators on debug builds.

This caught uninitialized access in the froxelizer.
2019-08-28 19:06:54 -07:00
Mathias Agopian
71e98d54f5 add missing includes 2019-08-27 18:03:45 -07:00
Ben Doherty
d4943cc70b Add #include preprocessing to filamat and matc (#1541)
* Add #include preprocessing to filamat and matc

* Update RELEASE_NOTES

* Fix RELEASE_NOTES

* Use final instead of virtual / override

* Clarify comments

* Use pure virtual for includer functions

* Use a callback instead of an interface

* Rename Includer.h to IncludeCallback.h

* Update comment
2019-08-23 11:10:36 -07:00
Mathias Agopian
76cca21c65 most StructureOfArray<> methods cannot be constexpr
This is because we can't create a constexpr SoA in the first place,
and it wouldn't be very useful if we could.

This change ripples into users of SoA.
2019-08-13 18:29:15 -07:00
Mathias Agopian
eac2ad7e05 fix some lint warnings in Hash.h 2019-08-12 11:09:24 -07:00
Gregory Popovitch
772af1e897 More fixes for building with vs2019/msvc (#1500)
* Update Froxelizer.h

Fix this error when building with msvc from vs2019
error C2926:  'filament::details::Froxelizer::FroxelEntry::<unnamed-tag>::offset': a default member initializer is not allowed for a member of an anonymous struct within a union

* Fix some compilation issues with vs2019/msvc

Program.cpp:
1>C:\greg\github\filament\filament\backend\src\Program.cpp(28,42): error C2610:  'filament::backend::Program::Program(void) noexcept': is not a special member function or comparison operator which can be defaulted
1>C:\greg\github\filament\filament\backend\src\Program.cpp(28,42): message :  exception specification does not match the implicitly declared specification.

GLUtils.h: __PRETTY_FUNCTION__ macro is clang specific. Use MSVC equivalent

Color.h: fix warning

* #1493 - inline constructor in definition as requested by @romainguy

* #1493  "move this #define inside the #else below" as requested

* #1493 revert last change which causes compilation failures on other platforms.

provide empty implementation of Program::Program() in Program.cpp

* More fixes for building with vs2019/msvc

* #1500 use consistent macro definition syntax (@bejado)

* #1500 simplify DEBUG_COMMAND macro as requested by @pixelflinger

* #1500 use `{ 0 }` which is accepted by Visual Studio  (` = 0 ` is not accepted)

* #1500 remove incorrect UTILS_RESTRICT alltogether
2019-08-09 16:38:31 -07:00
Mathias Agopian
c65239aec2 Implement a basic Texture cache for the framegraph
We're only caching textures (not render targets yet), and we're
evicting cache entries older than 30 allocations.

This should cut down on texture allocation / gl calls.

The cache should get in a stable state very quickly (less than half
second).
2019-08-09 15:01:04 -07:00
Mathias Agopian
111b3320c0 make sure we never execute a job smaller than requested 2019-07-09 16:12:15 -07:00
Mathias Agopian
665703dbd8 parallel_for now creates jobs in reverse order
this is because the JobSystem's queue works as a LIFO, by creating
jobs in reverse (memory) order, we attempt to help streaming to
the d-cache on that threads -- until the point where
jobs are stolen. 

we also execute the last job immediately instead of creating a job
for it -- since we're already in a job.
2019-07-09 16:12:15 -07:00
Pixelflinger
8170ca7cd1 improve JobSystem::parallel_for + minor optimizations
- parallel_for doesn't use recursion anymore to create the "leaf"
jobs, this is now done linearly on N thread (one thread per CPU).
This uses less stack space, and reduces miss-predicted branches.

- remove almost all SYSTRACE calls because they have a huge impact
on things like parallel_for() and are misleading. They can be
enabled again by setting HEAVY_SYSTRACE to true.
2019-07-09 16:12:15 -07:00
Mathias Agopian
2df639133b improvements to JobSystem
- we simplify the waiting code by using only a single
condition variable instead of two.

- wait() now behaves just like a looper, it will process jobs until
the one it's waiting for finishes -- before it could just sit there
(the idea was that the job would finish quickly, but that's not always
the case).

- we also make sure to never call notify_n() when it's not needed.
We track how many waiters we have and use that to decide if we need
to notify().

notify is pretty slow on all architectures, even on linux it's always
a syscall, so it's better to avoid it.

- don't use stand-alone fences, makes things ugly for no real benefit

- refactored the code a bit, hopefully it's more clear.
2019-07-09 16:12:15 -07:00
Mathias Agopian
fc1d39334f separate SpinLock out of Allocator.h 2019-07-09 16:12:15 -07:00
Mathias Agopian
af16e43830 add notify_n(int) to our condition variable
notify_n() wakes up n waiters, this is more fine grained than
notify_one() / notify_all(), and the implementation costs nothing.
2019-07-09 16:12:15 -07:00
Mathias Agopian
64c95c615a Fix a theoretical wrapping around issue in WSDQ
The Work-stealing dequeue indices could wrap around after ~2 billion
calls to steal(). This could probably be achieved in a few hours.
By using 64-bits indices, we avoid the problem entirely.
2019-07-01 14:25:26 -07:00
Mathias Agopian
31745f6025 fix benchamrk JobSystem de-initialization
JobSystem's API requires to call emancipate() for each adopt() before
destroying the JobSystem.
2019-07-01 14:25:26 -07:00
Mathias Agopian
12fde30f31 better comments and potential fix for work-stealing dequeue
It's better to use std::memory_order_seq_cst in pop() and
steal() because we rely on ordering of access to
mTop and mBottom members.
2019-06-28 13:08:56 -07:00
Ben Doherty
daddc25de8 Remove std::sstream from filamat (#1176) 2019-05-13 13:56:15 -07:00
Mathias Agopian
a781fb6f2b support compilers that don't have attribute(packed)
this fixes #1140
2019-04-24 22:08:14 -07:00
Philip Rideout
0076de25fa Configurability fixups per code review. 2019-03-27 09:24:44 -07:00
Ben Doherty
38e08131bc Update libs/utils/include/utils/compiler.h
Co-Authored-By: pixelflinger <pixelflinger@gmail.com>
2019-03-15 15:42:28 -07:00
Mathias Agopian
5383174df4 Use proper "restrict" syntax on visual studio 2019-03-15 15:42:28 -07:00
Mathias Agopian
2a72e8aee8 break dependency of Program on SamplerInterfaceBlock and SamplerBindingMap
Instead of passing SamplerInterfaceBlocks and a SamplerBindingMap to
Program, we now set a 'sampler group' per binding point:

Program::addSamplerGroup(...)

A sampler group here consists of a list of N 'Sampler' and a 'Sampler'
is just a unique name (unifrom name in the shader) and binding point in
the shader.

That's all the driver layer needs.

With this change we get rid of the code that re-created the uniform
names in the driver -- this should never have been done there. And we
also remove the hash-map lookups in vulkan and metal drivers.
2019-03-07 13:43:27 -08:00