Commit Graph

82 Commits

Author SHA1 Message Date
Mathias Agopian
844dde4e6d simple allocation debugger
This can be enabled by using the Tracking::Debug policy,
currently this just fills allocation on alloc() and free(),
which is useful to help detect access to uninitialized memory
and use after free.

Now enabled by default in libfilament allocators on debug builds.

This caught uninitialized access in the froxelizer.
2019-08-28 19:06:54 -07:00
Mathias Agopian
71e98d54f5 add missing includes 2019-08-27 18:03:45 -07:00
Ben Doherty
d4943cc70b Add #include preprocessing to filamat and matc (#1541)
* Add #include preprocessing to filamat and matc

* Update RELEASE_NOTES

* Fix RELEASE_NOTES

* Use final instead of virtual / override

* Clarify comments

* Use pure virtual for includer functions

* Use a callback instead of an interface

* Rename Includer.h to IncludeCallback.h

* Update comment
2019-08-23 11:10:36 -07:00
Mathias Agopian
76cca21c65 most StructureOfArray<> methods cannot be constexpr
This is because we can't create a constexpr SoA in the first place,
and it wouldn't be very useful if we could.

This change ripples into users of SoA.
2019-08-13 18:29:15 -07:00
Mathias Agopian
eac2ad7e05 fix some lint warnings in Hash.h 2019-08-12 11:09:24 -07:00
Gregory Popovitch
772af1e897 More fixes for building with vs2019/msvc (#1500)
* Update Froxelizer.h

Fix this error when building with msvc from vs2019
error C2926:  'filament::details::Froxelizer::FroxelEntry::<unnamed-tag>::offset': a default member initializer is not allowed for a member of an anonymous struct within a union

* Fix some compilation issues with vs2019/msvc

Program.cpp:
1>C:\greg\github\filament\filament\backend\src\Program.cpp(28,42): error C2610:  'filament::backend::Program::Program(void) noexcept': is not a special member function or comparison operator which can be defaulted
1>C:\greg\github\filament\filament\backend\src\Program.cpp(28,42): message :  exception specification does not match the implicitly declared specification.

GLUtils.h: __PRETTY_FUNCTION__ macro is clang specific. Use MSVC equivalent

Color.h: fix warning

* #1493 - inline constructor in definition as requested by @romainguy

* #1493  "move this #define inside the #else below" as requested

* #1493 revert last change which causes compilation failures on other platforms.

provide empty implementation of Program::Program() in Program.cpp

* More fixes for building with vs2019/msvc

* #1500 use consistent macro definition syntax (@bejado)

* #1500 simplify DEBUG_COMMAND macro as requested by @pixelflinger

* #1500 use `{ 0 }` which is accepted by Visual Studio  (` = 0 ` is not accepted)

* #1500 remove incorrect UTILS_RESTRICT alltogether
2019-08-09 16:38:31 -07:00
Mathias Agopian
c65239aec2 Implement a basic Texture cache for the framegraph
We're only caching textures (not render targets yet), and we're
evicting cache entries older than 30 allocations.

This should cut down on texture allocation / gl calls.

The cache should get in a stable state very quickly (less than half
second).
2019-08-09 15:01:04 -07:00
Mathias Agopian
111b3320c0 make sure we never execute a job smaller than requested 2019-07-09 16:12:15 -07:00
Mathias Agopian
665703dbd8 parallel_for now creates jobs in reverse order
this is because the JobSystem's queue works as a LIFO, by creating
jobs in reverse (memory) order, we attempt to help streaming to
the d-cache on that threads -- until the point where
jobs are stolen. 

we also execute the last job immediately instead of creating a job
for it -- since we're already in a job.
2019-07-09 16:12:15 -07:00
Pixelflinger
8170ca7cd1 improve JobSystem::parallel_for + minor optimizations
- parallel_for doesn't use recursion anymore to create the "leaf"
jobs, this is now done linearly on N thread (one thread per CPU).
This uses less stack space, and reduces miss-predicted branches.

- remove almost all SYSTRACE calls because they have a huge impact
on things like parallel_for() and are misleading. They can be
enabled again by setting HEAVY_SYSTRACE to true.
2019-07-09 16:12:15 -07:00
Mathias Agopian
2df639133b improvements to JobSystem
- we simplify the waiting code by using only a single
condition variable instead of two.

- wait() now behaves just like a looper, it will process jobs until
the one it's waiting for finishes -- before it could just sit there
(the idea was that the job would finish quickly, but that's not always
the case).

- we also make sure to never call notify_n() when it's not needed.
We track how many waiters we have and use that to decide if we need
to notify().

notify is pretty slow on all architectures, even on linux it's always
a syscall, so it's better to avoid it.

- don't use stand-alone fences, makes things ugly for no real benefit

- refactored the code a bit, hopefully it's more clear.
2019-07-09 16:12:15 -07:00
Mathias Agopian
fc1d39334f separate SpinLock out of Allocator.h 2019-07-09 16:12:15 -07:00
Mathias Agopian
af16e43830 add notify_n(int) to our condition variable
notify_n() wakes up n waiters, this is more fine grained than
notify_one() / notify_all(), and the implementation costs nothing.
2019-07-09 16:12:15 -07:00
Mathias Agopian
64c95c615a Fix a theoretical wrapping around issue in WSDQ
The Work-stealing dequeue indices could wrap around after ~2 billion
calls to steal(). This could probably be achieved in a few hours.
By using 64-bits indices, we avoid the problem entirely.
2019-07-01 14:25:26 -07:00
Mathias Agopian
31745f6025 fix benchamrk JobSystem de-initialization
JobSystem's API requires to call emancipate() for each adopt() before
destroying the JobSystem.
2019-07-01 14:25:26 -07:00
Mathias Agopian
12fde30f31 better comments and potential fix for work-stealing dequeue
It's better to use std::memory_order_seq_cst in pop() and
steal() because we rely on ordering of access to
mTop and mBottom members.
2019-06-28 13:08:56 -07:00
Ben Doherty
daddc25de8 Remove std::sstream from filamat (#1176) 2019-05-13 13:56:15 -07:00
Mathias Agopian
a781fb6f2b support compilers that don't have attribute(packed)
this fixes #1140
2019-04-24 22:08:14 -07:00
Philip Rideout
0076de25fa Configurability fixups per code review. 2019-03-27 09:24:44 -07:00
Ben Doherty
38e08131bc Update libs/utils/include/utils/compiler.h
Co-Authored-By: pixelflinger <pixelflinger@gmail.com>
2019-03-15 15:42:28 -07:00
Mathias Agopian
5383174df4 Use proper "restrict" syntax on visual studio 2019-03-15 15:42:28 -07:00
Mathias Agopian
2a72e8aee8 break dependency of Program on SamplerInterfaceBlock and SamplerBindingMap
Instead of passing SamplerInterfaceBlocks and a SamplerBindingMap to
Program, we now set a 'sampler group' per binding point:

Program::addSamplerGroup(...)

A sampler group here consists of a list of N 'Sampler' and a 'Sampler'
is just a unique name (unifrom name in the shader) and binding point in
the shader.

That's all the driver layer needs.

With this change we get rid of the code that re-created the uniform
names in the driver -- this should never have been done there. And we
also remove the hash-map lookups in vulkan and metal drivers.
2019-03-07 13:43:27 -08:00
Mathias Agopian
f2ba48f4a7 Program::shader() now only takes a void* + size
Program::shader() was taking a string before which didn't make sense
for spirv.  Now it's just a blob, in the case of GL/Metal, the blob
must be a null terminated c-string, and the size must include the
terminating null character.

This fixes an out-of-bound access in ShaderBuilder::getShader() (which
doesn't exist anymore), because it was creating a CString passing
a size that included the null terminating char, which is not was CString
expects. CString can now assert() in that case.


driver::Program now uses a std::vector<> for storage, which we should
fix at some point (b/c it's a public header). CString was not suited to
store binary blobs.
2019-03-04 19:00:26 -08:00
Mathias Agopian
d1aecf864f allocator's free() must take 2 parameters 2019-01-31 17:36:19 -08:00
Adrian Perez
7ba46ef5f1 Add explicit includes for types not implied in all stdlib impls 2019-01-30 16:08:57 -08:00
Mathias Agopian
8bdae79536 added rebind copy-ctor to STLAllocator
this should fix the build on windows debug
2019-01-30 16:05:49 -08:00
Mathias Agopian
0bdb6b8a8a workaround a compile error with some compilers 2019-01-30 16:05:49 -08:00
Mathias Agopian
ae6ab66af4 fix #755: a race condition cousing a deadlock
This reverts a JobSystem optimization that attempted to avoid signaling
a condition when there was no waiters. Unfortunately, there was a
race that caused the the signaling thread to miss that the waiter flag
was set, thus not signaling.
2019-01-29 15:40:25 -08:00
Mathias Agopian
07c1dbb5f4 fix typo causing a memory corruption in STLAllocator<>
We were allocating n*sizeof(n) instead of n*sizeof(T). Thankfully, 
we were not using STLAllocator anywhere.
2019-01-23 19:14:28 -08:00
Mathias Agopian
8c8cfb6528 Frame graph API
This is only the guts of the frame graph implementation. It supports
culling of passes, but doesn't create real resources yet.

API still in flux.
2019-01-18 13:07:54 -08:00
shartte
f676d3d3ea Add constexpr constructor to StaticString (#659)
Fixes a header usage problem for MSVC, which complains about StaticString not having a constexpr constructor thus making the "make" method not constexpr.
2018-12-24 08:51:18 +01:00
Mathias Agopian
cfb9c03226 Improve JobSystem, especially under contention
- only signal waitAndRelease() when the corresponding job finishes and
only if there is waitAndRelease() active -- instead of signaling 
every time a job ends.

- don't surrender time slice when attempting to steal a job and it fails
as long as some queue has jobs.

- check that we have to wait, because taking the lock

- add a benchmark

This change more than doubles the amount of jobs we can handle per
second (~965,000 jobs/s on Pixel3)
2018-12-14 16:01:17 -08:00
Mathias Agopian
d6de2bf426 get rid of JobSystem::reset()
It was only used to clear the master job, instead the master job is
cleared when waited on.
2018-12-14 14:48:33 -08:00
Mathias Agopian
e38870ebf5 Improve JobSystem::wait
JobSystem::waitAndRelease used to spin to wait for the job to finish,
usually this wasn't a problem because the spinning thread was
able to handle other jobs. However, in cases where no job was
available it would actually spin in burn cpu cycles.

we now use a (separate) condition variable to handle that case.
2018-12-13 10:50:52 -08:00
Mathias Agopian
2d6b43827e Improve StaticString hashing
In a lot of case the StaticString hash can be computed a compile time,
so we now take advantage of that.

Removed StaticString(const char*) ctor, and replaced it with a
StaticString::make() method.

Fixed a couple wrong uses of the old StaticString(const char*) ctor.
2018-12-12 17:47:21 -08:00
Pixelflinger
2529a34c36 code size optimizations
- make Profiler::readCounters() not inline as it didn't need to be, 
it's not performance critical and it's sufficiently large.

- don't inline hasExtensions(), same reason.
2018-12-11 17:44:03 -08:00
Pixelflinger
e624dff037 set cpu affinity of JobSystem's threads
this is to try to prevent threads from bouncing
between cores
2018-12-11 13:46:05 -08:00
Romain Guy
025f4a54f5 Use root utils namespace in Systrace.h 2018-12-11 13:45:37 -08:00
Philip Rideout
b4b12914a1 Add bitcast to utils. 2018-12-04 17:56:37 -08:00
Mathias Agopian
42a3d5ea33 use utils::Mutex instead of std::mutex for Allocators locking policy
on Android it's about 2x faster and uses much less instructions, which
is better for i-cache pressure.
2018-11-29 11:44:52 -08:00
Ben Doherty
fe4907d0ff Remove use of iostream in filamat (#549) 2018-11-29 10:35:37 -08:00
Mathias Agopian
cf5606e940 Fix another bug in AtomicFreeList
An assert checking invariants would sometime trigger, the problem
was a logic error that would be exposed by a race when running out
of space in the list.

The root of the problem is that in one place we were not remapping the 
-1 offset to nullptr, storing a pointer violating our invariants.

Also added more asserts!!!
2018-11-28 12:10:23 -08:00
Sebastian Hartte
6b030c9694 More documentation of the free list 2018-11-27 11:44:32 -08:00
Sebastian Hartte
df4a2bb86e Fixes #542 by moving the assert to after checking for a possible race condition 2018-11-27 11:44:32 -08:00
Pixelflinger
bc0f2f26f2 Tweak Profiler API so it works in multithread
Instead of using a singleton, Profiler has to be instantiated in the
scope/thread it is used.
2018-11-26 11:51:17 -08:00
shartte
9ab9333fa5 Fixes various MSVC issues in public headers (#523) 2018-11-26 09:17:19 -08:00
Mathias Agopian
9e39004b1f Add wait_for and wait_until to our Condition implementation
We're using timed condition variable in one place, but the STL version
pulls in a lot of code because it does clock calculations in 
"long double" (!!!!). Since we already had an implementation
of condition_variable, we just add the timed version.
This saves several KiB of code.

Also don't use unique_lock() lock/unlock because it can throw exceptions.
2018-11-21 13:46:07 -08:00
Mathias Agopian
5bcc4cfff1 Improve the slow-path of creating a program
This removes a lot of heap allocations/deallocations, and reduces
code size. Most improvements come from using CString and
StaticString instead of std::string and better using move
semantics.
2018-11-21 13:46:07 -08:00
Mathias Agopian
c1a7164084 Improve usage of CString
This cuts down 7 calls to free() to 1, when
instantiating a new program. There are still many low-hanging fruits
there.
2018-11-21 13:46:07 -08:00
Philip Rideout
0d476b9f6d Fix debug builds. 2018-11-19 15:45:40 -08:00