This commit introduces a CRC32 checksum to material packages to ensure
data integrity.
When a material is loaded, this checksum is verified. If the check
fails, an error is logged, and the material fails to load. For older
material packages without a CRC32 checksum, a warning is logged and
proceed.
BUGS=[373396840]
We link the behavior of certain precondition/postcondition checks to feature flag states.
- If provided *flag* is true, then this macro will assert when the *condition* is false.
- If provided *flag* is fase, then this macro will output a warning when the condition is false.
- If the condition is true, then neither of the above will happen
These two macros will enable us to provide less restrictive behavior for certain correctness assertions, allowing clients to modify their before enabling these assertions by default.
The addition JobSystem.cpp allows for defining
FILAMENT_TRACING_ENABLED across targets.
Addingin FILAMENT_TRACING_ENABLED to the #if in Tracing.h prevents
perfetto from being included.
* re-do "Use the Perfetto SDK instead of ATRACE" (#8701)
This time we create an entirely new private header: Tracing.h which
uses the perfetto SDK instead of ATRACE. The old Systrace.h is
unchanged to presever backward compatibility but is essentially
deprecated and no longer used within the filament repo.
The new TRACING_ macros use an explicit CATEGORY parameter, which is
declared in Tracing.h.
Moreover, tracing can be compiled out by defining
FILAMENT_TRACING_ENABLED to false before including Tracing.h
iOS tracing is still supported and still controlled via
FILAMENT_APPLE_SYSTRACE.
There are three perfetto categories defined:
- "filament/filament"
- "filament/jobsystem"
- "filament/gltfio"
The "filament/jobsystem" category is compiled out by default.
And they can be enabled in AGI / perfetto by adding:
```
data_sources {
config {
name: "track_event"
track_event_config {
disabled_categories: "*"
enabled_categories: "filament/filament"
enabled_categories: "filament/jobsystem"
enabled_categories: "filament/gltfio"
}
}
}
```
* Update libs/utils/include/private/utils/Tracing.h
Co-authored-by: Powei Feng <powei@google.com>
* Update libs/utils/src/android/Tracing.cpp
Co-authored-by: Powei Feng <powei@google.com>
* remove all references to SYSTRACE_TAG
---------
Co-authored-by: Powei Feng <powei@google.com>
We add rvalue version of append/insert/replace so that calling
those on a temporary yields to a temporary.
Also add string literal versions of those so that we can
append/insert/replace a literal without allocation, e.g.:
`CString foo = CString{ bar }.append("baz");`
This will not end-up creating a temporary CString for "baz".
clang optimizes code differently with very likely or unlikely
conditions, so we add a `VERY` version of these macros, and we
make use of it for assertions.
Perfetto has significantly less overhead. The User facing API is
mostly unchanged:
Here are the differences:
- SYSTRACE_ENABLE() does nothing on ANDROID, initializes systraces on darwin.
- SYSTRACE_DISABLE() is removed.
- A new "gltfio" tag is added.
- SYSTRACE_TAG *must* be defined before including `utils/Systrace.h`
- `utils/Systrace.h` should not be used from a public header
- the new SYSTRACE_TAG_DISABLE disables systrace at compile time
For android a data source MUST be created in the perfetto config:
```
data_sources {
config {
name: "track_event"
track_event_config {
enabled_categories: ["filament", "jobsystem", "gltfio"]
disabled_categories: "*"
}
}
}
```
This can for example be added to AGI's custom/advanced config.
FIXES=[407572663]
* Remove redundant qualifiers in filament public headers
* remove redundant qualifiers in filament implementation
* remove redundant qualifiers in libutils public headers
* remove redundant qualifier for libutils implementation
* remove redundant qualifiers for libmath
* use is_same_v<> instead of is_same<>
* bring back Builder::name()
we keep Builder::name() on all object, and forward to the MixIn class
that does the implementation, so that we have correct documentation, and
better IDE completion.
* add missing const parameters in filament's implementation
* various source cleanup
- missing includes
- missing const
- C cast style
- superfluous inline keyword
- when inserting an entry at a root other than zero, we need to update
the children count of the root's parent.
- the QuadTree array nodes need to be able to encode enough indices for
the largest "layer" in the tree. With 7 layers the largest one has
4096 entries, so we need 12 bits, not 8.
The previous fix attempt didn't work on some test. There are no known
bugs with AtomicFreeList other than tripping TSAN, and it's unclear
that TSAN isn't at fault.
However, switching to using a mutex works fine and doesn't appear to
be slower (it's actually faster with synthetic benchmarks on macOS)
BUGS=[377369108]
I do not think there was an actual error with AtomicFreeList, however
TSAN detected a data race when concurrent pop() happened. In that case,
there is indeed a race, where we can end-up reading data that is
already corrupted by the concurrent pop. However, that situation is
corrected by the following CAS. Somehow TSAN didn't see that.
The fix is strange and consists in replacing:
```
auto pNext = storage[offset].next;
```
with
```
auto s = storage[offset];
auto pNext = s.next;
```
In this PR we also adjust the memory ordering to be less strong. i.e.
we do not need `memory_order_seq_cst`, only the appropriate acquire or
release semantic.
In addition we also make `Node* next` a non-atomic variable again. It
should have been, but was change to placate an older version of TSAN.
BUGS=[377369108]
this change shouldn't have any impact on ARM, however, according
to cppreference it's not safe to mix seq_cst with other memory
orders:
"as soon as atomic operations that are not tagged memory_order_seq_cst
enter the picture, the sequential consistency guarantee for the program
is lost"
* improve parallel_for a bit
We get about 40% performance increase. The gain comes from not having
to copy the JobData structure each time we create a job, by using a
new emplaceJob() method, we can create the structure directly into
its destination.
* avoid calling wakeAll() when possible
wakeAll() is very expensive and not always needed when a job finishes
because there may not be anyone waiting on that job.
We now maintain a waiter count per job, and use that to determine if
we need to notify or not.
And now that the JobSystem overhead is lower, we can decrease the size
of the jobs, which improves the load balancing.
* mActiveJobs fixes
some comments claimed mActiveJobs needed to be modified before or after
accessing the WorkQueue; this couldn't be correct because there were no
guaranteed global ordering with the workQueue.
- reduce the number of calls to notify_one() and notify_all().
notify_one() is not only called when running a new job, and
notify_all() only when a job finishes.
- don't hold the condition lock while calling notify_*(), as it is not
strictly needed, and because notify_*() can be very slow, there can
be a lot of contention on this lock as a result; blocking the whole
jobsystem thread pool.
- add a new version of run() that takes an opaque thread id that can
be retrieved from a job's execute function; this is especially
intended to be used by parallel_for(); it's just a more efficient
version of run() that avoids a hashmap lookup.
Overall these change yield a significant performance boost:
- running + waiting a job: +200%
- running many jobs: +150%
- running many jobs in parallel: +50%
going forward, instead of using the printf style syntax for panics
we use the c++ stream syntax
The new macros that replace ASSERT_*CONDITON are
FILAMENT_CHECK_PRECONDITON
FILAMENT_CHECK_POSTCONDITION
FILAMENT_CHECK_ARITIHMETIC
Example usage:
FILAMENT_CHECK_PRECONDITON(condition) << "Message";
It's also now possible to define FILAMENT_PANIC_USES_ABSL=1 to redirect
all these calls to Abseil's CHECK() macro.
