Commit Graph
9871 Commits
Author SHA1 Message Date
Bartosz Taudul 2c6adfb416 Regenerate markdown manual. 2026-04-23 00:28:59 +02:00
Bartosz Taudul bde6e06cd7 Update manual. 2026-04-23 00:24:09 +02:00
Bartosz Taudul 4022494934 Update NEWS. 2026-04-22 22:07:39 +02:00
Bartosz Taudul 68ee8704d2 Merge pull request #1335 from siliceum/feature/check-macros-mismatch
Detecting macro definitions mismatches at link time
2026-04-21 18:12:32 +02:00
Bartosz Taudul 8d9a8494b8 Merge pull request #1344 from siliceum/fix/etw-compat
Fix TracyETW_compat.h structs and values based on docs
2026-04-21 18:11:01 +02:00
Clément Grégoire c1ab158f6c Update manual with mismatch detection info 2026-04-21 13:51:24 +02:00
Clément Grégoire ca076b4a60 Add TracyMangle.hpp file to centralize config name mangling
Also rename MANGLED_NAME_BASED_ON_DEFINES => MANGLED_NAME_BASED_ON_CONFIG
2026-04-21 13:28:48 +02:00
Clément Grégoire a4f245550b Macro incompatibility experiment
We redirect GetProfiler() (most likely used by any project consuming tracy, since it's used by `tracy::ScopedZone`) to its implementation which now has a different function name based on the macros that can impact ABI (and enabled/disabled).
That way, when linking with mismatched defines you'd get an error such as

> main.obj : error LNK2019: unresolved external symbol "int __cdecl GetProfiler_CFG_E0_OD0_DI0_ML0_F0_DHT0_TF0(void)" (?GetProfiler_CFG_E0_OD0_DI0_ML0_F0_DHT0_TF0@@YAHXZ) referenced in function "int __cdecl GetProfiler(void)" (?GetProfiler@@YAHXZ)

Or

>[build] /usr/bin/ld: CMakeFiles/app.dir/main.cpp.o: in function `GetProfiler()':
[build] /..../TracyProfiler.hpp:143: undefined reference to `GetProfiler_CFG_E1_OD0_DI0_ML0_F0_DHT0_TF0()'

Reason for going with acronym+0/1 instead of just acronym when enabled is for us to be able to tell users easily which define is wrong by just looking at the error if needed.

The only thing we don't really detect is user not having TRACY_ENABLE but tracy having been built with it. This is because macros become noops in that case, with no reference to `GetProfiler`.
There may be a way to do it by introducing a local variable into each TU, but I don't really like that idea.
We could also add pragma detect mismatch for a more user-friendly error on windows (https://learn.microsoft.com/en-us/cpp/preprocessor/detect-mismatch?view=msvc-170).
2026-04-21 13:28:48 +02:00
Clément Grégoire 1711d024dd Fix TracyETW_compat.h structs and values based on docs
https://learn.microsoft.com/fr-fr/windows/win32/api/evntprov/ns-evntprov-event_filter_event_id
https://learn.microsoft.com/fr-fr/windows/win32/etw/system-providers

Note: WinSDK does not use ULL in keyword constants so I removed them too.
2026-04-21 13:20:48 +02:00
Bartosz Taudul 217bdcf5a9 Merge pull request #1343 from siliceum/fix/better-tracefs-detection
Better tracefs detection
2026-04-21 11:55:10 +02:00
Clément Grégoire 4aac9e677d Fix formating/whitespaces in SysTraceStart 2026-04-21 09:51:43 +02:00
Clément Grégoire aba343e429 Pick first debugfs entry only 2026-04-21 09:51:42 +02:00
Bartosz Taudul 25f09bee2c Merge pull request #1342 from siliceum/fix/1337-respect-max-sample-rate
Fix #1337 : On Linux respect max sample rate
2026-04-20 19:33:14 +02:00
Clément Grégoire cb9ef7814e Use ReadFile and atoi 2026-04-20 18:05:31 +02:00
Clément Grégoire 8f208d732a Fix extra space 2026-04-20 18:04:41 +02:00
Clément Grégoire 8816dd0557 Allow finding tracefs through debugfs as a fallback for older kernels/systems that only mount debugfs 2026-04-20 15:04:33 +02:00
Clément Grégoire 0a5647b20f Use mnt_type instead of mnt_fsname for tracefs discovery
This is to be consistent with what libtracefs does: https://github.com/rostedt/libtracefs/blob/6fad6a14ba0d4c4b437d9e4eed7098d4bb07b4fc/src/tracefs-utils.c#L104
In theory one may mount tracefs with another name, though unlikely.
2026-04-20 15:00:37 +02:00
Clément Grégoire d48794024d Log why SysTrace does not start on Linux 2026-04-20 14:49:57 +02:00
Clément Grégoire 95b6fdeed3 Fixes #1337: On Linux, respect perf_event_max_sample_rate to avoid lost events
This may be especially useful for low performance machines. We also warn about this behaviour through TracyDebug which ends up in Messages.
2026-04-20 14:31:41 +02:00
Clément Grégoire e550e15ce6 Turn GetSamplingPeriod into SamplingFrequencyToPeriodNs
This forces to (re)use frequency values as input, which may be changed by the platform code later on. This way we have a single "source of truth" for sample freq.
Also removed the Win32 `GetSamplingInterval` which was a wrapper above `GetSamplingPeriod` but its value would be divided again anyway.
2026-04-20 14:29:35 +02:00
Bartosz Taudul 4e670fcaf5 Merge pull request #1338 from imsarllc/fix_32bit_arm_issue
Add static cast to uint64_t for std::min
2026-04-17 18:44:09 +02:00
Erik van Zwol 70fc86536c Add static cast to uint64_t for std::min
On 32-bit arm, phdr.p_vaddr is 32-bit, which causes a compilation
error because std::min expects both arguments to be of the same
type. Adding the static cast handles this case explicitly.
2026-04-17 09:45:43 -06:00
Bartosz Taudul 1c3691a57b Add llm chat topic. 2026-04-17 01:27:38 +02:00
Bartosz Taudul ebf3f02264 Merge pull request #1330 from bmilanich/cuda-graph-fallback
TracyCUDA: show GPU zones for CUDA Graph-launched kernels
2026-04-15 21:14:09 +02:00
Basil MilanichandClaude Opus 4.6 0a0566abc5 Update stale MEMORY2 comment to reflect cbid tracker removal
The comment still described consuming/not-consuming cudaCallSiteInfo
entries, but memory CBIDs are no longer tracked so no entry exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 10:28:13 -05:00
Basil MilanichandClaude Opus 4.6 58809f95ff Use atomic flag for retirement check to avoid mutex on hot path
Replace the mutex-guarded empty check in OnBufferCompleted with an
std::atomic<bool> dirty flag. The mutex is now only acquired when
there is actual retirement work to do. Also update stale comment
on cudaGraphCurrentLaunch that said "let them leak".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 10:23:42 -05:00
Basil MilanichandClaude Opus 4.6 8f6249c12f Restore insert_or_assign over operator[] for graphLaunchCache
operator[] on ConcurrentHashMap returns a reference after releasing the
read lock — the subsequent assignment happens with no lock held. This is
a latent data race if the map is ever accessed from multiple threads.

insert_or_assign performs the lookup and assignment atomically under a
single write lock, which is the correct pattern for a ConcurrentHashMap.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 10:23:42 -05:00
Basil MilanichandClaude Opus 4.6 da878d29d1 Address PR review feedback from slomp
- Add `using GraphID = uint32_t` typedef and use it throughout for
  graphId-typed variables (PersistentState, matchGraphActivityToAPICall,
  getGraphIdFromRecord, retirement set, buffer loop).

- Move matchError from matchGraphActivityToAPICall to caller sites
  (KERNEL, MEMCPY, MEMSET handlers). Keeping the error at the caller
  provides more debugging context about which activity kind failed.
  Remove the now-unnecessary `kind` parameter from the function.

- Replace insert_or_assign with operator[] assignment in
  matchGraphActivityToAPICall. Access to graphLaunchCache is
  single-threaded (CUPTI worker), so the simpler syntax is sufficient.
  Remove the insert_or_assign method from ConcurrentHashMap entirely.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 10:23:42 -05:00
Basil MilanichandClaude Sonnet 4.6 b22c8e86ba Remove memory API calls from cbid tracker maps
cudaMalloc/cudaFree (and driver equivalents) were tracked in
cbidRuntimeTrackers/cbidDriverTrackers, creating a cudaCallSiteInfo
entry on each API call. But the MEMORY2 handler never calls
matchActivityToAPICall (and never calls EmitGpuZone) — it only needs
the address, size, and timestamp from the activity record itself. Since
no activity handler consumes these entries, they leaked indefinitely.

Remove the 6 memory API CBIDs from both tracker maps so no entry is
created. This eliminates the leak with no change in visible behavior:
the MEMORY2 handler already operates independently of cudaCallSiteInfo.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 10:23:42 -05:00
Basil MilanichandClaude Sonnet 4.6 ec7805fd92 Retire cudaGraphCurrentLaunch entries on cudaGraphExecDestroy
Without retirement, the cache grows by one entry per unique exec handle
ever launched and never shrinks. While bounded by the number of distinct
execs in the application, long-running programs creating and destroying
many exec handles accumulate stale entries indefinitely.

Retirement mechanism:
- At cudaGraphExecDestroy (ENTER, while handle is still valid): call
  cuptiGetGraphExecId to translate exec handle → graphId and add to a
  pending-retirement set. Works for both runtime (cudaGraphExecDestroy)
  and driver (cuGraphExecDestroy) APIs. No new subscription needed —
  the existing cuptiEnableDomain already routes all API callbacks here.

- Deferral in OnBufferCompleted: erasure is not done immediately because
  cudaGraphExecDestroy does not wait for GPU completion. CUPTI may still
  have undelivered activity records for the last launch in its internal
  buffers. We defer the erase until a full buffer arrives that contains
  no records bearing the retired graphId, indicating all in-flight
  records have been delivered.

- getGraphIdFromRecord: new helper that extracts the graphId field from
  CONCURRENT_KERNEL / MEMCPY / MEMSET activity records (the three kinds
  that carry a graphId) for use in the per-buffer tracking.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 10:23:42 -05:00
Bartosz Taudul a64b9a2029 Proper mutex wrapper use.
In the JSON exception catch handler, m_jobsLock.lock() is called directly
on the mutex instead of through the jobsLock unique_lock. When the function
returns, jobsLock's destructor runs but it doesn't own the lock (it was
unlocked earlier at line 1134), and m_jobsLock is never released. This
causes a permanent deadlock the next time anything tries to acquire
m_jobsLock.
2026-04-11 17:29:44 +02:00
Bartosz Taudul 0269a196a4 Add TracyTaggedUserlandAddress.hpp to include list in meson. 2026-04-11 13:16:47 +02:00
Bartosz Taudul 010d25be06 Merge pull request #1332 from Brainzman/fix-missing-installed-include
Fix missing TracyTaggedUserlandAddress.hpp include due to it not being installed
2026-04-11 13:15:24 +02:00
Jonas Holzman 0de97789dd Fix missing TracyTaggedUserlandAddress.hpp due to it not being installed
Currently including the Tracy.hpp header from a set of installed Tracy
headers will result in the following error:

In file included from <...>/tracy/include/tracy/tracy/Tracy.hpp:133:
In file included from <...>/tracy/include/tracy/tracy/../client/TracyLock.hpp:9:
In file included from <...>/tracy/include/tracy/tracy/../client/TracyProfiler.hpp:18:
<...>/tracy/include/tracy/tracy/../client/../common/TracyQueue.hpp:6:10: fatal error: 'TracyTaggedUserlandAddress.hpp' file not found
    6 | #include "TracyTaggedUserlandAddress.hpp"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

Apparently introduced in f981330, which included the
TracyTaggedUserlandAddress.hpp header in TracyQueue.hpp without adding
it to the list of installed common header. Fixed by making the necessary
CMake change to install the header.

Ran into this issue while integrating Tracy as a dependency within
Blender[^1], where we use the latest main instead of stable for WoA
support, and use the install target to harvest the static lib and
headers for our libraries.

[^1]: https://projects.blender.org/blender/blender/pulls/156661
2026-04-11 12:27:40 +02:00
Bartosz Taudul f71c74aaf7 Change TRACY_ENABLE default to OFF to match documentation.
The documentation states that Tracy is disabled by default, but the
build system defaults were ON/true. Change CMake and Meson defaults to
OFF/false. Projects that need profiling enabled must now opt in
explicitly. Add explicit TRACY_ENABLE=ON / tracy_enable=true to CI
steps and the test project to preserve existing behavior.
2026-04-10 18:49:04 +02:00
Bartosz Taudul 82560c8c0e Fix emscripten CI job. 2026-04-10 17:06:34 +02:00
Basil MilanichandClaude Sonnet 4.6 12dc23f67e Add graphId recycle investigation test
Tests whether CUPTI recycles graphId values after cudaGraphExecDestroy,
which would be the only scenario where the graphLaunchCache in TracyCUDA
could serve stale entries for a non-matching exec handle.

Result (H100, CUDA 12, CUPTI): graphId is a monotonically increasing
counter that is never recycled. 22 create/instantiate/launch/destroy
cycles produced unique IDs ranging from 2 to 65 (incrementing by 3 per
cycle — one unit per node created during graph construction).

This confirms that the stale-cache concern raised in code review is not
a real risk in practice: two distinct exec handles always have distinct
graphIds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 09:34:51 -05:00
Basil MilanichandClaude Sonnet 4.6 c807367099 Add correlationId investigation test for CUDA Graph launches
Tests two questions:
1. Does relaunching the same cudaGraphExec produce a new correlationId
   each time, or is it reused?
2. Do two different cudaGraphExec handles from the same cudaGraph share
   a graphId?

Results on H100, CUDA 13.1:
- Each launch of the same exec handle gets a strictly unique, monotonically
  increasing correlationId. CPU callback corrId == GPU activity corrId.
  This is formally documented in cupti_activity.h:
    "Each graph launch is assigned a unique correlation ID that is
     identical to the correlation ID in the driver API activity record
     that launched the graph."
- graphId identifies the exec handle (instantiation), not the graph
  definition. Two cudaGraphInstantiate calls on the same graph produce
  different graphIds.

These findings confirm that the cudaGraphCurrentLaunch cache in
matchGraphActivityToAPICall is always refreshed by the first activity
of each new launch before the graphId fallback path is ever used.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 09:28:34 -05:00
Basil MilanichandClaude Sonnet 4.6 9e36e5d1ad Fix MEMORY2 handler: don't consume cudaCallSiteInfo entry
Calling matchActivityToAPICall in the MEMORY2 handler was consuming
the cudaCallSiteInfo entry for the graph launch correlationId. If a
graph mixes alloc nodes with kernel/memcpy nodes, all activities share
the same correlationId — consuming it here would cause
matchGraphActivityToAPICall to fail for the kernel/memcpy records that
follow, silently dropping their GPU zones.

Since apiCall is never used by the MEMORY2 handler (only address, size,
and timestamp from the activity record are needed), remove the call
entirely and leave the entry for the kernel/memcpy to consume and cache.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 16:01:41 -05:00
Basil MilanichandClaude Sonnet 4.6 b6222ae28b Fix MEMORY2 handler: don't gate memory tracking on API call correlation
CUpti_ActivityMemory3 has no graphId field, so graph-launched alloc
nodes and pre-profiling allocations can't be correlated to an API call.
The handler only needs address, size, and timestamp from the activity
record — apiCall is never used. Remove the early return so memory
tracking works in all cases.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 15:33:41 -05:00
Basil MilanichandClaude Sonnet 4.6 703df05529 Document known MEMORY2 limitation for graph-launched alloc nodes
CUpti_ActivityMemory3 has no graphId field, so matchGraphActivityToAPICall
cannot be applied. Graph-launched cudaGraphAddMemAllocNode emits multiple
MEMORY2 records sharing the launch correlationId; only the first is
tracked, subsequent ones fire a spurious matchError.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-07 15:30:27 -05:00
Basil MilanichandClaude Sonnet 4.6 c8ebc6f21e Review fixes: correct stale comments, fix fetch() missing lock, tidy helper
- Fix wrong comments on graph launch tracker entries: they claimed
  correlation works "via CUPTI_ACTIVITY_KIND_GRAPH_TRACE", but that
  approach was rejected (GRAPH_TRACE suppresses per-kernel records).
  The actual mechanism is the shared correlationId across all nodes
  in one graph launch.
- Fix ConcurrentHashMap::fetch() missing its read lock — a pre-existing
  data race now exercised by the new graph correlation hot path.
- Cache PersistentState::Get().cudaGraphCurrentLaunch in a local ref
  inside matchGraphActivityToAPICall instead of calling Get() twice.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 14:00:01 -05:00
Basil MilanichandClaude Sonnet 4.6 0b6e27934d ConcurrentHashMap: add insert_or_assign to replace erase+emplace
The cudaGraphCurrentLaunch cache update was acquiring the write lock
twice (once for erase, once for emplace). Wrapping
std::unordered_map::insert_or_assign under a single write lock lets
the caller do it in one operation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 13:48:53 -05:00
Basil MilanichandClaude Sonnet 4.6 d9a1cc06c1 Fix repro build: add -arch=native to use correct GPU architecture
NVCC 13.1 defaults to a PTX version incompatible with the installed
driver (580.105.08), causing kernels to silently fail with "provided
PTX was compiled with an unsupported toolchain". Use -arch=native so
NVCC auto-detects the target GPU (H100, sm_90) at build time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 13:33:05 -05:00
Basil MilanichandClaude Sonnet 4.6 f74dd21573 Refactor: extract matchGraphActivityToAPICall helper
The kernel, memcpy, and memset cases all had identical logic for
handling graph-launched activities. Extract it into a single helper
next to matchActivityToAPICall.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 13:08:39 -05:00
Basil MilanichandClaude Sonnet 4.6 4ccaea9f08 Expand repro: multiple graphs, multiple kernels, interleaved launches
Tests:
- Two distinct graphs (different graphIds) on the same stream
- Graph A: kernel + memcpy + kernel (3 nodes)
- Graph B: scale + add + scale (3 nodes)
- 5 interleaved launches of each, stressing the graphId cache
- Expected 30 graph GPU zones total

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 11:54:08 -05:00
Basil MilanichandClaude Sonnet 4.6 d36ca27041 Fix graph correlation: use shared correlationId, not GRAPH_TRACE
CUPTI discovery: all kernels launched by one cuGraphLaunch share the
same correlationId as the launch call itself. GRAPH_TRACE was the
wrong approach — enabling it suppresses per-kernel CONCURRENT_KERNEL
records entirely, replacing them with graph-level summaries.

New approach:
- Drop CUPTI_ACTIVITY_KIND_GRAPH_TRACE (it conflicts with CONCURRENT_KERNEL)
- Drop two-pass buffer processing (no longer needed)
- On the first kernel/memcpy/memset from a graph launch, matchActivityToAPICall
  succeeds (consuming the cuGraphLaunch entry) and the result is cached in
  cudaGraphCurrentLaunch[graphId]
- Subsequent operations from the same launch find the cached entry via graphId

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 11:36:14 -05:00
Basil MilanichandClaude Sonnet 4.6 2e5b076175 Also track cudaGraphLaunch runtime API for graph correlation
The repro uses cudaGraphLaunch (runtime API) not cuGraphLaunch (driver
API). Add cudaGraphLaunch_v10000 and its _ptsz variant to
cbidRuntimeTrackers so that graphs launched via the runtime API also
get their CPU call site captured for GPU zone correlation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 11:18:20 -05:00
Basil MilanichandClaude Sonnet 4.6 7bca9dcd90 Fix CUDA Graph GPU zones with proper cuGraphLaunch correlation
Replace the synthetic APICallInfo hack with proper correlation via
CUPTI_ACTIVITY_KIND_GRAPH_TRACE. When cuGraphLaunch fires an API
callback, its correlationId is stored in cudaCallSiteInfo. The
GRAPH_TRACE activity record carries the same correlationId plus the
graphId, which lets us build a graphId→APICallInfo map. Kernel/memcpy/
memset activities then look up this map via their graphId field.

Key changes:
- Add cuGraphLaunch/cuGraphLaunch_ptsz to cbidDriverTrackers so the
  API callback machinery captures the CPU call site
- Enable CUPTI_ACTIVITY_KIND_GRAPH_TRACE and handle it in
  DoProcessDeviceEvent to populate cudaGraphCurrentLaunch[graphId]
- Add cudaGraphCurrentLaunch map to PersistentState
- Two-pass buffer processing in OnBufferCompleted so GRAPH_TRACE
  records (which complete last on GPU) are processed before the
  kernel/memcpy/memset records that depend on them
- Replace graphId=0 fallback in kernel/memcpy/memset with proper
  cudaGraphCurrentLaunch lookup; fall through to matchError if
  the graphId is not found
- Update repro to include TracyCUDA headers and properly test
  GPU zone correlation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 11:03:58 -05:00
Bartosz Taudul 00a069d608 Perform meson build in mingw workflow. 2026-04-03 21:11:03 +02:00