The previous solution (75c173) didn't account for the fact that the text
to print may start with a space, in which case the text width calculation
results in 0. In effect, first word length was never greater than the
space left for printing, and the problem was still there.
Fix by walking through all initial spaces in firstWord.
If fwLen > left, then a line break is needed. In this case ignore initial
spaces in text.
TracyDebug fires from SysPower's ctor while it scans intel-rapl, which
runs as a Profiler member initializer -- before s_instance is set in
the Profiler ctor body. Under TRACY_MANUAL_LIFETIME without
TRACY_ON_DEMAND, the TracyInternalMessage path guarded this with
assert(ProfilerAvailable()), which aborted tracy-monitor whenever it
was run as root (only then is intel-rapl readable, so the log is
actually reached).
Soften the assert to an early-out, matching the TRACY_ON_DEMAND branch.
A TracyDebug issued before the profiler is up now silently skips
instead of aborting.
FindExternalImageRefresh already re-parsed /proc/<pid>/maps on miss,
but only one of three external decode paths used it. Switch the
DecodeCallstackPtrFastExternal and DecodeSymbolAddressExternal paths
over so symbol-name and file/line lookups stay fresh after the target
dlopens a library.
Rate-limit the re-parse to once per wall-clock second so samples
landing on permanently unresolvable regions (JIT, vDSO, stacks) do
not trigger a full parse each time.
perf_event_open(pid>0, cpu>=0) binds to a single task, so the previous
setup only sampled the target's main thread. In monitor mode, enumerate
/proc/<pid>/task/ and open one per-task event per existing thread with
cpu=-1; inherit=1 then covers every descendant. Self-profiling behavior
is preserved byte-for-byte: the iter list becomes (currentPid, i) for
each CPU, exactly what the old code did inline.
- Loop startup waitpid on EINTR; kill and reap the child on fatal error
or when interrupted, instead of leaking a ptrace-stopped process.
- Treat PTRACE_DETACH failure as fatal -- otherwise the child is stuck
stopped forever.
- Zero-initialize procName so the memcpy into ___tracy_magic_process_name
does not copy uninitialized stack past the NUL.
- Forward SIGINT to the child from the signal handler when in forked
mode, so Ctrl-C during a blocking waitpid unblocks cleanly.
- Preflight perf_event_open on the target before StartupProfiler so
permission failures surface with actionable guidance instead of
silently producing no samples.
- Also handle SIGHUP and SIGQUIT.
We redirect GetProfiler() (most likely used by any project consuming tracy, since it's used by `tracy::ScopedZone`) to its implementation which now has a different function name based on the macros that can impact ABI (and enabled/disabled).
That way, when linking with mismatched defines you'd get an error such as
> main.obj : error LNK2019: unresolved external symbol "int __cdecl GetProfiler_CFG_E0_OD0_DI0_ML0_F0_DHT0_TF0(void)" (?GetProfiler_CFG_E0_OD0_DI0_ML0_F0_DHT0_TF0@@YAHXZ) referenced in function "int __cdecl GetProfiler(void)" (?GetProfiler@@YAHXZ)
Or
>[build] /usr/bin/ld: CMakeFiles/app.dir/main.cpp.o: in function `GetProfiler()':
[build] /..../TracyProfiler.hpp:143: undefined reference to `GetProfiler_CFG_E1_OD0_DI0_ML0_F0_DHT0_TF0()'
Reason for going with acronym+0/1 instead of just acronym when enabled is for us to be able to tell users easily which define is wrong by just looking at the error if needed.
The only thing we don't really detect is user not having TRACY_ENABLE but tracy having been built with it. This is because macros become noops in that case, with no reference to `GetProfiler`.
There may be a way to do it by introducing a local variable into each TU, but I don't really like that idea.
We could also add pragma detect mismatch for a more user-friendly error on windows (https://learn.microsoft.com/en-us/cpp/preprocessor/detect-mismatch?view=msvc-170).
This is to be consistent with what libtracefs does: 6fad6a14ba/src/tracefs-utils.c (L104)
In theory one may mount tracefs with another name, though unlikely.
This forces to (re)use frequency values as input, which may be changed by the platform code later on. This way we have a single "source of truth" for sample freq.
Also removed the Win32 `GetSamplingInterval` which was a wrapper above `GetSamplingPeriod` but its value would be divided again anyway.
On 32-bit arm, phdr.p_vaddr is 32-bit, which causes a compilation
error because std::min expects both arguments to be of the same
type. Adding the static cast handles this case explicitly.
The comment still described consuming/not-consuming cudaCallSiteInfo
entries, but memory CBIDs are no longer tracked so no entry exists.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the mutex-guarded empty check in OnBufferCompleted with an
std::atomic<bool> dirty flag. The mutex is now only acquired when
there is actual retirement work to do. Also update stale comment
on cudaGraphCurrentLaunch that said "let them leak".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
operator[] on ConcurrentHashMap returns a reference after releasing the
read lock — the subsequent assignment happens with no lock held. This is
a latent data race if the map is ever accessed from multiple threads.
insert_or_assign performs the lookup and assignment atomically under a
single write lock, which is the correct pattern for a ConcurrentHashMap.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add `using GraphID = uint32_t` typedef and use it throughout for
graphId-typed variables (PersistentState, matchGraphActivityToAPICall,
getGraphIdFromRecord, retirement set, buffer loop).
- Move matchError from matchGraphActivityToAPICall to caller sites
(KERNEL, MEMCPY, MEMSET handlers). Keeping the error at the caller
provides more debugging context about which activity kind failed.
Remove the now-unnecessary `kind` parameter from the function.
- Replace insert_or_assign with operator[] assignment in
matchGraphActivityToAPICall. Access to graphLaunchCache is
single-threaded (CUPTI worker), so the simpler syntax is sufficient.
Remove the insert_or_assign method from ConcurrentHashMap entirely.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
cudaMalloc/cudaFree (and driver equivalents) were tracked in
cbidRuntimeTrackers/cbidDriverTrackers, creating a cudaCallSiteInfo
entry on each API call. But the MEMORY2 handler never calls
matchActivityToAPICall (and never calls EmitGpuZone) — it only needs
the address, size, and timestamp from the activity record itself. Since
no activity handler consumes these entries, they leaked indefinitely.
Remove the 6 memory API CBIDs from both tracker maps so no entry is
created. This eliminates the leak with no change in visible behavior:
the MEMORY2 handler already operates independently of cudaCallSiteInfo.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without retirement, the cache grows by one entry per unique exec handle
ever launched and never shrinks. While bounded by the number of distinct
execs in the application, long-running programs creating and destroying
many exec handles accumulate stale entries indefinitely.
Retirement mechanism:
- At cudaGraphExecDestroy (ENTER, while handle is still valid): call
cuptiGetGraphExecId to translate exec handle → graphId and add to a
pending-retirement set. Works for both runtime (cudaGraphExecDestroy)
and driver (cuGraphExecDestroy) APIs. No new subscription needed —
the existing cuptiEnableDomain already routes all API callbacks here.
- Deferral in OnBufferCompleted: erasure is not done immediately because
cudaGraphExecDestroy does not wait for GPU completion. CUPTI may still
have undelivered activity records for the last launch in its internal
buffers. We defer the erase until a full buffer arrives that contains
no records bearing the retired graphId, indicating all in-flight
records have been delivered.
- getGraphIdFromRecord: new helper that extracts the graphId field from
CONCURRENT_KERNEL / MEMCPY / MEMSET activity records (the three kinds
that carry a graphId) for use in the per-buffer tracking.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
In the JSON exception catch handler, m_jobsLock.lock() is called directly
on the mutex instead of through the jobsLock unique_lock. When the function
returns, jobsLock's destructor runs but it doesn't own the lock (it was
unlocked earlier at line 1134), and m_jobsLock is never released. This
causes a permanent deadlock the next time anything tries to acquire
m_jobsLock.
