Commit Graph

4 Commits

Author SHA1 Message Date
Basil Milanich
d9a1cc06c1 Fix repro build: add -arch=native to use correct GPU architecture
NVCC 13.1 defaults to a PTX version incompatible with the installed
driver (580.105.08), causing kernels to silently fail with "provided
PTX was compiled with an unsupported toolchain". Use -arch=native so
NVCC auto-detects the target GPU (H100, sm_90) at build time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 13:33:05 -05:00
Basil Milanich
4ccaea9f08 Expand repro: multiple graphs, multiple kernels, interleaved launches
Tests:
- Two distinct graphs (different graphIds) on the same stream
- Graph A: kernel + memcpy + kernel (3 nodes)
- Graph B: scale + add + scale (3 nodes)
- 5 interleaved launches of each, stressing the graphId cache
- Expected 30 graph GPU zones total

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 11:54:08 -05:00
Basil Milanich
7bca9dcd90 Fix CUDA Graph GPU zones with proper cuGraphLaunch correlation
Replace the synthetic APICallInfo hack with proper correlation via
CUPTI_ACTIVITY_KIND_GRAPH_TRACE. When cuGraphLaunch fires an API
callback, its correlationId is stored in cudaCallSiteInfo. The
GRAPH_TRACE activity record carries the same correlationId plus the
graphId, which lets us build a graphId→APICallInfo map. Kernel/memcpy/
memset activities then look up this map via their graphId field.

Key changes:
- Add cuGraphLaunch/cuGraphLaunch_ptsz to cbidDriverTrackers so the
  API callback machinery captures the CPU call site
- Enable CUPTI_ACTIVITY_KIND_GRAPH_TRACE and handle it in
  DoProcessDeviceEvent to populate cudaGraphCurrentLaunch[graphId]
- Add cudaGraphCurrentLaunch map to PersistentState
- Two-pass buffer processing in OnBufferCompleted so GRAPH_TRACE
  records (which complete last on GPU) are processed before the
  kernel/memcpy/memset records that depend on them
- Replace graphId=0 fallback in kernel/memcpy/memset with proper
  cudaGraphCurrentLaunch lookup; fall through to matchError if
  the graphId is not found
- Update repro to include TracyCUDA headers and properly test
  GPU zone correlation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 11:03:58 -05:00
Basil Milanich
6c6999bf01 Add CUDA Graph GPU zone repro
Minimal reproducer showing that CUDA Graph-launched kernels produce
0 GPU zones in Tracy. The repro creates a simple graph (2 kernels +
1 memcpy), launches it 10 times, and expects ~30 GPU zones. Without
the fallback patch, all activity records are dropped by matchError().

Tested on NVIDIA H100, CUDA 13.1.
2026-03-24 09:37:50 -05:00