Commit Graph

46 Commits

Author SHA1 Message Date
Powei Feng
3207c31721 renderdiff: disable webgpu + bloom due to flake (#9368)
WebGPU's rendering of bloom sems to be non-deterministic.  This
is not expected.  For now, we disable bloom+webgpu to avoid
hitting flakes in our CI.

This required a slight refactoring of the test configuration
parsing and execution scripts.

RDIFF_BRANCH=pf/renderdiff-disable-bloom-webgpu
2025-10-31 06:55:44 +00:00
Powei Feng
4547b69ccc backend-test: fix breakages on linux (#9350)
- On webgpu, do not error on linux when we're building headless
   only (without platform support for surface).
- For test/backend/test.sh, enable path for building on
   Linux + 64-bit x86
2025-10-23 21:12:25 +00:00
Powei Feng
6caf88d3aa github: add backend tests to presbumit (gl/vk) (#9344)
This commit enables running the backend tests for opengl and
vulkan on github CI. The script is located in test/backend.

We also add exceptions (SKIP_TEST) for the currently failing
tests.  Issues have been filed to track addressing these failed
tests.

We also disable a series of broken webgpu tests.
2025-10-22 10:21:55 -07:00
Powei Feng
0e1e5fb381 renderdiff: disable SSR for transmission (#9336)
SSR is not required for transmission tests, and it seems to be
flaky.

RDIFF_BRANCH=pf/renderdiff-disable-transmission
2025-10-17 13:51:35 -07:00
Powei Feng
b6df2b9b35 renderdiff: add transmission and ssao tests (#9232)
- Add transmission and ssao tests
 - Small mods to README.md
 - Add transmission models to gltf list
 - Change focal length to zoom in for more details

RDIFF_BRANCH=pf/renderdiff-add-transmission
2025-10-16 23:17:07 -07:00
Ben Doherty
271e639abe Revert "Make check-headers test use c++20" (#9316) 2025-10-14 14:12:34 -07:00
Powei Feng
6d061b5d01 renderdiff: add two params to local_test.sh (#9301)
- --no_rebuild will skip building the gltf_viewer again
 - --num_threads will limit the number of threads used in running
   the rendering test.

Both of these options are meant for debugging locally.
2025-10-08 13:01:09 -07:00
Powei Feng
53ddb3dd1c renderdiff: [viewer] enable frontend to be used with runID (#9277) 2025-10-08 18:36:30 +00:00
Powei Feng
fdec0f79a2 renderdiff: [viewer] fix magnifier (#9290)
- Fix the magnifier positioning for both compare and standalone
   mode. Simplified a lot of the logic and streamlined passing
   of states between components.
- Slight css adjustments
2025-10-03 21:44:53 +00:00
Powei Feng
10e63bf2cf renderdiff: [viewer] add Run ID as a way to pull artifacts (#9272)
The viewer supports pulling artifacts based on PR number, and now
we support providing Run ID as an alternative to identify the
renderdiff run on Github CI.
2025-09-30 21:32:36 +00:00
Powei Feng
0fa3de5ac7 renderdiff: add magnification glass of rendering (#9265)
- Show magnifying glass when diffing between two images
- Add checkbox to enable or disable feature
2025-09-27 00:19:26 +00:00
Powei Feng
d8e8aafad7 vk: fix VulkanPlatform CString usage (#9266) (#9260)
Since #9259, CString in VulkanPlatform fall into the literal
constructor path. But we really want the null-terminated
(char const*) path.  So we cast the strings to (char const*)
to enforce null-teriminated behavior.

RDIFF_BRANCH=pf/renderdiff-add-tolerance
2025-09-26 23:42:56 +00:00
Powei Feng
f233b20427 renderdiff: add a filter option for local testing (#9257)
This option will allow for rendering and comparing a single
image or files that match a wildcard-style string.

README.md has been updated
2025-09-25 21:18:35 +00:00
Powei Feng
decf316802 Make check-headers test use c++20 2025-09-23 21:28:37 -07:00
Powei Feng
60df8ac6b8 renderdiff: enable webgpu and improvements (#9219)
- Allow for presets to override selected models when presented
   in order.
 - Add 'gltf' for model search path
 - Add a few simple gltf models to the 'base' preset
 - Improve UI so that missing tests do not generate any html
   bits.
 - Add documentation on using the viewer
 - Add renderdiff documentation to the project webpage.

RDIFF_BRANCH=pf/renderdiff-enable-webgpu
2025-09-18 21:29:00 +00:00
Powei Feng
a36ad004b7 renderdiff: add ability to diff arbitrary rendered images (#9211)
Add a way to show the difference between two rendererd image.
This is a convenience feature to see the differences between
backends.

Refactor the ExpandedFailedResult to use this new view.
2025-09-16 00:27:42 +00:00
Powei Feng
679b08b5db github: add script and action for getting gltf assets (#9085)
- Script for getting models from glTF-Sample-Assets
- Add support for reading gltf models from a file.
2025-08-21 22:16:12 +00:00
Powei Feng
663a451031 renderdiff: enable vulkan (#9108)
RDIFF_BRANCH=pf/renderdiff-enable-vulkan
2025-08-20 06:10:01 +00:00
Powei Feng
ba0793ac18 Revert "[WIP] renderdiff: enable vulkan"
This reverts commit 2c154be0b3.
2025-08-14 16:19:46 -07:00
Powei Feng
2c154be0b3 [WIP] renderdiff: enable vulkan 2025-08-14 16:19:17 -07:00
Powei Feng
b2e4163ae1 renderdiff: enable multi-threading in rendering (#9057)
- Use threadpool to spawn gltf_viewer calls
- Make sure to use /tmp directories to avoid conflicts between
   different backend and models
2025-08-13 21:45:57 +00:00
Powei Feng
b129e5df4a renderdiff: fix upload issue second try (#8972)
- Move upload step to later so that comparison result is also
    included in the upload
 - Fix the wrong relative paths in the comparison_result.json
 - Fix viewer logic to pick only the latest run for a PR.
2025-07-22 18:47:18 +00:00
Powei Feng
d136d85402 renderdiff: fix compare.py 2025-07-16 15:03:44 -07:00
Ben Doherty
a44f6d9498 Update renderdiff README.md (#8961) 2025-07-16 14:56:20 -07:00
Powei Feng
b5aeee3dfd renderdiff: upload artifact even when comparison fails (#8960)
Previously, if a comparison fails then the artifacts are not
uploaded.  This is defeats the puprpose of having artifacts. We
fix that by moving the test logic into presubmit.yml.

local_test.sh is the way to repro a run from local machine.
2025-07-16 14:45:51 -07:00
Powei Feng
98eea205f3 renderdiff: viewer can pull artifacts given PR number (#8919)
This will allow for a faster flow then downloading manually.
Though it will require user to supply a github token.
2025-07-03 14:17:31 -07:00
Powei Feng
75d641ae8a renderdiff: fix artifact upload (#8879)
- Fix a problem where the output directory's name  has been
   changed
- Add git commit hash to the golden directory
2025-06-17 23:20:45 +00:00
Powei Feng
0ecf6c46e2 github: Fix commit message parsing (#8817) 2025-06-03 14:24:29 -07:00
Powei Feng
a68aaa114e renderdiff: fix breakage in parsing commit message (#8808) 2025-06-02 15:55:35 -07:00
Powei Feng
3da7dabb2a renderdiff: enable update goldens on commit merge (#8771) 2025-06-02 14:12:26 -07:00
Powei Feng
cd22478e4f renderdiff: add viewer for image differences (#8768)
- Modify the compare script to output more details of a
   comparison. This will include the source/golden directory,
   the comparison directory (the new renderings), and a file
   path to difference images if the golden does not match
   the rendered image.
 - The image_diff script can now output a TIFF that is the
   difference of two input TIFFs.
 - Add a viewer for examining the differences between rendered
   output and golden images.
 - The viewer consists of a simple server of web API endpoints
   for querying difference results (along with rendered images
   in TIFF).
 - And a web-based (html + lit-element) UI for looking at the
   rendered images and differences.
2025-06-02 19:37:46 +00:00
Powei Feng
361ba2afea renderdiff: separate comparison from rendering (#8733)
- Move the comparison logic into its own script
- Add entry point bash script to generate the renderings
- Separate the preamble bash logic into its own file
2025-05-14 09:48:16 -07:00
Powei Feng
e85dfe75c8 github: move test script to /test and same for /build (#8714)
- Moved get_mesa.sh to build/common/get-mesa.sh
- Moved check-headers.sh to test/check-headers/test.sh
- Moved check-metal-shaders.sh to test/check-metal-shaders/test.sh
2025-05-09 22:35:18 +00:00
Powei Feng
36e775902d renderdiff: script for updating golden images (#8709)
Adding a python script to enable updating new goldens into
a staging branch in the golden repo (filament-assets).

The same script can be used in github workflow to automatically
create a golden staging branch. This will be useful for users
without access to a mac (the only platform for generating
goldens as of now).
2025-05-08 22:07:17 +00:00
Powei Feng
28ecf5c35d renderdiff: add golden repo support (#8689)
- Add GoldenManager to manage access to the repo containing the
  goldens
- Add tif comparison code
- Enable comparison by default for actual test
2025-05-07 21:20:22 +00:00
Powei Feng
8070643ba5 ci: refactor to centralize version definitions (#8663)
- move CI only prerequisites to github actions
 - Add linux, mac prereq actions
 - Add an action for indicating dependency versions
 - Move ninja installation into its own script
2025-04-29 10:02:51 -07:00
Powei Feng
7c6b470650 Improve code-correctness script (#8659)
- Sort failures alphabetically
- Add way to suppress error when necessary
- Add way to run test on a subset of files for testing
- Fix case where number of files is less than number of workers
2025-04-28 17:00:40 +00:00
Powei Feng
7967157fbb renderdiff: fix breakage (#8648)
- Re-enable renderdiff test.
- Cache the mesa directory so that we're not spending time pulling
  and compiling it.
- Move python prereqs into the script and use venv.
2025-04-22 18:23:05 +00:00
Powei Feng
3fdb9f311a github: add code-correctness presubmit (#8595) 2025-04-04 15:17:07 -07:00
Powei Feng
df897b3fb2 osmesa: Enable Mesa/OSMesa rendering for mac (#8530)
This commit enables local running of the renderdiff tests and also
for github workflows.
2025-03-20 22:37:23 +00:00
Powei Feng
5f730ac173 osmesa: allow static linking of osmesa (#8486)
- Add path for static linking
- Fix get_mesa script.
2025-03-12 21:00:22 +00:00
Powei Feng
db30a9a50c viewer: add option to export TIFF in automation (#8472)
PPM does not store alpha channel, but TIFF does.  We add a method
to export RGBA to a TIFF file without compression.

We add the corresponding options to gltf_viewer and
AutomationEngine.

The default export format for both gltf_viewer and AutomationEngine
is now TIFF.
2025-02-27 18:54:37 +00:00
Powei Feng
e6b15a90e9 renderdiff: fix mesa deps (#8362)
There are some packages that cannot be installed with
 apt-get -y build-dep mesa

We manually add those packages wrt to the expected clang version.
2025-01-16 23:20:40 +00:00
Powei Feng
c435fc74c4 renderdiff: add initial code for updating goldens (#8349)
- Code for parsing tag in commit message
- Add placeholder for pulling and updating goldens in the script.
2025-01-16 07:08:48 +00:00
Powei Feng
65c30f3199 renderdiff: fix llvm-18 dep problem (#8347)
The public dep for mesa has moved on to beyond llvm-18, but the
version we're compiling against is not supported by this change.
We workaround by ensuring that the llvm dep is less than 18 but
consistent with the clang version.
2025-01-13 23:34:42 +00:00
Powei Feng
3d4fc7852b github: add software rasterizer job for GL to presubmit (#8158)
We use Mesa's gallium swrast to render as the driver with
Filament's backend set to GL. We provide a few scripts to parse
the tests (as jsons) and run gltf_viewer to produce the rendering.
2024-09-26 16:06:40 -07:00