docs: new page with mdbook (#8342)

- Use mdbook to generate a book from collection of documents
 - documents are
     - copied and processed from READMEs
     - original content stored in the docs_src folder
     - markdeep docs transformed into embedded html
 - Main logic is in script docs_src/build/run.py

docs_src/src/notes/README.md

@@ -0,0 +1,3 @@
+# Technical Notes
+
+Documents that pertain to components and use cases of the project.

docs_src/src/notes/debugging.md

@@ -0,0 +1,3 @@
+# Debugging
+
+Helpful documents for specific debugging needs.

docs_src/src/notes/libs.md

@@ -0,0 +1,3 @@
+# Libraries
+
+Collection of README.md from the `/libs` folder.

docs_src/src/notes/metal_debugging.md

@@ -0,0 +1,42 @@
+# Debugging Metal
+
+## Enable Metal Validation
+
+To enable the Metal validation layers when running a sample through the command-line, set the
+following environment variable:
+
+```
+export METAL_DEVICE_WRAPPER_TYPE=1
+```
+
+You should then see the following output when running a sample with the Metal backend:
+
+```
+2020-10-13 18:01:44.101 gltf_viewer[73303:4946828] Metal API Validation Enabled
+```
+
+## Metal Frame Capture from gltf_viewer
+
+To capture Metal frames from within gltf_viewer:
+
+### 1. Create an Info.plist file
+
+Create an `Info.plist` file in the same directory as `gltf_viewer` (`cmake/samples`). Set its
+contents to:
+
+```
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>MetalCaptureEnabled</key>
+    <true/>
+</dict>
+</plist>
+```
+
+### 2. Capture a frame
+
+Run gltf_viewer as normal, and hit the "Capture frame" button under the Debug menu. The captured
+frame will be saved to `filament.gputrace` in the current working directory. This file can then be
+opened with Xcode for inspection.

docs_src/src/notes/spirv_debugging.md

@@ -0,0 +1,199 @@
+# Investigating SPIRV-Cross / SPIRV-Tools issues
+
+There are 4 repositories at play here:
+- [KhronosGroup/glslang](https://github.com/KhronosGroup/glslang)
+- [KhronosGroup/spirv-tools](https://github.com/KhronosGroup/SPIRV-Tools)
+- [KhronosGroup/spirv-cross](https://github.com/KhronosGroup/SPIRV-Cross)
+- [KhronosGroup/SPIRV-Headers](https://github.com/KhronosGroup/SPIRV-Headers)
+
+Typically, the bug is present either in spirv-tools or spirv-cross.
+
+## Build and install command-line tools on PATH
+
+The goal is to replicate the bug outside of Filament, so we're going to use command-line versions of
+the SPIRV tools.
+
+### Clone and build each repo
+
+Note: Filament checks-out versions of these repositories inside `third_party/`; however, I've found
+it easiser to check out fresh copies separately so I can simply `git pull` to get the latest
+versions. Furthermore, Filament has modified some of these repositories locally for its own use
+case. Checking them out separately "proves" that the issue isn't Filament-specific.
+
+```
+git clone git@github.com:KhronosGroup/SPIRV-Tools.git
+git clone git@github.com:KhronosGroup/SPIRV-Cross.git
+git clone git@github.com:KhronosGroup/glslang.git
+git clone git@github.com:KhronosGroup/SPIRV-Headers.git SPIRV-Tools/external/SPIRV-Headers
+
+cd SPIRV-Tools/
+mkdir build && cmake . -G Ninja -B build
+ninja -C build
+cd ..
+
+cd SPIRV-Cross/
+mkdir build && cmake . -G Ninja -B build
+ninja -C build
+cd ..
+
+cd glslang/
+mkdir build && cmake . -G Ninja -B build
+ninja -C build
+cd ..
+```
+
+### Add directories to PATH
+
+```
+export PATH=`pwd`/SPIRV-Tools/build/tools:$PATH
+export PATH=`pwd`/glslang/build/StandAlone:$PATH
+export PATH=`pwd`/spirv-cross/build:$PATH
+```
+
+Ensure the following tools now exist on your PATH:
+
+1. `glslangValidator`
+2. `spiv-opt`
+3. `spirv-val`
+4. `spirv-cross`
+
+## Isolate the problematic GLSL shader
+
+First determine the Filament material and variant that causes the problem.
+
+What we want is the "raw" GLSL version of the shader, before any optimizations / cross-compilation
+happens.
+
+We can use the `--save-raw-variants` debug flag in matc to export each GLSL
+shader to a file. For example:
+
+```
+matc --save-raw-variants --optimize-size --variant-filter fog,ssr,vsm,stereo \
+        -a all -p all -o mymaterial.filamat mymaterial.mat
+```
+
+Files will be named like `mymaterial_0x05.frag` or `mymaterial_0x05.vert`.
+
+Note that gltfio material "templates" first go through a build step. After building gltfio, the
+gltfio Filament materials are output to:
+
+```
+out/cmake-release/libs/gltfio/*.mat
+```
+
+One of these materials can be compiled with the following command:
+
+```
+matc \
+    -TCUSTOM_PARAMS="// no custom params" \
+    -TCUSTOM_VERTEX="// no custom vertex" \
+    -TCUSTOM_FRAGMENT="// no custom fragment" \
+    -TDOUBLESIDED=false \
+    -TTRANSPARENCY=default \
+    -TSHADINGMODEL=unlit \
+    -TBLENDING=opaque \
+    --platform mobile --api metal -o temp.filamat \
+    unlit_opaque.mat
+```
+
+## Reproduce the compilation error
+
+The goal is to generate a .spv file that doesn't pass validation (through the spirv-val tool).
+
+Reproducing the error usually involves a few steps:
+
+1. Compile the raw GLSL shader into SPIR-V.
+
+```
+glslangValidator -V -o unoptimized.spv in.frag
+```
+
+2. Optimize for performance.
+
+```
+spirv-opt -Oconfig=optimizations.cfg unoptimized.spv -o optimized.spv
+```
+
+See [optimizations.cfg](optimizations.cfg) for a template. This file should contain the same list of optimizations that
+Filament employs. This should match the same optimizations specified in `GLSLPostProcessor`, for
+example, `GLSLPostProcessor::registerPerformancePasses` or `GLSLPostProcessor::registerSizePasses`.
+
+3. For shaders targeting Metal, convert relaxed ops to half.
+
+```
+spirv-opt \
+    --convert-relaxed-to-half \
+    --simplify-instructions \
+    --redundancy-elimination \
+    --eliminate-dead-code-aggressive \
+    optimized.spv \
+    -o half.spv
+```
+
+4. Finally, validate the final SPIR-V.
+
+```
+spirv-val half.spv
+```
+
+5. Sometimes validation will still pass, but still generate invalid shaders after cross-compiling.
+   In these cases, you'll need to cross compile to the target language and manually pick out errors
+   in the generated shader.
+
+```
+# for OpenGL
+spirv-cross optimized.spv > optimized.frag
+
+# for OpenGL ES
+spirv-cross --es optimized.spv > optimized.frag
+
+# for MSL
+spirv-cross --msl optimized.spv > optimized.metal
+```
+
+To invoke Apple's compiler to compile MSL, you can run:
+
+```
+xcrun -sdk macosx metal -c optimized.metal -o /dev/null
+```
+
+## Clean up the shader for a bug report
+
+These commands will run the preprocessor only on `in.frag`, and remove any empty lines.
+
+```
+glslangValidator -E in.frag > preprocessed.frag
+sed '/^$/d' preprocessed.frag > preprocessed_small.frag
+```
+
+You can also run `clang-format` on the preprocessed shader to make it easier to read:
+
+```
+clang-format -i preprocessed_small.frag
+```
+
+I always try to "whittle down" the shader to a smaller version that still reproduces the error. This
+might make it a bit easier on the Khronos team to diagnose the issue. I typically follow these steps
+in a loop until I'm satisfied:
+
+1. Delete an unnecessary part of the shader
+2. Run the steps to reproduce the error
+3. If the error still reproduces, repeat
+4. Otherwise, undo the change and make a smaller change
+
+There's also a [Reducer](https://github.com/KhronosGroup/SPIRV-Tools#reducer) tool that's part of
+SPIRV-Tools which can be used to automate these steps. I haven't experimented much with this, but it
+seems promising.
+
+## Submit an Issue with the relevant Khronos repository
+
+See some example issues that have been filed in the past:
+
+- https://github.com/KhronosGroup/SPIRV-Cross/issues/1935
+- https://github.com/KhronosGroup/SPIRV-Cross/issues/1088
+- https://github.com/KhronosGroup/SPIRV-Cross/issues/1026
+- https://github.com/KhronosGroup/SPIRV-Tools/issues/4452
+- https://github.com/KhronosGroup/SPIRV-Tools/issues/3406
+- https://github.com/KhronosGroup/SPIRV-Tools/issues/3099
+- https://github.com/KhronosGroup/SPIRV-Tools/issues/5044
+

docs_src/src/notes/spirv_debugging_optimizations.cfg

@@ -0,0 +1,33 @@
+--wrap-opkill
+--eliminate-dead-branches
+--merge-return
+--inline-entry-points-exhaustive
+--eliminate-dead-functions
+--private-to-local
+--scalar-replacement=0
+--ssa-rewrite
+--ccp
+--loop-unroll
+--eliminate-dead-branches
+--simplify-instructions
+--scalar-replacement=0
+--eliminate-local-single-store
+--if-conversion
+--simplify-instructions
+--eliminate-dead-code-aggressive
+--eliminate-dead-branches
+--merge-blocks
+--convert-local-access-chains
+--eliminate-local-single-block
+--eliminate-dead-code-aggressive
+--copy-propagate-arrays
+--vector-dce
+--eliminate-dead-inserts
+--eliminate-dead-members
+--eliminate-local-single-store
+--merge-blocks
+--ssa-rewrite
+--redundancy-elimination
+--simplify-instructions
+--eliminate-dead-code-aggressive
+--cfg-cleanup

docs_src/src/notes/tools.md

@@ -0,0 +1,3 @@
+# Tools
+
+Collection of README.md from the `/tools` folder.

docs_src/src/notes/versioning.md

@@ -0,0 +1,27 @@
+## Versioning
+
+Filament uses a 3-number versioning scheme that superficially resembles a [semantic
+version](https://semver.org/) but is actually more interesting because of our material system. Here
+are the guidelines:
+
+- Increment the **most significant** number only when making a non-backwards compatible API change,
+  or when introducing a major new API.
+- Increment the **middle number** only when making a non-backwards compatible change to the material
+  system. When this number gets bumped, users need to rebuild their mat files. Reset the middle
+  number to zero when the most significant number has been incremented.
+- Increment the **least significant** number each time a new release is published. Reset this number
+  to zero if one of the other two numbers have been incremented.
+
+## Material Versioning
+
+Additionally, the Filament renderer and material compiler internally contain a standalone integer
+called `MATERIAL_VERSION`, defined in `MaterialEnums.h`. This should be incremented every time we
+change the middle number in the public-facing version.
+
+When a material version mismatch is detected at run time, a panic is triggered, even in release
+builds. Therefore we should increment this only when making a serious breaking change to the
+material system (e.g. changing the size of a uniform block). Cosmetic shader changes usually do
+not merit a change to the material version number.
+
+Currently our material archives have two version chunks, one for "normal" materials and one for
+post-process materials. However for now these two numbers must be set to the same value.

docs_src/src/notes/vulkan_debugging.md

@@ -0,0 +1,15 @@
+# Debugging Vulkan
+
+## Enable Validation Logs
+
+Simply install the LunarG SDK (it's fast and easy), then make sure you've got the following
+environment variables set up in your **bashrc** file. For example:
+
+```
+export VULKAN_SDK='/path_to_home/VulkanSDK/1.3.216.0/x86_64'
+export VK_LAYER_PATH="$VULKAN_SDK/etc/explicit_layer.d"
+export PATH="$VULKAN_SDK/bin:$PATH"
+```
+
+As long as you're running a debug build of Filament, you should now see extra debugging spew in your
+console if there are any errors or performance issues being caught by validation.