test: use epsilon comparison for FMA-sensitive API tests (#6591)

When compiling with -march=znver4 (or any arch with FMA), GCC's default
-ffp-contract=fast contracts a*b+c into FMA opportunistically. The same
inline math function compiled in the shared library and in the test
binary can get different FMA contraction decisions due to different
optimization contexts, producing bit-different FP results.

Three API tests compare C++ direct calls (inlined into test TU) against
C API wrapper calls (through libassimp.so) using EXPECT_EQ (bit-exact),
which fails when the compiler contracts differently across TUs.

Verified via disassembly: the library uses vfnmadd FMA instructions
(3 roundings) while the test binary uses separate vmulss+vsubss
(6 roundings) for the same computation.

Replace EXPECT_EQ with Equal(epsilon) for the three affected tests:
- aiMatrix3FromToTest: use machine epsilon (~1.19e-7)
- aiMatrix4FromToTest: use machine epsilon (~1.19e-7)
- aiQuaternionFromNormalizedQuaternionTest: use 1e-4 because FMA
  differences in 1.0-x*x-y*y-z*z can flip a near-zero residual's sign,
  causing w=0 vs w=sqrt(tiny)≈1e-4

Fixes #6246

Co-authored-by: Chris de Claverie <declaverie@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Kim Kulling <kimkulling@users.noreply.github.com>

test/unit/AssimpAPITest_aiMatrix3x3.cpp

@@ -155,7 +155,7 @@ TEST_F(AssimpAPITest_aiMatrix3x3, aiMatrix3FromToTest) {
     const auto from = aiVector3D(1,2,1).Normalize(), to = aiVector3D(-1,1,1).Normalize();
     aiMatrix3x3::FromToMatrix(from, to, result_cpp);
     aiMatrix3FromTo(&result_c, &from, &to);
-    EXPECT_EQ(result_cpp, result_c);
+    EXPECT_TRUE(result_cpp.Equal(result_c, Epsilon));
 }
 
 TEST_F(AssimpAPITest_aiMatrix3x3, operatorTest) {

test/unit/AssimpAPITest_aiMatrix4x4.cpp

@@ -261,7 +261,7 @@ TEST_F(AssimpAPITest_aiMatrix4x4, aiMatrix4FromToTest) {
     const auto from = aiVector3D(1,2,1).Normalize(), to = aiVector3D(-1,1,1).Normalize();
     aiMatrix4x4::FromToMatrix(from, to, result_cpp);
     aiMatrix4FromTo(&result_c, &from, &to);
-    EXPECT_EQ(result_cpp, result_c);
+    EXPECT_TRUE(result_cpp.Equal(result_c, Epsilon));
 }
 
 TEST_F(AssimpAPITest_aiMatrix4x4, operatorTest) {

test/unit/AssimpAPITest_aiQuaternion.cpp

@@ -88,7 +88,10 @@ TEST_F(AssimpAPITest_aiQuaternion, aiQuaternionFromNormalizedQuaternionTest) {
     const auto qvec3 = random_unit_vec3();
     result_cpp = aiQuaternion(qvec3);
     aiQuaternionFromNormalizedQuaternion(&result_c, &qvec3);
-    EXPECT_EQ(result_cpp, result_c);
+    // Use a larger tolerance because FMA contraction differences in
+    // 1.0 - x*x - y*y - z*z can flip the sign of a near-zero residual,
+    // causing w = 0 vs w = sqrt(tiny) ≈ 1e-4.
+    EXPECT_TRUE(result_cpp.Equal(result_c, 1e-4f));
 }
 
 TEST_F(AssimpAPITest_aiQuaternion, aiQuaternionAreEqualTest) {