|
FhSim
3.1.0
Marine systems simulation
|
|
|
FhSim
3.1.0
Marine systems simulation
|
| ID | 0030 |
| Class | TEST |
| Severity | 2 |
| Status | ready |
| Models | Trawl/CenterWeight |
| Found | 2026-09-22 |
| Decision needed | Resolved: the drift was found (issue 0036). Re-recording is now safe; it needs only an owner's nod because it rewrites a committed baseline. |
Building at aa39979 with conan install . -s build_type=Release --build missing --update followed by conan build . makes SimObject.CenterWeight (tests/CenterWeight_Test.cpp, via RunAndCompareRegressionXmlTiming at tests/TestUtils.h:60) fail against the committed baseline tests/in/CenterWeight/CenterWeight_ref.csv:
The failure reproduces with the working tree clean (verified by stashing all local changes and rebuilding), so it is not caused by any local edit. The remaining variable is the dependency set: --update pulled fresh fhsim_marine_elements and fhsim_environment binaries, and CenterWeight is the one regression case that takes a seafloor forces element and an environment (src/trawl/CenterWeight.cpp:188-191).
Runs of the same binary against the same input do not always agree. The variation is intermittent, not systematic, which is the most important property of this bug and the easiest one to miss: a short check can easily draw two identical runs and conclude the case is deterministic.
Observed C/Pos_0 RMS values across separate invocations:
| Sample | C/Pos_0 RMS |
|---|---|
| first session, run 1 | 1.264859e+01 |
| first session, run 2 | 1.269238e+01 |
| second session, runs 1-2 | 1.268143e+01 (both) |
| third session, runs 1-5 | 1.268055e+01 (all five) |
| third session, run 6 | 1.263477e+01 |
So most runs within one session agree byte for byte and an occasional run diverges — roughly one in six in the third session. SimObject.CenterWeight_Jacobian passed in every run.
Environment.RandomSeed defaults to time(nullptr) (fhsim_environment/src/environment/util/EnvironmentSettings.cpp:76), and the default bathymetry is not flat: with no Bathymetry.Model given, ten randomly phased structures up to 3 m high are built over the stated depth. tests/in/CenterWeight/CenterWeight_in.xml sets Bathymetry.Depth = "55" and no seed, and CenterWeight takes seafloor forces — so every run integrates over a different seabed.
Confirmed directly. Without a seed, six consecutive runs gave C/Pos_0 RMS 1.264930e+01, 1.259474e+01, 1.273791e+01, 1.265492e+01, 1.267812e+01, 1.266141e+01. With one line added, RandomSeed = "1", six runs out of six gave 1.274191e+01.
That single line is now in the fixture, so the case is deterministic. It is still red, because the committed baseline was recorded under some other arbitrary draw — but it is now reproducibly red, which is the precondition this issue set for re-recording.
Filed upstream as fhsim_environment issue 0020 (a wall-clock seed default makes every model irreproducible unless the author knows to pin it) and locally as issue 0036.
Delete tests/in/CenterWeight/CenterWeight_ref.csv and CenterWeight.ref, re-run the suite once so ReferenceMode::CreateIfMissing records them against the pinned seed, inspect both before committing — they freeze whatever the model does today — and commit them together with the seed line. Confirm determinism first by running the case twenty times, not twice; the old intermittency showed on roughly one run in six, so a short check could easily have missed it.
Both were checked directly rather than reasoned about, because each would have implied a very different fix.
Not cross-test heap corruption. The suite shares one process, and several SimObjects in it write Jacobian entries at global state offsets into local-sized buffers (see the companion issue on local-versus-global state indices), which can overflow a workspace and corrupt the heap for later tests. That would have explained the intermittency. It is not the cause here: run alone under a --gtest_filter=SimObject.CenterWeight, with no other test in the process, the case still diverges on a minority of runs — nine of ten at 1.266699e+01 and one at 1.266421e+01.
Not concurrency. tests/in/CenterWeight/CenterWeight_in.xml:40 sets NumCores="1", so ModelStructure::OdeFcn takes the serial branch (fhsim/src/engine/model/ModelStructure.cpp:88-90) and the thread manager is never engaged. The trailing 0 4 n . arguments in tests/CMakeLists.txt:34 are verbosity and path settings, not a thread count.
What remains is a read of uninitialised or stale memory inside the single-threaded integration of this one model — which is the same signature as issue 0018 and is why the first possible fix below is a sanitiser run rather than a re-record.
ctest is red on a clean checkout, so conan build . fails and every subsequent piece of work has to distinguish its own breakage from this one. Worse, the run-to-run variation means the baseline cannot simply be re-recorded and trusted: a fresh recording would freeze one arbitrary draw of a wandering trajectory, and because the divergence is intermittent the case would then pass most of the time and fail occasionally – a far more expensive failure mode than the current reliable red.
The magnitude rules out floating-point noise. An RMS of 12.6 m on Pos_0 against a 1e-10 tolerance is a different trajectory, not a different rounding.
CenterWeight case would name it. Only once two runs agree byte for byte is re-recording meaningful.fhsim_marine_elements or fhsim_environment, pin the dependency versions in conanfile.py and re-record the baseline in the same commit, so the baseline and the dependency revision that produced it travel together.Doing 2 without 1 hides a live defect behind a fresh baseline, which is why this is marked blocked rather than ready.
Run the CenterWeight case at least twenty times in a loop and compare every output CSV byte for byte against the first. They currently differ on a minority of runs, so a two-run check is not sufficient to detect this and must not be used to declare the case fixed. All twenty agreeing is the precondition for re-recording the baseline.
Re-recording the baseline before the non-determinism is understood freezes a wrong trajectory and converts a hard failure into an intermittent one, which is strictly worse.
1.9.8