0403 — Fixtures with NumCpuCore 2 give different results run to run, so they cannot be byte-compared

ID	0403
Class	TEST
Severity	2
Status	ready

Models: TrawlCable, Mooring/MooringCable, Net/NetStructureWithConstraints

Found: 2026-10-20, feature m1-mare-ready-batch: comparing tests/out between two runs of identical code with tools/compare_results.py

Evidence

Two default ctest runs of the same build (several pairs of runs agree) give different result files for exactly these cases, and for nothing else (‘compare_results.py --exclude ’*_log.txt'`):

File	Differing values	Max abs	Where
TrawlCable/TrawlCable_res.txt	293358 of 453453	43	CableS/TopBridleOmega0_2 etc. (column RMS rel 0.16)
TrawlCable/TrawlCable_NoClumpWeight_res.txt	288955 of 440440	64	CableS/WarpQuater* (RMS rel 0.18)
TrawlCable/TrawlCable_BottomBridleArea_res.txt	6 of 91405	1e-7	CableE/WarpOmega9_2
MooringCable/MooringCable_res.txt	292416 of 849849	56	CableS/TopBridleOmega0_2 (RMS rel 0.15)
MooringCable/MooringCable_TenElements_res.txt	294145 of 589589	56	CableS/WarpOmega19_2 (RMS rel 0.15)
MooringCable/MooringCable_Slack10ppb_res.txt, _Slack2ppb_	2330 of 33649, 6255 of 33649	1e-16, 1e-9	Mooring/MidRopeOmega* (round-off scale, large relative)
MooringCable/MooringCable_Buoy, _BottomChainContact, _ContactMooringFirst, _ContactGearFirst, _Reversed	1 to 20 values	1e-8 to 1e-11	last printed digit
NetStructureWithConstraints/NetStructureWithConstraints_Teardown_res.txt	3 of 1067	1e-12	Net/orientation4_3

For TrawlCable_Smoke the first difference is the last printed digit at t = 1 ms, and it grows to 91 in CableS/BottomBridleOmega* by 1 s (a stiff, nearly chaotic rig). OMP_NUM_THREADS=1 does not remove it.

Cause, tested: every one of these fixtures sets NumCpuCore="2" (15 MooringCable_*, 3 TrawlCable_* inputs and NetStructureWithConstraints_Teardown_in.xml), which builds the CoRiBo constraint solver on a two-thread pool; the result depends on the thread scheduling (summation order). With NumCpuCore="1" in the input and nothing else changed, two runs of TrawlCable_Smoke (three runs), MooringCable_Buoy, TrawlCable_BottomBridleArea and the Teardown case are byte-identical. Valgrind memcheck of TrawlCable_Smoke (OMP_NUM_THREADS=1, --track-origins=yes) reports 0 errors, so it is not uninitialised memory. This is MARE-0075 (resolved: the owner decided that deterministic results are not required, documented on the model pages).

Effect

The result files of these cases cannot be byte-compared across a change (tools/compare_results.py, the feature's moves.md gate): every such change shows as "moved". They also cannot have a regression reference, so a real change of TrawlCable, MooringCable or the constrained net's solver path is only seen by the tolerance-based tests.

Possible fix

Keep the NumCpuCore="2" smoke fixtures (they run the multi-threaded path on purpose) and add NumCpuCore="1" twins of the cases that need a byte or regression comparison (TrawlCable, MooringCable, NetStructureWithConstraints_Teardown). The twin adds a regression reference per case with the usual output_rms_max. No source change. Making the pool's reduction deterministic would be a change in fhsim_coribo and was ruled out in MARE-0075.

Test that would prove it

Run each twin twice and require byte-identical result files (a test that runs a case twice and compares the series exactly), then add .ref and _ref.csv for them.

Risk

None for results: only new test inputs. Run time of the twins is that of the one-thread run (about 2 s each).