FhSim  3.1.0
Marine systems simulation
Loading...
Searching...
No Matches
0399 — NetStructureWithConstraints_Regression and _RegressionAppendices exceed their 30 s wall-time budget on CI (Linux, no visualisation)
ID 0399
Class TEST
Severity 2
Status ready

Models: NetStructureWithConstraints

Found: 2026-10-18, tag pipeline of v3.0.8 (feature seabed-contact, publish phase); deferred by the owner the same day

Evidence

tests/NetStructureWithConstraints_Regression_Test.cpp:18-22 and :39-43: both tests call RunAndCompareRegressionXmlTiming(..., maxRunTime = 30.0, ...). The fixture runs at fixed 1 ms steps (2 s, and 1 s for the appendices), so the step count is the same on every machine; only the wall time per step can differ.

In the v3.0.8 tag pipeline (fhsim_marine_elements pipeline 1863388) both tests fail with "Simulation wall-time budget exceeded" in all four Linux jobs that run the tests (gcc, Debian stable and old stable, Release and Debug, with_visualization=False). The visualisation jobs skip the tests and passed.

Test CI, job 3713770 (Linux Release, Debian stable) Local (32 cores, WSL2)
NetStructureWithConstraints_Regression > 30 s (budget) 0.45–5.4 s over five runs
NetStructureWithConstraints_RegressionAppendices > 30 s (budget) 0.13–1.4 s
NetStructureWithConstraints_RegressionThreads 14.5 s (passes) 28.6–36.9 s
NetStructureWithConstraints_FreePanelMatchesNetStructure 6974 ms under 50 ms (ratio about 290)
NetStructureWithConstraints_FreePanelMatchesNetStructureTwineCrossFlowFixed 5690 ms under 50 ms (ratio about 271)
CableRM_EndReactionsBalanceTheWeightInWater 23.8 s (passes) 7.5–8.2 s

The two FreePanel tests are serial runs of the same class; the local figures are from an r1 run.

Skipped on CI from 3.0.9: the four tests (_Regression, _RegressionAppendices, _FreePanelMatchesNetStructure, _FreePanelMatchesNetStructureTwineCrossFlowFixed) are skipped when CI=true, with a message naming this issue (SKIP_ON_CI_MARE0399 in tests/TestUtils.h). They still run locally. Resolving this issue must remove the skips.

So the serial runs are more than 6 to 290 times slower on CI, but the two-thread variant of the same net is faster there than locally. It is not a slow runner, and not CPU contention between pool threads.

Dependencies of the CI job: fhsim 3.3.6, fhsim_coribo 3.2.4, marenv 2.1.8, fhsim_environment 3.2.9 (all @sintef/stable). The local runs used fhsim_coribo 3.2.2 and the feature channel of marenv/env (the same source apart from tests and CI). A local run against fhsim_coribo 3.2.4 passes the whole suite (119 s), so the coribo patch is not the cause on its own.

The local wall time also varies by a factor of 10 from run to run, with the same binary and fixed steps.

Effect

The ME tag pipeline cannot be green on Linux, so a release has no Linux binaries without visualisation. The regression results themselves are not in question: the tests stop on the budget before comparing them.

Possible fix

Not known yet. Leads, in order:

  1. Measure the time per step on the CI image (Debian gcc 12/14, the CI profile): run the fixture outside ctest with a profiler or perf stat. A per-step slowdown of this size with fixed steps points to denormal (subnormal) numbers, which can cost about 100 times a normal floating-point operation, or to an exception or allocation path that only that build takes.
  2. If subnormals: find the state that decays towards zero (the free nodes far from the current, a damping term), and flush to zero or guard the term; or set FTZ/DAZ in the test runner.
  3. Compare with the CI timing of these tests before the seabed-contact feature (ME 3.0.1 tag pipeline, the whole suite took 42 s with fhsim_coribo 3.2.2).

Raising the budget hides the problem and is not a fix.

Test that would prove it

Both tests pass in the four Linux test jobs of the ME tag pipeline, with their 30 s budget, and the time per step on CI is within a few times the local one.

Risk

Low for the model results (the baselines are compared once the run finishes). If the cause is in the model (subnormals in a net state), real simulations on the same build are slow too.