0024 — The Jacobian checker's model rebuilds cannot suppress SimObject file output, so a file a SimObject writes during the run is clobbered or duplicated

ID	0024
Class	TEST
Severity	1
Status	ready
Models	—
Found	2026-09-25, filed from fhsim_fishery FISH-0050 (its option 3) on the owner's decision
Decision needed	— (Karl-Johan Reite decided 2026-09-25 to file this as ready)

Evidence

fhsim::test::RunTest builds the scenario again after the run whenever TestSpec::jacobianMode is not No: ProbeJacobianAvailability builds it once (src/testtools/TestRunner.cpp:53), and CheckJacobians builds it again for the initial and trajectory samples (src/testtools/JacobianChecker.cpp:658, :767, both through Scenario::Build, :585-591) and once more for the offset-shifted copy (:729). Every one of those builds runs each SimObject's setup in the test's working directory, and the trajectory samples also integrate the scenario. Nothing in JacobianCheckConfig (include/fhsim/testtools/JacobianChecker.h:45) or in the model build lets the checker tell a SimObject that this build is a checker build whose file output is not wanted.

Observed in fhsim_fishery (FISH-0050): SensorEchosounder opens its echogram with std::ios::out in FinalSetup, so every rebuild truncates the file. With the echogram tests' jacobianMode switched from No to Auto, the test that parses the echogram after RunTest returns reads 23 lines (header and the t = 0 ping) instead of 59 and fails with stod. The Echo-only fixtures have no analytic Jacobian, so the availability probe build alone is enough to truncate the file.

Effect

Any SimObject that writes a file during setup or integration has its output clobbered (truncating writers) or duplicated (appending writers) by the checker's rebuilds. A test that reads such a file after RunTest has to keep jacobianMode = No, so its scenario is never put through the Jacobian checker. fhsim_fishery's echosounder scenarios do so today; nothing is lost there yet because no object in them besides the exempted Auv declares an analytic Jacobian, but a future one would go unchecked.

Possible fix

Give the checker's builds (the availability probe, the sample builds and the shifted copy) a way to switch SimObject file output off, for example a model-build flag that SimObjects can query (a "no file output" property on the creator or model), set by ProbeJacobianAvailability and Scenario::Build. Alternative: run each checker build in a scratch working directory (ScopedWorkingDirectory exists in src/testtools), which needs no SimObject changes but only covers relative output paths.

Test that would prove it

A test SimObject in src/testtools/simobjects that writes a line per step to a relative file: run it through RunTest with jacobianMode = Yes and assert that the file afterwards holds exactly the run's lines. Downstream: switch fhsim_fishery's RunEchoCase (tests/AuvEchosounder_Test.cpp) to Auto; SimObject.Auv_EchoGram must pass.

Risk

A build flag is a SimObject-facing API addition; SimObjects that ignore it keep today's behaviour. The scratch-directory variant changes where relative input files resolve during checker builds unless the input paths are made absolute first.