FhSim  3.1.0
Marine systems simulation
Loading...
Searching...
No Matches
Testing report

Dated 2026-09-24, state of the checked-out commit fde92d0. Re-verified against the current source; see Review issues — fhsim_environment for the coverage gaps this page links to.

How the library is tested

Tests are gtest, target test_fhsim_environment (tests/CMakeLists.txt), registered as the single ctest test fhsim_environment, run from ${PLAYPEN_DIR}/bin with --gtest_filter=-*_NEEDS_GUI — so every case whose name ends _NEEDS_GUI is excluded from ctest and only runs when the binary is invoked directly with a real render window available. Run it with ctest -R fhsim_environment or directly as test_fhsim_environment from the playpen bin directory.

Two styles of test coexist:

  • Direct C++ unit tests with no XML (e.g. ParametricScalarField_Test.cpp, ParametricBathymetry_Test.cpp, ScalarFields_Test.cpp, ScalarFieldBake_Test.cpp, TransitioningWaveField_Test.cpp), constructing field-provider classes directly and often using the envtest::StubSimObjectCreator test double in tests/VisualFieldTestSupport.h to feed parameters without a full simulation.
  • fhsim::test::RunTest scenarios driven by tests/in/<ID>/<ID>_in.xml, with run output under tests/out/<ID>/. Most assert result.ok plus, where present, C++-side checks against the constructed SimObject. The scenarios of TestBuoy1, TestParticleField and TestDynamicWaves (TestModels_Test.cpp, added for ENV-0026) are also compared with a stored baseline, <ID>.ref and <ID>_ref.csv, the way other FhSim libraries do.

Auto-mode Jacobian checking: wired up, never engaged

TestSpec.jacobianMode defaults to Auto; RunTest runs CheckJacobians after the simulation whenever any SimObject in the scenario reports HasJacobians()==true (fhsim/src/testtools/TestRunner.cpp:328-332). In this library that condition is never met: a case-insensitive grep for jacobian across all of src/ and include/ returns zero hits, and no class overrides HasJacobians(), OdeJacobian() or HasPortJacobians(). Every RunTest() call here therefore runs with the Auto-mode machinery present but permanently idle — there is no "implicit via Auto" coverage to distinguish from "dedicated" coverage, because neither exists. This was re-verified directly (not just carried over from the audit) by grepping the current checkout.

Ports are used in exactly one place — TestBuoy1::FinalSetup declares outports Position and Velocity via AddOutport (src/testmodels/TestBuoy1.cpp:24-25) — but no port Jacobian is declared for them, and no other class in the library declares a port at all (confirmed: AddInport/AddOutport/DeclarePort appear nowhere else). So the FhSim port-Jacobian mechanism is not exercised anywhere in this library either.

What this means in practice: the two objects with real integrated ODE state, TestBuoy1 and TestParticleField (nonlinear gravity + buoyancy + quadratic-drag OdeFcn), fall back to the integrator's finite-difference Jacobian. Their RunTest scenarios pin the resulting trajectories against a stored baseline, but no Jacobian is compared.

Coverage table

One row per SimObject registered in src/fhsim_environment.cpp (ADD_SIMOBJECT_REF, not commented out).

SimObject Input name Tests What is asserted Jacobian implementation Jacobian verified Rating Issues
Environment Environment Environment_Test.cpp (4 cases), scenario tests/in/Environment; NetCdfSeafloorBathymetry_Test.cpp (9 cases) on a SINMOD-style grid the test writes itself Smoke run; fixed default random seed; flat bathymetry unless structures requested; structures produced when asked for. Bathymetry.Model = "NetCDF": smoke run in a simulation; constant depth read at and between nodes; 11034 m off the grid; reaches the facade, SeafloorForces and a seafloor-referenced scalar field (R124) and needs a readable file; a missing file throws instead of crashing (ENV-0031, resolved); a depth and a current varying along one file axis pin xc and u-velocity to NED north (ENV-0032, resolved); an input that lists the removed NetCdfSeafloor SimObject fails to load (R129) None (HasJacobians not overridden) N/A — Auto-mode never engages (no HasJacobians anywhere in this library) B —
VisualFieldPlane VisualFieldPlane VisualFieldPlane_Test.cpp + scenarios VisualFieldPlane, VisualFieldPlaneFlow, VisualFieldPlaneBadQuantity; FieldSampling_Test.cpp (sampled quantities, Terms, refresh schedule, sketch lines) Parameter validation (spacing×resolution, axis/cell limits, degenerate axes, Quantity/Terms/Axis/FieldName combinations, Refresh/RefreshPeriod, motion), Auto refresh resolution per field, case-insensitive enums, unknown Quantity fatal (ENV-0017); the quantities against the facade's queries None N/A B —
VisualFieldVolume VisualFieldVolume VisualFieldVolume_Test.cpp + scenarios VisualFieldVolume, VisualFieldPlaneFlow As VisualFieldPlane, plus derived-grid/spacing-vs-axis-limit attribution, Elevation refused, the smaller default grid of a field sampled again None N/A B —
VisualWakeSources VisualWakeSources scenario VisualFieldPlaneFlow; FieldSampling_Test.cpp (SketchWakes, SketchLines) The sketch of every registered wake and jet and its lines None N/A C —
TestBuoy1 TestBuoy1 TestModels_Test.cpp (1 case), scenario TestBuoy1 Two buoys for 2 s, compared with the stored baseline None; 6-state ODE (Position, Velocity), nonlinear gravity+buoyancy+quadratic-drag OdeFcn (src/testmodels/TestBuoy1.cpp:63) → integrator finite-difference fallback N/A C —
TestParticleField TestParticleField TestModels_Test.cpp (1 case), scenario TestParticleField A 2×2×2 block for 2 s, compared with the stored baseline None; per-particle ODE, same fallback N/A C —
TestDynamicWaves TestDynamicWaves TestModels_Test.cpp (1 case), scenarios TestDynamicWaves and TestDynamicWaves_Control A buoy probe compared with the stored baseline, and its heave differs from a control run in which the sea state is never applied Sea-state switching runs from AcceptedStep, not OdeFcn; no ODE state to speak of N/A C —
TestNetcageWake TestNetcageWake TestModels_Test.cpp (3 cases), scenario TestNetcageWake Smoke run; constructed directly, it registers a wake on the environment that attenuates GetCurrentVelocity behind the cage and not beside it; GetParticleVelocity is attenuated the same way (ENV-0029, resolved) None N/A B —

Rating legend: A behaviour asserted and the Jacobian verified where one exists; B partly; C smoke/regression only, or indirect; D untested. No SimObject in this library rates A, because none has an analytic Jacobian for Auto mode to verify in the first place — the ceiling here is B (thorough non-Jacobian assertions).

Field-provider kernels (not registered SimObjects)

Most of this library's real testing investment is in the field-provider classes Environment builds and hands out through EnvironmentProvider, not in the SimObjects themselves. They are not registered via ADD_SIMOBJECT_REF and are not counted in the table above, but are listed here per the audit's own note that this is where the coverage actually lives.

Kernel Tests What is asserted Derivative/Jacobian coverage
ParametricScalarField (Blob/Front/DepthBand/Seafloor/Clamp/Domain/Background/JSON features) ParametricScalarField_Test.cpp (21 cases) + ScalarFieldBake_Test.cpp (19 cases) + scenarios ParametricFields, ParametricFieldsBadFeature Gaussian blob amplitude/σ falloff, antisymmetric front, depth-band, seafloor-relative masking, clamp defaults, JSON parsing/validation errors, plus extensive baking-grid geometry and masking behaviour None — GetValue(position) is the only query method on ScalarField (include/fhsim_environment/ScalarField.h:83); no gradient method exists anywhere, by design (ENV-0028, resolved).
ParametricBathymetry ParametricBathymetry_Test.cpp (16 cases) Interpolation exactness, heading-rotation invariance, layer composition, transition partition-of-unity; SurfaceNormalOfIncline and RayIntersectionStraightDown (lines 159, 176) assert a hand-computed geometric normal/ray-intersection for one specific incline case Only incidentally, via those two geometry cases — not a general analytic-vs-finite-difference comparison
DepthLayeredScalarField Cases in ScalarFields_Test.cpp (part of its 21) Exact values at layer depths, linear interpolation between them, clamping and out-of-range reporting, malformed-profile rejection None
TransitioningWaveField TransitioningWaveField_Test.cpp (19 cases) Transition partition-of-unity and thread-safety among other wave-field behaviour None
NetCageWakeField (internal to the test models, R127) NetCageWakeField.* (6 cases) and TestNetcageWake.* (4 cases) in TestModels_Test.cpp The deficit equals the factor of the pre-3.0 model at pinned points (ENV-0034), the zero source velocity, depth measured from the cage position, SetPosition, the wake axis following the current at the cage, no wake in still water, and the current untouched 100 m to the side, a clamped current usable (R122) None

Jacobian implementation notes

None. No SimObject and no field-provider kernel in this library implements an analytic Jacobian or a gradient/derivative method of any kind (verified by grep, not carried over from the audit unchanged). ParametricBathymetry::ComputeSurfaceUnitNormal computes a geometric surface normal internally, but it is not a general derivative API and is not wired to the FhSim Jacobian mechanism.

Prioritised improvement list

None open.