|
FhSim
3.1.0
Marine systems simulation
|
|
|
FhSim
3.1.0
Marine systems simulation
|
| ID | 0008 |
| Class | BUG |
| Severity | 3 |
| Status | blocked |
| Models | — |
| Found | 2026-09-23, first end-to-end export of a model that uses the flag (fhsim_fishery's AuvSurvey scenario, fhsim_fmu pipeline) |
| Decision needed | Which directory is the authority for a relative path in a model attribute — the playpen's bin (what the existence test uses today, and where FhSim itself is run from), the directory of the model file given with -i, or the process working directory (what the classification and the copy use today)? The three disagree whenever the exporter is not run from bin, which is the normal case: fhsim_fmu/tools/export_fmus.py runs it in its own workspace. Second: should a file that is under the playpen's resources/ be copied into the FMU as well, rather than left in place and rewritten to ./../<path>? The documented promise is "except those located in `resources`", but only part of the playpen resources/ tree is bundled into the FMU's own resources/ (copyDefaultResources), so "leave it, it is already there" is not generally true either. |
--autocopy is documented in doc/user/using-the-api/fmu-export.md:127 as "Auto copy
files and folders referenced in the FhSim model attributes except those located in
<tt>resources</tt>. The copies go into <tt>resources/auto_copied/</tt>, and references in the FMU's own
fhsim model file are rewritten to point at the bundled files and directories".
It is implemented by FmuResourceCopier::CollectFileReferences (src/fmu/fhFMUexport/FmuResourceCopier.cpp:58-106), called from ResourceBundler.cpp:562-565 with paths.binPath — the directory the FhFmuExport executable itself lives in (ResourceBundler.cpp:48) — as its fhsimPath. For every Lib attribute other than Name, SimObject and LibName it does three things, against two different roots:
const fs::path asPath(value);
if (asPath.empty() || !fs::exists(fhsimPath / asPath)) { // line 75: relative to bin
continue;
}
const bool isFile = fs::is_regular_file(asPath); // line 79: relative to the CWD
const bool isDirectory = fs::is_directory(asPath); // line 80: relative to the CWD
const bool isInResources =
fs::equivalent(fhsimPath / fs::path("../resources") / asPath, fhsimPath / asPath); // line 82
and then copies with fs::copy(asPath, ...) (line 86, and lines 93 and 98 for a directory) — again relative to the CWD.
Two defects follow from those five lines.
The isInResources test answers true for any path that begins with ... The first operand is not canonicalised, so the leading .. of the attribute cancels the resources component the test just appended: for ../examples/input/…/x.json the first operand is bin/../resources/../examples/input/…/x.json, which names the same file as the second operand bin/../examples/input/…/x.json. fs::equivalent compares the files the two resolve to, finds one file, and returns true. A path relative to bin that steps out of bin — the spelling FhSim input files use, because FhSim is run from bin — is therefore always judged to be inside resources/ already, and is never copied.
**fs::equivalent throws whenever the first operand does not exist**, and CollectFileReferences catches nothing: the fs::filesystem_error travels out through ResourceBundler::Bundle (whose only try is around FmuPortMappingReader::Read, ResourceBundler.cpp:585-592) to main's handler at fhFMUExport.cpp:171-174, which prints it and returns EXIT_FAILURE.
All four consequences reproduced against the 3.1.0 playpens fhsim_fmu assembles, using a copy of that playpen's MassSpring.xml with one file attribute added to the mass object, DataFile="../examples/input/fhsim_base/FMU/MassSpring.xml":
From the visual playpen's bin the export succeeds and the FMU is wrong.
$ cd build/playpen-visual/playpen/bin $ ./FhFmuExport -i …/MassSpringData.xml -n …/VisAuto --fmi-version 2 --autocopy $ ls -A …/VisAuto/resources/auto_copied/ # empty $ grep -o 'DataFile="[^"]*"' …/VisAuto/resources/model.xml DataFile="./../../examples/input/fhsim_base/FMU/MassSpring.xml"
Nothing was copied, and the attribute was rewritten by the isInResources branch (line 89) to a path that leaves the archive: read from the bundled resources/ it is <fmu>/../examples/…, read from binaries/<platform>/ it is <fmu>/binaries/examples/…. The FMU contains no examples/ directory on either reading.
From any other working directory the attribute is left untouched.
$ cd /tmp/repro && …/playpen/bin/FhFmuExport -i …/MassSpringData.xml -n …/OtherCwd \
--fmi-version 2 --autocopy
$ grep -o 'DataFile="[^"]*"' …/OtherCwd/resources/model.xml
DataFile="../examples/input/fhsim_base/FMU/MassSpring.xml"
fs::exists(fhsimPath / asPath) still passes, but fs::is_regular_file(asPath) is false against the CWD, so neither branch fires and resources/auto_copied/ is created empty. This is the path the pipeline takes: fhsim_fmu/tools/export_fmus.py:170 runs the exporter with cwd=workspace, its own directory, never the playpen's bin.
From a playpen that has no resources/ directory the export aborts. The headless playpen fhsim_fmu assembles is exactly that — it has bin, examples and lib, and no resources:
$ cd build/playpen-headless/playpen/bin $ ./FhFmuExport -i …/MassSpringData.xml -n …/HeadlessAuto --fmi-version 2 --autocopy ERROR: filesystem error: cannot check file equivalence: No such file or directory [.../playpen/bin/../resources/../examples/input/fhsim_base/FMU/MassSpring.xml] [.../playpen/bin/../examples/input/fhsim_base/FMU/MassSpring.xml] $ echo $? 1
A file left in bin by an earlier run is enough to abort the export, with no .. in the attribute at all. With FileName="leftover_echogram.csv" and that file present in the visual playpen's bin — which is where an echosounder's own output lands when the model is run there — the first operand is bin/../resources/leftover_echogram.csv, which does not exist, and the same throw follows:
ERROR: filesystem error: cannot check file equivalence: No such file or directory [.../playpen/bin/../resources/leftover_echogram.csv] [.../playpen/bin/leftover_echogram.csv]
So --autocopy makes an export depend on what a previous run happened to leave behind.
The flag has no test and, until now, no user. grep -r autocopy over this repository finds it in ExportOptionsParser.cpp, FmuBuildKey.cpp, FmuResourceCopier.cpp and doc/user/using-the-api/fmu-export.md, and nowhere under tests/: tests/cmake/fmu_export.cmake never passes it. There is no test in fhsim_fmu/tests/ either, and fhsim_fmu/catalogue/ holds only its README.md, so no catalogue model exercises it. The first model to declare export_flags: ["--autocopy"] is fhsim_fishery's examples/input/FMU/AuvSurvey.xml, which is how this was found; the downstream consequence is recorded there as FISH-0044. The missing coverage is part of the finding, not background: each of the four behaviours above is one ctest case away from being caught, and none of them was.
A model whose attributes name data files cannot be exported as a working FMU with --autocopy, and the flag is the only mechanism the exporter offers for carrying such files. Either the export succeeds and ships an FMU whose model file points at data that is not in the archive — the SimObject then fails at the host's fmi2ExitInitializationMode, or, worse, proceeds silently with whatever it does when a file is missing — or the export exits 1 with a filesystem error that names two paths inside the playpen and nothing about the model attribute they came from.
For a published FMU matrix both halves are hit at once: the visual variant exports and is broken, the headless variant does not export at all, because the assembled headless playpen has no resources/ directory. And since the pipeline runs the exporter from its own workspace, even the variant that "succeeds" from bin leaves the attribute entirely unrewritten when the pipeline does it.
const fs::path resolved = fhsimPath / asPath; after the existence test and use resolved for is_regular_file, is_directory and the fs::copy calls. That alone makes the result independent of the working directory, and it is the half of the fix that needs no decision.fs::equivalent call with a comparison of fs::weakly_canonical(resolved) against fs::weakly_canonical(fhsimPath / "../resources") — lexically_relative and a check that the result does not begin with ... That cannot throw on a missing operand, it answers false for a file outside resources/ however the attribute spells it, and it does not need the playpen to have a resources/ directory at all.resources/auto_copied/ and rewrite every such attribute, dropping the isInResources branch. Simplest and most predictable — the FMU then carries what the model asks for, whatever the playpen looked like — at the cost of duplicating files that copyDefaultResources already bundled.Whichever is chosen, the fs::filesystem_error deserves a catch that names the attribute and its value: the present text gives two absolute playpen paths and leaves the reader to work out which Lib attribute produced them.
An --autocopy export in tests/cmake/fmu_export.cmake, which today has none. The fixture needs one model whose attribute names a file by a path relative to bin that steps out of bin (../resources/... and ../examples/... are both worth a case), exported twice: once with the working directory set to the playpen's bin and once from the build directory, asserting in both runs that the export succeeds, that the file appears under resources/auto_copied/ in the archive, and that the attribute in resources/model.xml names it there. A third case covers the throw: the same export with the named file present in bin and absent from resources/ must succeed rather than exit 1. The archive is already inspected this way by tests/fmu/CheckFmuArchive.cmake, so the assertions have a home.
The pipeline-side counterpart belongs in fhsim_fmu, which has no issues/ tracker of its own — which is why the coverage gap is recorded here. fhsim_fmu/tools/validate_fmus.py already extracts every FMU it validates, so the natural check there is that no path in an FMU's resources/model.xml escapes the archive.
Fixing the resolution changes what every --autocopy export produces, and the flag is off by default. Since nothing in either repository's tests uses it, and the only declared user is a model whose FMU does not work today, there is no working output to regress. The one thing to watch is a model that names a file inside the playpen's resources/ tree and relies on the current ./../<path> rewrite reaching the bundled copy: option 3 would move that file into auto_copied/ and rewrite the attribute accordingly, a different path in the archive than such a model expects today. No such model is known — the flag has no other user — but it is the one behaviour here that is not simply broken.
1.9.8