portable file path

[gitlzzz]

is this right without given the path filepath_files=descriptor_code_path, here

https://github.com/pol-sb/MatDBForge/blob/5dcc5be7c237ae4d1e4c5dfa6ec1c66e3b68924d/src/MatDBForge/workflows/simple_active_learning.py#L490-L493

[pol-sb]

Hey @gitlzzz,
In this case I think so. This is a ContainerizedCode, and in the build process of the image I am including all the executables in the image's path. If I loging into MN5 and load the image like this:

[user@login ~]$ module load singularity
load SINGULARITY/3.11.5 (PATH)

[user@login ~]$ singularity shell --contain --writable-tmpfs --nv /gpfs/projects/iciq72/psanz/containers/mdb.sif

WARNING: Fallback to underlay layer: /gpfs/projects/iciq72/psanz/containers/mdb.sif is located on a GPFS filesystem incompatible as overlay lower directory
WARNING: --writable-tmpfs disabled due to sandbox filesystem incompatibility with overlay
WARNING: underlay of /usr/bin/nvidia-smi required more than 50 (402) bind mounts
Singularity>

This returns a shell inside the image, from which I can run commands and check environment variables. If I check the $PATH variable:

Singularity> echo $PATH

/mdbforge/venv/bin:/mdbforge:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/mdbforge/julia-1.11.1/bin:/mdbforge/.local/bin:/mdbforge/MatDBForge/src/MatDBForge/active_learning/extrapolation/concave_hull_scripts:/mdbforge/MatDBForge/src/MatDBForge/active_learning/mace_code/combined:/mdbforge/MatDBForge/src/MatDBForge/active_learning/mace_code/descriptors:/mdbforge/MatDBForge/src/MatDBForge/active_learning/mace_code/committee:/mdbforge/MatDBForge/src/MatDBForge/active_learning/extrapolation/autoencoder_scripts:/mdbforge/MatDBForge/src/MatDBForge/active_learning/extrapolation/concave_hull_scripts:/mdbforge/MatDBForge/src/MatDBForge/active_learning/md

Also, if I write mdb_ and hit tab to autocomplete in the shell, it lists the following executables:

Singularity> mdb_
mdb_active_learning                  mdb_gen_configuration_file           mdb_init_setup                       mdb_monitor_al_loop                  mdb_train_autoencoder
mdb_autoencoder_get_latent_space.py  mdb_gen_init_db                      mdb_mace_eval_committee_configs.py   mdb_process_structure.py
mdb_check_descr_combined.py          mdb_get_concave_hull.py              mdb_mace_get_descriptors.py          mdb_run_dft_database
Singularity> mdb_

So the image can see and find the executable files. Also, I've been running the active learning loops like this for a while, and the calculation jobs work (so executables are being correctly located).

I am not sure if this is the correct way of building the image, but I feel it is convenient.
However, I need the filepath_files for the PortableCode instances because for these, I am copying the scripts from the local machine to the compute node, and I use a separate folder for every script and its auxiliary files.

Let me know what do you think!