[macOS] Enable QML / image-IO plugins to load on Apple Silicon

[NeverGET]

Description

Enables the QtAliceVision QML and image-IO plugins to load on macOS, so Meshroom's 3D viewer (SfM points, camera frustums, depth maps) and EXR image display work on Apple Silicon with the Metal RHI backend. Companion to the Meshroom macOS viewer PR (alicevision/Meshroom#3030) and built on AliceVision's macOS support (alicevision/AliceVision#2019).

Changes

• Embed link-path rpaths in installed plugins (CMAKE_INSTALL_RPATH_USE_LINK_PATH on Apple). The plugins link AliceVision frameworks and their dependencies (e.g. libceres) via @rpath but previously installed no rpath, so dyld fell back to system/Homebrew copies — notably a libceres lacking the glog symbols statically bundled into AliceVision's — and the plugins failed to load with "Symbol not found". Baking the link-time library directories into each plugin's rpath resolves the bundled copies. No-op where rpath isn't used.
• Placeholder normals for the SfM point cloud and camera entities. Their per-vertex-color material declares a vertexNormal input; on Metal RHI a geometry that omits it fails pipeline-state creation and crashes the renderer (e.g. on viewer resize). A constant placeholder satisfies the descriptor; it is inert because the material is unlit (vertex-color driven), so rendering is unchanged everywhere.
• Request Qt6GuiPrivate explicitly. Some Qt distributions (notably aqtinstall framework builds on macOS) don't auto-create the private-module targets via find_package(Qt6Gui), so Qt6::GuiPrivate is missing and configuration fails. Harmless where already provided.

Notes

@philippremy — on alicevision/Meshroom#3030 you offered to submit the rpath fix ("a very simple PR"); this implements it via CMAKE_INSTALL_RPATH_USE_LINK_PATH. Full credit to you for identifying that the right fix is build-time rpaths, not runtime env vars — happy to defer to your version or adjust.

Validated on Apple Silicon: rebuilt and confirmed the AliceVision, SfmDataEntity, and DepthMapEntity modules load with no install_name_tool patching and no DYLD_* overrides. One observation: with the current AliceVision install the baked rpaths also include /opt/homebrew/lib (AliceVision itself links some Homebrew transitive deps such as metis); it is ordered after the AliceVision lib dir so it is harmless, and would not appear with a fully self-contained AliceVision build.

[gemini-code-assist]

Code Review

This pull request introduces macOS-specific fixes for CMake configurations and Qt3D rendering. It explicitly requests Qt6GuiPrivate and configures RPATH usage on macOS to prevent plugin loading failures. Additionally, it adds placeholder normals buffers to CameraLocatorEntity and PointCloudEntity to prevent renderer crashes on the macOS Metal RHI backend. The review feedback highlights memory leaks in both entities due to QBuffer and QAttribute allocations lacking a parent QObject, and suggests parenting them to customGeometry to ensure proper cleanup.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The QBuffer and QAttribute instances are allocated on the heap using new but are not given a parent QObject (or QNode). In Qt3D, adding an attribute to a geometry does not transfer ownership, which leads to a memory leak when the entity or geometry is destroyed. Parenting them to customGeometry ensures they are properly cleaned up.

Suggested change

	auto normalDataBuffer = new QBuffer;
	normalDataBuffer->setData(normalData);
	auto normalAttribute = new QAttribute;
	auto normalDataBuffer = new QBuffer(customGeometry);
	normalDataBuffer->setData(normalData);
	auto normalAttribute = new QAttribute(customGeometry);

[gemini-code-assist]

The QBuffer and QAttribute instances are allocated on the heap without a parent QObject. Since PointCloudEntity::setData can be called multiple times to update the point cloud, this will leak memory on every update. Parenting them to customGeometry ensures they are automatically deleted when the geometry is destroyed or replaced.

    auto normalDataBuffer = new QBuffer(customGeometry);
    QByteArray normalData(reinterpret_cast<const char*>(normals.data()), npoints * 3 * static_cast<int>(sizeof(float)));
    normalDataBuffer->setData(normalData);
    auto normalAttribute = new QAttribute(customGeometry);

[philippremy]

Hi,

thanks for the PR. Because I am generally more into the build system stuff, I'll only comment on that for now ^^. While setting CMAKE_INSTALL_RPATH_USE_LINK_PATH might solve the issue for a local build/install, it essentially causes the resulting binaries to be non-redistributable. CMake will embed absolute and system-dependant paths for LC_RPATH in the Mach-O, which could (imho) break stuff easily. Relocate some of the dependent libaries and your libraries won't load anymore.

Take a look at this file in the Meshroom repo (at least I usually use MESHROOM_INSTALL_DIR to make my life easier). It pretty much explains how it expects the file structure to be for AliceVision and QtAliceVision. This obviously only applies to Unix systems (so most prominently Linux and macOS), because Windows basically expects the dynamic libraries to be just next to the executable, but that is another story (and very simplified). Starting from there you can calulate what the rpaths for the libraries should be. Linux has $ORIGIN and macOS provides @executable_path and @loader_path to allow referencing libraries in a relocatable fashion.

So my approach/recommendation would be to do the following:

1. Read a little bit about $ORIGIN, @executable_path and @loader_path to understand what they mean in relation to @rpath.
2. Build this project, look at the resulting structure after creating something similar to what Meshroom expects.
3. Calculate how the resulting libraries need to reference the AliceVision libraries (my guess(!) would be that we should prefer @loader_path and that it will probably be something like @loader_path/../../aliceVision/lib or @loader_path/../../../aliceVision/lib for macOS).
4. Check if this causes problems with the Qt libraries/frameworks, which QtAliceVision also links to (again, my guess would be that they'll be already loaded because PySide loads before QtAliceVision).
5. Once you figured this out, I'll recommend following the way we go for in AliceVision (the rpaths will obviously be different).

Sorry for doing this nit-picking here, but I think we should make the whole thing somewhat more robust if we are doing this overhaul...

[philippremy]

Otherwise I agree with the AI that we are probably leaking memory big time, I just found the Qt documentation on ownership as a reference, so maybe look into that :D.

[NeverGET]

Hi @philippremy — thanks a lot for the detailed guidance; the references were exactly what I needed.

Rpaths (redistributability): you were right that CMAKE_INSTALL_RPATH_USE_LINK_PATH bakes absolute, system-dependent paths — it was embedding /opt/homebrew/lib and the build-tree AliceVision path. I replaced it with relocatable @loader_path rpaths, following AliceVision's own pattern (src/CMakeLists.txt) and the standalone layout from Meshroom's setupEnvironment:

if(APPLE)
    set(CMAKE_MACOSX_RPATH 1)
    set(CMAKE_INSTALL_RPATH
        "@loader_path/../../../aliceVision/lib"   # qml/<Module>/  -> <root>/aliceVision/lib
        "@loader_path/../../aliceVision/lib"      # imageformats/  -> <root>/aliceVision/lib
        "@loader_path"
        "@loader_path/..")
endif()

Your guess was spot-on — qtPlugins/qml/<Module>/ needs ../../../aliceVision/lib and qtPlugins/imageformats/ needs ../../aliceVision/lib. To verify it's genuinely redistributable I built a standalone Meshroom.app in that layout and checked: the installed plugins carry only @loader_path rpaths (no absolute/Homebrew paths), all modules (AliceVision/SfmDataEntity/DepthMapEntity) load with no DYLD_* overrides, and they still load after moving the whole .app to a different location. Qt resolves via the embedded PySide6 (loaded first), as you expected.

Memory leak: fixed — the placeholder normal QBuffer/QAttribute are now parented to the geometry (new QBuffer(customGeometry) / new QAttribute(customGeometry)), so Qt owns and frees them per the object-tree ownership model. Thanks for the pointer.

One thing I noticed while in there: the existing position/color buffers in these two entities use the same unparented new pattern, so they share the ownership gap. I left them as-is to keep this PR in scope — happy to address them in a follow-up (or here) if you'd prefer.

Thanks again for the careful review!