[question] How to handle install of 3rd party python packages with conan?

[derived-coder]

What is the recommend approach of handling python modules which are needed by some C++ libraries for code generation?
E.g. I have a C++ library A which need some specific python module installed. How should the conan recipe for A should look like?

• I've read the CONTRIBUTING guide.

[memsharded]

E.g. I have a C++ library A which need some specific python module installed.

What is the "need" relation? C++ libraries cannot link with a python module directly. Is the C++ library linking with cpython and embeds the python interpreter, then using that to run the python module? Do you have a cpython Conan package?

[derived-coder]

It does not link against python modules. The library A uses python to generate C++ code via this python module. Does it clear up the use case?

[derived-coder]

@memsharded
ping 😇

[memsharded]

It does not link against python modules. The library A uses python to generate C++ code via this python module. Does it clear up the use case?

Not very clear yet. You mean the package recipe A uses python to generate C++ code? Then the library A does not need any python thing at all, just C++.

In that case, I think the simplest and probably more robust way is to install the 3rd party python packages as a prerequisite, together with Conan. Possibly create a requirements.txt and do pip install -r=requirements.txt with all the Python packages that are needed.

The problem is that Conan, at runtime is running in 1 python interpreter, in 1 environment. It is not possible to install simultaneously different python packages versions, as it is possible with Conan. So, if it is a python pip package, I suggest managing it as a python pip package at the same level as Conan.

[derived-coder]

@memsharded
Library A needs python in order to generate the C++ code. So no, the library needs this python modules as build require steps.
I see your point.

Another use case isa python library as itself. So no C++ code at all. And I see the problem, when you add multiple transitive layers. E.g. creating a conan python package P, which needs a certain python modules, and then consuming this library P in another python library B, which itself can be consumed. How do you handle then the requirements.txt ?

[memsharded]

The last efforts to implement this are in #11601.

I am closing this ticket, please follow up the ticket above for future progress on this. Thanks for the feedback.