Repository navigation
Can not install custom python dependency from custom git url #555
Description
Activity
Hi, thanks for raising this issue. This is a result of how the package resolution logic works in Lambda Builders. Essentially, if the pip workflow finds a package that is not built already (the langchain fork that is provided), it'll try to resolve some basic info from that package by looking at the
setup.pyfile. Since this file is non-existent, it fails like you see here.While we look for a proper way forward, would it be possible to build this package locally and reference it inside of the
requirements.txtfile as a local dependency withfile://instead of pointing to the git repository? This could be done as a step prior to runningsam build, or as part of a Makefile build workflow.Hello, thank you for the workaround. Indeed compiling my version of langchain into a wheel and passing it as a local file pointer worked.
I am sure that you're aware of the fact that
setup.pyis deprecated since years and nowadays most projects usepyproject.tomlas package configuration structure. So looking for the existence ofsetup.pymight be error prone.Also, in langchain package the
pyproject.tomlis not placed in the root of the repo but instead in a subfolder pointed by the#subdirectoryURL tag as specified in the pip documentation. I do not know if this inferred with the bug somehow.Reacted by Pavitra GolchhaThanks for confirming that building the project and then using the wheel file worked. As for moving away from setup.py, this would have to be something that would likely be taken in as a feature request since this is something that we don't currently check/handle.
Using the url fragments as you've done in the requirements file works fine however (though granted the project would probably have to have a
setup.pyfile as discovered)- added a commit that references this issue
on Jan 2, 2024 I would also like to see
sam buildrecognizingpyproject.tomlfiles. We tried publishing wheels through build artifacts, however can't download them because the code is in a private repo and they cannot be accessed through SSH. We had to resort to hand-rolling PKG-INFO which is another thing to keep up to date and duplicates the info thatpyproject.tomlalready provides.Reacted by Colin Murtaugh
Description:
I am trying to install a python dependency from a custom git repo, with a specific commit and a subdirectory specified.
Steps to reproduce:
I have the following line in my requirements.txt:
langchain @ git+https://github.com/mrtj/langchain.git@7b52eff#subdirectory=libs/langchainObserved result:
sam buildfails with the errorPythonPipBuilder:ResolveDependencies - Unable to retrieve name/version for packageExpected result:
sam buildworks.Note:
pip install -r requirements.txtdo correctly install the required version of the package.Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
{ "version": "1.98.0", "system": { "python": "3.8.13", "os": "macOS-12.5-x86_64-i386-64bit" }, "additional_dependencies": { "docker_engine": "20.10.23", "aws_cdk": "Not available", "terraform": "1.3.7" }, "available_beta_feature_env_vars": [ "SAM_CLI_BETA_FEATURES", "SAM_CLI_BETA_BUILD_PERFORMANCE", "SAM_CLI_BETA_TERRAFORM_SUPPORT", "SAM_CLI_BETA_RUST_CARGO_LAMBDA" ] }Add --debug flag to command you are runningDebug output: