Railway•17mo ago

Python Private package

so in my requirements.txt i have a private github package i need to install but i keep running into this error on deploy:

   fatal: could not read Username for 'https://github.com';: No such device or address
#10 8.518   fatal: clone of 'https://github.com/AugmentedLabsInc/archer-voice-bot'; into submodule path '/tmp/pip-req-build-l7oka1sr/archer-voice-bot' failed
#10 8.519   Failed to clone 'archer-voice-bot'. Retry scheduled
 
#10 8.706   fatal: could not read Username for 'https://github.com';: No such device or address
#10 8.709   fatal: clone of 'https://github.com/AugmentedLabsInc/archer-voice-bot'; into submodule path '/tmp/pip-req-build-l7oka1sr/archer-voice-bot' failed
#10 8.709   Failed to clone 'archer-voice-bot' a second time, aborting

   fatal: could not read Username for 'https://github.com';: No such device or address
#10 8.518   fatal: clone of 'https://github.com/AugmentedLabsInc/archer-voice-bot'; into submodule path '/tmp/pip-req-build-l7oka1sr/archer-voice-bot' failed
#10 8.519   Failed to clone 'archer-voice-bot'. Retry scheduled
 
#10 8.706   fatal: could not read Username for 'https://github.com';: No such device or address
#10 8.709   fatal: clone of 'https://github.com/AugmentedLabsInc/archer-voice-bot'; into submodule path '/tmp/pip-req-build-l7oka1sr/archer-voice-bot' failed
#10 8.709   Failed to clone 'archer-voice-bot' a second time, aborting

I saw that you guys don't support private packages for NPM but was wondering if this is the case for python as well

31 Replies

Percy•17mo ago

Project ID: c9469c41-0676-4e12-ab4a-0307986c6520

ricepapiOP•17mo ago

c9469c41-0676-4e12-ab4a-0307986c6520

Brody•17mo ago

how would you normally authorise a private python package down

ricepapiOP•17mo ago

git+https://username:[email protected]/AugmentedLabsInc/archer-voice.git to clarify this is inside the requirements.txt ty for the quick reply btw

Brody•17mo ago

plaintext passwords in a txt file? is that the only method you can use?

ricepapiOP•17mo ago

nope but just wanted to deploy something quickly to test

Brody•17mo ago

what's the recommended way to do it then

ricepapiOP•17mo ago

the way i usually do it is a pip install command with env vars of credentials after requirements.txt is alr installed

Brody•17mo ago

well tell me these commands and the environment variables required

ricepapiOP•17mo ago

pip install git+https://${GITHUB_USERNAME}:${GITHUB_TOKEN}@github.com/username/repository.git and it's inside a shell script:

#!/bin/bash
pip install -r requirements.txt
pip install git+https://${GITHUB_USERNAME}:${GITHUB_TOKEN}@github.com/username/repository.git

#!/bin/bash
pip install -r requirements.txt
pip install git+https://${GITHUB_USERNAME}:${GITHUB_TOKEN}@github.com/username/repository.git

which i call from docker what's the best way do this in railway?

Brody•17mo ago

okay first step, set those variables in your service variables second step, send me your railway.json file

ricepapiOP•17mo ago

ah i dont have have a railway.json file i just used the github deploy i can create one tho

Brody•17mo ago

every deployment gets a railway.json file, it would be in the details tab of your most recent deployment

ricepapiOP•17mo ago

o okay

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "NIXPACKS"
  },
  "deploy": {
    "numReplicas": 1,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "NIXPACKS"
  },
  "deploy": {
    "numReplicas": 1,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

Brody•17mo ago

do you only have one private package to install?

ricepapiOP•17mo ago

Brody•17mo ago

is this the package?

ricepapiOP•17mo ago

yes

Brody•17mo ago

have you set those as service variables?

ricepapiOP•17mo ago

yep

Brody•17mo ago

let me know how this goes

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "NIXPACKS",
    "nixpacksPlan": {
      "phases": {
        "private:install": {
          "dependsOn": ["install"],
          "cmds": ["pip install git+https://${GITHUB_USERNAME}:${GITHUB_TOKEN}@github.com/AugmentedLabsInc/archer-voice.git"]
        }
      }
    }
  },
  "deploy": {
    "numReplicas": 1,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

{
  "$schema": "https://railway.app/railway.schema.json",
  "build": {
    "builder": "NIXPACKS",
    "nixpacksPlan": {
      "phases": {
        "private:install": {
          "dependsOn": ["install"],
          "cmds": ["pip install git+https://${GITHUB_USERNAME}:${GITHUB_TOKEN}@github.com/AugmentedLabsInc/archer-voice.git"]
        }
      }
    }
  },
  "deploy": {
    "numReplicas": 1,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}

ricepapiOP•17mo ago

#10 6.164 Collecting git+https://musubipapi:****@github.com/AugmentedLabsInc/archer-voice.git (from -r requirements.txt (line 9))
#10 6.165   Cloning https://musubipapi:****@github.com/AugmentedLabsInc/archer-voice.git to /tmp/pip-req-build-j9g2xmdf
#10 6.172   Running command git clone --filter=blob:none --quiet 'https://musubipapi:****@github.com/AugmentedLabsInc/archer-voice.git' /tmp/pip-req-build-j9g2xmdf
 
#10 8.638   Resolved https://musubipapi:****@github.com/AugmentedLabsInc/archer-voice.git to commit 7d3357b171f8f967c979515e1b61b92de32e9102
#10 8.638   Running command git submodule update --init --recursive -q
 
#10 8.871   fatal: could not read Username for 'https://github.com';: No such device or address
#10 8.873   fatal: clone of 'https://github.com/AugmentedLabsInc/archer-voice-bot'; into submodule path '/tmp/pip-req-build-j9g2xmdf/archer-voice-bot' failed
#10 8.873   Failed to clone 'archer-voice-bot'. Retry scheduled
 
#10 9.059   fatal: could not read Username for 'https://github.com';: No such device or address
#10 9.062   fatal: clone of 'https://github.com/AugmentedLabsInc/archer-voice-bot'; into submodule path '/tmp/pip-req-build-j9g2xmdf/archer-voice-bot' failed
#10 9.062   Failed to clone 'archer-voice-bot' a second time, aborting
#10 9.072   error: subprocess-exited-with-error
#10 9.072
#10 9.072   × git submodule update --init --recursive -q did not run successfully.
#10 9.072   │ exit code: 1
#10 9.072   ╰─> See above for output.
#10 9.072
#10 9.072   note: This error originates from a subprocess, and is likely not a problem with pip.
#10 9.076 error: subprocess-exited-with-error
#10 9.076
#10 9.076 × git submodule update --init --recursive -q did not run successfully.
#10 9.076 │ exit code: 1
#10 9.076 ╰─> See above for output.
#10 9.076
#10 9.076 note: This error originates from a subprocess, and is likely not a problem with pip.
#10 9.082 WARNING: You are using pip version 22.0.4; however, version 23.2.1 is available.
#10 9.082 You should consider upgrading via the '/opt/venv/bin/python -m pip install --upgrade pip' command.
 
#10 ERROR: process "/bin/bash -ol pipefail -c python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt" did not complete successfully: exit code: 1
 
-----
> [stage-0  6/10] RUN --mount=type=cache,id=s/2e91e336-4412-4273-af7d-1ee57eb31be9-/root/cache/pip,target=/root/.cache/pip python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt:
#10 9.072   note: This error originates from a subprocess, and is likely not a problem with pip.
#10 9.076 error: subprocess-exited-with-error
#10 9.076
#10 9.076 × git submodule update --init --recursive -q did not run successfully.
#10 9.076 │ exit code: 1
#10 9.076 ╰─> See above for output.
#10 9.076
#10 9.076 note: This error originates from a subprocess, and is likely not a problem with pip.
#10 9.082 WARNING: You are using pip version 22.0.4; however, version 23.2.1 is available.
#10 9.082 You should consider upgrading via the '/opt/venv/bin/python -m pip install --upgrade pip' command.
-----
 
Dockerfile:20
-------------------
18 |     ENV NIXPACKS_PATH /opt/venv/bin:$NIXPACKS_PATH
19 |     COPY . /app/.
20 | >>> RUN --mount=type=cache,id=s/2e91e336-4412-4273-af7d-1ee57eb31be9-/root/cache/pip,target=/root/.cache/pip python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt
21 |
22 |     # private:install phase
-------------------
ERROR: failed to solve: process "/bin/bash -ol pipefail -c python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt" did not complete successfully: exit code: 1
 
Error: Docker build failed

#10 6.164 Collecting git+https://musubipapi:****@github.com/AugmentedLabsInc/archer-voice.git (from -r requirements.txt (line 9))
#10 6.165   Cloning https://musubipapi:****@github.com/AugmentedLabsInc/archer-voice.git to /tmp/pip-req-build-j9g2xmdf
#10 6.172   Running command git clone --filter=blob:none --quiet 'https://musubipapi:****@github.com/AugmentedLabsInc/archer-voice.git' /tmp/pip-req-build-j9g2xmdf
 
#10 8.638   Resolved https://musubipapi:****@github.com/AugmentedLabsInc/archer-voice.git to commit 7d3357b171f8f967c979515e1b61b92de32e9102
#10 8.638   Running command git submodule update --init --recursive -q
 
#10 8.871   fatal: could not read Username for 'https://github.com';: No such device or address
#10 8.873   fatal: clone of 'https://github.com/AugmentedLabsInc/archer-voice-bot'; into submodule path '/tmp/pip-req-build-j9g2xmdf/archer-voice-bot' failed
#10 8.873   Failed to clone 'archer-voice-bot'. Retry scheduled
 
#10 9.059   fatal: could not read Username for 'https://github.com';: No such device or address
#10 9.062   fatal: clone of 'https://github.com/AugmentedLabsInc/archer-voice-bot'; into submodule path '/tmp/pip-req-build-j9g2xmdf/archer-voice-bot' failed
#10 9.062   Failed to clone 'archer-voice-bot' a second time, aborting
#10 9.072   error: subprocess-exited-with-error
#10 9.072
#10 9.072   × git submodule update --init --recursive -q did not run successfully.
#10 9.072   │ exit code: 1
#10 9.072   ╰─> See above for output.
#10 9.072
#10 9.072   note: This error originates from a subprocess, and is likely not a problem with pip.
#10 9.076 error: subprocess-exited-with-error
#10 9.076
#10 9.076 × git submodule update --init --recursive -q did not run successfully.
#10 9.076 │ exit code: 1
#10 9.076 ╰─> See above for output.
#10 9.076
#10 9.076 note: This error originates from a subprocess, and is likely not a problem with pip.
#10 9.082 WARNING: You are using pip version 22.0.4; however, version 23.2.1 is available.
#10 9.082 You should consider upgrading via the '/opt/venv/bin/python -m pip install --upgrade pip' command.
 
#10 ERROR: process "/bin/bash -ol pipefail -c python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt" did not complete successfully: exit code: 1
 
-----
> [stage-0  6/10] RUN --mount=type=cache,id=s/2e91e336-4412-4273-af7d-1ee57eb31be9-/root/cache/pip,target=/root/.cache/pip python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt:
#10 9.072   note: This error originates from a subprocess, and is likely not a problem with pip.
#10 9.076 error: subprocess-exited-with-error
#10 9.076
#10 9.076 × git submodule update --init --recursive -q did not run successfully.
#10 9.076 │ exit code: 1
#10 9.076 ╰─> See above for output.
#10 9.076
#10 9.076 note: This error originates from a subprocess, and is likely not a problem with pip.
#10 9.082 WARNING: You are using pip version 22.0.4; however, version 23.2.1 is available.
#10 9.082 You should consider upgrading via the '/opt/venv/bin/python -m pip install --upgrade pip' command.
-----
 
Dockerfile:20
-------------------
18 |     ENV NIXPACKS_PATH /opt/venv/bin:$NIXPACKS_PATH
19 |     COPY . /app/.
20 | >>> RUN --mount=type=cache,id=s/2e91e336-4412-4273-af7d-1ee57eb31be9-/root/cache/pip,target=/root/.cache/pip python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt
21 |
22 |     # private:install phase
-------------------
ERROR: failed to solve: process "/bin/bash -ol pipefail -c python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt" did not complete successfully: exit code: 1
 
Error: Docker build failed

unfortunately still same issue -- i'll do some debugging myself, wasn't too familiar with railway.json ah wait i know the issue

Brody•17mo ago

do tell

ricepapiOP•17mo ago

i didn't remove the old git+.... line from requirements.txt -- i suspect thats it thanks for the help!

Brody•17mo ago

does it work?

ricepapiOP•17mo ago

yessir

Brody•17mo ago

my railway.json worked as is?

ricepapiOP•17mo ago

yep

Brody•17mo ago

thats awsome

ricepapiOP•17mo ago

i have some issues on my end but it deployed completely

Brody•17mo ago

sweet

Gaming

Programming

Python Private package