R
Railway•17mo ago
x4zx

Is it possible to specify PYTHONPATH in a project?

The thing is that when I run the application on my pc, I get the following error:
Traceback (last last last call):
File "F:\Projects\team works\keepa\keepa\venv\Lib\site-packages\disnake\ext\commands\common_bot_base.py", line 452, in _load_from_module_spec
spec.loader.exec_module(lib) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "F:\Projects\team works\keepa\src\discord\omnibytes\cogs\edit.py", line 12, in <module>
from utils.product import ProductData
File "F:\Projects\team works\keepa\src\discord\omnibytes\utils\product.py", line 10, in <module>
from src.monitors.product import Product
ModuleNotFoundError: No module named 'src'

The above exception was the immediate cause of the following exception:

Traceback (most recent call last):
File "F:\Projects\team works\keepa\src\discord\omnibytes\main.py", line 32, in <module>
bot.load_extensions('cogs')
File "F:\Projects\team works\keepa\src\discord\omnibytes\main.py", line 22, in load_extensions
self.load_extension(extension_path)
File "F:\Projects\team works\keepa\venv\Lib\site-packages\disnake\ext\commands\common_bot_base.py", line 523, in load_extension
self._load_from_module_spec(spec, name)
File "F:\Projects\team works\keepa\venv\Lib\site-packages\disnake\ext\commands\common_bot_base.py", line 455, in _load_from_module_spec
raise errors.ExtensionFailed(key, e) from e
disnake.ext.commands.errors.ExtensionFailed: The 'cogs.edit' extension raised the following error: ModuleNotFoundError: No module named 'src'
Traceback (last last last call):
File "F:\Projects\team works\keepa\keepa\venv\Lib\site-packages\disnake\ext\commands\common_bot_base.py", line 452, in _load_from_module_spec
spec.loader.exec_module(lib) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "F:\Projects\team works\keepa\src\discord\omnibytes\cogs\edit.py", line 12, in <module>
from utils.product import ProductData
File "F:\Projects\team works\keepa\src\discord\omnibytes\utils\product.py", line 10, in <module>
from src.monitors.product import Product
ModuleNotFoundError: No module named 'src'

The above exception was the immediate cause of the following exception:

Traceback (most recent call last):
File "F:\Projects\team works\keepa\src\discord\omnibytes\main.py", line 32, in <module>
bot.load_extensions('cogs')
File "F:\Projects\team works\keepa\src\discord\omnibytes\main.py", line 22, in load_extensions
self.load_extension(extension_path)
File "F:\Projects\team works\keepa\venv\Lib\site-packages\disnake\ext\commands\common_bot_base.py", line 523, in load_extension
self._load_from_module_spec(spec, name)
File "F:\Projects\team works\keepa\venv\Lib\site-packages\disnake\ext\commands\common_bot_base.py", line 455, in _load_from_module_spec
raise errors.ExtensionFailed(key, e) from e
disnake.ext.commands.errors.ExtensionFailed: The 'cogs.edit' extension raised the following error: ModuleNotFoundError: No module named 'src'
But when I type set PYTHONPATH=%PYTHONPATH%;F:\Projects\team works\keepa in the console, the error disappears.
47 Replies
Percy
Percy•17mo ago
Project ID: d1f48875-a047-4a4f-bc7a-d19d9446c8f5
x4zx
x4zxOP•17mo ago
d1f48875-a047-4a4f-bc7a-d19d9446c8f5
Brody
Brody•17mo ago
there are other ways you can do imports without modifying python path from your systems environment variables, please look into some methods
x4zx
x4zxOP•17mo ago
I don't really know what to try anymore. I get this file structure: keepa/ |----src/ |----discord/ |----omnibytes/ |----main.py |----cogs/ |----utils/ |----product.py |----monitors/ |----product.py |----amazon/ |----amazon_product.py |----amazon_resell_viability.py |----request_headers/ |----request_headers.py The function and class calls come from the product.py file and I've tried using relative file paths via ....monitors and so on. But no luck.
Brody
Brody•17mo ago
interesting, okay I'll see what I can do and get back to you
patod01
patod01•17mo ago
idk if it helps, but just in case. The way I work around this is changing internally the working folder with os.chdir(PATH), but I prefer doing this at bash level from a docker image and then I start my python app
Brody
Brody•17mo ago
sorry i dont know enough about python to come to a solution, im sure you arent the first to have this issue so id recommend maybe checking stack overflow?
patod01
patod01•16mo ago
I just saw a problem. @x4zx is trying to call src folder as a module, but since it's a folder, you must call it as a package. Did it ever work once?
x4zx
x4zxOP•16mo ago
No, but it does have a __init__.py file in each folder
patod01
patod01•16mo ago
could it be because of the disk F from where you are trying to execute your app? because you say that it gets fixed when you add the env variable, but it shouldn't be necessary at all
x4zx
x4zxOP•16mo ago
I think that's unlikely.
x4zx
x4zxOP•16mo ago
Because even on the hosting site this error occurs
patod01
patod01•16mo ago
and does it not work setting the env var like you do in windows? because, attending to the title's question, yes, you can specify that var for a service, and, for project level, I remember that was added recently that feature, which enables you to share environment variables through services
x4zx
x4zxOP•16mo ago
I just don't know how to do it or if it will help
Brody
Brody•16mo ago
setting that variable would be a hacky and non portable solution, please look into how you would correctly approach this issue
patod01
patod01•16mo ago
I think it's ok to try, it could be insightful and would lead to catch the real problem an then solve this easily and you can make it portable if you make a template of it xD but I understand your point that this is far from being the solution and must be still be resolved with that in mind...
patod01
patod01•16mo ago
go to the service in the dashboard and in the variables tab, select "New Variable" (button in the right side of the pic)
patod01
patod01•16mo ago
here you write the name "PYTHONPATH". For the value, it doesn't use variables like %VAR% or $VAR, so you must type by yourself the full path for every path needed
patod01
patod01•16mo ago
do you have a public repo of this? so I can try by myself to replicate the error. If it is private, we can still try to replicate the folder hierarchy and replicate the error trying to import dumb functions (this would be a better option, because it's not inherent to your app, but to the python interpreter)
x4zx
x4zxOP•16mo ago
The repository is private, but the hierarchy of files that are used is below: keepa/ |----src/ |----discord/ |----omnibytes/ |----main.py |----cogs/ |----utils/ |----product.py |----monitors/ |----product.py |----amazon/ |----amazon_product.py |----amazon_resell_viability.py |----request_headers/ |----request_headers.py
patod01
patod01•16mo ago
need to confirm. Is main.py the main file from where your app starts?
x4zx
x4zxOP•16mo ago
yep
patod01
patod01•16mo ago
ok, I'll fill the rest of the py files with dumb functions and call them from main.py does it simulate your import flows?
x4zx
x4zxOP•16mo ago
I think so.
patod01
patod01•16mo ago
does utils/product.py import something from monitors/product.py or vice versa? Or is everything imported by main.py?
x4zx
x4zxOP•16mo ago
The main problem arises from utils/product.py. A list of what it imports:
patod01
patod01•16mo ago
patod01
patod01•16mo ago
can you copy and paste the code of this pic in the chat, please?
x4zx
x4zxOP•16mo ago
from src.monitors.product import Product
from src.amazon.amazon_product import AmazonProduct
from src.request_headers.request_headers import RequestHeaders

from src.amazon.amazon_resell_viability import AmazonResellViability
from src.utils.httpx_api_request import httpx_api_request_with_error_handling
from src.utils.async_request_with_error_handling import async_request_with_error_handling
from src.monitors.product import Product
from src.amazon.amazon_product import AmazonProduct
from src.request_headers.request_headers import RequestHeaders

from src.amazon.amazon_resell_viability import AmazonResellViability
from src.utils.httpx_api_request import httpx_api_request_with_error_handling
from src.utils.async_request_with_error_handling import async_request_with_error_handling
Sorry, but it's past my bedtime. I'm very grateful for your help!
patod01
patod01•16mo ago
don't worry! tomorrow we can continue, but for me it looks like it's a virtual environment issue, so it will work in my pc with the little information I currently have ): And that might be the reason for needing the PYTHONPATH var anyway, I'll make the dummy functions and will go to bed too hahaha. See ya! hi! I'm back. I need your start command (:
x4zx
x4zxOP•16mo ago
Hello, sorry for the delay. Here is my run command: python src/discord/omnibytes/main.py - this is if run from the root directory
patod01
patod01•16mo ago
got it! give me a couple of minutes ok. Check this, @x4zx
patod01
patod01•16mo ago
GitHub
GitHub - patod01/test01
Contribute to patod01/test01 development by creating an account on GitHub.
patod01
patod01•16mo ago
try adding this to your main file
x4zx
x4zxOP•16mo ago
I also want to say that I moved the bot folder to the same level as the src folder and tried to run the bot. But I got the same error
patod01
patod01•16mo ago
send the new hierarchy
x4zx
x4zxOP•16mo ago
The structure didn't change, I tried running it - it didn't work. So I put it all back
patod01
patod01•16mo ago
ok
x4zx
x4zxOP•16mo ago
It helped.
patod01
patod01•16mo ago
is it working now? o: I thought I would take more steps to be honest hahaha
x4zx
x4zxOP•16mo ago
😅
patod01
patod01•16mo ago
I don't remember exactly all the changes I made, I'm waking up hahaha but let me explain some things that I did and why it works I added two files because you were importing them, though they didn't appear in the hierarchy you sent. That's all what's different now, what does the solution do is pretty much the same as the pythonpath env var i'm adding the current path to the look up table of python at start up. sys.path is the variable which keeps all the path from where python look for modules so that var is your best friend in normal cases this shouldn't be required if you run your app from the root folder, i.e., if you have your app.py file run from root like python app.py but if it sounds like a hacky way of doing the job, it is mandatory when you work with virtual environments in python, since you are moving the actual python path for the interpreter and if you work with docker, you'll have to do some pretty similar stuff and that's it
x4zx
x4zxOP•16mo ago
Got it. Thank you so much for your patience and help 🤩
patod01
patod01•16mo ago
so, basically, this error is because in product.py, src folder was effectively out of scope
patod01
patod01•16mo ago
you're welcome!
x4zx
x4zxOP•16mo ago
I want to run a bot on the hosting right now.
patod01
patod01•16mo ago
i hope it works hahah let me know what happens in any case, I'd recommend to learn docker, because this is the only way you ensure that what's working in your pc will work in the server, because you are basically deploying your pc to the server xD good luck!
Want results from more Discord servers?
Add your server