Bundled Python penv usage on MacOS and Windows

Hi,

I have a question on the PIO bundled python venv usage.

My environments are PIO/VSCode on Windows 10 and MacOS respectively.

  • On Windows 10, when I open the terminal panel for a PIO project in the VSCode, I notice the session is in the .platformio/penv venv as below.
    Screen Shot 2021-09-13 at 5.12.12 PM
  • On MacOS, it’s not in the bundled venv, but the MacOS system wide python run time.

And my question is, how can I make the MacOS environment pick the bundled venv as the python run time for the terminal?

Regards,
Yoonseok

Hi,
I’m closing this inquiry by what I think is the answer.

Not sure 100% if it’s the PIO’s design, but Windows installation set the .plaformio/penv/Scritps at the early part of %PATH%, so the bundled python was picked up on invocation, while my MacOS set .platformio/penv/bin at a later part of $PATH so the system wide python was picked up.

So it’s dependent on the %PATH% or $PATH environment variable. And as a convenient guard, I ended up using this steps before running the python code.

#!/bin/sh
which python | grep \.platformio\/penv > /dev/null
if [ $? -ne 0 ] ; then
    source ~/.platformio/penv/bin/activate
fi  
python ~/.platformio/packages/tool-espException/decodeException.py

Regards,
Yoonseok

This might be a bug, please open an issue at Issues · platformio/platformio-vscode-ide · GitHub

Thank you for your suggestion.

I’m not 100% sure if it is a bug or the like. But it could be justified as an enhancement at least.
Let’s open an issue for an enhancement.

@maxgerhardt

After playing around for some more time, I come to 100% that you’re correct and that it’s a bug. I noticed whenever a PIO terminal gets open, there is some algorithm running already which sets the PATH variable.

And that logic should put the ~/.platformio/python3 ahead of any system wide python directory in the PATH.

Thank you again, and I’ll update the issue.

Hi @yoonseok, @maxgerhardt !

I have the seem question but I lack knowledge and just wonder searching google for days.

I have MacOS setup (VSCode PlatformIO installer) so it create for me a /penv/… .

-----Deep looking in side

Library python script are located at "$HOME/.platformio/penv/lib/python3.9/site-packages/platformio"

Current I need to using util packages inside PlatformIO python Module.I could not import platfromIO cause install by using VSCode it create virtual env not a proper pip3 packages

I have a hack let me using the export Path to call and import Python method. I aware this it not a proper solution (no intellisence, no code formatting stuff).

pioPath = ".platformio/penv/lib/python3.9/site-packages/platformio
fullPath = os.path.join(os.path.expanduser('~'), pioPath)
# I using util package inside of platfromIO
import util
util.get_serial_ports()

Question is How could you export “platformio/penv/lib/python3.9/…” to be a proper Python package. that every python script is able to access.

Does the PlatfromIO inside VSCode gonna work when PlatfromIO installed by PIP3? (if so any documentation on how I could resolve it)

I think if you have a Python script that access the PlatformIO functionality outside of extra_scripts you would need to either

  • run it with either the Python interpreter that has the PlatformIO package installed ( ~/.platformio/python3/python` I think)
  • or install PlatformIO globally (I do this the quick&dirty way with sudo -H pip3 install platformio, then tell the PlatformIO VSCode extension to not use the built-in Python or PlatformIO version – do not have 2 core installed / used in parallel)

If you want to use the bundled python interpreter and the environment, then I suggest to use the python venv feature. As you may know, the PIO comes with a python3.x and also one virtual environment that is ~/.platformio/penv.

To use this, you just source it by either

. ~/.platformio/penv/bin/activate
or
source ~/.platformio/penv/bin/activate
Then you are in the PIO’s python environment where you can access all the pio related python libraries. In this environment, python command is symlinked to python3 and pip is actually the pip3, so you are in the python3 only environment. Once you’re in this environment, you can run pip install the required packages to this environment and access them also.

I guess you already know most of them, just in case this might make you rethink and discover.