I want to import an own python library in my extra_script.py but when building the code, an error is thrown
ImportError: cannot import name 'postpro'
I installed this python library successfully upfront using
pip install postpro and also with pip install postpro -U
I works well when using standard python libraries:
Import("env", "projenv")
import shutil
...
I am runnning macOS Catalina. thanks in advance for any hint
If you do a pip install in the MacOS terminal, that will add it to the Python 2.7 libraries, as that is what MacOS is using. When installing PlatformIO via VSCode, it brings its own self-contained Python3 environment which is separate from the system environment. The libs installed by your system’s pip will not be seen there.
The recommended way to install additional Python dependencies is to attempt an import, and if that fails, call into $PYTHONEXE -m pip install <package> to install it. This is showcased e.g. in
and you should add the same style of code in your extra script.
For a more direct way, you can also go into your home directory (/Users/<user> on Mac, idk?) and from there into the .platformio folder. Search this folder for the pip3 executable. Once you find its path, use this pip3 executable to execute the install command, and it will be installed. This is then however not well reproducable accross computers because every user of your extra script will have to do this too; Prefer the first method instead.
Thanks @maxgerhardt for the fast reply, but unfortunately both ways didn’t work for me.
I was able to install the library, but there is another issue, I cannot fix:
% /Users/dirk/.platformio/python3/bin/python3.8 -m pip install postpro
Requirement already satisfied: postpro in /Users/dirk/.platformio/python3/lib/python3.8/site-packages (0.6.5)
ERROR: Error while checking for conflicts. Please file an issue on pip's issue tracker: https://github.com/pypa/pip/issues/new
Traceback (most recent call last):File "/Users/dirk/.platformio/python3/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3021, in _dep_map
return self.__dep_map
File "/Users/dirk/.platformio/python3/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2815, in __getattr__
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
...
Maybe the library has errors itself, but it was build for python3.
Ehm well that’s new one for me. pip internall error. Does it only do that when you install that library? Does it go away with a .. -m pip uninstall postpro? The way you install it seems right to me if that python path is your $PYTHONEXE.
As a very experimental try, maybe updating pip helps? /Users/dirk/.platformio/python3/bin/python3.8 -m pip install --upgrade pip
update, @maxgerhardt: it was my fault, I called an internal function with an already used variable name.
so your answer was absolutely correct and very helpful. Thanks
I’m having a similar issue and I’m not sure how to implement this into a custom python test in the test_custom_runner.py file.
When I go to include a library that I haven’t installed I get an error like this:
UserSideException: Could not find custom test runner by this path -> .../test/embedded/test_custom_runner.py
But when I prefix my python code with this
# ---------------------- PIO SCRIPTING ----------------------
# https://docs.platformio.org/en/stable/scripting/examples/extra_python_packages.html
# Do not format the PIO scripting code. It must be before python imports
# noqa: off
Import("env") # type: ignore
# List installed packages
env.Execute("$PYTHONEXE -m pip list") # type: ignore
# Install custom packages from the PyPi registry
env.Execute("$PYTHONEXE -m pip install pkg1 pkg2") # type: ignore
# Install missed package
try:
from saleae import automation
except ImportError:
env.Execute("$PYTHONEXE -m pip install saleae") # type: ignore
# noqa: on
# ---------------------- PIO SCRIPTING ----------------------
from saleae import automation
from platformio.public import TestCase, TestCaseSource, TestStatus, UnityTestRunner
import os
import os.path
from datetime import datetime
# Code here
I get this error
Verbosity level can be increased via `-v, -vv, or -vvv` option
Collected 2 tests
NameError: Traceback (most recent call last):
File "/Users/freddy/.platformio/penv/lib/python3.11/site-packages/platformio/__main__.py", line 103, in main
cli() # pylint: disable=no-value-for-parameter
^^^^^
File "/Users/freddy/.platformio/penv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/freddy/.platformio/penv/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/Users/freddy/.platformio/penv/lib/python3.11/site-packages/platformio/cli.py", line 85, in invoke
return super().invoke(ctx)
^^^^^^^^^^^^^^^^^^^
File "/Users/freddy/.platformio/penv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/freddy/.platformio/penv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/freddy/.platformio/penv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/freddy/.platformio/penv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/freddy/.platformio/penv/lib/python3.11/site-packages/platformio/test/cli.py", line 137, in cli
runner = TestRunnerFactory.new(
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/freddy/.platformio/penv/lib/python3.11/site-packages/platformio/test/runners/factory.py", line 58, in new
mod = load_python_module(module_name, custom_runner_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/freddy/.platformio/penv/lib/python3.11/site-packages/platformio/compat.py", line 99, in load_python_module
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/Users/freddy/Library/Mobile Documents/com~apple~CloudDocs/Code/Arduino/ArduinoUnitPIO/test/embedded/test_custom_runner.py", line 5, in <module>
Import("env") # type: ignore
^^^^^^
NameError: name 'Import' is not defined
============================================================
An unexpected error occurred. Further steps:
* Verify that you have the latest version of PlatformIO using
`python -m pip install -U platformio` command
* Try to find answer in FAQ Troubleshooting section
https://docs.platformio.org/page/faq/index.html
* Report this problem to the developers
https://github.com/platformio/platformio-core/issues
============================================================
How do I interface with the virtual environment in this scenario?
I don’t think this is invoked as an SCons script, but rather as a regular Python import, so you don’t just have Import("env") available.
Do you need the env variable? It is available in the callbacks for your custom test runner class, e.g., seen in e.g.
https://docs.platformio.org/en/latest/advanced/unit-testing/frameworks/custom/api.html
def configure_build_env(self, env):
etc.
https://github.com/platformio/platformio-core/blob/develop/platformio/test/runners/unity.py
Update:
This works fine running with pio test which uses my own external installation of PIO with my own python packages. But I can’t get this work with the integrated test buttons in the pio venv without running source ~/.platformio/penv/bin/activate and pip install ....
How would I install a python package ‘logic2-automation’ in the pio virtual environment for testing? I understand how to do it for builds with this link, but this doesn’t seem to work with the testing environment
I’m not sure if I need the env variable. I was deriving this from the scripting tutorial. I only need a python package for the runner file. I don’t need to edit any C++ environment variables at the moment.
If I was to add
def configure_build_env(self, env):
env.Execute("$PYTHONEXE -m pip list")
# Install custom packages from the PyPi registry
env.Execute("$PYTHONEXE -m pip install saleae")
return env
Into class CustomTestRunner(UnityTestRunner): as a method, the original import at the top of the file import saleae would fail.
I ran a regular build with extra_scripts = extra_script.py which installed the package I need in the PIO environment, but this didn’t seem to transfer to the testing enviroment.