Advanced scripting: Lib error with subprocess.run and MinGW gdb

Hello,

I am trying to do some parsing after the elf file was created.
Win10, PIO version 5.1.1, robotdyn_blackpill_f303cc
For this I am trying to call gdb-python27 from MinGW (GNU gdb (GDB) 7.5, installed a few days ago).
Regardless of the input commands I provide, or even just calling the exe itself, I get an error in the python lib provided with PIO.

The error seems to be reproducable by just trying to create a subprocess of gdb. Adding arguments or setting other parameters (shell, text, or cwd) do not alter the behavior.

ret = subprocess.run(“C:/MinGW/bin/gdb-python27.exe”)

Error:

Starting postbuild, cmd C:/MinGW/bin/gdb-python27.exe --batch -ex “py inputFile = ‘firmware.elf’;outputFile = ‘dbg_readout.json’”-x C:/Users/Kevin/Documents/Projekte/FanControl/Control_V2/GDBReadoutDirect_27.py
cwd C:\Users\Kevin\Documents\Projekte\FanControl\Control_V2.pio\build\robotdyn_blackpill_f303cc
File “C:\Users\Kevin.platformio\python3\lib\site.py”, line 177
file=sys.stderr)
^
SyntaxError: invalid syntax
None
None
CompletedProcess(args=‘C:/MinGW/bin/gdb-python27.exe --batch -ex “py inputFile = 'firmware.elf';outputFile = 'dbg_readout.json'”-x C:/Users/Kevin/Documents/Projekte/FanControl/Control_V2/GDBReadoutDirect_27.py’, returncode=1)

The error occurs in site.py in the function

def addpackage(sitedir, name, known_paths):
“”“Process a .pth file within the site-packages directory:
For each line in the file, either combine it with sitedir to a path
and add that to known_paths, or execute it if it starts with 'import '.
“””

The error does not appear when I try to run the command from PyCharm with either my local python 3.9 or even the PIO provided 3.7 instance. Paths etc. are hard-coded in this case.

What could be the cause here? Executing an echo command works fine.

Are you sure that what you’re trying to achieve can’t be achieved with PlatformIO’s toolchain? The standard toolchain-gccarmnoneeabi package does have arm-none-eabi-gdb-py3.exe. env.Execute() and the $GDB environment variable (with a py3 suffix) are also usefull for that.

It seems to me as if the gdb-python27 accesses PlatformIO’s Python environment irregardles of possibly having Python2.7 compiled into it, but I’m not sure. You may be able to correct that by giving the subprocess.run command some sort of environment object which has corrected environment variables (e.g. regarding PYTHONPATH, PYTHONEXE etc.). Or, alternatively try PIO’s arm-none-eabi-gdb-py3 – this also removes an external dependency from the process.

For reference, this is how PlatformIO executes sub-commands with its Python version:

The parsing I am doing did not work with the arm gdb instances. I did modify the scripts slightly, but it was not easily adaptable.
I tried with env = None, and also by creating a bat file and executing that.

Are you sure that that will not cause subprocess to clone the current environment? I’d recommend to try doing something like

isolated_env = os.environ.copy()
# might throw exceptions if keys are not actually present, adapt as needed
del os.environ['PYTHONEXE']
del os.environ['PYTHONPATH']
# pass isolate_env

Printing os.environ might give you additional clues as to what environment variables might cause an issue.

EDIT: Woops, the code above modifies os.environ and not the copied isolated_env variable.

isolated_env = os.environ.copy()
# might throw exceptions if keys are not actually present, adapt as needed
del isolated_env['PYTHONEXE']
del isolated_env['PYTHONPATH']
# pass isolate_env

With the 3 versions of the gcc toolchain I get mixed results even starting gdb-py:

toolchain-gccarmnoneeabi@1.70201.0 → works
toolchain-gccarmnoneeabi@1.90201.191206 → ImportError: No module named site
toolchain-gccarmnoneeabi@1.90301.200702 →
Fatal Python error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named ‘encodings’
Current thread 0x00004bd0 (most recent call first):

So that does not seem to be too reliable.

But your hint was correct, deleting PYTHONPATH works. Or simply calling it like this:

ret = subprocess.run(f"call {batPath}", cwd = cwd, shell = True, env = {})