Cannot run remote unit tests from a github action : python setup.py egg_info did not run successfully

Hi, according to your article, I am trying to get remote unit testing (on target HW) to work triggered from a github action. I am not sure what is the most up-to-date yaml file to achieve this, but I am using this one

name: Unit Tests Remote Target
on: [push, pull_request]
jobs:
  UnitTests:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Cache pip
      uses: actions/cache@v2
      with:
        path: ~/.cache/pip
        key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
        restore-keys: |
          ${{ runner.os }}-pip-
    - name: Cache PlatformIO
      uses: actions/cache@v2
      with:
        path: ~/.platformio
        key: ${{ runner.os }}-${{ hashFiles('**/lockfiles') }}
    - name: Set up Python
      uses: actions/setup-python@v2
    - name: Install PlatformIO
      run: |
        python -m pip install --upgrade pip
        pip install --upgrade platformio
    - name: Run remote tests on the target platform
      run: platformio remote test -e esp32

An error occurs in the last step (Run remote tests on the target platform). This is the output from github action:

  platformio remote test -e esp32
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.10.2/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.10.2/x64/lib
********************************************************************************
If you like PlatformIO, please:
- follow us on Twitter to stay up-to-date on the latest project news > https://twitter.com/PlatformIO_Org
- star it on GitHub > https://github.com/platformio/platformio
- try PlatformIO IDE for embedded development > https://platformio.org/platformio-ide
********************************************************************************

Tool Manager: Installing platformio/contrib-pysite @ ~2.310.0
Collecting twisted==21.7.0
  Downloading Twisted-21.7.0.tar.gz (3.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.7/3.7 MB 66.5 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting cryptography<35.0.0,>=3.3
  Downloading cryptography-3.4.8.tar.gz (546 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 546.9/546.9 KB 84.3 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: still running...
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting pyopenssl<=21.0.0,>=16.0.0
  Downloading pyOpenSSL-21.0.0.tar.gz (175 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.7/175.7 KB 31.0 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting service_identity<=21.1.0,>=18.1.0
  Downloading service-identity-21.1.0.tar.gz (40 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.1/40.1 KB 13.1 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting Automat>=0.8.0
  Downloading Automat-20.2.0.tar.gz (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.7/61.7 KB 22.2 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error
  
  Γ— python setup.py egg_info did not run successfully.
  β”‚ exit code: 1
  ╰─> [53 lines of output]
      WARNING: The wheel package is not available.
        error: subprocess-exited-with-error
      
        Γ— python setup.py bdist_wheel did not run successfully.
        β”‚ exit code: 1
        ╰─> [6 lines of output]
            usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
               or: setup.py --help [cmd1 cmd2 ...]
               or: setup.py --help-commands
               or: setup.py cmd --help
      
            error: invalid command 'bdist_wheel'
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for m2r
      ERROR: Failed to build one or more wheels
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/site-packages/setuptools/installer.py", line 75, in fetch_build_egg
          subprocess.check_call(cmd)
        File "/opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/subprocess.py", line 369, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/opt/hostedtoolcache/Python/3.10.2/x64/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpsnb5q4xo', '--quiet', 'm2r']' returned non-zero exit status 1.
      
      The above exception was the direct cause of the following exception:
      
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-cufod7go/automat_97fa58c15650405e9db4e6bc10f2d4b1/setup.py", line 15, in <module>
          setup(
        File "/opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/site-packages/setuptools/__init__.py", line 152, in setup
          _install_setup_requires(attrs)
        File "/opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
          dist.fetch_build_eggs(dist.setup_requires)
        File "/opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/site-packages/setuptools/dist.py", line 806, in fetch_build_eggs
          resolved_dists = pkg_resources.working_set.resolve(
        File "/opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/site-packages/pkg_resources/__init__.py", line 766, in resolve
          dist = best[req.key] = env.best_match(
        File "/opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1051, in best_match
          return self.obtain(req, installer)
        File "/opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1063, in obtain
          return installer(requirement)
        File "/opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/site-packages/setuptools/dist.py", line 877, in fetch_build_egg
          return fetch_build_egg(self, req)
        File "/opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/site-packages/setuptools/installer.py", line 77, in fetch_build_egg
          raise DistutilsError(str(e)) from e
      distutils.errors.DistutilsError: Command '['/opt/hostedtoolcache/Python/3.10.2/x64/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpsnb5q4xo', '--quiet', 'm2r']' returned non-zero exit status 1.
      
      
      !!! m2r not found, long_description is bad, don't upload this to PyPI !!!
      
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

Γ— Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Error: 

Please ensure that the next packages are installed:

sudo apt install python3-dev libffi-dev libssl-dev

Error: Process completed with exit code 1.

Any hints on how to solve this ?

  • Remote testing (on this repo) from a platformio terminal works fine,
  • Native Unit tests from github actions (eg Strooom/logging repo) also work fine, (the .yaml file is simpler here)

CC @ivankravets when using the PIO-tutorial supplied YAML, pip seems to fail at the installation of markdown-to-reStructure pacakge.

1 Like

Could you replace these lines with

Does it work?

No,

this time Python 3.9.10 gets installed instead of 3.10.2 but the error I get looks exactly the same

(first part of log skipped)

  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error
  
  Γ— python setup.py egg_info did not run successfully.
  β”‚ exit code: 1
  ╰─> [53 lines of output]
      WARNING: The wheel package is not available.
        error: subprocess-exited-with-error
      
        Γ— python setup.py bdist_wheel did not run successfully.
        β”‚ exit code: 1
        ╰─> [6 lines of output]
            usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
               or: setup.py --help [cmd1 cmd2 ...]
               or: setup.py --help-commands
               or: setup.py cmd --help
      
            error: invalid command 'bdist_wheel'
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for m2r
      ERROR: Failed to build one or more wheels
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/site-packages/setuptools/installer.py", line 75, in fetch_build_egg
          subprocess.check_call(cmd)
        File "/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/subprocess.py", line 373, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/opt/hostedtoolcache/Python/3.9.10/x64/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpbj0gv7zt', '--quiet', 'm2r']' returned non-zero exit status 1.
      
      The above exception was the direct cause of the following exception:
      
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-tdv149w6/automat_5dc667c970804842bec033f63e724f94/setup.py", line 15, in <module>
          setup(
        File "/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/site-packages/setuptools/__init__.py", line 152, in setup
          _install_setup_requires(attrs)
        File "/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
          dist.fetch_build_eggs(dist.setup_requires)
        File "/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/site-packages/setuptools/dist.py", line 806, in fetch_build_eggs
          resolved_dists = pkg_resources.working_set.resolve(
        File "/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/site-packages/pkg_resources/__init__.py", line 766, in resolve
          dist = best[req.key] = env.best_match(
        File "/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1051, in best_match
          return self.obtain(req, installer)
        File "/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1063, in obtain
          return installer(requirement)
        File "/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/site-packages/setuptools/dist.py", line 877, in fetch_build_egg
          return fetch_build_egg(self, req)
        File "/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/site-packages/setuptools/installer.py", line 77, in fetch_build_egg
          raise DistutilsError(str(e)) from e
      distutils.errors.DistutilsError: Command '['/opt/hostedtoolcache/Python/3.9.10/x64/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpbj0gv7zt', '--quiet', 'm2r']' returned non-zero exit status 1.
      
      
      !!! m2r not found, long_description is bad, don't upload this to PyPI !!!
      
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

Γ— Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Error: 

Please ensure that the next packages are installed:

sudo apt install python3-dev libffi-dev libssl-dev

Error: Process completed with exit code 1.

Could you restart your CI? We deployed new packages of

Hi Ivan,
looks like PIO is now installing properly and running fine in the github-action-runner. :slight_smile:

I now hit this error
Error: -23 12:02:33 [error] You are not authorized! Please log in to PlatformIO Account.

Github actions has evolved a bit since your tutorial has been published. They now have several sections on Secrets… I put the PLATFORMIO_AUTH_TOKEN in

Settings
  Secrets
    Actions
      Repository secrets

Where is PIO expecting this secret ?

This definitely looks like the right envrionment variable to me.

Is there some code you added to copy this repository secret into this environment variable?

no, maybe I missed something in the tutorial. I thought that copying the token from the remote platformio setup to the repo secret was enough to make it work… Can you point me to an example, or a .yml file which is set up for remote testing from a github action ?

This piece of code seems to bridge the gap between the secret value and putting it into the environment variables.