Support for netbsd_amd64?

PlatformIO Core
1

I have been using pio with VSC on win10 for over a year on a number of esp32 projects. About 3 months ago I dropped vsc almost entirely and have been using pio cli under wsl-ubuntu. My native development platform is netbsd_amd64 for over 25 years and I really would like to use pio-cli here instead of the others. I have pio installed but when I try to start a project I get:

$ pio check -d CommandStation-EX 
Checking wemos_d1_mini32 > cppcheck (platform: espressif32; board: wemos_d1_mini32; framework:
----------------------------------------------------------------------------------------------
UnknownPackageError: Could not find the package with 'espressif/toolchain-xtensa-esp32 @ 8.4.0

I have gone through the .platformio/ installation files and the things I see different from FreeBSD are generally:

serialposix.py :
NetBSD - dtyU0 dtyU1 ...

configure.ac :
NetBSD - also uses  csh, tcsh
NetBSD - uses ctags

Can we get this running on netbsd_amd64? I would really appreciate it. Thanks

2

The edit box dropped the end of the UnknownPackageError at the plus mark. add
2021r2-patch5’ requirements for your system ‘netbsd_amd64’

3

Pretty much the same issue as described in Can't compile on FreeBSD (toolchain-xtensa32) · Issue #269 · platformio/platform-espressif32 · GitHub. The issue has not gained much traktion over the years, probably due to a lack of users or requests. :frowning:

4

Thanks. I am here to try to help it happen. I will try to follow the link you provided.

5 
git clone (https)://<mygithubusername>:<myaccesstoken>@github.com/trombik/platformio-netbsd-toolchain-x
-(yes, using the actual username and access token)
Cloning into 'platformio-netbsd-toolchain-x'...
remote: Support for password authentication was removed on August 13, 2021.
remote: Please see (url) for information on currently recommended modes of authentication.
fatal: Authentication failed for (url)

Same error when I tried to clone the freebsd -toolchain

6

I see it deleted the text indicating where my github username and access token were. The url should be correct, just deleted those items.

7

So the absolute simplest

git clone https://github.com/trombik/platformio-freebsd-toolchain-xtensa32.git toolchain-xtensa32

fails? That should be executable without any username or password or token. It’s a pubic repo.

Note though that this repo is outdated, it provides toolchain-xtensa32 (5.2.0) but you need toolchain-xtensa-esp32 (8.4.0)

https://github.com/trombik/xtensa-esp32-elf seems to be more up-to-date for building the toolchains / tools from scratch.

8

Got it. Will continue later. Thanks!

9

NetBSD pkgsrc has /usr/pkgsrc/devel/ninja-build (renamed due to a naming conflict with chat/ninja) and /usr/pkgsrc/devel/py-ninja and they are installed via pkgsrc as well as cmake so I modified the ninja script to point to the correct places there instead of /usr/local.

devel/ninja-build installs the following files under /usr/pkg/:

bin/ninja
share/doc/ninja/manual.asciidoc
share/ninja/bash-completion
share/ninja/ninja-mode.el
share/ninja/ninja.vim

I will take a look at what it needs to access cmake next. Thanks!

I think it would be good to create a /usr/pkgsrc/devel/platformio_support or such to auto install this stuff. The main platformio that I originally installed likely should go there also but I am not sure yet.

10

NetBSD pkgsrc doesn’t have esp32ulp and right now, I don’t need the co-processor support so can I ignore adding GitHub - trombik/platformio-freebsd-toolchain-esp32ulp ? I already had ninja and cmake installed and created the links first three links mentioned. I can provide the init.sh files needed to link to pkgsrc if desired. What next is required?

11

You keep adding packages and resolving error messages until you can compile an Arduino sketch at least. Arduino does not require ninja or cmake, so technically the tool-cmake and tool-ninja packages can be empty (just a folder with a package.json declaring the right name and system metadata, but no actual executables). The toolchain packages and esptoolpy (python based) should be the only ones you need to actually supply.

12

I copied .platformio/packages/* from the wsl ubuntu version that works there -

drwx------   8 kent  wheel   512 Feb 15 21:10 framework-arduino-avr
drwx------  10 kent  wheel   512 Feb 15 21:10 framework-arduino-megaavr
drwx------   7 kent  wheel   512 Feb 15 21:03 framework-arduino-samd
drwx------   7 kent  wheel   512 Feb 15 20:53 framework-arduino-samd-sparkfun
drwx------   6 kent  wheel   512 Feb 15 21:33 framework-arduinoespressif32
drwx------  10 kent  wheel   512 Feb 15 22:01 framework-arduinoststm32
drwx------   4 kent  wheel   512 Feb 15 22:09 framework-arduinoteensy
drwx------   4 kent  wheel   512 Feb 15 21:55 framework-cmsis
drwx------   3 kent  wheel   512 Feb 15 21:03 framework-cmsis-atmel
drwx------   4 kent  wheel   512 Feb 15 21:01 framework-cmsis@1.40500.0
drwx------   2 kent  wheel   512 Feb 15 23:27 tool-avrdude
drwxr-xr-x   5 kent  wheel   512 May  2 15:39 tool-cmake
drwx------   7 kent  wheel   512 Feb 15 21:33 tool-esptoolpy
drwx------   2 kent  wheel   512 Feb 16 00:56 tool-mkfatfs
drwx------   2 kent  wheel   512 Feb 16 00:56 tool-mklittlefs
drwx------   2 kent  wheel   512 Feb 16 00:56 tool-mkspiffs
drwxr-xr-x   3 kent  wheel   512 May  7 09:16 tool-ninja
drwx------   3 kent  wheel   512 Feb 15 21:03 tool-scons
drwx------   3 kent  wheel  1024 Feb 15 22:10 tool-teensy
drwx------   8 kent  wheel   512 Feb 15 21:10 toolchain-atmelavr
drwx------  10 kent  wheel   512 Feb 15 21:55 toolchain-gccarmnoneeabi
drwx------   8 kent  wheel   512 Feb 15 22:08 toolchain-gccarmnoneeabi-teensy
drwx------   6 kent  wheel   512 Feb 15 20:53 toolchain-gccarmnoneeabi@1.70201.0
drwx------   8 kent  wheel   512 May  7 09:20 toolchain-xtensa-esp32

and added the ones from above and swapped out xtensa-esp32-elf with the netbsd one -

tiamat.kent: toolchain-xtensa-esp32 {327} ll xtensa-esp32-elf
total 20
-rw-r--r--  1 kent  wheel   747 May  1 16:11 LICENSE
-rw-r--r--  1 kent  wheel  1156 May  1 16:11 README.md
lrwxr-xr-x  1 kent  wheel    16 May  7 09:25 bin -> /usr/pkgnew/bin/
-rwxr-xr-x  1 kent  wheel   100 May  1 16:11 clean.sh
lrwxr-xr-x  1 kent  wheel    20 May  7 09:25 include -> /usr/pkgnew/include/
-rwxr-xr-x  1 kent  wheel   627 May  7 09:25 init.sh
lrwxr-xr-x  1 kent  wheel    16 May  7 09:25 lib -> /usr/pkgnew/lib/
lrwxr-xr-x  1 kent  wheel    20 May  7 09:25 libexec -> /usr/pkgnew/libexec/
-rw-r--r--  1 kent  wheel   224 May  7 09:25 package.json
lrwxr-xr-x  1 kent  wheel    18 May  7 09:25 share -> /usr/pkgnew/share/

$  pio run
Processing wemos_d1_mini32 (platform: espressif32; board: wemos_d1_mini32; framework: arduino)
---------------------------------------------------------------------------------------------------------------------------------
Tool Manager: Installing espressif/toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
UnknownPackageError: Could not find the package with 'espressif/toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5' requirements for your system 'netbsd_amd64'

Where is this patch?

13

The package.json has to be directly in the toolchain-xtensa-esp32 folder, and then there should be bin/xtensa-esp32-elf-gcc relative to that, et cetara. The package.json should have systems array containing at least netbsd_amd64 (or only that).

Now the thing is, that even if you place the toolchain folder in ~/.platformio/packages, it will not be recognized as installed (I think it needs to have an additional special .piopm package manager file in that folder for that), it will try to go to the registry first. To resolve that, you have to edit the packages definition file of the platform, aka,

~/.platformio/platforms/espressif32/platform.json

There you will see

Which is where the “version” it wants comes from. To avoid the error and use a different folder as a package source instead, you change the version expression to be a symlink://<path> (one of many possible source expressions). That path should ideally not be inside ~/.platformio/packages, or it will get weird. You could e.g. place the packages in your home directory.

Other packages it complains about will need the same treatment.

Another point, is that you can see all packages with their downlink links by access the API

https://api.registry.platformio.org/v3/packages/espressif/tool/toolchain-xtensa-esp32

For e.g. espressif/toolchain-xtensa-esp32.

14

Thank you Very much! Just to see, I edited package.json, changing to netbsd_amd64 and -

$  pio run
Processing wemos_d1_mini32 (platform: espressif32; board: wemos_d1_mini32; framework: arduino)
---------------------------------------------------------------------------------------------------------------------------------
Tool Manager: Installing platformio/framework-arduinoespressif32 @ ~3.20017.0
Downloading  [####################################]  100%          
Unpacking  [####################################]  100%          
Tool Manager: framework-arduinoespressif32@3.20017.241212+sha.dcc1105b has been installed!
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/wemos_d1_mini32.html
PLATFORM: Espressif 32 (6.10.0) > WEMOS D1 MINI ESP32
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-arduinoespressif32 @ 3.20017.241212+sha.dcc1105b 
 - tool-esptoolpy @ 1.40501.0 (4.5.1) 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Converting CommandStation-EX.ino
sh: Cannot execute ELF binary /home/kent/.platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-g++
*** Error 2
AssertionError: :
  File "/home/kent/.platformio/penv/lib/python3.12/site-packages/platformio/builder/main.py", line 173:
    env.SConscript("$BUILD_SCRIPT")
  File "/home/kent/.platformio/packages/tool-scons/scons-local-4.8.1/SCons/Script/SConscript.py", line 620:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/home/kent/.platformio/packages/tool-scons/scons-local-4.8.1/SCons/Script/SConscript.py", line 280:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/home/kent/.platformio/platforms/espressif32/builder/main.py", line 368:
    target_elf = env.BuildProgram()
  File "/home/kent/.platformio/packages/tool-scons/scons-local-4.8.1/SCons/Util/envs.py", line 252:
    return self.method(*nargs, **kwargs)
  File "/home/kent/.platformio/penv/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 62:
    env.ProcessProgramDeps()
  File "/home/kent/.platformio/packages/tool-scons/scons-local-4.8.1/SCons/Util/envs.py", line 252:
    return self.method(*nargs, **kwargs)
  File "/home/kent/.platformio/penv/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 142:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "/home/kent/.platformio/packages/tool-scons/scons-local-4.8.1/SCons/Util/envs.py", line 252:
    return self.method(*nargs, **kwargs)
  File "/home/kent/.platformio/penv/lib/python3.12/site-packages/platformio/builder/tools/piobuild.py", line 349:
    env.ConvertInoToCpp()
  File "/home/kent/.platformio/packages/tool-scons/scons-local-4.8.1/SCons/Util/envs.py", line 252:
    return self.method(*nargs, **kwargs)
  File "/home/kent/.platformio/penv/lib/python3.12/site-packages/platformio/builder/tools/pioino.py", line 237:
    out_file = c.convert(ino_nodes)
  File "/home/kent/.platformio/penv/lib/python3.12/site-packages/platformio/builder/tools/pioino.py", line 85:
    return self.process(contents)
  File "/home/kent/.platformio/penv/lib/python3.12/site-packages/platformio/builder/tools/pioino.py", line 106:
    assert self._gcc_preprocess(contents, out_file)

So some progress.
It installed .platformio/packages/framework-arduinoespressif32 the same as with the Linux installation. I am not sure exactly what needs to go in “version”. Is it just a directory outside of .platformio or something more?

15

re - https://api.registry.platformio.org/v3/packages/espressif/tool/toolchain-xtensa-esp32

I see all of that but I don’t see exactly what I would need there.

16

The binaries are all useless because they’re for Windows/Mac/Linux, but you can get the needed package.json as a base file from any of them and then just edit the systems array to include netbsd_amd64, so it will be accepted by PlatformIO.

That looks like it’s at least trying to invoke the compiler. Is the provided xtensa-esp32-elf-g++ validy compiled for NetBSD AMD64? What happens when you try to execute

/home/kent/.platformio/packages/toolchain-xtensa-esp32/bin/xtensa-esp32-elf-g++ --version

?

19

I have installed ncurses-6.5 CRT screen handling and optimization package
It appears that ncursesw.h and panel.h are now in include/ncurses/ instead of include/ncursesw/.
Is there a new configure script that expects ncurses 6.5?

20

Sorry, I have absolutely no idea about that. If you’re stuck compiling the XTensa compiler, you should look into the instructions at

https://github.com/espressif/crosstool-NG/tree/esp-2021r2-patch5

21

Same with GitHub - espressif/crosstool-NG: crosstool-NG with support for Xtensa
It doesn’t support modern versions of ncurses which have the wide char support built in so ncursesw has been deprecated. It also doesn’t detect installed Python versions. I am looking into possible ways to fix it. Thanks

22

There are a few notes on ncurses and also Python in https://github.com/trombik/xtensa-esp32-elf/blob/main/devel/xtensa-esp-elf/Makefile.