Platformio remote test doesn't detect output from Serial

I have a raspberry pi running a remote platformio agent (in a virtualenv).

On my laptop I have a sample project that I want to remotely test.
When I run pio remote test -e d1_mini it starts to build, generates an error (somewhere during the build it says “No names found, cannot describe anything”. The build fails (I don’t know why, compilation and linking itself succeeds) but the test continues.

Next it says testing project remotely. nothing after that.

However, when I press ctrl+c to quit, it suddently does give me the output: processing, uploading…

Now the test report itself doesn’t get captured. It is like the board doesn’t reset. But when I press the reset button it still doesn’t capture the test report. Opening the serial monitor (remotely) does work.

Here’s the complete output (edited for readability):

$ pio remote test -v -e d1_mini
PIO Plus (https://pioplus.com) v2.5.4
Building project locally
Verbose mode can be enabled via `-v, --verbose` option
Collected 1 items

Processing * in d1_mini environment
-------------------------------------------------
Building...
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/d1_mini.html
PLATFORM: Espressif 8266 (Stage) 2.3.0-alpha.2 #3500fb2 > WeMos D1 R2 and mini
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES: toolchain-xtensa 2.40802.190218 (4.8.2), framework-arduinoespressif8266 8dd068e, tool-esptool 1.413.0 (4.13), tool-esptoolpy 1.20600.0 (2.6.0)       
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain+, Compatibility ~ soft
Found 46 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <mod1>
Generating .pio\build\d1_mini\core\core_version.h
Generating LD script .pio\build\d1_mini\ld\local.eagle.app.v6.common.ld
Compiling .pio\build\d1_mini\liba55\mod1\mod1.cpp.o
Compiling .pio\build\d1_mini\UnityTestLib\unity.c.o
Archiving .pio\build\d1_mini\libFrameworkArduinoVariant.a
fatal: No names found, cannot describe anything.
Compiling .pio\build\d1_mini\FrameworkArduino\StackThunk.cpp.o
...
Archiving .pio\build\d1_mini\libFrameworkArduino.a
Linking .pio\build\d1_mini\firmware.elf
Building .pio\build\d1_mini\firmware.bin
Retrieving maximum program size .pio\build\d1_mini\firmware.elf
Checking size .pio\build\d1_mini\firmware.elf
Memory Usage -> http://bit.ly/pio-memory-usage
DATA:    [===       ]  33.0% (used 27056 bytes from 81920 bytes)
PROGRAM: [==        ]  24.1% (used 251679 bytes from 1044464 bytes)
Creating BIN file ".pio\build\d1_mini\firmware.bin" using ".pio\build\d1_mini\firmware.elf"
======= [FAILED] Took 5.46 seconds =======
Testing project remotely

Aborted!
$ Verbose mode can be enabled via `-v, --verbose` option
Collected 1 items

Processing * in d1_mini environment
-------------------------------------------------
Uploading...
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/d1_mini.html
PLATFORM: Espressif 8266 2.2.3 > WeMos D1 R2 and mini
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES: framework-arduinoespressif8266 2.20502.0 (2.5.2), tool-esptool 1.413.0 (4.13), tool-esptoolpy 1.20600.0 (2.6.0), tool-mkspiffs 1.200.0 (2.0)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain+, Compatibility ~ soft
Found 0 compatible libraries
Scanning dependencies...
No dependencies
Configuring upload protocol...
AVAILABLE: espota, esptool
CURRENT: upload_protocol = esptool
Looking for upload port...
Auto-detected: /dev/ttyUSB0
Uploading .pio/build/d1_mini/firmware.bin
esptool.py v2.6
Serial port /dev/ttyUSB0
Connecting....
Chip is ESP8266EX
Features: WiFi
MAC: 5c:cf:7f:c3:b0:29
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 4MB
Compressed 255824 bytes to 186923...

Writing at 0x00000000... (8 %)
Writing at 0x00004000... (16 %)
Writing at 0x00008000... (25 %)
Writing at 0x0000c000... (33 %)
Writing at 0x00010000... (41 %)
Writing at 0x00014000... (50 %)
Writing at 0x0001c000... (66 %)
Writing at 0x00020000... (75 %)
Writing at 0x00024000... (83 %)
Writing at 0x00028000... (91 %)
Writing at 0x0002c000... (100 %)
Wrote 255824 bytes (186923 compressed) at 0x00000000 in 17.7 seconds (effective 115.9 kbit/s)...

Leaving...
Hard resetting via RTS pin...
Testing...


$ pio remote device monitor -b 74880 
PIO Plus (https://pioplus.com) v2.5.4
Starting Serial Monitor on DEVELOP:/dev/ttyUSB0
--- Miniterm on socket://localhost:50850  74880,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1384, room 16
tail 8
chksum 0x2d
csum 0x2d
v8dd068eb
~ld
test\test_embedded.cpp:29:test_mod1:PASS

-----------------------
1 Tests 0 Failures 0 Ignored
OK

--- exit ---

Mind that to generate this report, I have to manually abort after it says Testing project remotely but it resumes automatically. After uploading I also manually stop the test and (manually) open the serial monitor.

This post suggest there is an error with a git project. Do you rely or reference any such git projects in your platformio.ini or your project?

No dependencies. I only have some “blink” code to check if the code was uploaded and the ESP is running, which it does. My platformio.ini is nothing more then this:

[env:d1_mini]
platform = espressif8266
board = d1_mini
framework = arduino

I’m using VSCode in Win10. But Platformio CLI on the WSL shows the same issue.

Dependency Graph 
|-- <mod1>

You have however this dependency? What does it do?

So I created a new empty project, cleared all dependencies but still:

PS $ pio remote test -e d1_mini -v
PIO Plus (https://pioplus.com) v2.5.4
Building project locally
Verbose mode can be enabled via `-v, --verbose` option
Collected 1 items

Processing * in d1_mini environment
---------------------------------------------------------------------------------------------------------------------------------------------------------------
Building...
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/d1_mini.html
PLATFORM: Espressif 8266 2.2.3 > WeMos D1 R2 and mini
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES: toolchain-xtensa 2.40802.190218 (4.8.2), framework-arduinoespressif8266 2.20502.0 (2.5.2), tool-esptool 1.413.0 (4.13), tool-esptoolpy 1.20600.0 (2.6.0)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 28 compatible libraries
Scanning dependencies...
No dependencies
Compiling .pio\build\d1_mini\test\output_export.cpp.o
Compiling .pio\build\d1_mini\test\test_main.cpp.o
Generating LD script .pio\build\d1_mini\ld\local.eagle.app.v6.common.ld
Compiling .pio\build\d1_mini\UnityTestLib\unity.c.o
Archiving .pio\build\d1_mini\libFrameworkArduinoVariant.a
Compiling .pio\build\d1_mini\FrameworkArduino\Esp-frag.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\Esp-version.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\Esp.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\FS.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\FunctionalInterrupt.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\HardwareSerial.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\IPAddress.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\MD5Builder.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\Print.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\Schedule.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\ScheduledFunctions.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\StackThunk.cpp.o
Archiving .pio\build\d1_mini\libUnityTestLib.a
Compiling .pio\build\d1_mini\FrameworkArduino\Stream.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\StreamString.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\Tone.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\Updater.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\WMath.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\WString.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\abi.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\base64.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\cbuf.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\cont.S.o
Compiling .pio\build\d1_mini\FrameworkArduino\cont_util.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_app_entry_noextra4k.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_eboot_command.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_flash_utils.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_i2s.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_main.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_noniso.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_phy.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_postmortem.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_si2c.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_sigma_delta.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_timer.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_waveform.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_wiring.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_wiring_analog.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_wiring_digital.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_wiring_pulse.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_wiring_pwm.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\core_esp8266_wiring_shift.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\debug.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\gdb_hooks.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\heap.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\libb64\cdecode.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\libb64\cencode.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\libc_replacements.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\sntp-lwip2.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\spiffs\spiffs_cache.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\spiffs\spiffs_check.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\spiffs\spiffs_gc.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\spiffs\spiffs_hydrogen.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\spiffs\spiffs_nucleus.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\spiffs_api.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\spiffs_hal.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\sqrt32.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\time.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\uart.cpp.o
Compiling .pio\build\d1_mini\FrameworkArduino\umm_malloc\umm_malloc.cpp.o
Archiving .pio\build\d1_mini\libFrameworkArduino.a
Linking .pio\build\d1_mini\firmware.elf
Retrieving maximum program size .pio\build\d1_mini\firmware.elf
Checking size .pio\build\d1_mini\firmware.elf
Building .pio\build\d1_mini\firmware.bin
Creating BIN file ".pio\build\d1_mini\firmware.bin" using ".pio\build\d1_mini\firmware.elf"
Memory Usage -> http://bit.ly/pio-memory-usage
DATA:    [===       ]  32.9% (used 26924 bytes from 81920 bytes)
PROGRAM: [===       ]  25.3% (used 264200 bytes from 1044464 bytes)
================================================================= [FAILED] Took 5.40 seconds =================================================================
Testing project remotely

Now it hangs so I abort with <ctrl+c>

Aborted!
PS $ Verbose mode can be enabled via `-v, --verbose` option
Collected 1 items

Processing * in d1_mini environment
--------------------------------------------------------------------------------
Uploading...
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/d1_mini.html
PLATFORM: Espressif 8266 2.2.3 > WeMos D1 R2 and mini
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES: framework-arduinoespressif8266 2.20502.0 (2.5.2), tool-esptool 1.413.0 (4.13), tool-esptoolpy 1.20600.0 (2.6.0), tool-mkspiffs 1.200.0 (2.0)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 0 compatible libraries
Scanning dependencies...
No dependencies
Configuring upload protocol...
AVAILABLE: espota, esptool
CURRENT: upload_protocol = esptool
Looking for upload port...
Auto-detected: /dev/ttyUSB0
Uploading .pio/build/d1_mini/firmware.bin
esptool.py v2.6
Serial port /dev/ttyUSB0
Connecting....
Chip is ESP8266EX
Features: WiFi
MAC: 5c:cf:7f:c3:b0:29
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 4MB
Compressed 268352 bytes to 195553...

Writing at 0x00000000... (8 %)
Writing at 0x00004000... (16 %)
Writing at 0x00008000... (25 %)
Writing at 0x0000c000... (33 %)
Writing at 0x00010000... (41 %)
Writing at 0x00014000... (50 %)
Writing at 0x00018000... (58 %)
Writing at 0x0001c000... (66 %)
Writing at 0x00020000... (75 %)
Writing at 0x00024000... (83 %)
Writing at 0x00028000... (91 %)
Writing at 0x0002c000... (100 %)
Wrote 268352 bytes (195553 compressed) at 0x00000000 in 18.4 seconds (effective 116.6 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

So I press the reset button and the board starts again (blinking 5 times) but no test report.
Then I open serial monitor (remotely) et voilà:

PS $ pio remote device monitor -b 115200
PIO Plus (https://pioplus.com) v2.5.4
Starting Serial Monitor on DEVELOP:/dev/ttyUSB0
--- Miniterm on socket://localhost:50563  115200,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
sd␀l��|␀�d�|␃␄␄␄�␄d�␌c|��␂�␒�s�c�␄b��oo�doo���␌c␜x��l{d{lp�n�␐␃␄␄�␌l␌��␌␌␌c␌g�<␃d�␄␄�c��oo�␀l��l`␃�␓␓no␄d`␃␏␃o{���g␌␌c␌�␏l␎s��o␄␄c␌�␎l�␂�␂ll�␓�d`␃��g�␃test\test_main.cpp:32:test_led_builtin_pin_number:PASS
test\test_main.cpp:43:test_led_state_high:PASS
test\test_main.cpp:45:test_led_state_low:PASS
test\test_main.cpp:43:test_led_state_high:PASS
test\test_main.cpp:45:test_led_state_low:PASS
test\test_main.cpp:43:test_led_state_high:PASS
test\test_main.cpp:45:test_led_state_low:PASS
test\test_main.cpp:43:test_led_state_high:PASS
test\test_main.cpp:45:test_led_state_low:PASS
test\test_main.cpp:43:test_led_state_high:PASS
test\test_main.cpp:45:test_led_state_low:PASS

-----------------------
11 Tests 0 Failures 0 Ignored 
OK

Okay, I explicitely added the test port and test speed to platformio.ini and now the test finish.

It does say however that the build has failed, while there is no build error.

So, what was the problem? It seems that PIO picks an invalid test port.

I think PIO indeed picks up an invalid test port.

The other issue remains though: when performing the test, the build says “failed” but it actually builds just fine (hence the passed test).