Teensy unit testing doesn't execute

Advanced Solutions Unit Testing
1

I am trying to get unit testing to work with my teensy 3.1 (programmed with a Segger J-Link over SWD for debugging purposes).

I can compile, upload and debug code just fine, and I can compile unit tests, and apparently successfully upload the binary, but the debugging gets stuck at the Testing… phase

=================== [test/test_embedded > teensy31] Testing… (3/3) ===================
If you don’t see any output for the first 10 secs, please reset board (press reset button)

After I see this in the console, I can reset my board and nothing happens. Not sure how to troubleshoot this. I am using the latest PIO (4.0.0 something, just ran pio upgrade --dev)

I also was trying to get Native testing to work with this teensy project, but I can’t figure out how to fake the libraries to work for Teensyduino. Any tips there would be very helpful too :slight_smile:

2

What exact code are you running for the unit tests?

3

I am just using a single test case which I assume should pass (32==32), just to see if I can get testing to work in the first place

#include "Arduino.h"
#include <unity.h>

void test_function_quantize_edo_pitch(void)
{
    TEST_ASSERT_EQUAL(32, 32);
}

int main(int argc, char **argv)
{
    UNITY_BEGIN();
    RUN_TEST(test_function_quantize_edo_pitch);
    UNITY_END();
}

and this is what is in my platform.ini file:

[env:teensy31]
platform=teensy
framework=arduino
upload_protocol = jlink
debug_tool = jlink
board=teensy31
board_build.f_cpu=120000000L
build_flags= -UUSB_SERIAL -DUSB_MIDI_SERIAL -lstdc++
test_ignore = test_desktop 

[env:native]
platform = native
test_ignore = test_embedded
build_flags = -std=gnu++11

[platformio]
src_dir=./src
4

It’s kind of weird to overwrite the main() function and not use setup() and loop(). Can you try the code from Unit Testing of a “Blink” Project — PlatformIO latest documentation?

5

I based this on one of the platformio examples: platformio-examples/test_calculator.cpp at develop · platformio/platformio-examples · GitHub

The arduino-mock example also overwrites main: platformio-examples/test_my_service.cpp at develop · platformio/platformio-examples · GitHub

but, regardless, I putting everything in setup, and still no dice.

6

And if you use the exact same code as linked with a standard platformio.ini


[env:teensy31]
platform = teensy
framework = arduino
board = teensy31

? Also I have the feeling the delay(2000); was put in because the Teensy doesn’t auto-reset on port or spits out the results too fast? What happens when you just open miniterm.py (or any other serial monitor) at 9600 or 115200 baud and push the reset button?

7

I had been monitoring the serial on miniterm already, and nothing came out until I added delay(2000). I guess it took a second for the Serial to initialize, so it was probably running the tests, I just couldn’t see any of them!

It still does not pop out in the platformIO terminal that is triggering the tests, but I bet its because I am using a Segger Jlink, and its trying to listen for serial through semihosting or something by default.
I think i need to change this option to get that to work, time to experiment! Redirecting...

8

Might be related to Teensy 3.1 correct serial initialisation | Teensy Forum, i.e. maybe a while(!Serial) {} in setup() and/or a delay works.

9

So I think what is happening is that I need to change the test_port, but I am getting an error when I try to do that:

I see the device in my device list:

pio device list                                                                                                                                                                                                
/dev/cu.usbmodem42949672951
---------------------------
Hardware ID: USB VID:PID=16C0:0489 SER=4294967295 LOCATION=0-4
Description: Teensy MIDI

but when i set:

test_port = /dev/tty.usbmodem42949672951

I get an error immediately after compilation is finished:

================ [test/test_embedded > teensy31] Testing… (3/3) ================
If you don’t see any output for the first 10 secs, please reset board (press reset button)

[Errno 2] could not open port /dev/tty.usbmodem42949672951: [Errno 2] No such file or directory: ‘/dev/tty.usbmodem42949672951’

I think what might be happening is that the serial port disappears for a second while the Teensy is being programmed, and PIO is looking for the serial port and failing immediately when it can’t find it. Is there a way to adjust the timeout here while it waits for the port to be available? I can view this port fine on minicom and view the results there, but PIO doesn’t see it. I tried the above while(!Serial) technique too, and that did not change the results.

update: same thing happens when I set the monitor_port to the same port. If I do upload and monitor, it will fail to monitor the port, but if I just choose monitor, where the device does not reboot, it will monitor fine.

10

yea, I think the bug is happening here: platformio-core/embedded.py at 61db0f1d6a89a621281b12e1a913635905d0e035 · platformio/platformio-core · GitHub

The python Serial object is erroring out immediately, instead of waiting for the timeout period of 600 seconds. I believe this happens because the serial object disappears, and the timeout is probably a timeout for connection. I think if some extra code was injected here to try connecting a few times before erroring out, this would fix this problem.

11

I think the serial modem identification just changes when you reboot and it activates its USB Serial? What are the two /dev/tty* you see from this device?

12

the USB Serial identification stays the same over reboots. I have rebooted my computer a few times in the past day and the serial device name stays the same:

/dev/cu.usbmodem42949672951

I have tried both using cu. and tty. in the test_port variable to no avail.
I believe this is happening because I am using a Segger J-link on a Teensy and getting teensy serial information from the USB port on the teensy (this is the port that is listed above). When the device is programmed with the Segger J-link, the USB is taken offline while the teensy is reprogrammed, and the /dev/tty device momentarily disappears during programming, but reappears shortly after. In this moment, the pio test command errors out, expecting the Serial device to be online, instead of waiting and trying again a few seconds later.

If I use minicom to continuously poll the port, I can see when the device goes offline, and then it reappears a second later and I can read the test results fine.

I believe there needs to be a test_port_timeout var that lets you adjust a timeout before failure when serial cannot be accessed immediately after programming.

13

You might want to look at bit closer at the examples you were linking… they’re not for embedded processors like the teensy! :open_mouth: :wink:

ie. calculator/test/test_desktop/test_calculator.cpp was for unit testing in the the native desktop environment… hence why it uses main(). The arduino-mock example was also for the native environment. The version for embedded processors, calculator/test/test_embedded/test_calculator.cpp uses setup() and loop()… and setup has a 2 second delay at the start…

// NOTE!!! Wait for >2 secs
// if board doesn't support software reset via Serial.DTR/RTS
delay(2000);