Arduinostm32 built-in libraries + CMSIS NN RTOS

ta1db · June 9, 2020, 7:38am

Hi,

Default arduinostm32 (stm32duino) built-in libraries are listed here: , however when I initiate a ststm32 project with arduino framework I have only some of them as seen here:

My first question is why these two (arduinostm32 and stm32duino built-ins) are different
Second question is, how can I add the missed stm32duino built-in libraries to framework-arduinostm32 built-ins.
Third question, how can I add stm32duino non-built-in libraries such as CMSIS / NN RTOS as well,

I am asking the third question because when I look to

.platformio/packages/framework-arduinostm32

folder I see a CMSIS folder inside
framework-arduinostm32
and inside of this CMSIS folder there are the libraries I want to use besides Arduino standard libraries and functions
framework-arduinostm32-CMSIS
I can of-course simply include these to my Eclipse project but I don’t want to destroy some other thing while trying to add these new libraries therefore I want to ask and learn the correct way if possible.
For the information of others, libraries folder includes the built-in libraries listed by pio home:
framework-arduinostm32-libraries

maxgerhardt · June 9, 2020, 6:37pm

Where are they different? The screenshot matches the STM32Duino provided library folders 1:1:

(Refer to your initial screenshot)

Which missed ones?

ta1db · June 10, 2020, 12:43am

Hi,

Thanks for your clarification; I was reading the API reference to learn the library functions but it seems that stm32duino / wiki API section is not up-to-date.

OK, I will ask this (a more complete reference to learn the library) and the third question in stm32duino forum but I appreciate if you can make your own comments; I want to use CMSIS NN and RTOS libraries if possible without leaving the stm32duino core.

maxgerhardt · June 10, 2020, 6:53pm

Sadly the Arduino core build script is hardcoded to just use the CMSIS device header (arm_cm3.h for ARM Cortex-M3 CMSIS abstractions) and the DSP library…

https://github.com/stm32duino/Arduino_Core_STM32/blob/master/tools/platformio-build.py#L194-L217

So eiither

A modification to the build script and the Arduino core logic can be done to compile the unused CMSIS stuff
the libraries can be excluded externally as a, well, library. I do remember showing people how to include and configure CMSIS-RTOS2 in STM32Cube project a year ago (GitHub - maxgerhardt/OSProject: Project that uses RTOS on the stm32l476g_discovery board), the same stuff plus the neural network CMSIS file shouldn’t be too hard to externally included and still run Arduino. BUT it will inevitably have side effects, especially the RTOS. The RTOS will need a base timer so that it can schedule task-switches. This may disrupt internally used timers in Arduino-STM32, e.g. for millis() or PWM – creating a PWM output which maps to a timer used by the RTOS will kill it.

I’ll try and spin up a refernce project to give a reference implementation for Arduino STM32 + CMSIS RTOS2 + NN.

maxgerhardt · June 10, 2020, 10:10pm

Huh, CMSIS-RTOS2 integrates rather nicely in the STM32Duino core. They even provided nice systick handler hook functions so that I don’t even have to modify their core. Since it uses the Systick (didn’t find a general timer implementation for STM32 that fits here… only Cortex-A stuff) it also doesn’t mess up the Arduino internals.

Please have a look at GitHub - maxgerhardt/CMSIS-RTOS2-ArduinoSTM32: CMSIS-RTOS2 library integrated with the STM32 Arduino core and PlatformIO. It shows an example platformio.ini for a Nucleo F103RB. You can use the blinky and print example code for testing.

Does that work for your board? Which board are you using?

After that, CMSIS-NN shouldn’t be much of a challenge to also include as a library…

ta1db · June 11, 2020, 6:10am

My flagship is STM32F746G-Discovery board and I want to use its LCD/touch screen with LvGL as a PiO project as well.
CMSIS NN is very important for Machine Learning / AI applications.

Thank you very much for your efforts and for the great work you’re doing.
I have a Nucleo F103RB as well, I will study and test your code on this board first and on the flagship afterwards.

EDIT:
Tested BlinkAndPrint.cpp as main.cpp on both nucleo_f103rb and disco_f746ng using Eclipse CDT, symptoms are same on both targets:

Build process finished with 0 errors, 2 warnings,

But the LED blinks and serial port prints “The time is : [some number like 400000]” as expected.

In debug mode build process finish with only 1 warning, with the second one

|implicit declaration of function '

in handlers.c, program starts normally with F8, led blinks but if I try to suspend the execution interrupt fails.

I guess I have to use your BlinkAndPrint.cpp as it is, and call it from the main.cpp existing in the Arduino core ?

A Question:
I would like to combine LvGL GUI Library ported to STM32F746G-DISCO to this project for GUI development; do you see a possible conflict?

maxgerhardt · June 11, 2020, 1:38pm

Ha, than it actually works

I’ve also observed this warning but the compiler is actually wrong about it / the intent of the programmer is correct. The kernel code wants to set the structure to zero starting at the offset of kernel but extending beyond it. This is perfectly fine.

Also observed that one but I think that’s fixable.

You mean the debugger cannot halt the MCU anymore? Any error message?

I don’t particularily see why it shouldn’t be acomplishable – reference project seems to be using the STM32HAL which STM32Duino also provides….

I think I’ll be working on getting rid of the warnings in the CMSIS library and even making it arduino compatible, then CMSIS-NN.

ta1db · June 11, 2020, 1:52pm

Yes, exactly. Just a pop up “IDE Error” window saying “Interrupt failed”. It is interesting that when I start the debug session another main.cpp from cores/arduino folder opens up in the window besides to the src/main.cpp

This is great. Thanks.

maxgerhardt · June 11, 2020, 7:31pm

At the start of the debugging session it automatically halts in the main() function of the firmware, which is in the main.cpp of the STM32Duino core. That’s normal. After continuing it should hit your breakpoints. I have not observed this error when debugging my STM32F103RB target…

CMSIS-RTOS2 is now Arduino-IDE compatible!

I think there’s a memory leak somewhere though that I have to debug before starting a pull-request into STM32Duino. E.g., the Blinky example just stops printing at some point. Stack overflow / out of mem / something not freed? Bonus points if you find it before I do. After that the library can just be copied into Arduino_Core_STM32/libraries at main · stm32duino/Arduino_Core_STM32 · GitHub and is available to all STM32Duino users.

Also the CMSIS-NN Arduino library is done – STM32Duino already exposes the CMSIS DSP library which CMSIS-NN depends on. Just putting all source and header files into src/ and adding some declerative library.properties / library.json makes the library usable. An example can be found at GitHub - maxgerhardt/CMSIS-NN-ArduinoSTM32: CMSIS-NN as a Arduino-compatible library. – simple functions like a ReLU on a constant input vector works for my F103RB, larger examples like the CIFAR10 are meant for better M4 and M7 cores, so I couldn’t test it yet, but it compiles.

ta1db · June 12, 2020, 4:54am

This is going to be an excellent work and I am happy to be a part of it.

In my trial I realized that the time value printed on the serial port started as “The time is: 40000” and gradually reached to “The time is: 200000” and was increasing further, I think this was an indication of a leak. I’ll try to get the bonus

Not only myself, all the community will thank to your contribution to embedded machine learning / artificial intelligence works

EDIT
Tested WithRTOS.ino as main.cpp on stm32f746G-discovery board using Eclipse CDT it works as expected; the LED blinks and serial port prints Input vector and Transformed Vector regularly. There was 3 warnings

|dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]|pio-f746-arduino||line 425, external location: C:\users\murat\.platformio\packages\framework-arduinoststm32\CMSIS\CMSIS\DSP\Include\arm_math.h|C/C++ Problem|

+100 infos (several, I can send detailed info if you ask)

in build process, just 1 (same) 'implicit declaration of function ’ warning in debug session. PlatformIO Debugger on Eclipse CDT starts mormally but interrupt fails on “Suspend” as I’ve told previously. Anyhow good job, God bless you!..

BTW. CMSIS Library Reference and Arduino Software Language Reference are the primary guides, however Can you recommend any additional application note, blog, and/or book with examples to follow up and make experiments with your CMSIS DSP-NN RTOS2 libraries. First I want to make a simple pattern generator with impulse, sine, sawtooth, square wave outputs… after that I want to make an oscilloscope with spectrum display, all with SM32F7-Disco. I plan to use LvGL library for GUI development. On the other hand I plan to make experiments on predictive maintenance ML/AI IoT (+LoRa LoRa WAN) applications and on Software Defined Radio using your CMSIS DSP-NN libraries again.
I see a similar work (pio-cmsis-nn) of you started 16 months ago with mbed framework; may be you update this work with recent arduinostm32 framework.

maxgerhardt · June 12, 2020, 9:37pm

You are still using an old version of the library – PlatformIO does not update the libraries it has once downloaded in .pio\libdeps\<environment> without you deleting it (or running pio lib update). Once you delete .pio\libdeps in the project folder the RTOS wranings should be gone.

Those come directly from the used CMSIS DSP library included in STM32Duino – surely the compiler has some right to complain about that, but I can’t momentarily change that . Maybe a pull request can be done to STM32Duino to update to the latest version of the library which doesn’t exhibit these warnings anymore.

For me it’s extermely deterministic on a F103RB on multiple runs, after some exact time (14.93 minutes) it crashes the exact same way where it spews out null bytes and 0xff bytes.

The time is: 895000<\r><\n>
The time is: 896000<\r><\n>
The timeL<1><\0> ?<1><\0> T<\0><\0><\r><\n>
T<\0><\0><\0>???? is: 898000<\r><\n>
<\0><\0><\0><\0><\0><\0><\0><\0><\0><\0><\0>00<\r><\n>

Not yet sure why but investigating.

The RTOS2 comes with some example files to test out the functionality: CMSIS_5/CMSIS/RTOS2/RTX/Template at develop · ARM-software/CMSIS_5 · GitHub

Same for the CMSIS-DSP: https://github.com/ARM-software/CMSIS_5/tree/develop/CMSIS/DSP/Examples/ARM

Includes samples for convolution, finite-impulse-response filters, fourier transform, signal generation with sin/cos etc.

Out of my head I can’t think of a good other blog / project source which explicitly is about the RTOS or DSP… well despite oscilliscope projects maybe for the DSP. RTOS is used pretty universally and always taken for granted.

For the neural network part I think I’m expanding the CIFAR10 demo to one based completely on STM32Duino with a bigger STM32F4 board, a OV7670 camera for input, an LCD for output of the camera picture plus the categorized output of the CIFAR10 neural network, which is able to differentiate 10 different classes.

ta1db · June 13, 2020, 8:51am

I am using the old version on purpose; when I updated pio in past weeks I started getting project initialization errors related to platform version requirements, on my mac osx machine I could resolve this simply by updating the gcc-arm-none-eabi toolchain to a 2020 version however on my windows 7 machine I couldn’t resolve this issue and I deleted everything and installed pio from scratch using Installer Script Method this time, previously I didn’t care the Warning Here . My pio version is the version which is what the Installer Script installed. I am afraiding to touch it but I will try updating.

I will really appreciate this work too, same as how I do for your other efforts and help.

BTW can you recommend any data set for fast fourier transformed images of audio and electrical signal recordings to distinguish mechanical and electrical hazards for predictive maintenance applications?

maxgerhardt · June 17, 2020, 8:06pm

It’s just about the library version here, not PIO – your PIO version can stay the same. Just removing the .pio\libdeps\ folder should be enough to trigger a re-download of the git repository.

I have also identified the issue that makes it crash after a while – the way STM32Duino calls the osSystickHandler() as a C function with the expectation of it returning to the caller is not given in the CMSIS-RTOS2 IRQ file implemention of that function. The STM32Duino’s push {r3, lr} is not cleared by a pop in some cases, leading to the stack growing from bottom to top until it crashes with the heap growing downards and other important memory. Fix stack overflow · Issue #1 · maxgerhardt/CMSIS-RTOS2-ArduinoSTM32 · GitHub is tracking this.

Indeed I think getting more people in AI, machine learning and neural networks with easily-accessible frameworks (Arduino) and libraries for CPU cores which are well capable of it (STM32) should be a good and interesting thing

Interesting thoughts, I don’t have any experience on predictive maintence though. Maybe an open-source example would also be good here. FFT example stuff is however included in the CMSIS-DSP libs.

ta1db · June 22, 2020, 2:44pm

Hi @maxgerhardt

I am sad because I use platformIo on Eclipse and do not want to switch to Arduino IDE; after your making the libraries Arduino IDE compatible I started getting error messages, I returned to my starting point again.

Is it not possible to retain paltaformio compatibility as well; what do you recommend ?

maxgerhardt · June 23, 2020, 10:30pm

PlatformIO compatibility is unchanged. Eclipse works flawlessly for me

What’s the error? Project is cleaned an .pio\libdeps folder deleted, then the project is rebuilt?

ta1db · June 24, 2020, 6:05am

I’m very glad to hear that! perhaps I’m doing a mistake; today I’ll try again carefully and let you know. Thanks.