Making Neuton.ai tinyML working on an arduino nano 33 BLE bloard

charlie.gallopin · July 24, 2023, 4:19pm

Hello everyone !

I am currently trying the make a TinyML using Neuton.ai to work on an Arduino board.

I Think I have 2 issues.
1. Including the libraries properly.
2. Using the right flags to compile it.

Here is the folder structure.

Compiling .pio\build\nano33ble\FrameworkArduino\wiring_digital.cpp.o
In file included from lib\neuton\include/neuton/neuton.h:15,
                 from src\main.cpp:4:
lib\neuton\include/neuton/nn/neuton_nn.h:13:10: fatal error: neuton_generated/neuton_user_types.h: No such file or directory
 #include <neuton_generated/neuton_user_types.h>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
*** [.pio\build\nano33ble\src\main.cpp.o] Error 1

Here is the code in neuton_nn.h

/**
 *
 * @defgroup neuton_nn Neural Network
 * @{
 * @ingroup neuton
 *
 * @brief
 *
 */
#ifndef _NEUTON_NN_H_
#define _NEUTON_NN_H_

#include <neuton_generated/neuton_user_types.h>
#include <neuton/neuton_types.h>

#ifdef   __cplusplus
extern "C"
{
#endif

main.cpp

#include <Arduino.h>
#include <Arduino_LSM9DS1.h>
#include <CircularBuffer.h>
#include <neuton/neuton.h>

void setup()
{
    Serial.begin(115200);
    Serial.println("Started");

    if (!IMU.begin())
    {
        Serial.println("Failed to initialize IMU!");
        while (1)
            ;
    }
    Serial.println("setup");

    /** Setup Neuton */
    neuton_nn_setup();
}

void loop()
{
}

In the setup() I am calling neuton_nn_setup(); whichis define in libneuton_arm_cm4.a

Here is platformio.ini

[env:nano33ble]
board = nano33ble
framework = arduino
monitor_speed = 115200
platform = nordicnrf52@9.5.0 
lib_deps = 
	arduino-libraries/Arduino_LSM9DS1@^1.1.1
	rlogiacco/CircularBuffer@^1.3.3

lib_ldf_mode = chain+

build_unflags = 
	-mfloat-abi=softfp
	-Os
build_flags =
	-mfloat-abi=hard
	-Ilib/neuton/include
	-Ilib/neuton_generated

I successfully the only way of making it working was to modify neuton_nn.h to use relative path to the neuton_user_types.h file.

What is the correct way, without having to modify the library download from neuton.ai ?

Second issue :
Of no libraries issues, i get tones of errors like this

Archiving .pio\build\nano33ble\libFrameworkArduino.a
Indexing .pio\build\nano33ble\libFrameworkArduino.a
Linking .pio\build\nano33ble\firmware.elf
c:/users/cg/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.181220/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: error: .pio\build\nano33ble\src\main.cpp.o uses VFP register arguments, .pio\build\nano33ble\firmware.elf does not
c:/users/cg/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.181220/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: failed to merge target specific data of file .pio\build\nano33ble\src\main.cpp.o
c:/users/cg/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.181220/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: error: .pio\build\nano33ble\libf9b\libWire.a(Wire.cpp.o) uses VFP register arguments, .pio\build\nano33ble\firmware.elf does not
c:/users/cg/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.181220/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: failed to merge target specific data of file .pio\build\nano33ble\libf9b\libWire.a(Wire.cpp.o)
c:/users/cg/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.181220/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: error: .pio\build\nano33ble\lib2c1\libArduino_LSM9DS1.a(LSM9DS1.cpp.o) uses VFP register arguments, .pio\build\nano33ble\firmware.elf does not
c:/users/cg/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.181220/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: failed to merge target specific data of file .pio\build\nano33ble\lib2c1\libArduino_LSM9DS1.a(LSM9DS1.cpp.o)
c:/users/cg/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.181220/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: error: .pio\build\nano33ble\libFrameworkArduinoVariant.a(variant.cpp.o) uses VFP register arguments, .pio\build\nano33ble\firmware.elf does not
c:/users/cg/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.181220/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: failed to merge target specific data of file .pio\build\nano33ble\libFrameworkArduinoVariant.a(variant.cpp.o)

So i tried to force -mfloat-abi=hard and disable -mfloat-abi=softfp in the platformio.ini but the error still persist.

Can you please help me ?

maxgerhardt · July 24, 2023, 4:27pm

Better give it a libary.json manifest in which you can nicely specify the build flags and thus the include paths.

This is indeed the issue, but you’ll have to adapt the linker flags too and that can only be done via scripting, see Changing -mfloat-abi build option not working - #2 by maxgerhardt and Unable to solve error ".o uses VFP register arguments, .elf does not" - #6 by maxgerhardt

charlie.gallopin · July 25, 2023, 8:33am

I am pleasantly surprised by the reactivity, thank you

I have some errors :

src\main.cpp:4:10: fatal error: neuton/neuton.h: No such file or directory
 #include <neuton/neuton.h>
          ^~~~~~~~~~~~~~~~~
compilation terminated.
*** [.pio\build\nano33ble\src\main.cpp.o] Error 1

main.cpp

#include <Arduino.h>
#include <Arduino_LSM9DS1.h>
#include <CircularBuffer.h>
#include <neuton/neuton.h>

library.json for neuton lib

{
  "name": "neuton",
  "version": "0.0.0+20230724142225",
  "keyword": "neuton, ai, ML, tinyML",
  "description": "Generated fron neuton.ai website",
  "platforms": "*",
  "build": {
    "scrDir": ".",
    "includeDir": "./include/neuton"
  }
}

library.json for neuton_generated lib

{
  "name": "neuton_generated",
  "version": "0.0.0+20230724142225",
  "keyword": "neuton, ai, ML, tinyML",
  "description": "Generated fron neuton.ai website",
  "platforms": "*",
  "build": {
    "scrDir": ".",
    "includeDir": "."
  }
}

folder structure

any hint ?

maxgerhardt · July 25, 2023, 8:54am

Well if the library is included as #include <neuton/neuton.h> and you have the folder structure

Then the icnlude dir is just include. Which is already the default.

charlie.gallopin · July 25, 2023, 9:25am

I now have this error

In file included from lib\neuton\include/neuton/neuton.h:15,
                 from src\main.cpp:4:
lib\neuton\include/neuton/nn/neuton_nn.h:13:10: fatal error: neuton_generated/neuton_user_types.h: No such file or directory
 #include <neuton_generated/neuton_user_types.h>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
*** [.pio\build\nano33ble\src\main.cpp.o] Error 1

tried with this in library.json of neuton_generated lib

{
  "name": "neuton_generated",
  "version": "0.0.0+20230724142225",
  "keyword": "neuton, ai, ML, tinyML",
  "description": "Generated fron neuton.ai website",
  "platforms": "*",
  "build": {
    "scrDir": ".",
    "includeDir": "."
  }
}

What is the best solution ? without having to modify the source code, or the folder strucutre ?

maxgerhardt · July 25, 2023, 10:43am

I think you should remove lib/neuton_generated/library.json and just move lib/neuton_generated → lib/neuton/neuton_generated and then add "build": { "flags": ["-I...."] } into the library.json (docs) of lib_neuton that do -Iinclude/neuton/ so that neuton_generated/neuton_user_types can be found.

charlie.gallopin · July 25, 2023, 6:17pm

The library is correctly implemented now ! Thank you.
I am now facing the VFP register issue… there is tone of them.

c:/users/cg/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.181220/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: error: .pio\build\nano33ble\firmware.elf uses VFP register arguments, C:\Users\cg\.platformio\packages\framework-arduino-mbed@3.5.4\variants\ARDUINO_NANO33BLE\libs\libmbed.a(RTX_Config.o) does not
c:/users/cg/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.181220/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: failed to merge target specific data of file C:\Users\cg\.platformio\packages\framework-arduino-mbed@3.5.4\variants\ARDUINO_NANO33BLE\libs\libmbed.a(RTX_Config.o)

I have search the web for solutions realted to this issue. The solution is to find a version precompiled with -mfloat-abi=hard

I don’t understand where to start this issue.

Is it platform or framework related ?

Could I compile this myself ? maybe the sources are on a github ?
This is tricky ^^
Thank you in advance for your help

charlie.gallopin · July 31, 2023, 1:57pm

So it seems like the arduino mbed os has been compiled with FPU disabled on this platform.
I successfully recompiled mbeb_os with FPU enabled, but at the end, I still have errors with libcc_310_core.a and other libcc_310 file realted. I couldn’t find the sources files to recompile it, and at this point it is really to complexe.

I’ll use another platform. the Teensy4.0 has arduino with FPU enabled by default on it’s cortex MCU.

But still, i have one last issue.

c:/users/cg/.platformio/packages/toolchain-gccarmnoneeabi@1.80201.181220/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: .pio\build\nano33ble\src\main.cpp.o: in function `setup':
C:\Users\cg\Documents\PlatformIO\Projects\Arduino Nano BLE NeutonAI/src/main.cpp:33: undefined reference to `neuton_nn_setup'

neuton_nn_setup is supposed to be defined in libneuton_arm_cm4.a located in ./lib/neuton/lib/

Am I doing something wrong ?

maxgerhardt · July 31, 2023, 1:59pm

Missing some

extern "C" {
#include <libneuton.h>
}

construct? You’re intermixing C++ and C code possibly.

charlie.gallopin · July 31, 2023, 2:59pm

Thank you
Yes the library is written in C.

I finally made it work by including the library with a flag.

{
  "name": "neuton",
  "version": "0.0.0+20230724142225",
  "keyword": "neuton, ai, ML, tinyML",
  "description": "Generated fron neuton.ai website",
  "platforms": "*",
  "build": {
    "scrDir": ".",
    "flags": ["-Iinclude/neuton","-Ineuton_generated","-Ilib","-Llib","-lneuton_arm_cm4"] 
  }
}

I it the right way ?

maxgerhardt · July 31, 2023, 3:02pm

If the only target is cortex_m4, sure. Otherwise you would have to do an “extraScript”, read the current architecture and link the right library. Should be fine with only Nano 33 BLE in mind.