Portenta H7 SDRAM errors

Hi there,

I’m trying to use the Portenta H7s for at-edge machine learning inferencing. Due to the size of our model, we cannot fit its tensor arena in the standard SRAM of the M7 core, and as such need to use the 8MB of SDRAM (which is what drew us to use the Portentas). However, we have discovered through our testing that when allocating memory in SDRAM and performing a read/write test on that allocated memory, there are random bytes that cannot be correctly written to.

This is our current methodology: allocate X bytes of SDRAM, loop through each byte, write a constant value (e.g. 0x55), then loop over that array again and check if that byte equals the expected value of 0x55. If that loop sees a value that is not 0x55, it sets a flag indicating there was an error and increments a counter, which is then logged after the test.

We are seeing about 70% success with a 1kB memory allocation in SDRAM where there are no errors in the test, but when we increase the allocation to something like 400kB, there are 100+ errors consistently, and we never get a boot with 0 errors in SDRAM. This is a major problem for us because if there are unwritable bytes in SDRAM allocating the tensors for our model will cause a panic and hard fault.

Here’s a link to our firmware used to test this: https://code.toddr.org/s/y1i1IRT3.

When I use the Arduino IDE using the same exact code, the tests pass with no errors, but since the Arduino IDE is not really appropriate for any non-trivial project I’d really like to use PlatformIO for proper debugging support and such.

Thanks for reading through this

If it works fine in the Arduino IDE, then PlatformIO should be capable of it too, in theory.

Please provide the following info:

  • the ArduinoCore-mbed core version you’re using in the Arduino IDE (Tools->Boards->Board Manager)
  • the value of each setting in the Arduino IDE “Tools” menu
  • the full platformio.ini
  • the output of the “Build” PlatformIO project task

ArduinoCore-mbed version: v4.0.2

Arduino tools menu:

  • Board - Portenta H7
  • Flash split - 1MB m7, 1MB m4
  • Target core - main core
  • Security setting - none
  • Port - dev/cu.usbmodem11301

platformio.ini

; PlatformIO Project Configuration File
;
;   Build options: build flags, source filter
;   Upload options: custom upload port, speed and extra flags
;   Library options: dependencies, extra library storages
;   Advanced options: extra scripting
;
; Please visit documentation for the other options and examples
; https://docs.platformio.org/page/projectconf.html

[env:portenta_h7_m7]
platform = ststm32
board = portenta_h7_m7
framework = arduino
; debug_tool = stlink
; upload_protocol = stlink

(I use an STLink v3 to debug the main project this was going into, but reverted to DFU for simplicity here)

Full output of clean build:

 *  Executing task in folder sdram_raw: platformio run 

Processing portenta_h7_m7 (platform: ststm32; board: portenta_h7_m7; framework: arduino)
----------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/ststm32/portenta_h7_m7.html
PLATFORM: ST STM32 (14.2.0) > Arduino Portenta H7 (M7 core)
HARDWARE: STM32H747XIH6 480MHz, 511.35KB RAM, 768KB Flash
DEBUG: Current (cmsis-dap) External (cmsis-dap, jlink, stlink)
PACKAGES: 
 - framework-arduino-mbed @ 2.4.1 
 - tool-dfuutil @ 1.9.200310 
 - toolchain-gccarmnoneeabi @ 1.70201.0 (7.2.1)
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 29 compatible libraries
Scanning dependencies...
Dependency Graph
|-- Portenta_SDRAM
Building in release mode
Compiling .pio/build/portenta_h7_m7/src/main.cpp.o
Compiling .pio/build/portenta_h7_m7/lib10a/ea_malloc/malloc_freelist.c.o
Compiling .pio/build/portenta_h7_m7/libc4a/Portenta_SDRAM/SDRAM.cpp.o
Compiling .pio/build/portenta_h7_m7/libc4a/Portenta_SDRAM/ram_internal.c.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduinoVariant/variant.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/Interrupts.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/Serial.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/Tone.cpp.o
In file included from /Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/ea_malloc/malloc_freelist.c:6:0:
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/ea_malloc/ll.h:73:0: warning: ignoring #pragma mark  [-Wunknown-pragmas]
 #pragma mark - List Manipulation -
 
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/ea_malloc/ll.h:139:0: warning: ignoring #pragma mark  [-Wunknown-pragmas]
 #pragma mark - Foreach -
 
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/ea_malloc/ll.h:272:0: warning: ignoring #pragma mark  [-Wunknown-pragmas]
 #pragma mark - Init -
 
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/ea_malloc/ll.h:296:0: warning: ignoring #pragma mark  [-Wunknown-pragmas]
 #pragma mark - Add -
 
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/ea_malloc/ll.h:331:0: warning: ignoring #pragma mark  [-Wunknown-pragmas]
 #pragma mark - Delete -
 
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/ea_malloc/malloc_freelist.c:11:0: warning: ignoring #pragma mark  [-Wunknown-pragmas]
 #pragma mark - Definitions -
 
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/ea_malloc/malloc_freelist.c:42:0: warning: ignoring #pragma mark  [-Wunknown-pragmas]
 #pragma mark - Prototypes -
 
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/ea_malloc/malloc_freelist.c:46:0: warning: ignoring #pragma mark  [-Wunknown-pragmas]
 #pragma mark - Declarations -
 
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/ea_malloc/malloc_freelist.c:51:0: warning: ignoring #pragma mark  [-Wunknown-pragmas]
 #pragma mark - Private Functions -
 
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/ea_malloc/malloc_freelist.c:79:0: warning: ignoring #pragma mark  [-Wunknown-pragmas]
 #pragma mark - APIs -
 
Archiving .pio/build/portenta_h7_m7/lib10a/libea_malloc.a
Indexing .pio/build/portenta_h7_m7/lib10a/libea_malloc.a
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/USB/PluggableUSBDevice.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/USB/USBCDC.cpp.o
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/Portenta_SDRAM/SDRAM.cpp: In member function 'int SDRAMClass::begin(uint32_t)':
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/Portenta_SDRAM/SDRAM.cpp:8:65: warning: format '%x' expects argument of type 'unsigned int', but argument 2 has type 'uint32_t {aka long unsigned int}' [-Wformat=]
  printf("FMC_SDRAM_DEVICE->SDCMR: %x\n", FMC_SDRAM_DEVICE->SDCMR);
                                                                 ^
/Users/trylaarsdam/.platformio/packages/framework-arduino-mbed/libraries/Portenta_SDRAM/SDRAM.cpp:28:83: warning: format '%d' expects argument of type 'int', but argument 2 has type 'long unsigned int' [-Wformat=]
   printf("malloc_addblock: allocate %d bytes\n", SDRAM_END_ADDRESS - start_address);
                                                                                   ^
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/USB/USBSerial.cpp.o
Archiving .pio/build/portenta_h7_m7/libc4a/libPortenta_SDRAM.a
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/WMath.cpp.o
Indexing .pio/build/portenta_h7_m7/libc4a/libPortenta_SDRAM.a
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/abi.cpp.o
Archiving .pio/build/portenta_h7_m7/libFrameworkArduinoVariant.a
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/api/Common.cpp.o
Indexing .pio/build/portenta_h7_m7/libFrameworkArduinoVariant.a
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/api/IPAddress.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/api/PluggableUSB.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/api/Print.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/api/Stream.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/api/String.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/arm_hal_random.c.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/as_mbed_library/variant.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/itoa.c.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/main.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/mbed/platform/cxxsupport/mstd_mutex.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/pinToIndex.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/wiring.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/wiring_analog.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/wiring_digital.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/wiring_pulse.cpp.o
Compiling .pio/build/portenta_h7_m7/FrameworkArduino/wiring_shift.cpp.o
Archiving .pio/build/portenta_h7_m7/libFrameworkArduino.a
Indexing .pio/build/portenta_h7_m7/libFrameworkArduino.a
Linking .pio/build/portenta_h7_m7/firmware.elf
Checking size .pio/build/portenta_h7_m7/firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  11.5% (used 60152 bytes from 523624 bytes)
Flash: [=         ]  15.0% (used 117728 bytes from 786432 bytes)
Building .pio/build/portenta_h7_m7/firmware.bin
Adding dfu suffix to firmware.bin
dfu-suffix (dfu-util) 0.9

Copyright 2011-2012 Stefan Schmidt, 2013-2014 Tormod Volden
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

Suffix successfully added to file
=============================================== [SUCCESS] Took 7.12 seconds ===============================================
 *  Terminal will be reused by tasks, press any key to close it. 

So the most recent platform version is 15.6.0 using ArduinoCore-mbed 4.0.2 per releases.

Let’s go for the low hanging fruit first and equalize the used Arduino core version. To do a quick update, open a PlatformIO core CLI and do

pio pkg update -g -p ststm32

and retry uploading the PlatformIO sketch.

Updating the platform version seems to have fixed the issue. I guess that’s what I get for not updating my packages!

Thanks for your help

For safety and reproduction purposes you should also try and pin the platform version in the platformio.ini to ensure everyone else on the project is getting the same results. Per docs

platform = ststm32@15.6.0

to make sure.

1 Like