Memory management

Hi all!

I keep trying to work with ESP32 till now, but have some issues.

I have the sketch using a telegram bot, and sanding http & https posts, but every 14+/- days I have the issue:

  • [E][ssl_client.cpp:33] _handle_error(): [start_ssl_client():190]: (-32512) SSL - Memory allocation failed
  • [E][WiFiClientSecure.cpp:132] connect(): start_ssl_client: -32512

The ESP32 keeps works but all things with HTTP & HTTPS are broken.

Firstly I didn’t configure the platformio.ini file and after compiling code I had very small Flash residue:

RAM:   [=         ]  13.3% (used 43512 bytes from 327680 bytes)
Flash: [=======   ]  74.8% (used 979794 bytes from 1310720 bytes)

it was strange, and I googled that used Flash could be expanded by:
board_build.partitions = huge_app.csv

Now it looks better:

RAM:   [=         ]  13.3% (used 43512 bytes from 327680 bytes)
Flash: [===       ]  31.1% (used 978742 bytes from 3145728 bytes)

But I’m not sure that “Memory allocation failed” appeared due to the not enough Flash space, so I decided to ask it here instead of waiting two weeks:

  • will the increase of Flash solve the problem of memory allocation?

  • is any way to increase RAM?

  • will the increase of Flash interfere with data saved by:

    #include <Preferences.h>
    Preferences eeprom;
    eeprom.putLong(“name1”, val1);

Thanks in advance!

No, the SSL client tries to allocate its working memory on the heap, which lives in RAM. Flash is for constant, unchanging data – a firmware ususally never writes into Flash during runtime (needs special re-programming routine anyways), except in the cases of Flash-based filesystems or non-volatile storage where they have to store the data in persistent memory.

For the ESP32 yes, you can buy an ESP32 module which has a built-in PSRAM (pseudo-static RAM) chip extension, such as a ESP32-WROVER-B (adafruit, amazon).

The Preferences library uses a the NVS (non-volatile storage) partition of the flash. It is a pre-allocated section of flash as determined by the partition table (default.csv). The application lives in a separate partition than NVS, so by writing things to NVS, you do not decrease the amount of Flash available to the application by the number of bytes written to the NVS. You do however cause a slight increase in flash usage since you use the Preferences library, and its program code will be stored in Flash.

If you use this different partition table for your application, it shifts the available space for the application, but it does not solve the problem that you’ve run out of RAM.

This behavior is extremely indicative of a memory leak. You can add as much RAM as you physically can, it will just delay the moment in time that this problem occurs. The source of your problem is your code.

It is extremely critical that you clean-up / free resources after using them. If e.g. an object allocates memory using the new opperator or malloc(), but after it’s done using it (or when the object is destroyed) it does not delete or free() the memory again, that chunk of memory will be forever locked up as “used” although no one can possible use the memory again. These errors accumulate over time, say if for every telegram message a connection is made, which dynamically allocates 20 kBytes of memory, but the client is never stopped so it never gives back that 20 kBytes, then you’re losing 20 kByte of dynamic memory in every message. At ~320 kBytes of RAM, a few connection attempts more and your ESP32 is seemingly “out of memory” because of this 20kbyte per message memory leak.

For a SSL/TLS connection with the WiFiClientSecure library it’s imperative that you call the .stop() function on it so that it releases its internal resources.

Depending on what library you’re using, you might either be forgetting to clean up its resources by forgetting to call a function in there (or using the objects of the library in a completely wrong way), or the library itself might have a bug. Check the library’s github page for possible known issues.

You can verify that you have a memory leak by monitoring the amount of free RAM on the heap. Just have your sketch periodically print or send the value of ESP.getFreeHeap() (ESP32 Arduino: Getting the Free Heap - techtutorialsx).

If you see a strictly decreasing curve over time, your system has a memory leak and is slowly starving out memory.

If you can accelerate operation of your firmware (e.g., sending more messages per second or triggering the bot periodically), you should be able to starve it faster / see an effect faster.

1 Like

Dear maxgerhardt a lot of thanks for your comprehensive answer!

It really helps me to understand what is going on inside of ESP32 memory.
Now I detect ESP.getFreeHeap(), ESP.getMaxAllocHeap(), ESP.getMinFreeHeap() each hours and see that MaxAllocHeap constantly decreases.

day FreeHeap()	MaxAllocHeap() MinFreeHeap()
1)	232404	113792	155100
2)	231068	113336	153548
2)	221432	106084	142232
2)	219380	106084	140648

Have to find the way of leak memory detection?

Seems I’ve already found the reason for the memory leak.
The script below shows how even a little string formatted with *storePrintf() function eats all RAM

void setup() {
Serial.begin(115200);
}

String random_string(std::size_t length) {
String CHARACTERS = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
String random_string;
for (int i = 0; i < length; ++i) {
    random_string += CHARACTERS[random(0, CHARACTERS.length())];
}
return random_string;
}

char *storePrintf(const char *fmt, ...) {
va_list arg;
va_start(arg, fmt);
size_t sz = snprintf(NULL, 0, fmt, arg);
char *buf = (char *) malloc(sz + 1);
vsprintf(buf, fmt, arg);

va_end(arg);
return buf;
}

void loop() {
int n = 0;
while (n < 100000) {

    //String tmp0 = storePrintf("str %d,", n); // ---->  the reason of MEMORY LEAK

    String tmp = random_string(random(500, 1800));
    Serial.printf("$%d %d %d;", tmp.length(), ESP.getFreeHeap(), ESP.getMaxAllocHeap());
    n++;

    delay(200);
}
delay(5000);
}

when code **String tmp0 = storePrintf(“str %d,”, n); ** ubcommented it shows next:

but commenting this string solve the problem:

I googled that after all calls to malloc() must be a call to free() resources. But I haven’t an idea where I should do this.

Great, you’ve pin-pointed the exact point of the memory leak! I’m however unsure of the entire urpose of the storePrintf() function… since it just calls printf in a really more complicated way, why is storePrintf not equivalent to just printf()? If it does not have additional logic besides printing, the function implementation should be removed and there should just be a macro

#define storePrintf printf

instead of the function declaration char *storePrintf(const char *fmt, ...);.

Otherwise, you should add a

free(buf);

after the va_end() in the function.

1 Like

Dear maxgerhardt,
I used to use this function for string formatting, e.g for Telegram bot,

    String responce = storePrintf("Titration finished. KH = <b>%.2f</b>\nFree Heap() = %i(max free %i, min %i)", KH * KH_index, 
             ESP.getFreeHeap(),
             ESP.getMaxAllocHeap(),
             ESP.getMinFreeHeap()
             );
            Serial.println(responce);
     bot.sendMessage(chat_id, responce , "html");

When I try to free(buf); I gets error:
image

so, I found that the next function, working well without memory leaking

template<typename... Args>
String string_sprintf(const char *format, Args... args) {
    int length = snprintf(nullptr, 0, format, args...);
    assert(length >= 0);

    char *buf = new char[length + 1];
    snprintf(buf, length + 1, format, args...);

    std::string str(buf);
    delete[] buf;
    return str.c_str();
}

Thanks for your help!

1 Like

Oh oh sorry I have to rethink that – there are multiple things wrong with that

Since the variable is returned no way you can deallocate it. However, when the function returns a char* and the left side of the expression is a String, that should have… never worked.

The tricky piont here is that you dynamically allocate a string of the required size and put it into a String object, and so when the String object is destroyed it must free that initially malloced memory. The problem is that whatever const char* you give into a String object it always wants to do make an internal copy of it which is again mallocd (see sourcecode) – you can’t initialize a String object from a verbatim pointer that it will again free in the destructor.

You should also be able to change it to

String storePrintf(const char *fmt, ...) {
va_list arg;
va_start(arg, fmt);
size_t sz = snprintf(NULL, 0, fmt, arg);
char *buf = (char *) malloc(sz + 1);
vsprintf(buf, fmt, arg);
va_end(arg);
// copy into string object
String s(buf);
//now the buf content have been duplicated inside the String, destroy it
free(buf);
return s;
}

although your std::string collection also looks valid if you’ve validated it…

With the return types its still extremely funny though, because you return a const char*, that was created from a std::string, which is somehow auto-converted into a String, although all these 3 things are different. But okay.

1 Like
String storePrintf(const char *fmt, ...) {
va_list arg;
va_start(arg, fmt);
size_t sz = snprintf(NULL, 0, fmt, arg);
char *buf = (char *) malloc(sz + 1);
vsprintf(buf, fmt, arg);
va_end(arg);
// copy into string object
String s(buf);
//now the buf content have been duplicated inside the String, destroy it
free(buf);
return s;
}

this code, unfortunately, leads to error:

23:20:51.580 -> rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
23:20:51.580 -> configsip: 0, SPIWP:0xee
23:20:51.580 -> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
23:20:51.580 -> mode:DIO, clock div:2
23:20:51.580 -> load:0x3fff0018,len:4
23:20:51.580 -> load:0x3fff001c,len:1044
23:20:51.580 -> load:0x40078000,len:10124
23:20:51.626 -> load:0x40080400,len:5828
23:20:51.626 -> entry 0x400806a8
23:20:51.766 -> CORRUPT HEAP: multi_heap.c:431 detected at 0x3ffb8470
23:20:51.766 -> abort() was called at PC 0x40088c74 on core 1
23:20:51.766 -> 
23:20:51.766 -> ELF file SHA256: 0000000000000000
23:20:51.814 -> 
23:20:51.814 -> Backtrace: 0x40084ea0:0x3ffb1d70 0x40085115:0x3ffb1d90 0x40088c74:0x3ffb1db0 0x40089348:0x3ffb1dd0 0x4008149d:0x3ffb1df0 0x40081549:0x3ffb1e10 0x40081615:0x3ffb1e30 0x400831ed:0x3ffb1e50 0x4000bedd:0x3ffb1e70 0x400d152b:0x3ffb1e90 0x400d162c:0x3ffb1ec0 0x400d1641:0x3ffb1ee0 0x400d1683:0x3ffb1f00 0x400d0d4d:0x3ffb1f20 0x400d0e5a:0x3ffb1f70 0x400d22ed:0x3ffb1fb0 0x40086125:0x3ffb1fd0
23:20:51.814 -> 
23:20:51.814 -> Rebooting...
23:20:51.814 -> ets Jun  8 2016 00:22:57

I’ve been testing for 6 hours the variant proposed in the previous answer and seems it works well…
I will let know the long-testing results.

also, I’ve already try return char, without .c_str(), but it lead to the error:

Probably because there’s a String constructor that works with const char* but not std::string. If you have verified working code, it should be it good though.

1 Like

@igorlab
Which plot tool did you use? is it within vscode?