Dynamic generation of rather complex JSON - error of memory allocation for a string

Good time, dear!

I want to send JSON with readings of some sensors to the server. In addition to the data itself, it contains a time stamp, as well as minimums and maximums for a certain period. In general, a rather large JSON is obtained - more than 1.5Kb.

I do it like this:

Each piece of line is created like this:
image

That is, we allocate memory for the string, then create it using printf, and destroy the parameters created in the previous step (free).
There are a lot of such nested malloc and free. In the end, the last resulting row is also deleted after being sent to the server. And everything seems to work.
But the next command after the last free finds out that memory blocks allocated for completely different tasks (much earlier) are damaged, and then the controller restarts completely with an error

CORRUPT HEAP: multi_heap.c:438 detected at 0x4f676944

It happens here:

As I understand it, the problem is in multiple confusing memory allocations for strings. Because in other cases (without nested calls to malloc_stringf), everything works wonderfully. But I have no idea how else the problem can be solved. Maybe someone came across a similar one?

File: rSensor.zip — Яндекс Диск

In your malloc_stringf() function you call malloc with len+1 but you haven’t initialized or set len to a valid number. Since len is a local variable it will have a not predictable value.

But I would recommend not using malloc() here, because the allocation and freeing could lead into a memory fragmentation and this will lead into a program fault someday… Finding the reason of such a problem is realy annoying.

the len value is determined on the first call to vsprintf with a null buffer, then it returns the size of the buffer for the string.

image

unfortunately, I have no idea how to get by with static variables, otherwise I will have to allocate 100500 buffers of 2 kilobytes each, no stack will be enough. in addition, there are also conditions under which it is necessary to use 100% dynamic memory

Sorry, I missed that vsnprintf call :sweat_smile:

Hm I don’t see an error. You also va_copy the variable argument list so it should be fine for a second vsnprintf call. Maybe memset-ing the ret buffer with all 0 before calling the last vsnprintf makes a difference?

Otherwise I would just add debug-printfs to see at which point the buffer of interest was created or overwritten.

The page Heap Memory Debugging - ESP32 - — ESP-IDF Programming Guide latest documentation also has good tips with HEAP_TRACE_ALL.

vsnprintf is able to return a negative value if a error occurs:

The number of characters written if successful or negative value if an error occurred. If the resulting string gets truncated due to buf_size limit, function returns the total number of characters (not including the terminating null-byte) which would have been written, if the limit was not imposed.

en.cppreference.com/w/c/io/vfprintf

If this happens, len would be large number because of the unsigned datatype. Maybe add a error check just to get sure this is not the problem?

No, for espressiff32 it is not. There, the behavior is changed so that there will never be a negative value. This is specifically done to estimate the buffer size. Specified in the function comments.

It turned out that I forgot to specify * in one function when passing a pointer to a class instance. In the result, not the pointer, but the entire instance itself, hit the stack. Of course, the stack ended unexpectedly. Part of the heap was rubbed along with the stack.
I sprinkle ashes on my head.

1 Like

Tricky – glad you found it!

An addition I yet had to write was that you could use PlatformIOs “Inspect” tool (in PIO Home) to let cppcheck run over the code and tell you what it thinks. Maybe it would have catched that… maybe not.

Okay, thanks, I’ll keep in mind.