ESP32, RMT ws2812 and WIFI

I ran into the same problem with different RMT libraries, it works fine until you start the WIFI, then the LEDs flicker, display the wrong colors, and change without anything written to the RMT. I found many similar problems on the internet but not yet a solution. Any Ideas or suggestions on where I can find a working library? Currently, I just
#include “esp32-hal.h”,
and set the
#define NR_OF_ALL_BITS 24 * NUM_LEDS
#define BUILTIN_RGBLED_PIN GPIO_WS2812_PIN

rmt_data_t led_data [NR_OF_ALL_BITS] {0};
rmt_obj_t * rmt_send {nullptr};
Manipulating the structures and sending them to the RMT, not using a library.

It works perfectly for many hours, but if you enable WIFI, all go bananas.