No, why? Without an SVD files you don’t get the pretty list of the peripheral register values. Debugging still works perfectly normal with variable inspection, line-by-line stepping etc. This is a purely an addition to normal debugging.
This is an issue on Espressif’s side since they don’t provide one. See Provide *.SVD for use with editors (IDFGH-299) · Issue #2214 · espressif/esp-idf · GitHub and Fix detection of some registers · Issue #6 · MabezDev/idf2svd · GitHub.
There are currently community-made converter tools written in Rust to convert Espressif’s header files into SVD files. See GitHub - MabezDev/idf2svd. There also seems to be a more corrected version with a base file + patches at GitHub - esp-rs/esp32: Peripheral access crate for the ESP32.