This is a special case. Usually, for different frameworks for a microcontroller, a library written for one framework is not compatible with another framework, if that library uses some API functions that don’t exist in the other framework.
Arduino-ESP32 and ESP-IDF are a special case: Arduino-ESP32 is an extension on top of the ESP-IDF code. When building ESP-IDF, you can actually specify that the Arduino core is built, too, and thus you gain compatibility with all Arduino-ESP32 libraries. This was also discussed at Using esp-idf library within the Arduino Framework (ESP32) - #2 by maxgerhardt.
An example of such a configuration is seen at platform-espressif32/examples/espidf-arduino-blink at develop · platformio/platform-espressif32 · GitHub.
That also actually means that if you write an Arduino-ESP32 firmware, you do have full access to the underlying ESP-IDF framework that was prebuilt / statically compiled and linked for the the Arduino core. So there isn’t a lot of difference…
Of course you can also just use plain ESP-IDF with its set of “libraries”, or rather components as we would call them in ESP-IDF.
The hello-world example for ESP-IDF is at platform-espressif32/examples/espidf-hello-world at develop · platformio/platform-espressif32 · GitHub and an example with a component (AWS-IoT) is at platform-espressif32/examples/espidf-aws-iot at develop · platformio/platform-espressif32 · GitHub. The entire documentation for it is at Espressif IoT Development Framework — PlatformIO latest documentation.