Correct. Debugging AVR chips with the Atmel-ICE tool is unsupported and quite frankly a mess, thanks to Microchip not releasing an open source GDB server, but only looking after their nice proprietary Micrchip Studio environment. See related posts like https://community.platformio.org/t/updi-debugging-avr128dx-with-platformio/32807/4?u=maxgerhardt.
The program “avarice” could potentially use your atmel-ice to connect to an ATTiny85. See https://community.platformio.org/t/atmega32-debugging-with-avarice-and-avr-gdb/15960/2?u=maxgerhardt and https://github.com/avrdudes/avarice.