PIO Issues with Partition Offset

I’m having an issue where it appears PIO isn’t respecting the partition offset I set in menuconfig. When I turn on verbose logging while using ESP-IDF with Arduino as a component, PIO fails to upload with the error:

esptool write_flash: error: argument : Detected overlap at address: 0x8000 for file: /Documents/ESP/firmware/.pio/build/device-1/partitions.bin

If I reduce log level to info I don’t get this error, and am able to upload. This seems to track with the offset being at 0x8000 as mentioned here. However, I have set the partition offset to 0x10000 in menuconfig, and can see this reflected in the sdkconfig:

CONFIG_PARTITION_TABLE_OFFSET=0x10000

When I do reduce log level to info and upload, I get a boot loop that indicates that it’s not finding the partition table. I think it being overwritten, potentially due to this same issue:

rst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:6584
load:0x40078000,len:15784
load:0x40080400,len:5136
entry 0x40080708
I (105) boot: Chip Revision: 3
I (121) boot_comm: chip revision: 3, min. bootloader chip revision: 0
I (74) boot: ESP-IDF 3.40001.200521 2nd stage bootloader
I (74) boot: compile time 17:18:41
I (75) boot: Enabling RNG early entropy source…
I (77) qio_mode: Enabling default flash chip QIO
I (82) boot: SPI Speed : 80MHz
I (85) boot: SPI Mode : QIO
I (88) boot: SPI Flash Size : 16MB
E (91) flash_parts: partition 0 invalid magic number 0x6e9
E (96) boot: Failed to verify partition table
E (100) boot: load partition table error!

My platformio.ini file:

[env]
board = esp-wrover-kit
platform = espressif32
framework = arduino, espidf
platform_packages =
framework-arduinoespressif32 @ GitHub - espressif/arduino-esp32: Arduino core for the ESP32
build_unflags =
-Os
build_flags =
-w
-O2
-DBOARD_HAS_PSRAM
-DESP32
-mfix-esp32-psram-cache-issue
-DCORE_DEBUG_LEVEL=3
board_build.f_cpu = 240000000L
board_build.f_flash = 80000000L
board_build.partitions = partitions.csv
board_build.flash_mode = qio
board_upload.flash_size = 16MB
board_upload.maximum_size = 16777216

My partitions.csv file:

#Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x11000, 0x4000,
otadata, data, ota, , 0x2000,
phy_init, data, phy, , 0x1000,
factory, app, factory, , 2M,
ota_0, app, ota_0, , 2M,
ota_1, app, ota_1, , 2M,

Flashing directly with esptool.py works as expected with the offsets specified. So it seems that the issue is PIO doesn’t pass those parameters correctly, and therefore it’s not possible to use PIO with a custom partitions table.

I ended up migrating from PIO to ESP-IDF with Arduino as a component using the IDF VS Code extension and all works well.