If you have harware pullups in place, why pullup the pin yourself? This will change the impendence based on the internal microcontroller pullup.
Also, issue Wrover Kit SD · Issue #1131 · espressif/arduino-esp32 · GitHub talks about settung up the SPI pins prior to the SD_MMC call. Maybe the helps?
As a further debugging step, one could set up an ESP-IDF project to see if there was an issue with the SDMMC driver functions (which are also used by arduino) which may have been fixed.