Trying to use a static .a library, but the linker cannot find it

I am trying to use a static .a library. I add a build flag in the platfomrio.ini file as follow:

build_flags = -IC:\Users\zhengwei\.platformio\lib\MycroPythonEmbedded\py -LC:/Users/zhengwei/.platformio/lib/MycroPythonEmbedded/lib -lmicropython

the .h files are in C:\Users\zhengwei.platformio\lib\MycroPythonEmbedded\py, the .a file is in C:\Users\zhengwei.platformio\lib\MycroPythonEmbedded\lib

but I kept getting this error :xtensa-esp32-elf/bin/ld.exe: cannot find -lmicropython

I am using the latest ESP-IDF framework.

replacing the \ with / makes not difference, below won’t help either:
build_flags = -I$PROJECT_CORE_DIRlib\MycroPythonEmbedded\py -L$PROJECT_CORE_DIRlib\MycroPythonEmbedded\lib -lmicropython

Does any one know what mistake I was making?

So what’s the name of that .a library? It’s expecting libmicropython.a as a result from giving it that flag, is this what the file is called?

Also the last full linker command would be hellpful, which can be seen by using “Verbose Build” (or pio run -v).

It may also be the case that for the ESP-IDF framework, additionally linked libraries must be added through the CMakeLists.txt file rather than build_flags.

Thank you very much.
It was me missing the trailing “”, the .a file is at
C:\Users\zhengwei\.platformio\lib\MycroPythonEmbedded\lib\micropython.a

It can find the .a file, now I am facing some new errors, great.:grin:

Very interesting that it’s still found if its name is micropython.a and not libmicropython.a o_O. Usually you have to use that name or a different linker command (see Failing to link external library) for that to work.

If it doesn’t come from the linker command (so that problem is solved) then you can maybe open a new topic for it.

Sorry, it terns out the error is still there.
When added the trailing , it did not give me the cannot find -lmicropython error, no matter what I name this .a file or even remove it.
instead, it give me undefined reference error.

the pio run -v gave me this huge chunk:

xtensa-esp32-elf-g++ -o .pio\build\firebeetle32\firmware.elf -T esp32_out.ld -u esp_app_desc -u pthread_include_pthread_impl -u pthread_include_pthread_cond_impl -u pthread_include_pthread_local_storage_impl -u app_main -T esp32.project.ld -T esp32.peripherals.ld -u call_user_start_cpu0 -u ld_include_panic_highint_hdl -T esp32.rom.ld -T esp32.rom.libgcc.ld -T esp32.rom.syscalls.ld -T esp32.rom.newlib-data.ld -T esp32.rom.newlib-funcs.ld -u vfs_include_syscalls_impl -u newlib_include_locks_impl -u newlib_include_heap_impl -u newlib_include_syscalls_impl -u newlib_include_pthread_impl -u __cxa_guard_dummy -u __cxx_fatal_exception -Wl,--Map=G:/project/Embedded/esp32/MicroPythonTest/MicroPythonEmbedding/test/MPTest/.pio/build/firebeetle32/MPTest.map -Wl,--cref -Wl,--gc-sections -Wl,--undefined=uxTopUsedPriority -Wno-frame-address -fno-lto -fno-rtti -mlongcalls -nostdlib .pio\build\firebeetle32\esp-idf\src\main.cpp.o .pio\build\firebeetle32\esp-idf\src\mpHelper.c.o .pio\build\firebeetle32\esp-idf\src\wifiStart.c.o -L.pio\build\firebeetle32 "-LC:\users\zhengwei\.platformio\platforms\espressif32\builder\Userszhengwei.platformiolibMycroPythonEmbeddedlib -lmicropython" -LC:\users\zhengwei\.platformio\packages\framework-espidf\components\esp_wifi\lib_esp32 -LC:\users\zhengwei\.platformio\packages\framework-espidf\components\esp_wifi\lib_esp32 -L.pio\build\firebeetle32\esp-idf\esp32 -L.pio\build\firebeetle32\esp-idf\esp32\ld -LC:\users\zhengwei\.platformio\packages\framework-espidf\components\esp32\ld -LC:\users\zhengwei\.platformio\packages\framework-espidf\components\xtensa\esp32 -LC:\users\zhengwei\.platformio\packages\framework-espidf\components\esp_rom\esp32\ld -Wl,--start-group .pio\build\firebeetle32\lib5d7\libesp32.a .pio\build\firebeetle32\libe4c\libMycroPythonEmbedded.a .pio\build\firebeetle32\esp-idf\esp_ringbuf\libesp_ringbuf.a .pio\build\firebeetle32\esp-idf\asio\libasio.a .pio\build\firebeetle32\esp-idf\coap\libcoap.a .pio\build\firebeetle32\esp-idf\esp_adc_cal\libesp_adc_cal.a .pio\build\firebeetle32\esp-idf\console\libconsole.a .pio\build\firebeetle32\esp-idf\nghttp\libnghttp.a .pio\build\firebeetle32\esp-idf\esp-tls\libesp-tls.a .pio\build\firebeetle32\esp-idf\esp_gdbstub\libesp_gdbstub.a .pio\build\firebeetle32\esp-idf\tcp_transport\libtcp_transport.a .pio\build\firebeetle32\esp-idf\protobuf-c\libprotobuf-c.a .pio\build\firebeetle32\esp-idf\esp_http_server\libesp_http_server.a .pio\build\firebeetle32\esp-idf\esp_http_client\libesp_http_client.a .pio\build\firebeetle32\esp-idf\esp_https_ota\libesp_https_ota.a .pio\build\firebeetle32\esp-idf\mdns\libmdns.a .pio\build\firebeetle32\esp-idf\protocomm\libprotocomm.a .pio\build\firebeetle32\esp-idf\esp_local_ctrl\libesp_local_ctrl.a .pio\build\firebeetle32\esp-idf\esp_websocket_client\libesp_websocket_client.a .pio\build\firebeetle32\esp-idf\sdmmc\libsdmmc.a .pio\build\firebeetle32\esp-idf\fatfs\libfatfs.a .pio\build\firebeetle32\esp-idf\wear_levelling\libwear_levelling.a .pio\build\firebeetle32\esp-idf\freemodbus\libfreemodbus.a .pio\build\firebeetle32\esp-idf\expat\libexpat.a .pio\build\firebeetle32\esp-idf\jsmn\libjsmn.a .pio\build\firebeetle32\esp-idf\json\libjson.a .pio\build\firebeetle32\esp-idf\mqtt\libmqtt.a .pio\build\firebeetle32\esp-idf\spiffs\libspiffs.a .pio\build\firebeetle32\esp-idf\ulp\libulp.a .pio\build\firebeetle32\esp-idf\openssl\libopenssl.a .pio\build\firebeetle32\esp-idf\libsodium\liblibsodium.a .pio\build\firebeetle32\esp-idf\wifi_provisioning\libwifi_provisioning.a .pio\build\firebeetle32\esp-idf\unity\libunity.a .pio\build\firebeetle32\esp-idf\spi_flash\libspi_flash.a .pio\build\firebeetle32\esp-idf\log\liblog.a .pio\build\firebeetle32\esp-idf\mbedtls\mbedtls\library\libmbedx509.a .pio\build\firebeetle32\esp-idf\heap\libheap.a .pio\build\firebeetle32\esp-idf\lwip\liblwip.a .pio\build\firebeetle32\esp-idf\efuse\libefuse.a .pio\build\firebeetle32\esp-idf\esp_common\libesp_common.a .pio\build\firebeetle32\esp-idf\cxx\libcxx.a .pio\build\firebeetle32\esp-idf\app_trace\libapp_trace.a .pio\build\firebeetle32\esp-idf\espcoredump\libespcoredump.a .pio\build\firebeetle32\esp-idf\esp_rom\libesp_rom.a .pio\build\firebeetle32\esp-idf\newlib\libnewlib.a .pio\build\firebeetle32\esp-idf\wpa_supplicant\libwpa_supplicant.a .pio\build\firebeetle32\esp-idf\xtensa\libxtensa.a .pio\build\firebeetle32\esp-idf\freertos\libfreertos.a .pio\build\firebeetle32\esp-idf\pthread\libpthread.a .pio\build\firebeetle32\esp-idf\esp_eth\libesp_eth.a .pio\build\firebeetle32\esp-idf\nvs_flash\libnvs_flash.a .pio\build\firebeetle32\esp-idf\driver\libdriver.a .pio\build\firebeetle32\esp-idf\vfs\libvfs.a .pio\build\firebeetle32\esp-idf\app_update\libapp_update.a .pio\build\firebeetle32\esp-idf\tcpip_adapter\libtcpip_adapter.a .pio\build\firebeetle32\esp-idf\bootloader_support\libbootloader_support.a .pio\build\firebeetle32\esp-idf\mbedtls\mbedtls\library\libmbedtls.a .pio\build\firebeetle32\esp-idf\esp32\libesp32.a .pio\build\firebeetle32\esp-idf\mbedtls\mbedtls\library\libmbedcrypto.a .pio\build\firebeetle32\esp-idf\soc\libsoc.a .pio\build\firebeetle32\esp-idf\esp_event\libesp_event.a .pio\build\firebeetle32\esp-idf\esp_wifi\libesp_wifi.a -lcoexist -lcore -lespnow -lmesh -lnet80211 -lphy -lpp -lrtc -lsmartconfig -lhal -lstdc++ -lgcov -lc -lm -lgcc -Wl,--end-group

Without the trailing , i always get cannot find -lmicropython error, even when naming the file libmicropython.a.

The verbose build gave me this:
xtensa-esp32-elf-gcc -o .pio\build\firebeetle32\bootloader.elf -Wl,--Map=G:/project/Embedded/esp32/MicroPythonTest/MicroPythonEmbedding/test/MPTest/.pio/build/firebeetle32/bootloader/bootloader.map -Wl,--cref -Wl,--gc-sections -Wno-frame-address -fno-lto -fno-rtti -mlongcalls -nostdlib -T esp32.peripherals.ld -T esp32.rom.ld -T esp32.rom.newlib-funcs.ld -T esp32.rom.libgcc.ld -T esp32.bootloader.ld -T esp32.bootloader.rom.ld -Wl,--start-group .pio\build\firebeetle32\bootloader\esp-idf\soc\libsoc.a .pio\build\firebeetle32\bootloader\esp-idf\micro-ecc\libmicro-ecc.a .pio\build\firebeetle32\bootloader\esp-idf\spi_flash\libspi_flash.a .pio\build\firebeetle32\bootloader\esp-idf\main\libmain.a .pio\build\firebeetle32\bootloader\esp-idf\log\liblog.a .pio\build\firebeetle32\bootloader\esp-idf\xtensa\libxtensa.a .pio\build\firebeetle32\bootloader\esp-idf\efuse\libefuse.a .pio\build\firebeetle32\bootloader\esp-idf\bootloader_support\libbootloader_support.a -L.pio\build\firebeetle32 -LC:\users\zhengwei\.platformio\platforms\espressif32\builder\frameworks\Userszhengwei.platformiolibMycroPythonEmbeddedlib -LC:\users\zhengwei\.platformio\packages\framework-espidf\components\esp32\ld -LC:\users\zhengwei\.platformio\packages\framework-espidf\components\xtensa\esp32 -LC:\users\zhengwei\.platformio\packages\framework-espidf\components\esp_rom\esp32\ld -LC:\users\zhengwei\.platformio\packages\framework-espidf\components\bootloader\subproject\main -Wl,--end-group -lmicropython -lhal

Neither has -LC:\Users\zhengwei\.platformio\lib\MycroPythonEmbedded\lib in them.

I am totally lost.

Well having errors for undefined references is better than not finding a library, that just means a function isn’t there in its expected form (or you have a name mangling problem).

Still

Looks unhealthy with quotes. Have you tried the alternative

build_flags = -IC:\Users\zhengwei\.platformio\lib\MycroPythonEmbedded\py -LC:/Users/zhengwei/.platformio/lib/MycroPythonEmbedded/lib -l:micropython.a.a

Also what exact unreferenced errors do oyu get and where can this MycroPythonEmbedded library be found that you’re using?

Thank you very much, this worked! I am sure the linker found the micropython.a, because I get this error now:

c:/users/zhengwei/.platformio/packages/toolchain-xtensa32/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: C:\users\zhengwei\.platformio\lib\MycroPythonEmbedded\lib/micropython.a(network_lan.o):(.literal.init_lan_rmii+0x4): undefined reference to `phy_rmii_configure_data_interface_pins'
......
c:/users/zhengwei/.platformio/packages/toolchain-xtensa32/bin/../lib/gcc/xtensa-esp32-elf/8.2.0/../../../../xtensa-esp32-elf/bin/ld.exe: C:\users\zhengwei\.platformio\lib\MycroPythonEmbedded\lib/micropython.a(network_lan.o): in function `lan_active':
network_lan.c:(.text.lan_active+0x16): undefined reference to `esp_eth_enable'

There are 200 lines of error, all like these either undefined reference or in function ‘some _function’ undefined reference to `other_function’.

I use nm micropyton.a to see what’s inside this lib. It seen the undefined reference is in that lib like:

network_lan.o:
00000000 r allowed_args$9261
         U esp_config_obj
         U esp_eth_disable
         U esp_eth_enable
         U esp_eth_init
         U esp_ifconfig_obj
00000000 t get_lan
00000000 R get_lan_obj
         U gpio_pad_select_gpio

I am not sure what that means, I assume the function is in the .a file.

The micropython.a was compiled by myself. Since I want to use it as a library not a pre-built firmware. But, I cannot get micropython built in PIO. So I renamed the main function of the ESP32 port of the micropython, compiled it into a .a file using ESP-IDF tool chain (ESP-IDFv3.3 this time, there should be no difference in the binary output?).

But if you compile micropython against ESP-IDF v3.3 and in PlatformIO you use framework = espidf with the latest platform version, you’re getting ESP-IDF 4.0.1 (refer Releases · platformio/platform-espressif32 · GitHub).

That also makes sense because in v3.3 you have the esp_eth_init() function: Ethernet - - — ESP-IDF Programming Guide release-v3.3 documentation

But that’s not existent in v4.0.1: https://docs.espressif.com/projects/esp-idf/en/v4.0.1/api-reference/network/esp_eth.html

There it’s esp_eth_driver_install and esp_eth_start().

So I guess you need to recompile Micropython against that exact ESP-IDF version with the same sdkconfig configuration as you’re using in the PlaformIO project.

Thank you very much, that is very helpful.
I build the micropython using esp-idf 4.0.1, but it gives me a new set of undefined references.
Like:

xQueueCreateMutexStatic
ble_hs_util_ensure_addr
ble_hs_id_gen_rnd
os_mbuf_append
os_mbuf_copydata
and more...

but I was able to trim the ble off from the micropython, I don’t even know how I did it, just a lots of experiments, at last only one undefined reference remained:

undefined reference to xQueueCreateMutexStatic`

Any Ideas how to resolve this? :grinning:

This depends on the FreeRTOSConfig.h of the ESP-IDF, may also be handled in the sdkconfig.h. There’s some macros to enable this that can be looked up.

maxgerhardt, You are amazing!

I found

#if( configSUPPORT_STATIC_ALLOCATION == 1 )

    #define xSemaphoreCreateMutexStatic( pxMutexBuffer ) xQueueCreateMutexStatic( queueQUEUE_TYPE_MUTEX, ( pxMutexBuffer ) )

#endif /* configSUPPORT_STATIC_ALLOCATION */

in semphr.h, and this:

#if( ( configUSE_MUTEXES == 1 ) && ( configSUPPORT_STATIC_ALLOCATION == 1 ) )

    QueueHandle_t xQueueCreateMutexStatic( const uint8_t ucQueueType, StaticQueue_t *pxStaticQueue )

    {
    ....
    }

#endif /* configUSE_MUTEXES */

This in queue.c

in the .platformio\packages\framework-espidf\components\freertos

so I added:

#define configSUPPORT_STATIC_ALLOCATION 1

in the FreeRTOSConfig.h, and it builds and links!

But, now I am facing this:

Error: The program size (1238096 bytes) is greater than maximum allowed (1048576 bytes)

Isn’t my ESP32 has 4MB of flash? Moreover, the micropython.a is 2.6MB, why the firmware is 1.2MB?

Anyway,

Since the original problem was solved, I am gonna make this solved.

Greate thanks to dear maxgerhardt, you help me out step by step.

Hope this post is useful for others.

Well you might have a 4MB flash but you still have a partition table which splits up the flash into NVS, App0, App1, SPIFFS / LittleFS etc, and so if you exceed the pregiven size for your partition then that’s going to happen.

By default, as noted in the docs, for ESP-IDF you will have

which gives you 1MB of app space.

You can check the above documentation to supply your own partition table file with maybe 1.5M so thta it works.

But the .a is an archive file with a few debug / symbol information, not only the binary code, so it’s larger than the underlying code. Also, depending in link time optimization, not all functions may be referenced / unreachable in the firmware – so the compiler can optimize those out an they won’t take up space.

Thank you very much maxgerhardt, I learned a lot from here.