ESP-IDF pyparsing expecting end of text

malja · September 21, 2021, 10:33pm

Hi guys, I have a ESP project with following settings:

platformio.ini

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = espidf

debug_tool = esp-prog
debug_init_break = tbreak app_main
upload_protocol = esp-prog
monitor_speed = 115200

build_flags =
    -DCOMPONENT_EMBED_FILES=components/ss_mqtt/certs/ca.pem:components/ss_mqtt/certs/client.pem:components/ss_mqtt/certs/key.pem
    -Og
    -ggdb3
build_unflags = -Os

pio system info

--------------------------  --------------------------------------------
PlatformIO Core             5.2.1b2
Python                      3.9.2-final.0
System Type                 darwin_x86_64
Platform                    macOS-10.16
File System Encoding        utf-8
Locale Encoding             UTF-8
PlatformIO Core Directory   /Users/username/.platformio
PlatformIO Core Executable  /Users/username/.platformio/penv/bin/platformio
Python Executable           /Users/username/.platformio/penv/bin/python
Global Libraries            0
Development Platforms       1
Tools & Toolchains          12
--------------------------  --------------------------------------------

pio platform update

Platform espressif32
--------
Updating platformio/espressif32               3.3.2                              [Up-to-date]
Updating platformio/toolchain-xtensa32        2.50200.97 @ ~2.50200.0            [Up-to-date]
Updating platformio/toolchain-esp32ulp        1.22851.191205 @ ~1.22851.0        [Up-to-date]
Updating platformio/framework-espidf          3.40300.0 @ ~3.40300.0             [Up-to-date]
Updating platformio/tool-esptoolpy            1.30100.210531 @ ~1.30100.0        [Up-to-date]
Updating platformio/tool-openocd-esp32        2.1000.20210721 @ ~2.1000.0        [Up-to-date]
Updating platformio/tool-mkspiffs             2.230.0 @ ~2.230.0                 [Up-to-date]
Updating platformio/tool-cmake                3.16.4 @ ~3.16.0                   [Up-to-date]
Updating platformio/tool-ninja                1.9.0 @ ^1.7.0                     [Up-to-date]

python version

% /Users/username/.platformio/penv/bin/python --version
Python 3.9.2

pyparsing version

% /Users/username/.platformio/penv/bin/python -m pip show pyparsing
Name: pyparsing
Version: 2.3.1
Summary: Python parsing module
Home-page: https://github.com/pyparsing/pyparsing/
Author: Paul McGuire
Author-email: ptmcg@users.sourceforge.net
License: MIT License
Location: /Users/username/.platformio/penv/lib/python3.9/site-packages
Requires: 
Required-by:

Build process ends with following error message:

Traceback (most recent call last):
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/entity.py", line 156, in _get_infos_from_file
    results = parser.parseString(info.content, parseAll=True)
  File "/Users/username/.platformio/penv/lib/python3.9/site-packages/pyparsing.py", line 1814, in parseString
    raise exc
  File "/Users/username/.platformio/penv/lib/python3.9/site-packages/pyparsing.py", line 1808, in parseString
    se._parse( instring, loc )
  File "/Users/username/.platformio/penv/lib/python3.9/site-packages/pyparsing.py", line 1548, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/username/.platformio/penv/lib/python3.9/site-packages/pyparsing.py", line 3722, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/Users/username/.platformio/penv/lib/python3.9/site-packages/pyparsing.py", line 1552, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/Users/username/.platformio/penv/lib/python3.9/site-packages/pyparsing.py", line 3502, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pyparsing.ParseException: Expected end of text (at char 153608), (line:2513, col:50)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/ldgen.py", line 178, in <module>
    main()
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/ldgen.py", line 154, in main
    mapping_rules = generation_model.generate_rules(sections_infos)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/generation.py", line 382, in generate_rules
    root_node.insert(entity, sections, target, entities)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/generation.py", line 257, in insert
    self.add_node_child(entity, sections, target, sections_db)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/generation.py", line 130, in add_node_child
    child.insert(entity, sections, target, sections_db)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/generation.py", line 149, in insert
    self.add_node_child(entity, sections, target, sections_db)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/generation.py", line 130, in add_node_child
    child.insert(entity, sections, target, sections_db)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/generation.py", line 149, in insert
    self.add_node_child(entity, sections, target, sections_db)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/generation.py", line 189, in add_node_child
    obj_sections = sections_db.get_sections(self.parent.name, self.name)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/entity.py", line 199, in get_sections
    obj = self._match_obj(archive, obj)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/entity.py", line 187, in _match_obj
    objs = self.get_objects(archive)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/entity.py", line 180, in get_objects
    self._process_archive(archive)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/entity.py", line 168, in _process_archive
    parsed = self._get_infos_from_file(stored)
  File "/Users/username/.platformio/packages/framework-espidf/tools/ldgen/entity.py", line 158, in _get_infos_from_file
    raise ParseException('Unable to parse section info file ' + info.filename + '. ' + p.msg)
pyparsing.ParseException: Unable to parse section info file /Users/username/Projects/SmartSense/main_esp_unit/.pio/build/esp32dev/esp-idf/esp_event/libesp_event.a. Expected end of text (at char 0), (line:1, col:1)
*** [.pio/build/esp32dev/esp32.project.ld] Error 1
esptool.py v3.1
Merged 1 ELF section
============================================================================== [FAILED] Took 103.98 seconds ==============================================================================

I tried installing pyparsing 2.2.0 as suggested in other topics, but it did not chage anything. I tried adding UTF8 support, but as far as I know, it was already added to pyparsing and my path does not contain non-ASCII characters.

Next I tried to update both platformio and esp-idf to latest version. Again, nothing helped.

Do you have any idea what could be done?

maxgerhardt · September 22, 2021, 9:51am

This should have been resolved in ESP-IDF compilation fails on Mac due to pyparsing version · Issue #561 · platformio/platform-espressif32 · GitHub. But I also see that the error message is slightly different

When the observed error was

linker script generation failed for //.platformio/packages/framework-espidf/components/esp32/ld/esp32.project.ld.in
ERROR: failed to parse //.platformio/packages/framework-espidf/components/esp_event/linker.lf
Expected end of text, found ‘i’ (at char 0), (line:1, col:1)
*** [.pio/build/esp32dev/esp32.project.ld] Error 1

Have you tried installing different pyparsing versions, from the lower and higher end of the version range (>=2.0.3,<2.4.0)? And you did a complete project clean everytime time between them?

malja · September 23, 2021, 10:28am

I tried installing 2.1.0, 2.2.0, 2.3.0 and 2.4.1. None of them works.

Just to make sure everything was done properly, first I uninstalled current version with:

$ /Users/username/.platformio/penv/bin/python -m pip uninstall pyparsing

Then installed specified version with:

$  /Users/username/.platformio/penv/bin/python -m pip install pyparsing==2.3.0

Then cleared the .pio directory in my project (calling Clean task). Lastly I started new Build task.

The error message is still the same.

maxgerhardt · September 23, 2021, 5:07pm

No idea what could be causing this is if the proper versions are installed. Please comment in the linked issue above so that the devs may help you.

fmiku · June 19, 2023, 9:10am

I know, it’s an old post but today I faced with same problem with the pyparsing 3.1 after an update, The solution is to go back to pyparsing 3.0.0, The key is do this in the “current” diectroy of the framwork like ~/.platformio/penv/.espidf-5.0.2/bin !

batileo · June 20, 2023, 9:26am

Hi, i am faced with the same issue. I use windows 11:

File “C:\Users\USERNAME.platformio\packages\framework-espidf\tools\ldgen\ldgen\entity.py”, line 154, in _get_infos_from_file
raise ParseException('Unable to parse section info file ’ + info.filename + '. ’ + p.msg)
pyparsing.exceptions.ParseException: Unable to parse section info file D:/Tools_ext/ESP32Test/.pio/build/esp32-c3-devkitm-1/esp-idf/esp_event/libesp_event.a. Expected end of text (at char 0), (line:1, col:1)
*** [.pio\build\esp32-c3-devkitm-1\sections.ld] Error 1

I tried to un-/reinstall vscode, platformIO and the included esp-idf platform.
I didnt install python3 by myself as platformIO does have it own distribution.

As mentioned above (by fmiku) I installed pyparsing via:
C:\Users\USERNAME.platformio\penv.espidf-5.0.2\Scripts\python -m pip install pyparsing==3.0.0
(Location: c:\users\USERNAME.platformio\penv.espidf-5.0.2\lib\site-packages)

still the same error.

regards

Sebastian

maxgerhardt · June 20, 2023, 11:24am

Can you do a clean build by removing that folder and rebuilding?

batileo · June 20, 2023, 1:50pm

Hi max,

i managed to move forward without deleting the folder you mentioned. I was a bit “frustrated” and i executed activate.bat (beside of many other things) and build the project again. Now I’m getting a different error message and I hope I haven’t made things worse.

Generating project linker script .pio\build\esp32-c3-devkitm-1\sections.ld
linker script generation failed for C:\Users\USERNAME\.platformio\packages\framework-espidf\components\esp_system\ld\esp32c3\sections.ld.in
ERROR: failed to parse <_io.TextIOWrapper name='C:/Users/USERNAME/.platformio/packages/framework-espidf/components/app_trace/linker.lf' mode='r' encoding='cp1252'>
Could only specify one archive file in one mapping fragment, found 'archive'  (at char 20), (line:2, col:1)
*** [.pio\build\esp32-c3-devkitm-1\sections.ld] Error 1

the content of the linker.lf file:

[mapping:app_trace]
archive: libapp_trace.a
entries:
    app_trace (noflash)
    port_uart (noflash)
    app_trace_util (noflash)
    if APPTRACE_MEMBUFS_APPTRACE_PROTO_ENABLE:
      app_trace_membufs_proto (noflash)
    if APPTRACE_DEST_JTAG = y:
      port (noflash)
    if APPTRACE_SV_ENABLE = y:
      SEGGER_SYSVIEW (noflash)
      SEGGER_RTT_esp (noflash)
      SEGGER_SYSVIEW_Config_FreeRTOS (noflash)
      SEGGER_SYSVIEW_FreeRTOS (noflash)

[mapping:app_trace_driver]
archive: libdriver.a
entries:
    if APPTRACE_SV_TS_SOURCE_GPTIMER = y:
      gptimer (noflash)
    else:
      * (default)

add: i deleted the folder ‘.pio’ you mentioned just to ensure.

sebastian

batileo · June 20, 2023, 4:55pm

Update:
haven’t solved my issue so far. it looks like the issue is linked to pyparsing version (which was already stated by the thread starter).
I tried different version as well: 2.2.0, 2.4.0, 3.0.0, 3.1.0 and get diffrent error messages. If I remember right somewere in the PlatformIO docu it was stated that each packages need it own environment. So how do i know which one is used by building the project in the Platform IDE?

Which is the correct version to build the esp-idf project?

regards

sebastian

ivankravets · June 20, 2023, 5:19pm

It has been fixed in the latest version of the espressif32 dev-platform. See Release 6.3.2 · platformio/platform-espressif32 · GitHub

Please type pio pkg update in the project.