Update Libraries not possible

schawen · October 29, 2020, 1:53pm

Hello,
when opening platformio and go to the Libraries tab (indicating that there are 99+ updates available),
I get two popup windows showing this error message:

Libraries: Updates
PIO Core Call Error: 
"Error: Traceback (most recent call last):
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\platformio\__main__.py", line 109, in main
 cli() # pylint: disable=no-value-for-parameter
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\click\core.py", line 829, in __call__
 return self.main(*args, **kwargs)
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\click\core.py", line 782, in main
 rv = self.invoke(ctx)
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\platformio\commands\__init__.py", line 44, in invoke
 return super(PlatformioCLI, self).invoke(ctx)
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\click\core.py", line 1259, in invoke
 return _process_result(sub_ctx.command.invoke(sub_ctx))
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\click\core.py", line 1259, in invoke
 return _process_result(sub_ctx.command.invoke(sub_ctx))
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\click\core.py", line 1066, in invoke
 return ctx.invoke(self.callback, **ctx.params)
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\click\core.py", line 610, in invoke
 return callback(*args, **kwargs)
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\click\decorators.py", line 21, in new_func
 return f(get_current_context(), *args, **kwargs)
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\platformio\commands\lib\command.py", line 247, in lib_update
 lm = LibraryPackageManager(storage_dir)
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\platformio\package\manager\library.py", line 30, in __init__
 PackageType.LIBRARY, package_dir or get_project_global_lib_dir()
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\platformio\package\manager\base.py", line 54, in __init__
 self.package_dir = self.ensure_dir_exists(package_dir)
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\platformio\package\manager\base.py", line 98, in ensure_dir_exists
 assert os.path.isdir(path)
AssertionError

============================================================

An unexpected error occurred. Further steps:

* Verify that you have the latest version of PlatformIO using
 `pip install -U platformio` command

* Try to find answer in FAQ Troubleshooting section
 https://docs.platformio.org/page/faq.html

* Report this problem to the developers
 https://github.com/platformio/platformio-core/issues

============================================================"

I don’t have a clue about what goes wrong and what to do.

Tried this

pip install -U platformio

C:\WINDOWS\system32>pip install -U platformio
Fatal error in launcher: Unable to create process using ‘“d:\python27\python.exe” “D:\Python27\Scripts \pip.exe” install -U platformio’

maxgerhardt · October 29, 2020, 1:58pm

schawen:

 PackageType.LIBRARY, package_dir or get_project_global_lib_dir()
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\platformio\package\manager\base.py", line 54, in __init__
 self.package_dir = self.ensure_dir_exists(package_dir)
 File "C:\Users\MiKo\.platformio\penv\lib\site-packages\platformio\package\manager\base.py", line 98, in ensure_dir_exists
 assert os.path.isdir(path)
AssertionError

Somehow it fails hard while checking if a global library folder exists… or some other lib folder…

You should upgrade the PIO core to the latest dev version as there are some important bugfixes (as well as the VSCode plugin if you haven’t). Ope a PIO terminal, execute pio upgrade --dev, and restart VScode. Then retry.

schawen · October 29, 2020, 2:04pm

pio upgrade --dev
Fatal error in launcher: Unable to create process using ‘“d:\python27\python.exe” “D:\Python27\Scripts\pio.exe” upgrade --dev’

OK, python27 is no longer installed on this PC, there is only python 3.8 available.
How/where can I change this?

maxgerhardt · October 29, 2020, 2:11pm

Where did you enter this? In a normal shell or in a PIO terminal? Only in the PIO terminal you have the internally used Python 3.x version available. Python 2.7 support is deprecated and subject to soon removal.

Anyways, please first try the other route of going into the PlatformIO VSCode extension settings and check “Use development core”. Might need to wait a bit and restart VScode.

schawen · October 29, 2020, 2:28pm

Yes, I executed “pio upgrade --dev” in a pio terminal.

I’ve copied the Settings in JSON format:
“platformio-ide.useBuiltinPython”: true
“platformio-ide.useDevelopmentPIOCore”: true

But after restart and switching to “libraries” I get this error:

PIO Core Call Error: "b'
Error: Traceback (most recent call last):
 File "c:\users\miko\.platformio\penv\lib\site-packages\platformio\__main__.py", line 109, in main
 cli() # pylint: disable=no-value-for-parameter
 File "c:\users\miko\.platformio\penv\lib\site-packages\click\core.py", line 829, in __call__
 return self.main(*args, **kwargs)
 File "c:\users\miko\.platformio\penv\lib\site-packages\click\core.py", line 782, in main
 rv = self.invoke(ctx)
 File "c:\users\miko\.platformio\penv\lib\site-packages\platformio\commands\__init__.py", line 44, in invoke
 return super(PlatformioCLI, self).invoke(ctx)
 File "c:\users\miko\.platformio\penv\lib\site-packages\click\core.py", line 1259, in invoke
 return _process_result(sub_ctx.command.invoke(sub_ctx))
 File "c:\users\miko\.platformio\penv\lib\site-packages\click\core.py", line 1259, in invoke
 return _process_result(sub_ctx.command.invoke(sub_ctx))
 File "c:\users\miko\.platformio\penv\lib\site-packages\click\core.py", line 1066, in invoke
 return ctx.invoke(self.callback, **ctx.params)
 File "c:\users\miko\.platformio\penv\lib\site-packages\click\core.py", line 610, in invoke
 return callback(*args, **kwargs)
 File "c:\users\miko\.platformio\penv\lib\site-packages\click\decorators.py", line 21, in new_func
 return f(get_current_context(), *args, **kwargs)
 File "c:\users\miko\.platformio\penv\lib\site-packages\platformio\commands\lib\command.py", line 290, in lib_update
 json_result[storage_dirs[0]] if len(storage_dirs) == 1 else json_result
 File "c:\users\miko\.platformio\penv\lib\site-packages\platformio\__main__.py", line 97, in <lambda>
 click.echo = lambda *args, **kwargs: _safe_echo(0, *args, **kwargs)
 File "c:\users\miko\.platformio\penv\lib\site-packages\platformio\__main__.py", line 91, in _safe_echo
 click_echo_origin[origin](*args, **kwargs)
 File "c:\users\miko\.platformio\penv\lib\site-packages\click\utils.py", line 272, in echo
 file.write(message)
 File "C:\Users\MiKo\.platformio\python3\lib\encodings\cp1252.py", line 19, in encode
 return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u2714' in position 6997: character maps to <undefined>

============================================================

An unexpected error occurred. Further steps:

* Verify that you have the latest version of PlatformIO using
 `pip install -U platformio` command

* Try to find answer in FAQ Troubleshooting section
 https://docs.platformio.org/page/faq.html

* Report this problem to the developers
 https://github.com/platformio/platformio-core/issues

============================================================"

maxgerhardt · October 29, 2020, 2:40pm

Does it still show some libraries with updates in the GUI (like in Library inspection throws utf-8 decoding error · Issue #1583 · platformio/platformio-home · GitHub) are not all? A decoding error was recently fixed in the most recent version… What does pio --version output?

schawen · October 29, 2020, 2:54pm

No, it shows no libraries:

PIO Core Call Error: "b''\n\nError: Traceback (most recent call last):\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\__main__.py\", line 109, in main\r\n cli() # pylint: disable=no-value-for-parameter\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\commands\\__init__.py\", line 44, in invoke\r\n return super(PlatformioCLI, self).invoke(ctx)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 1259, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 1259, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\decorators.py\", line 21, in new_func\r\n return f(get_current_context(), *args, **kwargs)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\commands\\lib\\command.py\", line 290, in lib_update\r\n json_result[storage_dirs[0]] if len(storage_dirs) == 1 else json_result\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\__main__.py\", line 97, in <lambda>\r\n click.echo = lambda *args, **kwargs: _safe_echo(0, *args, **kwargs)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\__main__.py\", line 91, in _safe_echo\r\n click_echo_origin[origin](*args, **kwargs)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\utils.py\", line 272, in echo\r\n file.write(message)\r\n File \"C:\\Users\\MiKo\\.platformio\\python3\\lib\\encodings\\cp1252.py\", line 19, in encode\r\n return codecs.charmap_encode(input,self.errors,encoding_table)[0]\r\nUnicodeEncodeError: 'charmap' codec can't encode character '\\u2714' in position 6997: character maps to <undefined>\r\n\r\n============================================================\r\n\r\nAn unexpected error occurred. Further steps:\r\n\r\n* Verify that you have the latest version of PlatformIO using\r\n `pip install -U platformio` command\r\n\r\n* Try to find answer in FAQ Troubleshooting section\r\n https://docs.platformio.org/page/faq.html\r\n\r\n* Report this problem to the developers\r\n https://github.com/platformio/platformio-core/issues\r\n\r\n============================================================"

schawen · October 29, 2020, 2:54pm

PS D:\Dokumente\PlatformIO\Projects\THP\THP-Sender_089> pio --version 
Fatal error in launcher: Unable to create process using '"d:\python27\python.exe"  "D:\Python27\Scripts\pio.exe" --version'

maxgerhardt · October 29, 2020, 3:01pm

Okay that looks really, really, broken. Btw, the

message is refering to the Unicode character “” (U+2714). Such a character isn’t contained in any of the libraries in C:\Users\<user>\.platformio\lib right?

I suggest you do a clean uninstall and reinstall of PlatformIO.

Backup any data you might need later (e.g. the global library folder C:\Users\<user>\.platformio\lib or any modified packages in there, if you have any).
Uninstall the PlatformIO extension in VScode.
- this should not touch projects stored in Documents\PlatformIO\Projects, but you can make a backup to make sure
Remove the entire C:\Users\<user>\.platformio folder
Remove any old Python 2.7 installation cleanly and make sure that python on the commandline does not invoke python 2.7.
In VSCode, install the latest PlatformIO extension again
In the extension settings, again use the latest development core version
After a VSCode restart, the displayed Core version must be 5.0.2b3 in the PIO Home start page
Check if Library Updates are still broken.
If they aren’t, re-add your old lib folder. Does it crash now?

schawen · October 29, 2020, 9:02pm

Followed all the steps, but get same error after I copied the previously saved folder “…platformio\lib” back to the path.

ivankravets · October 29, 2020, 9:04pm

Should be fixed in the latest PIO Core Dev ($ pio upgrade --dev)

schawen · October 30, 2020, 12:39am

I missed this:

PS D:\Dokumente\PlatformIO\Projects\THP\THP-Sender_089> pio upgrade --dev
Please wait while upgrading PlatformIO ...
PlatformIO has been successfully upgraded to 5.0.2rc1
Release notes: https://docs.platformio.org/en/latest/history.html

But even after this change, i get the same error:

schawen · October 30, 2020, 12:53am

I checked the gui settings again and discoverd that there are two places where the
“Use development PIOCore” is used ( User / Workspace ).
One is set, the other one is not set:

second image in next post (because I’m a new user)

schawen · October 30, 2020, 12:54am

schawen · October 30, 2020, 1:08am

I’ve now set both places to “on” and restarted VS.
After the gui installed some things a restart was requested.

Checking the libraries/upload again I see this for about one minute:

*** see next post ****

schawen · October 30, 2020, 1:10am

after about one minute, I get the same error:

PIO Core Call Error: "b''\n\nError: Traceback (most recent call last):\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\__main__.py\", line 109, in main\r\n cli() # pylint: disable=no-value-for-parameter\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\commands\\__init__.py\", line 44, in invoke\r\n return super(PlatformioCLI, self).invoke(ctx)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 1259, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 1259, in invoke\r\n return _process_result(sub_ctx.command.invoke(sub_ctx))\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\decorators.py\", line 21, in new_func\r\n return f(get_current_context(), *args, **kwargs)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\commands\\lib\\command.py\", line 290, in lib_update\r\n json_result[storage_dirs[0]] if len(storage_dirs) == 1 else json_result\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\__main__.py\", line 97, in <lambda>\r\n click.echo = lambda *args, **kwargs: _safe_echo(0, *args, **kwargs)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\__main__.py\", line 91, in _safe_echo\r\n click_echo_origin[origin](*args, **kwargs)\r\n File \"c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\utils.py\", line 272, in echo\r\n file.write(message)\r\n File \"C:\\Users\\MiKo\\.platformio\\python3\\lib\\encodings\\cp1252.py\", line 19, in encode\r\n return codecs.charmap_encode(input,self.errors,encoding_table)[0]\r\nUnicodeEncodeError: 'charmap' codec can't encode character '\\u2714' in position 6997: character maps to <undefined>\r\n\r\n============================================================\r\n\r\nAn unexpected error occurred. Further steps:\r\n\r\n* Verify that you have the latest version of PlatformIO using\r\n `pip install -U platformio` command\r\n\r\n* Try to find answer in FAQ Troubleshooting section\r\n https://docs.platformio.org/page/faq.html\r\n\r\n* Report this problem to the developers\r\n https://github.com/platformio/platformio-core/issues\r\n\r\n============================================================"

or:

PIO Core Call Error: "b''
Error: Traceback (most recent call last):
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\__main__.py\", line 109, in main
cli() # pylint: disable=no-value-for-parameter
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 829, in __call__
return self.main(*args, **kwargs)
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 782, in main
rv = self.invoke(ctx)
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\commands\\__init__.py\", line 44, in invoke
return super(PlatformioCLI, self).invoke(ctx)
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\core.py\", line 610, in invoke
return callback(*args, **kwargs)
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\decorators.py\", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\commands\\lib\\command.py\", line 290, in lib_update
json_result[storage_dirs[0]] if len(storage_dirs) == 1 else json_result
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\__main__.py\", line 97, in <lambda>
click.echo = lambda *args, **kwargs: _safe_echo(0, *args, **kwargs)
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\platformio\\__main__.py\", line 91, in _safe_echo
click_echo_origin[origin](*args, **kwargs)
File "c:\\users\\miko\\.platformio\\penv\\lib\\site-packages\\click\\utils.py\", line 272, in echo
file.write(message)
File "C:\\Users\\MiKo\\.platformio\\python3\\lib\\encodings\\cp1252.py\", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u2714' in position 6997: character maps to <undefined>

============================================================

An unexpected error occurred. Further steps:
* Verify that you have the latest version of PlatformIO using
`pip install -U platformio` command
* Try to find answer in FAQ Troubleshooting section
https://docs.platformio.org/page/faq.html
* Report this problem to the developers
https://github.com/platformio/platformio-core/issues

============================================================

Hmmm …

schawen · October 30, 2020, 2:04pm

These are all folders in directory C:\Users\MiKo.platformio\lib

AccelStepper
Adafruit ADXL343
Adafruit BME280 Library
Adafruit BusIO
Adafruit Circuit Playground
Adafruit FONA Library
Adafruit GFX Library
Adafruit GPS Library
Adafruit ILI9341
Adafruit LED Backpack Library
Adafruit MQTT Library
Adafruit SleepyDog Library
Adafruit SSD1306
Adafruit STMPE610
Adafruit TouchScreen
Adafruit Unified Sensor
Adafruit Zero DMA Library
Adafruit Zero FFT Library
Adafruit Zero PDM Library
ArduinoJson
DallasTemperature
DHT sensor library
DS3232RTC
ESPAsyncUDP
ILI9488_ID4667
LiquidCrystal_ID887
Mouse
NtpClientLib_ID727
OneWire_ID1
PubSubClient
RTC
RTClib
SD
TFT_eSPI
Time_ID44
WaveHC
WebSockets
WifiManager_ID567
XPT2046_Touchscreen

After deleting the first half off all lib folders, the error didn’t occur any more.
Restored folder by folder until the error was back.

It’s the ArduinoJson library that seems to be responsible for that error.

Changed to github with this problem because here I wasn’t allowed to add further replies as “new user” …

Libraries: Updates (complete library management) · Issue #1598 · platformio/platformio-home · GitHub

maxgerhardt · October 30, 2020, 2:26pm

Yeah the description contains these Unicode chars:

github.com

bblanchon/ArduinoJson/blob/6.x/library.json#L4


      
          {
            "name": "ArduinoJson",
            "keywords": "json, rest, http, web",
            "description": "A simple and efficient JSON library for embedded C++. ArduinoJson supports ✔ serialization, ✔ deserialization, ✔ MessagePack, ✔ fixed allocation, ✔ zero-copy, ✔ streams, ✔ filtering, and more. It is the most popular Arduino library on GitHub ❤❤❤❤❤. Check out arduinojson.org for a comprehensive documentation.",
            "homepage": "https://arduinojson.org/?utm_source=meta&utm_medium=library.json",
            "repository": {
              "type": "git",
              "url": "https://github.com/bblanchon/ArduinoJson.git"
            },
            "version": "6.20.0",
            "authors": {
              "name": "Benoit Blanchon",
              "url": "https://blog.benoitblanchon.fr"
            },

There’s the checkmark we’ve been looking for.

ivankravets · October 30, 2020, 3:54pm

Could revert back that library and try the latest PlatformIO Core RC2? pio upgrade --dev. Does it work now?