Does the Platformio registry scrape the arduino library registry or the espressif component registry?

I have a library that I uploaded to both the arduino library registry and the espressif component registry.

I didn’t upload it to the platformio registry but yet it got added there - however only version 1.0.0 got added, the update 1.0.1 did not get picked up.

My questions:
does PlatformIO registry automatically add libraries from other registries? if not, how did the library get added and how/when are updates to the library in whatever other registry it was taken from be pulled in?

No scraping - you have to do it on your own using pio pkg

@arduhe then how did my library end up in the platformio registry?

Good point - at least until now I always had to use pio pkg for publishing new versions of my library. Haven’t seen any automated takeover.

Is your library listed here library-registry/repositories.txt at main · arduino/library-registry · GitHub ?

@ivankravets yes, as stated I added the library to both the arduino library registry (which you linked) as well as the espressif component registry

We keep our registry in sync with Arduino TXT file to let developers easily migrate from Arduino IDE to the PlatformIO.

@ivankravets but this is a problem if you scrape only the initial upload when it’s added but not subsequent updates. The Arduino library registry automatically picks up on version updates in the source repositories (whenever the version in the manifest is increased and a new version tag is pushed), the PlatformIO registry seems to not do that so it will always stay at the very first version of the library pulled.

If you pull libraries added to the Arduino library registry automatically into the PlatformIO registry, then you need to make sure to also monitor the source repositories for updates in the same way the Arduino library registry is doing it, or alternatively not pull them into the PlatformIO registry in the first place and instead just rely on the Arduino library registry when a lib_deps dependency is encountered that isn’t present in the PlatformIO registry.

Could you point to the library that was imported from the Arduino public list and was not updated in time?

Once you manually publish the new version, the package will never be handled by the crawler that pulls updates automatically.

it’s mabuware/mabutrace and I see that it has been updated to the latest version in the meantime, but it took several days. Seems the like the PlatformIO crawler doesn’t run as often as the Arduino library crawler perhaps (The Arduino library registry usually picks up on new updates within a day)?

The crawler checks for updates every 24h.

hmm, not sure if this crawler works. I want to use the esp-fs-webserver library and in the Arduino IDE I see the version 2.0.9 (Aug 2025) as the latest version, exactly like on github. But in Platformio the version is 1.0.6 (3 years old !!!).

Is there any way to update this or to manually integrate this library?

I think if the author self publishes to the PIO registry once, it gets taken off the crawler.

Irregardless, you can still make PlatformIO use the latest stable 2.0.9 by pointing to

lib_deps =
  https://github.com/cotestatnt/esp-fs-webserver/archive/refs/tags/2.0.9.zip

Fixed! The author changed the “master” branch. That was the issue.