Is it limited to 6 month that the life of old library on PlatformIO system?

asukiaaa · November 22, 2020, 7:41am

Is is specifigation of PlatformIO system that not latest library that was released over than 6 month ago is removed from library list?
If yes, how about to suggest to use url of git with tag rather than specifying version of library on explanation of lib_deps.

I’m happy if the limitation is removed on system of PlatformIO.

Thank you.

I share cases that got problem and resolve it by using url and tag of github.

I cannot install libary which was released over 6 months ago.

lib_deps = utils_asukiaaa@1.0.0

Library Manager: Installing utils_asukiaaa @ 1.0.0
Warning! Could not find the package with 'utils_asukiaaa @ 1.0.0' requirements for your system 'linux_x86_64'

I can install librarh with was released about 6 month ago.

lib_deps = utils_asukiaaa@1.0.1

Library Manager: utils_asukiaaa @ 1.0.1 is already installed

I could install 1.0.0 with using github url and tag.

lib_deps = https://github.com/asukiaaa/utils_asukiaaa.git#1.0.0

Library Manager: utils_asukiaaa @ 1.0.0+sha.36cc23b is already installed

Versions that are registered to PlatformIO seems less than or equal to 6 month ago.

Screenshot from 2020-11-22 16-24-43

Thank you.

maxgerhardt · November 22, 2020, 11:38am

Counter-example: ArduinoJSON

We hold versions of ArduinoJSON from more than 3 years ago.

I don’t know when you registered your library with the PlatformIO library system, so it could have been that at that time, you alredy releaed 1.0.1 (so starting may 13th 2020).

Do you need this old version to be referencable?

maxgerhardt · November 22, 2020, 11:44am

Ah it could be that ArduinoJSON does use the new pio package publish mechanism so it might be a false-positive. @ivankravets should know the answer to OP’s question?

pfeerick · November 22, 2020, 12:18pm

I don’t think so… I haven’t used the pio package publish mechanism yet, and elapsedMillis is showing it’s history, back to 2016…

I suspect it is more your first thought, as the PIO registry only goes back as far as 1.0.4, which is when it was first added to the registry.

ivankravets · November 22, 2020, 7:29pm

We don’t import ALL versions from VCS. PlatformIO Registry contains only versions that were published by its owner. Also, there is a legacy “crawler” which monitors ONLY new releases.

asukiaaa · November 22, 2020, 11:04pm

Thank you for sharing examples and information.
I understood that some library keeps old versions more than 6 month ago.

Thank you for the information.
Does it means I can keep and use old versions by registering library with using pio package publish command?

Does the crawler keeps only latest version of library?

Other case, I wanted to use version 1.1.1 of WiredControler_asukiaaa but Version control system of PlatformIO keeps only latest version of WiredController_asukiaaa.

Screenshot from 2020-11-23 08-00-58

pfeerick · November 22, 2020, 11:19pm

Did this library only get added to the PIO Library Registry at version 1.1.2? As Ivan said above, the legacy crawler only monitors NEW releases… This is one of the reasons they are moving to the new library model, as it puts control over what versions are pushed and when they are pushed to the PIO Library Registry in the hands of the library authors, rather than the delayed crawler + requiring moderation, etc.

asukiaaa · November 23, 2020, 12:00am

Thank you for reply but I’m not sure what you want to know.
I released 1.1.2 and it become latest version.
I don’t use pio package release command.
1.1.2 was released 1 month after 1.1.1 so crawler may detected 1.1.1 but it was removed from version list of PlatformIO,
Screenshot from 2020-11-23 08-00-08

I want to know the way how to keep old versions on VCS of platformIO.

pfeerick · November 23, 2020, 12:21am

What I want to know is what was the version of WiredControler_asukiaaa at the time the library was ADDED to the registry. If it was 1.1.2, you’re not going to get versions prior to that, since the crawler only picks up subsequent changes. Hence the changes to the Library Registry, and the pio package options, allowing for control of what versions are published.

btw, have a look at other libraries… it’s clear that the crawler doesn’t keep only the last version… i.e. As I menttioned above, I have not used the new mechanism for maintaining elapsedMillis, yet the last few versions are present, going back to 2016. You’ll also see that the first recorded version (1.0.4) is the version that was current at the time it was added to the PIO Library Registry.

asukiaaa · November 23, 2020, 9:23am

Does is mean the date that the library was registered as Arduino library?
It is July 18 2020.
Please add WiredController_asukiaaa #9052

I’m not sure why you can say clear because my library is not.
I want to know the difference.

pfeerick · November 23, 2020, 9:44am

No, the date you added it to the PlatformIO Library Registry! i.e. When did you do platformio lib register https://raw.githubusercontent.com/asukiaaa/WiredController_asukiaaa/master/library.properties in order to get the library added to the PlatformIO Library Registry?

If that was about the same time as you asked for it to be added to the Arduino IDE libraries list, since 1.1.2 was the ‘current’ version at the time (and had been since Jan 8 2020), the PIO crawler will not go back in time and look at the earlier versions. Only any versions added since then.

I say clear because if you look at other libraries on the PIO registry, you’ll find plenty which go back further than six months. Therefore suggesting that ‘six months’ is not the difference you should be looking for.

I will give an example again, using elapsedMillis, because I know the timeline

v1.0.0 - Nov 8, 2013   <- not visible on PIO Library Registry
v1.0.1 - Apr 4, 2015   <- not visible on PIO Library Registry
v1.0.2 - Jul 4, 2015   <- not visible on PIO Library Registry
v1.0.3 - Jul 24, 2015  <- not visible on PIO Library Registry
v1.0.4 - Sep 8, 2016   <- added to PIO, first visible version on PIO Library Registry
v1.0.5 - Aug 28, 2019  <- visible on PIO Library Registry
v1.0.6 - Dec 2, 2019   <- visible on PIO Library Registry

How do I know with certainly that 1.0.4 was the version that the library was added to the PIO registry? Because that version committed the PIO specific library.json file, as part of how I (Ivan, actually, through a PR) added it to the registry. As you can see by looking at the ‘Versions’ heading on the registry for elapsedMillis, only 1.0.4, 1.0.5 and 1.0.6 are shown. In other words, 1.0.4 was the version at the time the library was added, and 1.0.5 and 1.0.6 have been released since that date.

asukiaaa · November 23, 2020, 10:30am

Thank you for the information.

I have not ever executed pio lib register command for my libraries but some library has old version on system of PlatformIO.
I’m still not sure the requirements to hold versions on the system.

Do you mean that version history will start after adding library.json and executing pio lib register command?

pfeerick · November 23, 2020, 11:15am

If you didn’t, it means someone else did. Anyone can run that command, and add a valid library to the system… or at least could. The new library specification (i.e. were lib_deps = elapsedMillis has become lib_deps = pfeerick/elapsedMillis) means that I (pfeerick) am the owner of the library (elapsedMillis), and thus only I or someone I give access can make changes to it on the PIO Library Registry.

Yes, version history started after the pio lib register… or, in other words, when it was first added to the registry.

With the publish option, it does look like you can specify custom release times/dates, and can push specific versions, so it should be possible to just push the missing versions yourself.

Read the snippet about publishing, and then see how you go with the pio publish command. Because you want to publish older versions, you’ll want to have them downloaded, and then publish those specific versions.

asukiaaa · November 25, 2020, 12:33pm

Thank you for the information.

I tried to publish 1.0.0 of urils_asukiaaa.

pio package publish https://github.com/asukiaaa/utils_asukiaaa/archive/1.0.0.zip --owner asukiaaa --released-at "2020-04-29 14:09:00"

However failed and showed the following errors.

Error: Traceback (most recent call last):
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/__main__.py", line 109, in main
    cli()  # pylint: disable=no-value-for-parameter
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/commands/__init__.py", line 44, in invoke
    return super(PlatformioCLI, self).invoke(ctx)
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/commands/package.py", line 87, in package_publish
    archive_path = p.pack()
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/package/pack.py", line 120, in pack
    with FileUnpacker(src) as fu:
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/package/unpack.py", line 152, in __enter__
    self._archiver = self._init_archiver()
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/package/unpack.py", line 144, in _init_archiver
    with open(self.path, "rb") as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'https://github.com/asukiaaa/utils_asukiaaa/archive/1.0.0.zip'

============================================================

An unexpected error occurred. Further steps:

* Verify that you have the latest version of PlatformIO using
  `pip install -U platformio` command

* Try to find answer in FAQ Troubleshooting section
  https://docs.platformio.org/page/faq.html

* Report this problem to the developers
  https://github.com/platformio/platformio-core/issues

============================================================

I also tried the following command but also failed.

cd /tmp
wget https://github.com/asukiaaa/utils_asukiaaa/archive/1.0.0.zip
pio package publish ./1.0.0.zip --owner asukiaaa --released-at "2020-04-29 14:09:00"

I used version5.0.3.

pio --version
PlatformIO, version 5.0.3

Do you have any idea to avoid the errors?

maxgerhardt · November 25, 2020, 1:19pm

What did that command output? Have you previously logged in with pio login?

asukiaaa · November 25, 2020, 1:32pm

It was same that I shwed on previous post.

pio package publish ./1.0.0.zip --owner asukiaaa --released-at "2020-04-29 14:09:00"
Error: Traceback (most recent call last):
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/__main__.py", line 109, in main
    cli()  # pylint: disable=no-value-for-parameter
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/commands/__init__.py", line 44, in invoke
    return super(PlatformioCLI, self).invoke(ctx)
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/asuki/.local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/commands/package.py", line 87, in package_publish
    archive_path = p.pack()
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/package/pack.py", line 120, in pack
    with FileUnpacker(src) as fu:
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/package/unpack.py", line 152, in __enter__
    self._archiver = self._init_archiver()
  File "/home/asuki/.local/lib/python3.8/site-packages/platformio/package/unpack.py", line 144, in _init_archiver
    with open(self.path, "rb") as fp:
FileNotFoundError: [Errno 2] No such file or directory: './1.0.0.zip'

============================================================

An unexpected error occurred. Further steps:

* Verify that you have the latest version of PlatformIO using
  `pip install -U platformio` command

* Try to find answer in FAQ Troubleshooting section
  https://docs.platformio.org/page/faq.html

* Report this problem to the developers
  https://github.com/platformio/platformio-core/issues

============================================================

I logged in but failed uploading as same state.

maxgerhardt · November 25, 2020, 1:33pm

It can’t find the file. Are you executing that command in the same directory where the 1.0.0.zip file is? Have you tried it without the ./ prefix?

asukiaaa · November 25, 2020, 1:35pm

I also tried without ./ prefix but same result.

maxgerhardt · November 25, 2020, 1:35pm

What does ls return?

asukiaaa · November 25, 2020, 1:35pm

ls 1.0.0.zip 
1.0.0.zip