What to do to reduce unecessary plugin "compatibility" releases

At pulpcon 2021 the problem came up that the N+1 forward compatibility with pulpcore causes a lot of unnecessary plugin releases. These are unnecessary in the sense that they are only required to bump the upper bound of compatibility.

As a problem example, here’s a pulp_file commit for a pulp_file release that sets the upper bound at release time.

There was a session at pulpcon discussing ideas for this and I’ll try to summarize them.

Two ideas we should do no matter what

  1. Merge breaking changes in the plugin API very early in the pulpcore release cycle. So if we are removing this in pulpcore 3.20, we should be removing all of those things say 5-7 calendar days after we release pulpcore 3.19. This gives plugin writers a long time to update their code for compatibility.

  2. Update the CI to show that there are deprecations emitted as each plugin runs its test suite. The goal here is to make it as clear as possible to plugin writers their plugin is not compatible with the upcoming pulpcore release.

Additional ideas:

Only make breaking changes in specific releases

This approach would have say every 3 pulpcore releases or perhaps everyone divisible evenly by 5, 3.20, 3.25, etc allow breaking changes. This would allow plugins to declare compatibility up until the next “3rd” or “5th” release which would reduce the number of plugin releases significantly.

I believe this idea had the most consensus ^.

Make the pulpcore version number apply to the plugin API

In this case, when we wanted to introduce breaking plugin API changes we would be releasing Pulp 4. At least one concern resonated especially with me. @dralley pointed out that if we left the api at /pulp/api/v3/ it would be confusing so we’d have /pulp/api/v4/. Even if upgrading is easy, it’s hard to ask all users to switch from using /pulp/api/v3/ to /pulp/api/v4/ unless /pulp/api/v4/ is meaningfully a new, backwards incompatible API.

3 Likes

This sounds great to me. No complaints here.

+1 to the no-matter-what ideas, I would just emphasize that the deprecations need to be loud. They are present and visible in CI now but the job is green and no one notices. I do not think it’s the best approach to fail the main job for that but ideally this would be a loud but non-blocking check in CI.

Note to myself, the additional ideas do not necessarily exclude each other.
I do not have strong opinion one way or the other but I’ll add some perspective to both:

Additional idea #1 Only make breaking changes in specific releases
Pros:

  • every N releases to allow breaking changes in plugin API gives predictability for plugin writers
  • gives a lot of time to address incompatibilities if deprecations are happening gradually in pulpcore and addressed asap by plugin maintainers

Cons:

  • potentially all breaking changes are getting “collected” closer to The Release, or plugin writers postpone to address deprecations till last moment (making deprecations loud helps with that to an extent but we are still humans :slight_smile: )
  • what to do if we do not have breaking changes for Nth release ready but it would be convenient to do it in the N+1 one, do we wait another N releases because we missed the breaking-change release?

Addiitonal idea #2 Make the pulpcore version number apply to the plugin API
Pros:

  • same +'s as in the idea #1, with additional +. We still can wait but no less than N releases before introducing breaking changes and we do not need to stick to every N releases in case we “missed” the breaking release.
  • it’s easier to use the next X release as an upper limit for pulpcore requirement than remember and follow the convention that every N release is a breaking one (especially, if you are not living in pulp world only and maintain/develop other software)

Cons:

  • the issue with potentially not addressing deprecations early enough due to pulpcore or maintainers being late stays

  • inconsistency between pulpcore major verison and plugins major versions and REST API version can be confusing.

    Maybe I miss something but I’m not sure why REST API version (which is what we refer to as /pulp/api/v3/) should be necessarily changed when the plugin API has breaking changes. The idea #2 is a clear no-go for me if REST API version changes with every major release but I’m not sure it’s necessary.

If we make backwards incompatible changes to the plugin API every 5th release, these changes would happen about every 7 months. I like this plan, but maybe we should go with every 4th release?

Maybe you can specify the version the plugin should be compatible with and all deprecations that are found for that release will be breaking CI. But then we should still show the ones you can already know of.

I propose we make the next pulpcore release to contain breaking plugin API changes to be 3.20. We don’t need a “forever” change in policy before we’ve tried this idea. If we can agree on 3.20 it will give these benefits:

  • Plugins can declare their next releases as compatible with <3.20
  • We can try before committing to something forever

What sorts of CI changes do we need to have the release automation declare 3.20 instead of 3.next?

+1, 3.20 is basically 3 releases ahead which is not too short, not too long period to try it out.

To my knowledge the automation does not set requirements, it’s done manually by a plugin maintainer.

With all positive feedback and no negative feedback heard I’ve written this blog post to make it an official thing: https://github.com/pulp/pulpproject.org/pull/399

Additionally, the Roles work has plugin API deprecations and eventual removals as part of it and that is declaring in those deprecations they will be removed in 3.20.

1 Like

+1 to this. I’d still discuss though every which release there will be breaking changes in plugin API.

At the pulpcore meeting we decided to try this. This has been announced publicly here: Increasing Pulpcore Compatibility with Plugin Versions | software repository management

When we are preparing the pulpcore==3.20 release, we’ll need to decide what the next “plugin API breaking change will be”, e.g. pulpcore 3.24 or 3.25.