Recently I spent a long time (maybe 2 days) fixing the upgrade job. Thanks to fabricio and dkliban for all their help during that time, but it still took a while. The CI/CD group met today to discuss how to simplify the CI in terms of maintenance, and one idea is to no longer have an upgrade codepath.
We spend a lot of time on the upgrade CI, and I don’t think it has provided any value in a long time. We originally created it to give a smoke test that a data migration wouldn’t fail, but that is such a rare event, we can address those issues as normal bugfix requests from users. Additionally, I’d like to move the upgrade testing to the katello and various downstreams similar to what we already do with FIPS.
So I propose that the plugin template have its upgrade job removed and we will be able to maintain what we have with significantly less effort. We have additional CI improvements planned and continuing to have this job will make each of those efforts more difficult, so even keeping what we have is not that feasible either.
Obviously having more testing is better, so this isn’t a best practices proposal. Instead it’s built around what the project’s developer time can support and also believing that the cost/benefit causes it to not make sense.
I wanted to post here to ask for feedback on how others feel about this change being made soon?