Katello/Pulp3 Integration meeting

Minutes
#78

2023-02-22

Overview

  • Katello Schedule

Pulp

  • Pulpcore

  • RPM

  • Ansible

  • Pulp Container

  • OSTree

  • Pulp CLI

    • pulp-cli/pulp-glue split caused some Fun Times in CI

  • Pulp Replica

  • When should we hold this meeting?

    • [ian] Katello is moving the prior meeting back 15 minutes.
    • [ggainey] cheers wildly

Katello

QE

Build Team

#79

2023-03-01

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Fixed https://github.com/pulp/pulp_rpm/issues/2961
      • Backported to 3.19
    • Improving end-to-end metadata integrity tests
    • Have not been able to reproduce the module API appearance differences yet? Tried with 3.17, 3.19, 3.17 → 3.19
      • Array-of-strings vs JSON
      • Iballou modified katello to handle both scenarios
    • Lots of new BZs incoming :slightly_frowning_face:
      • Help prioritizing?
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica
    • Merging into pulpcore “really soon” (for core/3.23)
    • Needs some plugin support as well
    • Needs content-protection before integration w/ katello
    • Discussion: what about ACS?
      • It works to the degree that ACS is set up ‘downstream’

Katello

  • Dep-solving discussions - prob needs some resolution
    • Small incremental improvements in workflow/docs
    • Needs an epic for katello/4.9

QE

Build Team

#80

2023-03-08

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
    • Release coming #soon - mostly w/ domain support
  • Pulp Replica
    • Will be in core/3.23
    • Rpm-support-pr is opened
  • Review meeting cadence
    • can/should we move to every-other-week?
    • Sjha - maybe try every-other. Meeting is useful
    • Dralley, vsedmik - +1
    • Add new QE rep (sweta)
    • AI: ggainey to move this to every-other-week as an experiment

Katello

  • Starting to test out pulp replica w.r.to POP and smart proxy use cases.
    • Sjha has a card
  • Domain work is also exciting - yay!
  • How big of a no-no is a flag on a repository telling us if repo has been synced with immediate and bits exist on filesystem?
    • More specifically - knowing that all content-artifacts exist on disk
  • “How much disk space is being taken up by a repository”
    • The perennial question
    • Dralley has a tool for rpm-repos that “sort of” answers this
    • COPR support may want something similar
    • “Its Very Complicated”
      • Artifacts are deduplicated
      • Metadata takes space
      • Postgres takes space
      • On-demand doesn’t take space (but could)
      • Deduplicating filesystems make things even more interesting

QE

  • No major updates
    • Lots of BZ work (compressed comps file)
1 Like
#81

2023-03-22

Overview

  • Katello Schedule

Pulp

Katello

  • Looks like 4.8 should be out “soon”, is that correct? Yes. Nightlies are green and so is 4.8 after pulp_rpm 3.19.2. We have an action item to re-record our VCRs assuming format of data returned is changed.
    • Does katello turn on pulp-analytics in the upstream? - No
    • Still opt-in on installer
  • Working on refining epics for Next-Y
    • Doesn’t look like many Pulp impacts (yet)
  • katello/4.8 looks to be unblocked now, woohoo!

QE

  • no updates at all - all ON_QA 6.13 BZs have been resolved
#82

2023-04-05

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Shweta picked up incremental-export-sync after switching from on_demand-to-immediate
#83

2023-04-19

Overview

  • Katello Schedule

Pulp

Katello

  • Any upgrade wishes for the next few Katello releases?
    • core/3.23 has domain work
    • core/3.25 will have django4
    • katello/4.9 branching in “a month”, core/3.22
    • Maybe katello/4.10 will aim for core/3.25(ish)

QE

#84

[missed posting previous minutes]

2023-05-03

Overview

  • Katello Schedule

Pulp

  • Pulpcore
    • Query of all changes from last two weeks
    • core/3.24 released
      • Having Some Issues, will update
    • core/3.25 is coming #soon (May-ish)
      • Moves to Django4.2, psycopg3
      • katello/4.Next should prob think about being core/3.25 or later
  • RPM
    • Fix for the profiles issue that Ian reported on Slack is ongoing
    • 3.20 release today
      • No domains yet, alas
  • Ansible
  • Pulp Container
    • Plugin leadership change, goes to Lubos!
    • Ggainey will add him to invite
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Core-repair API will be in-use for capsules
    • Nudge lmjachky about container-issues involving repair
    • Katello will need fixes “soon”
      • Added prio-list label on repair related issues
    • Thoughts about large/manyrepo/huge-dataset systems - per-repo, or global-repair?
    • Discussion needs to happen! Poke lmjachky?

QE

  • Nothing to yell about
  • Sat6.13 released today!
    • And there was much rejoicing
#85

2023-05-17

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • 3.17.18 released
    • 3.21.0 release soon
      • Mostly domains, core/3.25 required
  • Ansible
  • Pulp Container
    • Podman-push-repo-versioning is changing
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

QE

Build Team

  • core/3.25 and Django4 is coming - heads up!
#86

2023-05-31

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Sat-next Snap1 is out, no pulp issues yet
#87

2023-06-14

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Gunicorn “fix”?
    • Cfg-change to auto-restart workers “regularly” (not a pulp-change, a foreman-installer change)
#88

2023-06-28

Overview

  • Katello Schedule

Pulp

Katello

2159105 – Remove deprecated & not working API endpoints from APIdoc (as an example)

QE

#89

2023-07-12

Overview

  • Katello Schedule

Pulp

Katello

  • 2219885 – Satellite using the feature 'Alternate Content Source' is getting errors
    • Not quite clear on what is going on here
    • Is there some “tuning” we need to/can do here (esp on the capsule side)
      • Eg, don’t currently have access to optimize/no-optimize on acs refreshes
  • Finishing up Sat-Next testing
  • Still waiting on full pulp-rpms for core/3.28 and friends
    • Some packaging snafu in-progress
  • Hey - what is going on with Flatpak support?
    • Talk to pulp_container gang for discussion-status
  • katello/satellite smart-proxy-sync discussions
    • Would like to insure capsule-N-2 works against satellite-N, always

QE

  • Sat-Next verifications continue apace
  • QE adding a bunch of smart-proxy-sync test cases
2 Likes
#90

2023-07-26

Overview

  • Katello Schedule

Pulp

Katello

QE

  • RE ipanova’s pulp_container issue - QE tests are passing without?
    • Vsedmik to check test-versions, maybe poke ipanova as/if needed
  • https://bugzilla.redhat.com/show_bug.cgi?id=2142689
    • May not be reproducible any more - vsedmik to do some investigation
    • Will prob eventually close? Depends on Sat-Current-Y testing
#91

2023-08-09

Overview

  • Katello Schedule

Pulp

Katello

QE

  • some help needed to reproduce/test 2125366 – Capsule sync fails with Error: It is not possible to add more than one advisory with the same id to a repository version

    • We have an upstream test - let’s get the link for QE to look at

  • How to track 2122872 and 2229788 (gunicorn memory leak Fun)

    • 2122872 originally had a workaround that we were going to move forwards with
      • It was cloned to 6.13.2 as 211957 and pushed, now CLOSED-ERRATA
      • Then we found a better fix, and we want to push that to Next-Z? And Sat-Next
      • We also intend to revert the workaround as it’s no longer needed
    • 2229788 was created for the “actual” fix
    • TL;DR how should this be tracked
      • Dalley suggestion:
        • Leave 2122872 tracking the “meta-issue” of gunicorn memory consumption, including the new lru_cache fix, and the reversion, for Sat-Next
        • Use 2122872 to track the improved fix and the reversion for Next-Z?
        • In both cases, we just test that the gunicorn memory consumption issue is resolved as expected
    • ‘2872 - used to handle the mitigation/restart Fun
    • ‘9788 - used for the ‘real’ fix
    • ‘1957 - cloned ‘2872 mitigation for 6.13 release
    • ‘9788 won’t fix 6.11/6.12 cases (since it’s fixing something that didn’t happen until 6.13)
    • What we settled on:
      • Keep 9788 for separately tracking the lru_cache fix (and set flags to clone it for 6.13)
      • Create a new bug to track the relaxed installer defaults (6.13 cloning optional)
        • Created 2230584
        • flags: 6.14.0?, 6.13.z?
      • QE these together with the original bug, so that QE effort is not duplicated
        • (the “actual” fix + relaxation) for each release
      • Email Brad, Justin and Vijay Singh (QE mngrs), Shweta (QE contact for already delivered mitigations on 6.13) and Vlad (QE just for tracking), and Odilon (delivery) so everyone knows what to expect
      • The new BZs should be triaged on Aug 10th triage meeting, Vlad or William will provide the info.
#92

2023-09-06

Overview

  • Katello Schedule

Pulp

Katello

  • Quba42 and pulp_deb/3.0
    • discussion/investigation in progress Right Now
    • Trying to see if we can fit into katello/4.10 release
    • Stay tuned!
  • BZ RE “tell users if module-streams have expired”?
    • Is this even available? (looks like Not)

QE

  • Couple BZs still being verified
#93

(Catching up on notes - usual poster was unavailable)

2023-10-18

Overview

  • Katello Schedule

Pulp

Katello

  • 4.10 GA “imminent”, a lot of new core/3.28 users incoming!
  • 4.11 will be branching Real Soon Now
    • Once that happens, will be upgrading Pulp again
    • ggainey to update support-matrix - 4.11 versions same as 4.10

QE

  • Reviewed import/export perf-BZs from a test standpoint, looks good
    • QE ready to get the releases w/ the fixes

2023-10-04

Overview

  • Katello Schedule

Pulp

Katello

2023-09-20

Overview

  • Katello Schedule

Pulp

Katello

  • Need backport of https://github.com/pulp/pulp_ansible/pull/1561/ to fix syncing ansible content to smart proxies for Katello 4.10 (GAing next week)
    • Chatting with Gerrod about it now, awaiting opinions from Matthias
    • A backport to 0.18.0 would be ideal, but we could work with a release if it’s compatible with Pulpcore 3.28.
#94

2023-11-01

Overview

  • Katello Schedule

Pulp

Katello

  • PulpCon schedule shared to the rest of the team, yay!
  • Pulpcore upgrade
    • Trying to get 3.40 into katello/4.11
    • Timeline is VERY SHORT
    • Client-api needs to work for N-1
    • any surprises will prob abort the effort

QE

  • Concern about core-update and N/N-1 api access requirement?
    • Iballou to experiment pre-snap/pre-QE
    • Vsedmik to coord w/iballou to get test-scenarios run automatically
    • Dalley proposes to run old-bindings against new-core and give feedback
    • Also katello VCR runs

Build Team

  • If core/3.40 is a Bridge Too Far, should we just choose core/3.39 from the start?
1 Like
#95

2023-11-15

Overview

  • Katello Schedule

Pulp

Katello

  • “Goin crazy on core/3.39 pkg-building Fun”
  • Final fit-and-finish in progress on that effort

QE

  • “So far so good”
  • Testing against N-1 setup w/ core/3.39, also “so far so good”
#96

2023-12-13

Overview

  • Katello Schedule

Pulp

  • Last meeting of the year!
    • Is this meeting still useful?
    • Is the cadence still useful?
    • Does the meeting time still work?
    • ggainey to schedule 6 months in 2024, if so
  • Pulpcore
  • RPM
    • Zstd publishing support added
      • Works for RHEL8.1+/9/10 (nut not RHEL7, no yum support there)
      • Not merged/released into pulp_rpm
      • Not the default, just a new option available
      • Could be exposed by a katello-UI widget “at some point”
    • Publishing with checksums weaker than sha256 removed
      • see 13-NOV minutes
      • Will be in pulp_rpm/3.25
      • Katello will need to decide what to do w/ the csum-choice-pulldown at publish button
      • EL5 may not support sha256 - but it has other, larger, issues w/ “modern” katello
      • Mirroring metadata works fine for older/ancient distributions
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica
  • N-1 problem
    • Is the communication between Satellite and its Capsules isolated at all?
    • There is a decent amount of separation - maybe we can make changes in just-smart-proxy comms to deal with this

Katello

  • Pulpcore nightly builds & Katello’s new stricter upgrade plan
    • Pulp upgrade was WAY more than anticipated
    • In future, will select the pulp-release for a branch at the branch-point, or very close to
    • Pretty close to having up-to-date pulpcore “nightly”
    • Env updates (ie, python3.next-X) - even more conservative

QE

#97

2024–01-10

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Working on Sat-Next BZs as our priority
    • Dalley out Tuesday - Friday next week
    • 2 weeks before BZ freeze (24-JAN)
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Time to think about Pulp versions for:
    • Katello 4.12 (need decision around Feb 1st)
    • Katello 4.13 / Satellite 6.16
    • Currently at core/3.43, next potential breaking-change release is core/3.55
    • Suggestion: Pick whatever is current as of 30-JAN
      • Are there any features that are not already in Pulp that are desired/required for katello-4.12/13?
      • Iballou to investigate
    • Def need to understand build-implications (if any)
      • Python-version-issues (3.11) already resolved
        • Modularity-fun ensued, but is resolved

QE

  • not many updates - Sat-Next BZs under verification - 5 ON_QA / 3 VERIFIED

2024-01-24

Overview

  • Katello Schedule

Pulp

Katello

QE

  • we are approx in the first third of Sat-Next verification