Katello/Pulp3 Integration meeting

Minutes
#72

2023-01-11

Overview

  • Katello Schedule

Pulp

  • Pulpcore
    • Query of all changes from last week
    • core/3.22 released, 3.22.1 Real Soon Now to fix a migration issue
    • Should katello-next update to 3.22?
      • Leaning towards “no”, depends on features
      • When does katello need to make a call?
    • A couple backports went out yesterday for various BZs
      • core/3.18 and core/3.21 updated
  • RPM
    • Modulemd is proposing to add/be-changed-to .xml (instead of yaml) - does anything in katello (applicability?) look at the modulemd yaml directly?
      • Nope! I didn’t even know we had access to that. [Ian]
    • RHEL moving forwards with a change to stop shipping un-compressed comps.xml
      • Or maybe not. We’re talking through some issues with the DNF team
      • May impact which pulp versions support future versions of RHEL 9.Y
      • We can/will do backports if necessary
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Meeting Review
    • Is this useful?
    • Is the cadence useful?
    • Is the format useful?
    • Is the invite-list correct?
    • Is “katello issues” link above useful at all? (suspect not)
      • Current query not relevant
      • No current equivalent
    • Think about these questions, decisions next week

Katello

QE

  • No updates
  • BZ-verification underway for next-Y and next-Z

Build Team

#73

2023-01-18

Overview

  • Katello Schedule

Pulp

  • Meeting Review
    • Is this useful?
      • Does Ansible need to actually be here?
      • Maybe just add report to the doc prior to the mtg
    • Is the cadence useful?
      • Seems to be
    • Is the format useful?
    • Is the invite-list correct?
      • Corrected with input
  • Pulpcore
  • RPM
    • Scheduling a discussion about possibly restructuring the incremental update features to be more foolproof, possibly eliminate depsolving
      • RHEL8, libsolv, and module Sadness
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI

Katello

QE

Katello/Pulp-Community Integration meeting minutes
#74

2023-01-25

Overview

  • Katello Schedule

Pulp

Katello

  • 2162591 – Oracle JDK rpms no longer work
  • Upcoming CDN changes that block crawling – any updates needed?
    • Will Pulp also change to block crawling for protected repositories?
      • pulp doesn’t do index.htmls, either to sync or as part of its repos - unaffected
1 Like
#75

2023-02-1

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Monday: meeting to discuss dependency solving issues and future
    • Dalley action items:
      • Need to hunt down HCaaS use case investigation data re: incremental update
      • Tying up all of the “RPM name” issues / BZs
    • Zstd compressed metadata support probably being added soon
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

QE

  • 6.12.2 BZs verification
  • 6.11 → 6.12 upgrade - pulpcore migration fail - is it familiar to anyone?
    • “AttributeError: ‘Settings’ object has no attribute ‘CONTENT_ORIGIN’"
      • If CONTENT_ORIGIN isn’t set, your Satellite isn’t installed correctly
    • Try “sudo -u pulp PULP_SETTINGS=’/etc/pulp/settings.py’ /usr/bin/pulpcore-manager migrate”
#76

2023-02-08

Overview

  • Katello Schedule

Pulp

  • Lots of folk prepping for/travelling to/at FOSDEM and CfgMgtCamp
  • Pulpcore
  • RPM
    • Tying up all of the “RPM name” issues / BZs
      • There’s 2 more now than there was earlier, so now trying to tie those together as well
      • ~4 BZs, 1 other customer case, ~2 upstream issues
      • At this point resolving all the confusion is collectively a high priority
      • Biggest problem is that there’s some overlap with a perennially very difficult issue, “relative path problem” which may require architectural adjustments to come to a solution that makes everyone happy
      • On the bright side it might be more addressable now than it once was
    • 3.19 released
      • Includes RBAC support
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Looking less likely that we’ll move to Pulpcore 3.22 for katello/4.8
    • Build Gang may be making progress? Iballou to check
    • Maybe start thinking about doing a core/3.xx upgrade in the katello/4.9 branch
  • An rpm-name-issue reproducer can be found 2162591 – Oracle JDK rpms no longer work
    • Difference between katello and hammer uploads?

QE

  • Expect questions from QE next week to Pulp on testing
#77

2023-02-15

Overview

  • Katello Schedule

Pulp

Katello

QE

#78

2023-02-22

Overview

  • Katello Schedule

Pulp

  • Pulpcore

  • RPM

  • Ansible

  • Pulp Container

  • OSTree

  • Pulp CLI

    • pulp-cli/pulp-glue split caused some Fun Times in CI

  • Pulp Replica

  • When should we hold this meeting?

    • [ian] Katello is moving the prior meeting back 15 minutes.
    • [ggainey] cheers wildly

Katello

QE

Build Team

#79

2023-03-01

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Fixed https://github.com/pulp/pulp_rpm/issues/2961
      • Backported to 3.19
    • Improving end-to-end metadata integrity tests
    • Have not been able to reproduce the module API appearance differences yet? Tried with 3.17, 3.19, 3.17 → 3.19
      • Array-of-strings vs JSON
      • Iballou modified katello to handle both scenarios
    • Lots of new BZs incoming :slightly_frowning_face:
      • Help prioritizing?
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica
    • Merging into pulpcore “really soon” (for core/3.23)
    • Needs some plugin support as well
    • Needs content-protection before integration w/ katello
    • Discussion: what about ACS?
      • It works to the degree that ACS is set up ‘downstream’

Katello

  • Dep-solving discussions - prob needs some resolution
    • Small incremental improvements in workflow/docs
    • Needs an epic for katello/4.9

QE

Build Team

#80

2023-03-08

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
    • Release coming #soon - mostly w/ domain support
  • Pulp Replica
    • Will be in core/3.23
    • Rpm-support-pr is opened
  • Review meeting cadence
    • can/should we move to every-other-week?
    • Sjha - maybe try every-other. Meeting is useful
    • Dralley, vsedmik - +1
    • Add new QE rep (sweta)
    • AI: ggainey to move this to every-other-week as an experiment

Katello

  • Starting to test out pulp replica w.r.to POP and smart proxy use cases.
    • Sjha has a card
  • Domain work is also exciting - yay!
  • How big of a no-no is a flag on a repository telling us if repo has been synced with immediate and bits exist on filesystem?
    • More specifically - knowing that all content-artifacts exist on disk
  • “How much disk space is being taken up by a repository”
    • The perennial question
    • Dralley has a tool for rpm-repos that “sort of” answers this
    • COPR support may want something similar
    • “Its Very Complicated”
      • Artifacts are deduplicated
      • Metadata takes space
      • Postgres takes space
      • On-demand doesn’t take space (but could)
      • Deduplicating filesystems make things even more interesting

QE

  • No major updates
    • Lots of BZ work (compressed comps file)
1 Like
#81

2023-03-22

Overview

  • Katello Schedule

Pulp

Katello

  • Looks like 4.8 should be out “soon”, is that correct? Yes. Nightlies are green and so is 4.8 after pulp_rpm 3.19.2. We have an action item to re-record our VCRs assuming format of data returned is changed.
    • Does katello turn on pulp-analytics in the upstream? - No
    • Still opt-in on installer
  • Working on refining epics for Next-Y
    • Doesn’t look like many Pulp impacts (yet)
  • katello/4.8 looks to be unblocked now, woohoo!

QE

  • no updates at all - all ON_QA 6.13 BZs have been resolved
#82

2023-04-05

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Shweta picked up incremental-export-sync after switching from on_demand-to-immediate
#83

2023-04-19

Overview

  • Katello Schedule

Pulp

Katello

  • Any upgrade wishes for the next few Katello releases?
    • core/3.23 has domain work
    • core/3.25 will have django4
    • katello/4.9 branching in “a month”, core/3.22
    • Maybe katello/4.10 will aim for core/3.25(ish)

QE

#84

[missed posting previous minutes]

2023-05-03

Overview

  • Katello Schedule

Pulp

  • Pulpcore
    • Query of all changes from last two weeks
    • core/3.24 released
      • Having Some Issues, will update
    • core/3.25 is coming #soon (May-ish)
      • Moves to Django4.2, psycopg3
      • katello/4.Next should prob think about being core/3.25 or later
  • RPM
    • Fix for the profiles issue that Ian reported on Slack is ongoing
    • 3.20 release today
      • No domains yet, alas
  • Ansible
  • Pulp Container
    • Plugin leadership change, goes to Lubos!
    • Ggainey will add him to invite
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Core-repair API will be in-use for capsules
    • Nudge lmjachky about container-issues involving repair
    • Katello will need fixes “soon”
      • Added prio-list label on repair related issues
    • Thoughts about large/manyrepo/huge-dataset systems - per-repo, or global-repair?
    • Discussion needs to happen! Poke lmjachky?

QE

  • Nothing to yell about
  • Sat6.13 released today!
    • And there was much rejoicing
#85

2023-05-17

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • 3.17.18 released
    • 3.21.0 release soon
      • Mostly domains, core/3.25 required
  • Ansible
  • Pulp Container
    • Podman-push-repo-versioning is changing
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

QE

Build Team

  • core/3.25 and Django4 is coming - heads up!
#86

2023-05-31

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Sat-next Snap1 is out, no pulp issues yet
#87

2023-06-14

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Gunicorn “fix”?
    • Cfg-change to auto-restart workers “regularly” (not a pulp-change, a foreman-installer change)
#88

2023-06-28

Overview

  • Katello Schedule

Pulp

Katello

2159105 – Remove deprecated & not working API endpoints from APIdoc (as an example)

QE

#89

2023-07-12

Overview

  • Katello Schedule

Pulp

Katello

  • 2219885 – Satellite using the feature 'Alternate Content Source' is getting errors
    • Not quite clear on what is going on here
    • Is there some “tuning” we need to/can do here (esp on the capsule side)
      • Eg, don’t currently have access to optimize/no-optimize on acs refreshes
  • Finishing up Sat-Next testing
  • Still waiting on full pulp-rpms for core/3.28 and friends
    • Some packaging snafu in-progress
  • Hey - what is going on with Flatpak support?
    • Talk to pulp_container gang for discussion-status
  • katello/satellite smart-proxy-sync discussions
    • Would like to insure capsule-N-2 works against satellite-N, always

QE

  • Sat-Next verifications continue apace
  • QE adding a bunch of smart-proxy-sync test cases
2 Likes
#90

2023-07-26

Overview

  • Katello Schedule

Pulp

Katello

QE

  • RE ipanova’s pulp_container issue - QE tests are passing without?
    • Vsedmik to check test-versions, maybe poke ipanova as/if needed
  • https://bugzilla.redhat.com/show_bug.cgi?id=2142689
    • May not be reproducible any more - vsedmik to do some investigation
    • Will prob eventually close? Depends on Sat-Current-Y testing
#91

2023-08-09

Overview

  • Katello Schedule

Pulp

Katello

QE

  • some help needed to reproduce/test 2125366 – Capsule sync fails with Error: It is not possible to add more than one advisory with the same id to a repository version

    • We have an upstream test - let’s get the link for QE to look at

  • How to track 2122872 and 2229788 (gunicorn memory leak Fun)

    • 2122872 originally had a workaround that we were going to move forwards with
      • It was cloned to 6.13.2 as 211957 and pushed, now CLOSED-ERRATA
      • Then we found a better fix, and we want to push that to Next-Z? And Sat-Next
      • We also intend to revert the workaround as it’s no longer needed
    • 2229788 was created for the “actual” fix
    • TL;DR how should this be tracked
      • Dalley suggestion:
        • Leave 2122872 tracking the “meta-issue” of gunicorn memory consumption, including the new lru_cache fix, and the reversion, for Sat-Next
        • Use 2122872 to track the improved fix and the reversion for Next-Z?
        • In both cases, we just test that the gunicorn memory consumption issue is resolved as expected
    • ‘2872 - used to handle the mitigation/restart Fun
    • ‘9788 - used for the ‘real’ fix
    • ‘1957 - cloned ‘2872 mitigation for 6.13 release
    • ‘9788 won’t fix 6.11/6.12 cases (since it’s fixing something that didn’t happen until 6.13)
    • What we settled on:
      • Keep 9788 for separately tracking the lru_cache fix (and set flags to clone it for 6.13)
      • Create a new bug to track the relaxed installer defaults (6.13 cloning optional)
        • Created 2230584
        • flags: 6.14.0?, 6.13.z?
      • QE these together with the original bug, so that QE effort is not duplicated
        • (the “actual” fix + relaxation) for each release
      • Email Brad, Justin and Vijay Singh (QE mngrs), Shweta (QE contact for already delivered mitigations on 6.13) and Vlad (QE just for tracking), and Odilon (delivery) so everyone knows what to expect
      • The new BZs should be triaged on Aug 10th triage meeting, Vlad or William will provide the info.