Katello/Pulp3 Integration meeting

Minutes
#75

2023-02-1

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Monday: meeting to discuss dependency solving issues and future
    • Dalley action items:
      • Need to hunt down HCaaS use case investigation data re: incremental update
      • Tying up all of the “RPM name” issues / BZs
    • Zstd compressed metadata support probably being added soon
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

QE

  • 6.12.2 BZs verification
  • 6.11 → 6.12 upgrade - pulpcore migration fail - is it familiar to anyone?
    • “AttributeError: ‘Settings’ object has no attribute ‘CONTENT_ORIGIN’"
      • If CONTENT_ORIGIN isn’t set, your Satellite isn’t installed correctly
    • Try “sudo -u pulp PULP_SETTINGS=’/etc/pulp/settings.py’ /usr/bin/pulpcore-manager migrate”
#76

2023-02-08

Overview

  • Katello Schedule

Pulp

  • Lots of folk prepping for/travelling to/at FOSDEM and CfgMgtCamp
  • Pulpcore
  • RPM
    • Tying up all of the “RPM name” issues / BZs
      • There’s 2 more now than there was earlier, so now trying to tie those together as well
      • ~4 BZs, 1 other customer case, ~2 upstream issues
      • At this point resolving all the confusion is collectively a high priority
      • Biggest problem is that there’s some overlap with a perennially very difficult issue, “relative path problem” which may require architectural adjustments to come to a solution that makes everyone happy
      • On the bright side it might be more addressable now than it once was
    • 3.19 released
      • Includes RBAC support
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Looking less likely that we’ll move to Pulpcore 3.22 for katello/4.8
    • Build Gang may be making progress? Iballou to check
    • Maybe start thinking about doing a core/3.xx upgrade in the katello/4.9 branch
  • An rpm-name-issue reproducer can be found 2162591 – Oracle JDK rpms no longer work
    • Difference between katello and hammer uploads?

QE

  • Expect questions from QE next week to Pulp on testing
#77

2023-02-15

Overview

  • Katello Schedule

Pulp

Katello

QE

#78

2023-02-22

Overview

  • Katello Schedule

Pulp

  • Pulpcore

  • RPM

  • Ansible

  • Pulp Container

  • OSTree

  • Pulp CLI

    • pulp-cli/pulp-glue split caused some Fun Times in CI

  • Pulp Replica

  • When should we hold this meeting?

    • [ian] Katello is moving the prior meeting back 15 minutes.
    • [ggainey] cheers wildly

Katello

QE

Build Team

#79

2023-03-01

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Fixed https://github.com/pulp/pulp_rpm/issues/2961
      • Backported to 3.19
    • Improving end-to-end metadata integrity tests
    • Have not been able to reproduce the module API appearance differences yet? Tried with 3.17, 3.19, 3.17 → 3.19
      • Array-of-strings vs JSON
      • Iballou modified katello to handle both scenarios
    • Lots of new BZs incoming :slightly_frowning_face:
      • Help prioritizing?
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica
    • Merging into pulpcore “really soon” (for core/3.23)
    • Needs some plugin support as well
    • Needs content-protection before integration w/ katello
    • Discussion: what about ACS?
      • It works to the degree that ACS is set up ‘downstream’

Katello

  • Dep-solving discussions - prob needs some resolution
    • Small incremental improvements in workflow/docs
    • Needs an epic for katello/4.9

QE

Build Team

#80

2023-03-08

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
    • Release coming #soon - mostly w/ domain support
  • Pulp Replica
    • Will be in core/3.23
    • Rpm-support-pr is opened
  • Review meeting cadence
    • can/should we move to every-other-week?
    • Sjha - maybe try every-other. Meeting is useful
    • Dralley, vsedmik - +1
    • Add new QE rep (sweta)
    • AI: ggainey to move this to every-other-week as an experiment

Katello

  • Starting to test out pulp replica w.r.to POP and smart proxy use cases.
    • Sjha has a card
  • Domain work is also exciting - yay!
  • How big of a no-no is a flag on a repository telling us if repo has been synced with immediate and bits exist on filesystem?
    • More specifically - knowing that all content-artifacts exist on disk
  • “How much disk space is being taken up by a repository”
    • The perennial question
    • Dralley has a tool for rpm-repos that “sort of” answers this
    • COPR support may want something similar
    • “Its Very Complicated”
      • Artifacts are deduplicated
      • Metadata takes space
      • Postgres takes space
      • On-demand doesn’t take space (but could)
      • Deduplicating filesystems make things even more interesting

QE

  • No major updates
    • Lots of BZ work (compressed comps file)
1 Like
#81

2023-03-22

Overview

  • Katello Schedule

Pulp

Katello

  • Looks like 4.8 should be out “soon”, is that correct? Yes. Nightlies are green and so is 4.8 after pulp_rpm 3.19.2. We have an action item to re-record our VCRs assuming format of data returned is changed.
    • Does katello turn on pulp-analytics in the upstream? - No
    • Still opt-in on installer
  • Working on refining epics for Next-Y
    • Doesn’t look like many Pulp impacts (yet)
  • katello/4.8 looks to be unblocked now, woohoo!

QE

  • no updates at all - all ON_QA 6.13 BZs have been resolved
#82

2023-04-05

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Shweta picked up incremental-export-sync after switching from on_demand-to-immediate
#83

2023-04-19

Overview

  • Katello Schedule

Pulp

Katello

  • Any upgrade wishes for the next few Katello releases?
    • core/3.23 has domain work
    • core/3.25 will have django4
    • katello/4.9 branching in “a month”, core/3.22
    • Maybe katello/4.10 will aim for core/3.25(ish)

QE

#84

[missed posting previous minutes]

2023-05-03

Overview

  • Katello Schedule

Pulp

  • Pulpcore
    • Query of all changes from last two weeks
    • core/3.24 released
      • Having Some Issues, will update
    • core/3.25 is coming #soon (May-ish)
      • Moves to Django4.2, psycopg3
      • katello/4.Next should prob think about being core/3.25 or later
  • RPM
    • Fix for the profiles issue that Ian reported on Slack is ongoing
    • 3.20 release today
      • No domains yet, alas
  • Ansible
  • Pulp Container
    • Plugin leadership change, goes to Lubos!
    • Ggainey will add him to invite
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Core-repair API will be in-use for capsules
    • Nudge lmjachky about container-issues involving repair
    • Katello will need fixes “soon”
      • Added prio-list label on repair related issues
    • Thoughts about large/manyrepo/huge-dataset systems - per-repo, or global-repair?
    • Discussion needs to happen! Poke lmjachky?

QE

  • Nothing to yell about
  • Sat6.13 released today!
    • And there was much rejoicing
#85

2023-05-17

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • 3.17.18 released
    • 3.21.0 release soon
      • Mostly domains, core/3.25 required
  • Ansible
  • Pulp Container
    • Podman-push-repo-versioning is changing
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

QE

Build Team

  • core/3.25 and Django4 is coming - heads up!
#86

2023-05-31

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Sat-next Snap1 is out, no pulp issues yet
#87

2023-06-14

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Gunicorn “fix”?
    • Cfg-change to auto-restart workers “regularly” (not a pulp-change, a foreman-installer change)
#88

2023-06-28

Overview

  • Katello Schedule

Pulp

Katello

2159105 – Remove deprecated & not working API endpoints from APIdoc (as an example)

QE

#89

2023-07-12

Overview

  • Katello Schedule

Pulp

Katello

  • 2219885 – Satellite using the feature 'Alternate Content Source' is getting errors
    • Not quite clear on what is going on here
    • Is there some “tuning” we need to/can do here (esp on the capsule side)
      • Eg, don’t currently have access to optimize/no-optimize on acs refreshes
  • Finishing up Sat-Next testing
  • Still waiting on full pulp-rpms for core/3.28 and friends
    • Some packaging snafu in-progress
  • Hey - what is going on with Flatpak support?
    • Talk to pulp_container gang for discussion-status
  • katello/satellite smart-proxy-sync discussions
    • Would like to insure capsule-N-2 works against satellite-N, always

QE

  • Sat-Next verifications continue apace
  • QE adding a bunch of smart-proxy-sync test cases
2 Likes
#90

2023-07-26

Overview

  • Katello Schedule

Pulp

Katello

QE

  • RE ipanova’s pulp_container issue - QE tests are passing without?
    • Vsedmik to check test-versions, maybe poke ipanova as/if needed
  • https://bugzilla.redhat.com/show_bug.cgi?id=2142689
    • May not be reproducible any more - vsedmik to do some investigation
    • Will prob eventually close? Depends on Sat-Current-Y testing
#91

2023-08-09

Overview

  • Katello Schedule

Pulp

Katello

QE

  • some help needed to reproduce/test 2125366 – Capsule sync fails with Error: It is not possible to add more than one advisory with the same id to a repository version

    • We have an upstream test - let’s get the link for QE to look at

  • How to track 2122872 and 2229788 (gunicorn memory leak Fun)

    • 2122872 originally had a workaround that we were going to move forwards with
      • It was cloned to 6.13.2 as 211957 and pushed, now CLOSED-ERRATA
      • Then we found a better fix, and we want to push that to Next-Z? And Sat-Next
      • We also intend to revert the workaround as it’s no longer needed
    • 2229788 was created for the “actual” fix
    • TL;DR how should this be tracked
      • Dalley suggestion:
        • Leave 2122872 tracking the “meta-issue” of gunicorn memory consumption, including the new lru_cache fix, and the reversion, for Sat-Next
        • Use 2122872 to track the improved fix and the reversion for Next-Z?
        • In both cases, we just test that the gunicorn memory consumption issue is resolved as expected
    • ‘2872 - used to handle the mitigation/restart Fun
    • ‘9788 - used for the ‘real’ fix
    • ‘1957 - cloned ‘2872 mitigation for 6.13 release
    • ‘9788 won’t fix 6.11/6.12 cases (since it’s fixing something that didn’t happen until 6.13)
    • What we settled on:
      • Keep 9788 for separately tracking the lru_cache fix (and set flags to clone it for 6.13)
      • Create a new bug to track the relaxed installer defaults (6.13 cloning optional)
        • Created 2230584
        • flags: 6.14.0?, 6.13.z?
      • QE these together with the original bug, so that QE effort is not duplicated
        • (the “actual” fix + relaxation) for each release
      • Email Brad, Justin and Vijay Singh (QE mngrs), Shweta (QE contact for already delivered mitigations on 6.13) and Vlad (QE just for tracking), and Odilon (delivery) so everyone knows what to expect
      • The new BZs should be triaged on Aug 10th triage meeting, Vlad or William will provide the info.
#92

2023-09-06

Overview

  • Katello Schedule

Pulp

Katello

  • Quba42 and pulp_deb/3.0
    • discussion/investigation in progress Right Now
    • Trying to see if we can fit into katello/4.10 release
    • Stay tuned!
  • BZ RE “tell users if module-streams have expired”?
    • Is this even available? (looks like Not)

QE

  • Couple BZs still being verified
#93

(Catching up on notes - usual poster was unavailable)

2023-10-18

Overview

  • Katello Schedule

Pulp

Katello

  • 4.10 GA “imminent”, a lot of new core/3.28 users incoming!
  • 4.11 will be branching Real Soon Now
    • Once that happens, will be upgrading Pulp again
    • ggainey to update support-matrix - 4.11 versions same as 4.10

QE

  • Reviewed import/export perf-BZs from a test standpoint, looks good
    • QE ready to get the releases w/ the fixes

2023-10-04

Overview

  • Katello Schedule

Pulp

Katello

2023-09-20

Overview

  • Katello Schedule

Pulp

Katello

  • Need backport of https://github.com/pulp/pulp_ansible/pull/1561/ to fix syncing ansible content to smart proxies for Katello 4.10 (GAing next week)
    • Chatting with Gerrod about it now, awaiting opinions from Matthias
    • A backport to 0.18.0 would be ideal, but we could work with a release if it’s compatible with Pulpcore 3.28.
#94

2023-11-01

Overview

  • Katello Schedule

Pulp

Katello

  • PulpCon schedule shared to the rest of the team, yay!
  • Pulpcore upgrade
    • Trying to get 3.40 into katello/4.11
    • Timeline is VERY SHORT
    • Client-api needs to work for N-1
    • any surprises will prob abort the effort

QE

  • Concern about core-update and N/N-1 api access requirement?
    • Iballou to experiment pre-snap/pre-QE
    • Vsedmik to coord w/iballou to get test-scenarios run automatically
    • Dalley proposes to run old-bindings against new-core and give feedback
    • Also katello VCR runs

Build Team

  • If core/3.40 is a Bridge Too Far, should we just choose core/3.39 from the start?
1 Like