Katello/Pulp3 Integration meeting

Minutes
#61

September 07, 2022

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • 3.18 blockers in the process of being merged
      • Still minor cleanup to finish up pre-release
    • Considering some changes that would allow pulp_rpm 3.18 to be used alongside pulpcore 3.18
      • Currently, as mentioned last week, it contains a feature that relies on pulpcore 3.19
    • Coordination going on with certain BZs / customer cases, hotfixes
  • Migration
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI

Katello

  • Testathon in-progress, so changes are on-hold for two weeks
  • Waiting patiently for core/3.21
  • RDU Tean meetups next week - should coord w/ pulp RDU meetup

QE

  • next-Z snap 1 bz verification underway
  • next-Y snap 9 bz verification
  • Feature testing
  • Automation failure fixes
  • Waiting on static ouia-id to get into a build
    • Some missing OUIA-ID for cv, sent BZ back to dev
    • Pending - Host details page OUIA-ID PR
  • Question: how long is “too long” to publish a CV where we have added 1 erratum and incrementally updating (ie, we’re doing depsolving)
    • Rhel8_baseos + Rhel8_appstream
      • Created a cv with a filter of before June 30th
      • Publish and did incremental update
        • Incremental update time: 10min 15 sec
        • Publishing cv after inc update: 10min 15 sec
      • Created a filter that includes up to July 31st
      • Publish and did incremental update
        • Incremental update time: 10min 15 sec
        • Publishing cv after inc update: 10min 15 sec
    • rhel7
      • Created a cv with a filter of before June 30th
      • Publish and did incremental update
        • Incremental update time: 9min 9 sec
        • Publishing cv after inc update: 13min 23 sec
      • Created a filter that includes up to July 31st
      • Publish and did incremental update
        • Incremental update time: 8min 49 sec
        • Publishing cv after inc update: 18min 35 sec
        • [Iballou] It would be interesting to see the breakdown between “pulp time” and “katello time” for this
        • Katello-indexing should be much faster now, would be good to know if that isn’t the case
    • Note: depSolving is not selected for these runs, but iirc, doing filtering and publishing does the depsolving?
      • In general usage, it does not - katello only asks Pulp to depsolve, if the user asks for it explicitly
      • Incrementals depsolve by default, so they do
#62

September 21, 2022

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • 3.18.2 is out, I had a preliminary agreement from Sat that they would be OK with having this in 6.12 - 3.18 is just a generally a small bugfix release that includes some migrations
    • 3 still aligned to sat-next that aren’t complete, 1 in danger of not making the cutoff
  • Migration
    • New release 0.17 , that doesn’t affect katello (plugin-eol-dates)
  • Ansible
    • 0.15 releasing today
      • Requires core/3.21
      • Bugfixes and 3 new features
  • Pulp Container
    • Pulp-container 2.9 and 2.10 is out with bugfixes for sat-next
  • OSTree
  • Pulp CLI

Katello

  • core/3.21 RPMs “almost done”
    • This will be for katello/4.7
  • Question: what needs to happen for https://github.com/pulp/pulpcore/pull/3188?
    • Multiple issues, needs multiple CHANGES/ entries
    • Discussion ensued around multiple-commits or not

QE

#63

September 28, 2022

Overview

  • Katello Schedule

Pulp

Katello

QE

  • next-Z snap 2 bz verification underway
    • Sign off this Friday 9/30
  • next-Y snap 12 bz verification
    • Expecting 2 more snaps
    • ggainey to work w/ ltran to test potential deadlock fixes in ‘real’ environments
  • Feature testing & automation
  • Automation failure fixes
  • Ouia-id for both cv and host details are in
    • Any missing IDs will prob be in later releases
#64

October 5, 2022

Overview

  • Katello Schedule

Pulp

Katello

QE

  • next-next-Z is scoped
  • next-Y snap 13 bz verification
    • One last snap incoming
  • Feature testing & automation
  • Automation failure fixes
  • Ltran last day is 10/7
    • Lai to tell ggainey who the right QE contact will be for this mtg
#65

October 12, 2022

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Um…who will be covering Pulp/this mtg going forward?
    • Katello, Pulp Sat Liason to work w/ QE mgt to get definite answer

Build Team

#66

October 19, 2022

Overview

  • Katello Schedule

Pulp

Katello

  • Pulp Telemetry – are you more interested in users outside of Katello?

    • We’re defaulting it to be off for now, but it can be turned on via custom hiera (Puppet).
    • If you do want Katello user telemetry, we’ll need to figure out how to advertise turning it on.
    • Pulp is interested in katello installs, yes
    • Let’s start with “exposed but opt-in via custom hiera”, sociallize w/ katello/foreman community, and see where we go from there

  • Working on pulpcore 3.21 upgrades. Gems are available. Thanks! Osousa looking into pulp_deb release.

    • Branching for katello-next is Monday(ish)
1 Like
#67

November 2, 2022 (26-OCT skipped)

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Continuing to work on errata issues
    • Progress being made on concurrent repo syncs
    • Some discussion about deprecating default profiles upstream
      • Thinking about how that impacts us as we’ve just added a related feature (to show profile metadata)
    • Lots of BZ cleanup happening (yay dralley!)
  • Migration
    • QE testing for 2to3-migration-version release
    • ggainey working w/ vsedmik
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI

Katello

  • AWS RHUI – What is missing in Katello for basic support?
    • Only the two special headers?
  • https://github.com/pulp/pulp_container/issues/1122
    • Updated client-libraries can’t talk to old-smart-proxy
    • E.g., 3.21-client can’t talk back to 3.18-pulp
    • AI: sjha to pull a group together to talk through this complicated issue
      • Include vsedmik
  • Reminder – does Pulp always copy an erratum if all of its packages are copied too?
  • Telemetry changes are merged and will be off by default in katello 4.7

QE

  • 3 BZs to be verified for 2to3-migration-version (1 FailedQA but fix PR is already up (thanks ggainey), 2 under test)
#68

November 16, 2022 (9-NOV skipped for PulpCON)

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Domains-in-rpm planning started
    • BZs in progress
  • Migration
    • 0.11.3 released for last 6.9.10 zstream
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI

Katello

  • Conflicting Meetings make moderators Sad

QE

#69

November 30, 2022

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • More progress on dropping the libmodulemd dependency
    • Lots of BZs being escalated by support, trying to investigate
    • ppicka will be leaving RH, RPM Team somewhat undercapacity for a bit
  • Migration
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI

Katello

QE

Build Team

November 23, 2022

Overview

  • Katello Schedule

Pulp

  • Pulpcore
    • Query of all changes from last week
    • Pulpcore 3.22.0 scheduled for Nov 30, but will likely slip by one week
      • All tests converted to Pytest and build team should be able to run them on the RPMs they build.
  • RPM
    • Considering allowing users to skip treeinfo file. How is katello using the skip types?
  • Migration
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI

Katello

  • Pulp_deb monkey patch in the works. Should make it to 4.7 GA.
#70

December 7, 2022

Overview

  • Katello Schedule

Pulp

Katello

QE

#71

(NOTE: this meeting will be held next on 11-JAN-2023! Happy New Year, everyone!)

December 14, 2022

Overview

  • Katello Schedule

Pulp

  • Pulpcore
    • Query of all changes from last week
    • core/3.22 discussions
      • go/nogo tomorrow
      • May not contain the domains work?
    • Merged two PRs that that revealed unexpected plugin impacts
    • Incremental filesystem import/export
      • Not implemented yet
      • Causing support heartaches
  • RPM
    • 2 BZs nearing resolution
      • Dup-advisory
      • modulemd-multiple-get
    • Pulp_rpm is mostly ggainey and dalley
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • No meeting 21/28-DEC or 04-JAN-2023
    • ggainey to cancel invites

Katello

QE

  • testing deadlock BZ - reproducible in 6.12.0 with rhscl, NOT reproducible in 6.12.1 with bigger rhel7 repo (5 CVs, on_demand on SAT side, immediate on CAPS side) - so far looks good!
#72

2023-01-11

Overview

  • Katello Schedule

Pulp

  • Pulpcore
    • Query of all changes from last week
    • core/3.22 released, 3.22.1 Real Soon Now to fix a migration issue
    • Should katello-next update to 3.22?
      • Leaning towards “no”, depends on features
      • When does katello need to make a call?
    • A couple backports went out yesterday for various BZs
      • core/3.18 and core/3.21 updated
  • RPM
    • Modulemd is proposing to add/be-changed-to .xml (instead of yaml) - does anything in katello (applicability?) look at the modulemd yaml directly?
      • Nope! I didn’t even know we had access to that. [Ian]
    • RHEL moving forwards with a change to stop shipping un-compressed comps.xml
      • Or maybe not. We’re talking through some issues with the DNF team
      • May impact which pulp versions support future versions of RHEL 9.Y
      • We can/will do backports if necessary
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Meeting Review
    • Is this useful?
    • Is the cadence useful?
    • Is the format useful?
    • Is the invite-list correct?
    • Is “katello issues” link above useful at all? (suspect not)
      • Current query not relevant
      • No current equivalent
    • Think about these questions, decisions next week

Katello

QE

  • No updates
  • BZ-verification underway for next-Y and next-Z

Build Team

#73

2023-01-18

Overview

  • Katello Schedule

Pulp

  • Meeting Review
    • Is this useful?
      • Does Ansible need to actually be here?
      • Maybe just add report to the doc prior to the mtg
    • Is the cadence useful?
      • Seems to be
    • Is the format useful?
    • Is the invite-list correct?
      • Corrected with input
  • Pulpcore
  • RPM
    • Scheduling a discussion about possibly restructuring the incremental update features to be more foolproof, possibly eliminate depsolving
      • RHEL8, libsolv, and module Sadness
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI

Katello

QE

Katello/Pulp-Community Integration meeting minutes
#74

2023-01-25

Overview

  • Katello Schedule

Pulp

Katello

  • 2162591 – Oracle JDK rpms no longer work
  • Upcoming CDN changes that block crawling – any updates needed?
    • Will Pulp also change to block crawling for protected repositories?
      • pulp doesn’t do index.htmls, either to sync or as part of its repos - unaffected
1 Like
#75

2023-02-1

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Monday: meeting to discuss dependency solving issues and future
    • Dalley action items:
      • Need to hunt down HCaaS use case investigation data re: incremental update
      • Tying up all of the “RPM name” issues / BZs
    • Zstd compressed metadata support probably being added soon
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

QE

  • 6.12.2 BZs verification
  • 6.11 → 6.12 upgrade - pulpcore migration fail - is it familiar to anyone?
    • “AttributeError: ‘Settings’ object has no attribute ‘CONTENT_ORIGIN’"
      • If CONTENT_ORIGIN isn’t set, your Satellite isn’t installed correctly
    • Try “sudo -u pulp PULP_SETTINGS=’/etc/pulp/settings.py’ /usr/bin/pulpcore-manager migrate”
#76

2023-02-08

Overview

  • Katello Schedule

Pulp

  • Lots of folk prepping for/travelling to/at FOSDEM and CfgMgtCamp
  • Pulpcore
  • RPM
    • Tying up all of the “RPM name” issues / BZs
      • There’s 2 more now than there was earlier, so now trying to tie those together as well
      • ~4 BZs, 1 other customer case, ~2 upstream issues
      • At this point resolving all the confusion is collectively a high priority
      • Biggest problem is that there’s some overlap with a perennially very difficult issue, “relative path problem” which may require architectural adjustments to come to a solution that makes everyone happy
      • On the bright side it might be more addressable now than it once was
    • 3.19 released
      • Includes RBAC support
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Looking less likely that we’ll move to Pulpcore 3.22 for katello/4.8
    • Build Gang may be making progress? Iballou to check
    • Maybe start thinking about doing a core/3.xx upgrade in the katello/4.9 branch
  • An rpm-name-issue reproducer can be found 2162591 – Oracle JDK rpms no longer work
    • Difference between katello and hammer uploads?

QE

  • Expect questions from QE next week to Pulp on testing
#77

2023-02-15

Overview

  • Katello Schedule

Pulp

Katello

QE

#78

2023-02-22

Overview

  • Katello Schedule

Pulp

  • Pulpcore

  • RPM

  • Ansible

  • Pulp Container

  • OSTree

  • Pulp CLI

    • pulp-cli/pulp-glue split caused some Fun Times in CI

  • Pulp Replica

  • When should we hold this meeting?

    • [ian] Katello is moving the prior meeting back 15 minutes.
    • [ggainey] cheers wildly

Katello

QE

Build Team

#79

2023-03-01

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Fixed https://github.com/pulp/pulp_rpm/issues/2961
      • Backported to 3.19
    • Improving end-to-end metadata integrity tests
    • Have not been able to reproduce the module API appearance differences yet? Tried with 3.17, 3.19, 3.17 → 3.19
      • Array-of-strings vs JSON
      • Iballou modified katello to handle both scenarios
    • Lots of new BZs incoming :slightly_frowning_face:
      • Help prioritizing?
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica
    • Merging into pulpcore “really soon” (for core/3.23)
    • Needs some plugin support as well
    • Needs content-protection before integration w/ katello
    • Discussion: what about ACS?
      • It works to the degree that ACS is set up ‘downstream’

Katello

  • Dep-solving discussions - prob needs some resolution
    • Small incremental improvements in workflow/docs
    • Needs an epic for katello/4.9

QE

Build Team

#80

2023-03-08

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
    • Release coming #soon - mostly w/ domain support
  • Pulp Replica
    • Will be in core/3.23
    • Rpm-support-pr is opened
  • Review meeting cadence
    • can/should we move to every-other-week?
    • Sjha - maybe try every-other. Meeting is useful
    • Dralley, vsedmik - +1
    • Add new QE rep (sweta)
    • AI: ggainey to move this to every-other-week as an experiment

Katello

  • Starting to test out pulp replica w.r.to POP and smart proxy use cases.
    • Sjha has a card
  • Domain work is also exciting - yay!
  • How big of a no-no is a flag on a repository telling us if repo has been synced with immediate and bits exist on filesystem?
    • More specifically - knowing that all content-artifacts exist on disk
  • “How much disk space is being taken up by a repository”
    • The perennial question
    • Dralley has a tool for rpm-repos that “sort of” answers this
    • COPR support may want something similar
    • “Its Very Complicated”
      • Artifacts are deduplicated
      • Metadata takes space
      • Postgres takes space
      • On-demand doesn’t take space (but could)
      • Deduplicating filesystems make things even more interesting

QE

  • No major updates
    • Lots of BZ work (compressed comps file)
1 Like