Katello/Pulp3 Integration meeting

Minutes
#96

2023-12-13

Overview

  • Katello Schedule

Pulp

  • Last meeting of the year!
    • Is this meeting still useful?
    • Is the cadence still useful?
    • Does the meeting time still work?
    • ggainey to schedule 6 months in 2024, if so
  • Pulpcore
  • RPM
    • Zstd publishing support added
      • Works for RHEL8.1+/9/10 (nut not RHEL7, no yum support there)
      • Not merged/released into pulp_rpm
      • Not the default, just a new option available
      • Could be exposed by a katello-UI widget “at some point”
    • Publishing with checksums weaker than sha256 removed
      • see 13-NOV minutes
      • Will be in pulp_rpm/3.25
      • Katello will need to decide what to do w/ the csum-choice-pulldown at publish button
      • EL5 may not support sha256 - but it has other, larger, issues w/ “modern” katello
      • Mirroring metadata works fine for older/ancient distributions
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica
  • N-1 problem
    • Is the communication between Satellite and its Capsules isolated at all?
    • There is a decent amount of separation - maybe we can make changes in just-smart-proxy comms to deal with this

Katello

  • Pulpcore nightly builds & Katello’s new stricter upgrade plan
    • Pulp upgrade was WAY more than anticipated
    • In future, will select the pulp-release for a branch at the branch-point, or very close to
    • Pretty close to having up-to-date pulpcore “nightly”
    • Env updates (ie, python3.next-X) - even more conservative

QE

#97

2024–01-10

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Working on Sat-Next BZs as our priority
    • Dalley out Tuesday - Friday next week
    • 2 weeks before BZ freeze (24-JAN)
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Time to think about Pulp versions for:
    • Katello 4.12 (need decision around Feb 1st)
    • Katello 4.13 / Satellite 6.16
    • Currently at core/3.43, next potential breaking-change release is core/3.55
    • Suggestion: Pick whatever is current as of 30-JAN
      • Are there any features that are not already in Pulp that are desired/required for katello-4.12/13?
      • Iballou to investigate
    • Def need to understand build-implications (if any)
      • Python-version-issues (3.11) already resolved
        • Modularity-fun ensued, but is resolved

QE

  • not many updates - Sat-Next BZs under verification - 5 ON_QA / 3 VERIFIED

2024-01-24

Overview

  • Katello Schedule

Pulp

Katello

QE

  • we are approx in the first third of Sat-Next verification
#98

2024-02-07

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • 2253381 investigation continues, now w/ reproducer on customer site
    • 2246247 repo-delete can cause a large memory-spike, dalley investigating
    • 2207867 2151657 repositories can contain duplicate NEVRA if the file paths are different
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • next-Y seems to be moving along well, curr-X has been verified
  • next-Y+1 features in-progress (repo-repair, container-upload for examples )
    • core/3.39 dev-boxes for katello devs - work in progress
  • Flatpak support being discussed

QE

Build Team

#99

Can you expand on this? In particular I would be interested in the versions involved.

#100

This PR Taught https-download to trust system certstore. by ggainey · Pull Request #4951 · pulp/pulpcore · GitHub taught Pulp to know about the system-truststore-certs. Prior to that, trying to access a Remote via an HTTPS proxy that used a different CA than the Remote (which is the usual case!) would fail. (A long time back, you couldn’t even try, because aiohttp didn’t support https-proxies at all until…3.8? Maybe?)

This change requires py3.11 - prior to that, one had to depend on basically monkeypatching an internal attribute to get it to work. See fixes #3036 -Setup the SSL Context correctly for https proxies by parthaa · Pull Request #3038 · pulp/pulpcore · GitHub for what that looked like.

1 Like
#101

2024-02-21

Overview

  • Katello Schedule

Pulp

Katello

QE

2 Likes
#102

2024-03-06

Overview

  • Katello Schedule

Pulp

Katello

  • Choosing pulp-versions for katello413
    • Will wait till post-4.12-GA (19-MAR) to choose
    • Will grab whatever-is-newest
  • Pulp-container: exposing labels/flags?
    • Pulp hoping to merge the work this week
      • Will update supported branches based on the Katello requirements if they pick up the latest available pulp-container version
    • Should make it into katello/4.14 UI
    • Ping iballou as soon as it’s released

QE

  • mostly done with next-Y (two more BZs should land in today’s snap)
  • one to be verified in next-Z
  • Katello-flatpak-test-support still under discussion
#103

2024-03-20

Overview

  • Katello Schedule

Pulp

Katello

  • Orphaned pulp content - #19 by gvde - Support - TheForeman discussion
  • Locking in Katello 4.13 Pulpcore versions imminently
    • Planning to pickup pulp-container after 2.19 assuming it releases not too close to Katello 4.13 branching
    • core/3.49
    • May upgrade to container/2.20 if we can
    • Check in with pulp_rpm at the same time - there may be a new release that can be grabbed as well
  • Plan has begun for integrating with new container manifest labels et. al.

QE

  • Sat-Next/stream : everything VERIFIED!
    • Cheering happened
#104

2024-04-03

Overview

  • Katello Schedule

Pulp

Katello

  • Good discussions w/ ATIX/pulp_deb
  • When is the next pulp_container happening?
    • Iballou to ping lmjachky directly
  • Upgrades looking good so far
  • Query RE “protected repo version” : Github PR ?
    • Can we add a force/cascade flag?
    • Pro: Hard to get from repo-versions to have-distributions for some plugins
    • Con: Distribution is not hard-linked to repo-version
    • Suggestion: can we add an API to make it easier to go from “list of repo-versions” to “distributions pointing to them”
    • Observation: current error-msg doesn’t give us any help
    • AI: sjha to open an RFE - let’s move discussion there
1 Like
#105

2024-04-17

Overview

  • Katello Schedule

Pulp

Katello

  • Core-upgrade is going well
    • Waiting for releases to be RPM-d
    • Should have 3.49 branch soon
    • Looking into capsule-sync issues - may not be Pulp
  • Container-push questions
    • Talk to lmjachky

QE

  • Sat-Current BZs looking good
  • Current testing looks good for Pulp
  • Artifact-to-file linkage question
#107

2024-05-01

Overview

  • Katello Schedule

Pulp

  • Pulpcore
    • Query of all changes from last two weeks
    • core/3.53 and 3.49.6 released
    • Q: recent pulp-container NFS benchmarks
      • New perf-test, or just reporting what Pulp team discovered?
      • Unsure - but probably based on Pulp numbers
  • RPM
    • RE Performance
      • RHEL repos grow over time so can affect numbers
      • Some discussion ensued
  • Ansible
  • Pulp Container
    • Upgrade issue - v1-manifest-support bit us unexpectedly
      • Needed a katello fix
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • QE will be picking up container-push testing “soon”
  • Sat-6.15 release may result in new issues (due to Actual Users)
  • Sat-next testing starting
  • Discussion RE Jira transition
    • Slack integration exists - talk to iballou!
#108

2024-05-15

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Not much to report
  • label/annotation testing of manifests starting next week
    • Expect incoming pings from QE
1 Like
#109

2024-06-12

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM

    • hyu@ addressed another advisory-merge-edge-case (#3587)

    • Publishing checksums - SHA1 Fun in katello

      • Katello addressed the issue
      • Katello now off of SHA1 and also removed deprecated functionality
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
    • Reverted allowing requests-2.32 due to https regressions
  • Pulp Replica
  • Virtual PulpCon will be happening in November-ish
  • Possible in-person team mtg - location/feasibility tbd
    • Sync up w/ local katello/foreman/qe folk?
  • Bindings-generation-discussion continues apace in discourse
    • Current decision: not going to do this “now”
    • won’t block core/3.55
    • Isn’t a “breaking change” in the context of the plugin-api - so doesn’t really need to wait on a core-breaking-change release
    • Lots of discussion still to be had in discourse thread

Katello

  • Container push support nearing completion
    • Under “heavy testing”, yay!
  • Next up: upgrading to rpm/3.26

QE

  • Verifications for Sat-Next are in-train
1 Like
#110

2024-06-26

Overview

  • Katello Schedule

Pulp

Katello

  • Repo creation errors out with Ruby bindings · Issue #3639 · pulp/pulp_rpm · GitHub
    • Blocking RPM 3.27 upgrade
    • I’d offer to regenerate the bindings myself but we’d need to upload them somewhere, and they’ll likely get overwritten by the next z-stream upgrade of pulp-rpm.
    • Collides with delivery-pipeline process
    • Can we get some more problem-determination from the Ruby side?
  • Katello would like to be on more-recent-rpm if possible

QE

  • Sat-Current verifications in progress, no blockers AFAIK
  • Some discussion about/around automating testing a scenario that requires a HUGE amount of data - prob will end up in performance-test-harness
1 Like
#111

2024-07-10

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • ACS/refresh bug updated, work in progress
  • Ansible
  • Pulp Container
    • core/3.55 compat-release incoming
  • OSTree
  • Pulp CLI
  • Pulp Replica
  • “Fun with Jira” continues - please nag us if it looks like we’re missing/messing up something (poke dalley)
    • Discussion ensues
  • Pulp team will be in Brno 2-6 SEP
  • Thinking about a Virtual PulpCON, first week(ish) of NOV

Katello

  • rpm/3.26 packaged, being consumed by katello
    • AI: Dalley to add 3.26 to pulp_rpm supported versions

QE

  • kudos to Ina, Luboš, Ian for fast fix of the header issue in Sat-Curr-Z
#112

2024-07-24

Overview

  • Katello Schedule

Pulp

Katello

2 Likes
#113

2024-08-07

Overview

  • Katello Schedule

Pulp

  • Pulpcore

  • RPM

    • Investigating publication-checksum-behavior issue
      • Syncing sha1-repo and then publishing resulted in sha1-publication
      • This is “expected behavior” - but may not make sense with current reality
      • Katello will adjust by requesting specific-csum-on-publish

  • Ansible

  • Pulp Container

  • OSTree

  • Pulp CLI

  • Pulp Replica

Katello

QE

#114

2024-08-21

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Discuss use of is_modular flag in katello
      • To be discussed “soon”, but not needed immediately
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Publish and Promote slow with 3.11/4.13 - #11 by gvde - Support - TheForeman
    • Pointers for adding Pulp DB indexes? We need to make sure it won’t break the next DB migration which will officially create the indexes.
  • Container content questions
    • Do child manifests and manifest lists both get tags?
      • Seems both can be tagged
      • Lubos: Depends
    • Diff between labels and annotations?
      • Labels come from image configs
      • Annotations come from image manifests
      • Annotations were created for the OCI spec
    • Do child manifests and manifest lists get annotations
    • Should image-mode hosts be attached to manifests or manifest lists?
      • bootc images used – perhaps they can be manifests or lists?
      • Lubos+Ina: bootc implements container native tooling, so whatever the tag is pointing to it will figure things out; thus, we can operate on lists and regular manifests
    • Should is-flatpak be shown on manifest lists or just manifests?
      • It can apply to either
  • Container bug (?)
    • Katello will open an issue w/ reproducer for below so it can be investigated

1600×236 117 KB

QE

  • https://issues.redhat.com/browse/SAT-23051 - bumped into this one - looks like the fix is present in stream snap 71 but the card is still InProgress. Should be in Review instead?
    • There is a Big Automation Hole that needs to be filled by Not-The-Pulp-Team
#115

2024-09-18

Overview

  • Katello Schedule

Pulp

Katello

  • Pulpcore upgrade “soon” - will be coordinating w/ some community needs
    • Decision in “a month(ish)”
  • RHEL10 client/repo testing in progress with early releases

QE

  • last (hopefully) 6.16.0 bug has been verified :tada:
    • thanks to Luboš for help with pulpcore-manager to seed 100k pulp tasks
#116

2024-10-30

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Backports released for some fixes
      • Which caused bindings-issues that also needed a backport
    • Security review done
  • Ansible
  • Pulp Container
  • OSTree
    • Released a fix for the bindings issue(?)
      • Please, test the build with pulp-ostree 2.4.4.
    • Humberto will be the main contact.
  • Pulp CLI
  • Pulp Replica

Katello