Katello/Pulp3 Integration meeting

2023-06-14

Overview

  • Katello Schedule

Pulp

Katello

QE

  • Gunicorn “fix”?
    • Cfg-change to auto-restart workers “regularly” (not a pulp-change, a foreman-installer change)

2023-06-28

Overview

  • Katello Schedule

Pulp

Katello

2159105 – Remove deprecated & not working API endpoints from APIdoc (as an example)

QE

2023-07-12

Overview

  • Katello Schedule

Pulp

Katello

  • 2219885 – Satellite using the feature 'Alternate Content Source' is getting errors
    • Not quite clear on what is going on here
    • Is there some “tuning” we need to/can do here (esp on the capsule side)
      • Eg, don’t currently have access to optimize/no-optimize on acs refreshes
  • Finishing up Sat-Next testing
  • Still waiting on full pulp-rpms for core/3.28 and friends
    • Some packaging snafu in-progress
  • Hey - what is going on with Flatpak support?
    • Talk to pulp_container gang for discussion-status
  • katello/satellite smart-proxy-sync discussions
    • Would like to insure capsule-N-2 works against satellite-N, always

QE

  • Sat-Next verifications continue apace
  • QE adding a bunch of smart-proxy-sync test cases
2 Likes

2023-07-26

Overview

  • Katello Schedule

Pulp

Katello

QE

  • RE ipanova’s pulp_container issue - QE tests are passing without?
    • Vsedmik to check test-versions, maybe poke ipanova as/if needed
  • https://bugzilla.redhat.com/show_bug.cgi?id=2142689
    • May not be reproducible any more - vsedmik to do some investigation
    • Will prob eventually close? Depends on Sat-Current-Y testing

2023-08-09

Overview

  • Katello Schedule

Pulp

Katello

QE

  • some help needed to reproduce/test 2125366 – Capsule sync fails with Error: It is not possible to add more than one advisory with the same id to a repository version

    • We have an upstream test - let’s get the link for QE to look at
  • How to track 2122872 and 2229788 (gunicorn memory leak Fun)

    • 2122872 originally had a workaround that we were going to move forwards with
      • It was cloned to 6.13.2 as 211957 and pushed, now CLOSED-ERRATA
      • Then we found a better fix, and we want to push that to Next-Z? And Sat-Next
      • We also intend to revert the workaround as it’s no longer needed
    • 2229788 was created for the “actual” fix
    • TL;DR how should this be tracked
      • Dalley suggestion:
        • Leave 2122872 tracking the “meta-issue” of gunicorn memory consumption, including the new lru_cache fix, and the reversion, for Sat-Next
        • Use 2122872 to track the improved fix and the reversion for Next-Z?
        • In both cases, we just test that the gunicorn memory consumption issue is resolved as expected
    • ‘2872 - used to handle the mitigation/restart Fun
    • ‘9788 - used for the ‘real’ fix
    • ‘1957 - cloned ‘2872 mitigation for 6.13 release
    • ‘9788 won’t fix 6.11/6.12 cases (since it’s fixing something that didn’t happen until 6.13)
    • What we settled on:
      • Keep 9788 for separately tracking the lru_cache fix (and set flags to clone it for 6.13)
      • Create a new bug to track the relaxed installer defaults (6.13 cloning optional)
        • Created 2230584
        • flags: 6.14.0?, 6.13.z?
      • QE these together with the original bug, so that QE effort is not duplicated
        • (the “actual” fix + relaxation) for each release
      • Email Brad, Justin and Vijay Singh (QE mngrs), Shweta (QE contact for already delivered mitigations on 6.13) and Vlad (QE just for tracking), and Odilon (delivery) so everyone knows what to expect
      • The new BZs should be triaged on Aug 10th triage meeting, Vlad or William will provide the info.

2023-09-06

Overview

  • Katello Schedule

Pulp

Katello

  • Quba42 and pulp_deb/3.0
    • discussion/investigation in progress Right Now
    • Trying to see if we can fit into katello/4.10 release
    • Stay tuned!
  • BZ RE “tell users if module-streams have expired”?
    • Is this even available? (looks like Not)

QE

  • Couple BZs still being verified

(Catching up on notes - usual poster was unavailable)

2023-10-18

Overview

  • Katello Schedule

Pulp

Katello

  • 4.10 GA “imminent”, a lot of new core/3.28 users incoming!
  • 4.11 will be branching Real Soon Now
    • Once that happens, will be upgrading Pulp again
    • ggainey to update support-matrix - 4.11 versions same as 4.10

QE

  • Reviewed import/export perf-BZs from a test standpoint, looks good
    • QE ready to get the releases w/ the fixes

2023-10-04

Overview

  • Katello Schedule

Pulp

Katello

2023-09-20

Overview

  • Katello Schedule

Pulp

Katello

  • Need backport of https://github.com/pulp/pulp_ansible/pull/1561/ to fix syncing ansible content to smart proxies for Katello 4.10 (GAing next week)
    • Chatting with Gerrod about it now, awaiting opinions from Matthias
    • A backport to 0.18.0 would be ideal, but we could work with a release if it’s compatible with Pulpcore 3.28.

2023-11-01

Overview

  • Katello Schedule

Pulp

Katello

  • PulpCon schedule shared to the rest of the team, yay!
  • Pulpcore upgrade
    • Trying to get 3.40 into katello/4.11
    • Timeline is VERY SHORT
    • Client-api needs to work for N-1
    • any surprises will prob abort the effort

QE

  • Concern about core-update and N/N-1 api access requirement?
    • Iballou to experiment pre-snap/pre-QE
    • Vsedmik to coord w/iballou to get test-scenarios run automatically
    • Dalley proposes to run old-bindings against new-core and give feedback
    • Also katello VCR runs

Build Team

  • If core/3.40 is a Bridge Too Far, should we just choose core/3.39 from the start?
1 Like

2023-11-15

Overview

  • Katello Schedule

Pulp

Katello

  • “Goin crazy on core/3.39 pkg-building Fun”
  • Final fit-and-finish in progress on that effort

QE

  • “So far so good”
  • Testing against N-1 setup w/ core/3.39, also “so far so good”

2023-12-13

Overview

  • Katello Schedule

Pulp

  • Last meeting of the year!
    • Is this meeting still useful?
    • Is the cadence still useful?
    • Does the meeting time still work?
    • ggainey to schedule 6 months in 2024, if so
  • Pulpcore
  • RPM
    • Zstd publishing support added
      • Works for RHEL8.1+/9/10 (nut not RHEL7, no yum support there)
      • Not merged/released into pulp_rpm
      • Not the default, just a new option available
      • Could be exposed by a katello-UI widget “at some point”
    • Publishing with checksums weaker than sha256 removed
      • see 13-NOV minutes
      • Will be in pulp_rpm/3.25
      • Katello will need to decide what to do w/ the csum-choice-pulldown at publish button
      • EL5 may not support sha256 - but it has other, larger, issues w/ “modern” katello
      • Mirroring metadata works fine for older/ancient distributions
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica
  • N-1 problem
    • Is the communication between Satellite and its Capsules isolated at all?
    • There is a decent amount of separation - maybe we can make changes in just-smart-proxy comms to deal with this

Katello

  • Pulpcore nightly builds & Katello’s new stricter upgrade plan
    • Pulp upgrade was WAY more than anticipated
    • In future, will select the pulp-release for a branch at the branch-point, or very close to
    • Pretty close to having up-to-date pulpcore “nightly”
    • Env updates (ie, python3.next-X) - even more conservative

QE

2024–01-10

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Working on Sat-Next BZs as our priority
    • Dalley out Tuesday - Friday next week
    • 2 weeks before BZ freeze (24-JAN)
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Time to think about Pulp versions for:
    • Katello 4.12 (need decision around Feb 1st)
    • Katello 4.13 / Satellite 6.16
    • Currently at core/3.43, next potential breaking-change release is core/3.55
    • Suggestion: Pick whatever is current as of 30-JAN
      • Are there any features that are not already in Pulp that are desired/required for katello-4.12/13?
      • Iballou to investigate
    • Def need to understand build-implications (if any)
      • Python-version-issues (3.11) already resolved
        • Modularity-fun ensued, but is resolved

QE

  • not many updates - Sat-Next BZs under verification - 5 ON_QA / 3 VERIFIED

2024-01-24

Overview

  • Katello Schedule

Pulp

Katello

QE

  • we are approx in the first third of Sat-Next verification

2024-02-07

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • 2253381 investigation continues, now w/ reproducer on customer site
    • 2246247 repo-delete can cause a large memory-spike, dalley investigating
    • 2207867 2151657 repositories can contain duplicate NEVRA if the file paths are different
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • next-Y seems to be moving along well, curr-X has been verified
  • next-Y+1 features in-progress (repo-repair, container-upload for examples )
    • core/3.39 dev-boxes for katello devs - work in progress
  • Flatpak support being discussed

QE

Build Team

Can you expand on this? In particular I would be interested in the versions involved.

This PR Taught https-download to trust system certstore. by ggainey · Pull Request #4951 · pulp/pulpcore · GitHub taught Pulp to know about the system-truststore-certs. Prior to that, trying to access a Remote via an HTTPS proxy that used a different CA than the Remote (which is the usual case!) would fail. (A long time back, you couldn’t even try, because aiohttp didn’t support https-proxies at all until…3.8? Maybe?)

This change requires py3.11 - prior to that, one had to depend on basically monkeypatching an internal attribute to get it to work. See fixes #3036 -Setup the SSL Context correctly for https proxies by parthaa · Pull Request #3038 · pulp/pulpcore · GitHub for what that looked like.

1 Like

2024-02-21

Overview

  • Katello Schedule

Pulp

Katello

QE

2 Likes

2024-03-06

Overview

  • Katello Schedule

Pulp

Katello

  • Choosing pulp-versions for katello413
    • Will wait till post-4.12-GA (19-MAR) to choose
    • Will grab whatever-is-newest
  • Pulp-container: exposing labels/flags?
    • Pulp hoping to merge the work this week
      • Will update supported branches based on the Katello requirements if they pick up the latest available pulp-container version
    • Should make it into katello/4.14 UI
    • Ping iballou as soon as it’s released

QE

  • mostly done with next-Y (two more BZs should land in today’s snap)
  • one to be verified in next-Z
  • Katello-flatpak-test-support still under discussion

2024-03-20

Overview

  • Katello Schedule

Pulp

Katello

  • Orphaned pulp content - #19 by gvde - Support - TheForeman discussion
  • Locking in Katello 4.13 Pulpcore versions imminently
    • Planning to pickup pulp-container after 2.19 assuming it releases not too close to Katello 4.13 branching
    • core/3.49
    • May upgrade to container/2.20 if we can
    • Check in with pulp_rpm at the same time - there may be a new release that can be grabbed as well
  • Plan has begun for integrating with new container manifest labels et. al.

QE

  • Sat-Next/stream : everything VERIFIED!
    • Cheering happened

2024-04-03

Overview

  • Katello Schedule

Pulp

Katello

  • Good discussions w/ ATIX/pulp_deb
  • When is the next pulp_container happening?
    • Iballou to ping lmjachky directly
  • Upgrades looking good so far
  • Query RE “protected repo version” : Github PR ?
    • Can we add a force/cascade flag?
    • Pro: Hard to get from repo-versions to have-distributions for some plugins
    • Con: Distribution is not hard-linked to repo-version
    • Suggestion: can we add an API to make it easier to go from “list of repo-versions” to “distributions pointing to them”
    • Observation: current error-msg doesn’t give us any help
    • AI: sjha to open an RFE - let’s move discussion there
1 Like

2024-04-17

Overview

  • Katello Schedule

Pulp

Katello

  • Core-upgrade is going well
    • Waiting for releases to be RPM-d
    • Should have 3.49 branch soon
    • Looking into capsule-sync issues - may not be Pulp
  • Container-push questions
    • Talk to lmjachky

QE

  • Sat-Current BZs looking good
  • Current testing looks good for Pulp
  • Artifact-to-file linkage question

2024-05-01

Overview

  • Katello Schedule

Pulp

  • Pulpcore
    • Query of all changes from last two weeks
    • core/3.53 and 3.49.6 released
    • Q: recent pulp-container NFS benchmarks
      • New perf-test, or just reporting what Pulp team discovered?
      • Unsure - but probably based on Pulp numbers
  • RPM
    • RE Performance
      • RHEL repos grow over time so can affect numbers
      • Some discussion ensued
  • Ansible
  • Pulp Container
    • Upgrade issue - v1-manifest-support bit us unexpectedly
      • Needed a katello fix
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • QE will be picking up container-push testing “soon”
  • Sat-6.15 release may result in new issues (due to Actual Users)
  • Sat-next testing starting
  • Discussion RE Jira transition
    • Slack integration exists - talk to iballou!