Katello/Pulp3 Integration meeting

Minutes
#113

2024-08-07

Overview

  • Katello Schedule

Pulp

  • Pulpcore

  • RPM

    • Investigating publication-checksum-behavior issue
      • Syncing sha1-repo and then publishing resulted in sha1-publication
      • This is “expected behavior” - but may not make sense with current reality
      • Katello will adjust by requesting specific-csum-on-publish

  • Ansible

  • Pulp Container

  • OSTree

  • Pulp CLI

  • Pulp Replica

Katello

QE

#114

2024-08-21

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Discuss use of is_modular flag in katello
      • To be discussed “soon”, but not needed immediately
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Publish and Promote slow with 3.11/4.13 - #11 by gvde - Support - TheForeman
    • Pointers for adding Pulp DB indexes? We need to make sure it won’t break the next DB migration which will officially create the indexes.
  • Container content questions
    • Do child manifests and manifest lists both get tags?
      • Seems both can be tagged
      • Lubos: Depends
    • Diff between labels and annotations?
      • Labels come from image configs
      • Annotations come from image manifests
      • Annotations were created for the OCI spec
    • Do child manifests and manifest lists get annotations
    • Should image-mode hosts be attached to manifests or manifest lists?
      • bootc images used – perhaps they can be manifests or lists?
      • Lubos+Ina: bootc implements container native tooling, so whatever the tag is pointing to it will figure things out; thus, we can operate on lists and regular manifests
    • Should is-flatpak be shown on manifest lists or just manifests?
      • It can apply to either
  • Container bug (?)
    • Katello will open an issue w/ reproducer for below so it can be investigated

1600×236 117 KB

QE

  • https://issues.redhat.com/browse/SAT-23051 - bumped into this one - looks like the fix is present in stream snap 71 but the card is still InProgress. Should be in Review instead?
    • There is a Big Automation Hole that needs to be filled by Not-The-Pulp-Team
#115

2024-09-18

Overview

  • Katello Schedule

Pulp

Katello

  • Pulpcore upgrade “soon” - will be coordinating w/ some community needs
    • Decision in “a month(ish)”
  • RHEL10 client/repo testing in progress with early releases

QE

  • last (hopefully) 6.16.0 bug has been verified :tada:
    • thanks to Luboš for help with pulpcore-manager to seed 100k pulp tasks
#116

2024-10-30

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Backports released for some fixes
      • Which caused bindings-issues that also needed a backport
    • Security review done
  • Ansible
  • Pulp Container
  • OSTree
    • Released a fix for the bindings issue(?)
      • Please, test the build with pulp-ostree 2.4.4.
    • Humberto will be the main contact.
  • Pulp CLI
  • Pulp Replica

Katello

#117

2024-11-13

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Jira/github communications have been suboptimal
    • Discussion ensues
  • Ansible
  • Pulp Container
    • 2.22 released
  • OSTree
  • Pulp CLI
  • Pulp Replica
  • Pyproject.toml is being experimented with
    • In discussion with odilon

Katello

  • Ian pushing for self-built API bindings for next(ish)-katello-release
    • ggainey cheers wildly
  • Phoenix very likely soon to start team-triaging Pulp Jiras
    • Satellite vs pulp, do the subtask creation, etc

QE

  • Nothing to mention yet
#118

2024-12-11

Overview

  • Katello Schedule

Pulp

  • Pulpcore

    • Query of all changes from last two weeks
    • core/3.20 is officially EOL now
    • Pulp-certguard and python-rhsm
      • Actually isn’t supported?
      • See internal-slack-thread
      • Investigation starting into “can Pulp just deal w/ what it needs directly”
      • Working-group will happen “soon”
    • Need to talk about the PRN migration
      • Want to “get the ball rolling” in Q1
      • No huge time-pressure “now” - but would be good to know what’s happening before it becomes time-critical
      • “Migration” will be…exciting? (lengthy, but not exactly “risky”)
      • RPM advanced copy API?
    • Migration-squashing discussing

  • RPM

  • Ansible

  • Pulp Container

  • OSTree

  • Pulp CLI

  • Pulp Replica

  • Will next meet 8-JAN-2025! Happy Holidays all!

Katello

  • Working on switching from is_bootable/is_flatpak to type field
    • does it go to Sat-Next?
    • Yes - but additive, current-boolean-fields continue to exist
    • May want to add test-scenarios for type-field-testing
  • Working on filtering pulp’s flatpak index to present to registered clients
  • Katello going through + triaging Pulp Jiras (some are quite old)

QE

  • Sat-Next-Z signoff in-progress
  • Discussion RE pure-pulp-robotello-tests and under which team they should be reported
1 Like
#119

2025-01-08

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Curating packages/streams in modules?
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Katello likely to start PRN migration in the next release (or n+1)
    • Branching will happen Feb-18
    • Will be looking to update to core/newest as part of that
      • Wild cheering ensues
    • Also still hoping to take on bindings-generation
      • More cheering
  • Artifact matching query issue hit with flatpak index
    • Might be just old-dev-box weirdness
  • Discussion RE backporting fix that involves a migration
    • Ping pbrochado directly to moe forward w this

QE

2 Likes
#120

2025-01-22

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • New version will need to be released for 3.70 compatibility
      • Ought to finalize PRN support also
      • Advanced-copy-api is missing PRN support - will be addressed ^^
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
    • 0.30 released
  • Pulp Replica
  • Feb-5 mtg will be cancelled - PTO and CfgMgtCamp interfering
    • Katello to work w/ Pulp in matrix/slack to decide on final next-katello-versions of Pulp code

Katello

  • New Pulp upgrade can start in ~ 1 month
    • Feb 18th potential decision-date
  • Pulp-container flatpack index - gerrod’s fix seems good!
    • Just needs 2.22 backport

QE

  • approaching Next-Sat-Z sign-off, no issues directly linked to pulp
#121

2025-02-19

Overview

  • Katello Schedule

Pulp

Katello

  • Katello planning to upgrade Pulp in the next month
    • Need pulp_deb to be core/3.70 available
  • Future research: downstream questions on “content access for Things that are not ‘registered hosts’”
1 Like
#122

2025-03-05

Overview

  • Katello Schedule

Pulp

Katello

  • Tell katello “things that we’d like to use postgres-16 for”
  • Switch to relying on PRNs is recognized as “would be a great idea” by PM - but “not a top priority”

QE

1 Like
#123

2025-03-19

Overview

  • Katello Schedule

Pulp

  • Pulpcore
    • Query of all changes from last two weeks
    • core/3.74 released
    • ACS performance enhancement released/backported
      • We really need some way to reproduce the memory issue w/ ACS, to get a handle on it
      • Needs a satellite-box to repro - “pulp by itself” doesn’t currently show. vsedmik/dalley to work together on that
    • Currently core/3.73 is the “supported branch” post-3.69 - let us know if that’s not right!
      • core/3.73 is A Good Idea
  • RPM
    • rpm/3.29 released
    • Wants createrepo_c/1.2.1+ to solve a FIPS-compliance issue reported by community
      • Backported to 3.28
      • Lots of discussion ensues
  • Ansible
  • Pulp Container
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

QE

  • TY for the pulp-ansible-0.21.10 release (should come into Sat 6.16.4 next snap)
  • Spinning up an instance for dalley’s ACS-memory-issue testing
  • How to update field using pulp-cli
#124

2025-04-02

Overview

  • Katello Schedule

Pulp

Katello

  • https://issues.redhat.com/browse/SAT-32421
    • Discovered the day before the proposed next-Y signoff, but it looks like we’ll get at least one more snap
    • Discussing it with Gerrod
    • container/2.23 and 2.24 might be consumable? Diff is less scary than it sounds
      • Only non-bugfix changes are migration squashing and partial domain support (and the bugfix changes are in 2.22 also)
    • Can we get away with backporting the migration in this special case?
    • Pulp needs to get https://issues.redhat.com/browse/PULP-449 into current sprint
  • Katello planning to allow certificate-authentication - does Pulp have any plans for cert-auth for api-workers?
    • Pulp has done zero investigation

QE

  • Please fix the problem raised above, Soon
1 Like
#125

2025-04-16

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Sync-memory-investigation continues
  • Ansible
  • Pulp Container
    • Now fully domainized!
  • OSTree
  • Pulp CLI
  • Pulp Replica

Katello

  • Pulp versions updated in our dependency-matrix
    • ggainey cheers wildly
    • Everything seems to be working!
  • Moving to all PRNs - under investigation
  • There is now pulp_python interest in katello/Satellite

QE

  • Just a heads-up for SatLab VM users (Daniel) - VM SLA has been changed from 40-years to 3 months
#126

2025-04-30

Overview

  • Katello Schedule

Pulp

Katello

  • Katello may start using “internal” repositories to host contents for other Foreman plugins down the road
  • Pulp upgrade to 3.73 is complete
  • DevConf.US booth is proposed - includes Pulp

QE

#127

[Two reports for the price of one!]

2025-05-14

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • Pulp_rpm 3.30.0 released
      • Repo_gpgcheck, gpgcheck, package_checksum_type, metadata_checksum_type dropped
        • The options don’t do anything, if you try to use them
        • I believe you dropped these a while back (they’ve been deprecated a while) but it’s maybe worth double checking

Katello

  • 4.17 branching today. Will use 3.73 which is now branched: Index of /pulpcore/3.73
    • Be prepared for influx of core/3.73 users
  • Q: setuptools min-version and pulp_python
    • Check in w/ odilhao - Ian will get back to Jitka on this ASAP
    • Other plugins as well?
    • [Ian] We can’t bump up setuptools until it is in RHEL 9, currently we have 68.2 available.
  • Phoenix will prob inherit the Pulp-QE Team
    • Iballou to review attendee list for changes

QE

  • New folk introduced - yay for more QE!

2025-05-28

Overview

  • Katello Schedule

Pulp

Katello

  • Katello 4.17 release in process.
    • core/3.73 and friends
  • Support FlatPak repo “soon”
  • Community question: can one use file: URLs for pulp_ansible?
    • Not unless the API files are available, and prob not then
    • Pulp_ansible expects either “live” API or to be talking to a git-server
  • Confirmed that Phoenix team now contains all Pulp QE as well

QE

  • latest (snap 101) pulp-ansible is python3.12-pulp-ansible-0.24.6-1.el9pc.noarch.rpm
1 Like
#128

2025-06-11

Overview

  • Katello Schedule

Pulp

Katello

  • What happens when Pulp syncs a repository when a package’s modularity state changes? Assume the package has the same checksum as before.
    • A question came up - a non-modular repo was turned modular, but a skip-metadata-check sync seems to have not updated the modularity state
    • Not a great way to deal with this edge-case, alas - “remove all traces and resync”
    • Collides with “Immutable content”
  • Rocky 9 devel repo cannot be synchronized - Support - TheForeman
    • Community wondering if 404s in broken repositories (Rocky) can be worked around somehow
      • “Keep going when the repo is broken” keeps coming up
      • there are reasons we don’t do that (mostly, “you end up with a broken repo you can’t install from and have to go back to the third-party-repo-owner anyway”)
  • Upgrade to Katello 4.17 - katello:correct_repositories fails - #2 by ingcarl - Support - TheForeman
    • RPM client bindings issue: package_signing_fingerprint cannot be nil
    • We are all confused how this is even possible - iballou waiting on user and maybe attempting to reproduce

QE

  • glitch: distribution url of a file-type repo is empty
    • Sounds like a timing-hole between katello/pulp/test
    • Not reliably reproducible
    • Sometimes “waiting” in the test “fixes it” - but maybe not?
      • wait timeout is 120s
    • We are all sad now
2 Likes
#129

A lot of meetings failed to be posted, for various “out of office” reasons - catch up all at once here

2025-08-20

Overview

  • Katello Schedule

Pulp

Katello

QE

  • https://issues.redhat.com/browse/SAT-36451 - first vs next curl (200 vs 404)
    • The fix works on naked-Pulp-instance, fails on first-attempt in katello-install env
    • Pbrochado and vsedmik continue to bang their heads against it
    • [ggainey] this is an edge-case of an edge-case of an error-condition - not worth blocking a release or doing a fire drill on the pulp side right now [pbrochad]+

2025-08-06

Overview

  • Katello Schedule

Pulp

Katello

  • https://issues.redhat.com/browse/SAT-19863 Is there anything pulp can do about this or does this require a redesign on katello’s end. Does pulp replication resolve this if we instead invest in that effort? My understanding is replication creates a new repository for every upstream distribution to be replicated, something our current design driven by katello also does - causing the data to balloon in above tables.
    • Those tables are just Doing Their Job, tracking what content is in which repository. If you make 10 repos where Sat has 1, you’re going to get 10x duplication
  • It’s Pulp & plugins upgrade time for Katello, what versions should we take?
    • core/3.85 is happening Soon (next week?) You may want to be on core/3.84?
    • Having /v4/ available would be a Really Good Thing
  • Any plans to support ostree flatpaks: flathub ?
    • No movement happening on the pulp side of things

QE

  • createrepo_c /pulp_rpm bump - still working on reproducing/testing that change

2025-07-23

Overview

  • Katello Schedule

Pulp

Katello

QE

2025-07-09

Overview

  • Katello Schedule

Pulp

Katello

QE

2025-06-25

Overview

  • Katello Schedule

Pulp

Katello

  • More active planning for /v4/
    • Need some more solidity around “when will /v4/ be ready in Pulp”
    • Aiming at katello/4.19 (releasing end of Q4-ish)

QE

  • Some odd non-reproduceable failures in KS repos sync in provisioning pipelines
    • Need actual pulp-logs to figure out what happened
#130

2025-09-03

Overview

  • Katello Schedule

Pulp

Katello

  • libcomps/SIGSEGV/py3.11 issue
    • New createrepo_c/py3.11 does not show problems in 6.17
    • “Maybe” createrepo_c downgrade addressed the issue
  • One week out from upgrading pulp components (3.85+) into katello/4.19

QE

  • final verifications for Sat-Next (2 in Testing, 1 in RP-upstream (should be in Testing too probably since it’s in 3.73))
#131

2025-09-17

Overview

  • Katello Schedule

Pulp

Katello

  • Thanks for the help with the libcomps issue!
  • Any requests for Pulpcon presentations besides a general Katello update?
    • Examples of good/bad things happening “in the real world” pulp usecases
  • Pulpcore 3.85 upgrade 
  "versions": {
    "deb": "3.7.0",
    "rpm": "3.32.0",
    "core": "3.85.1",
    "file": "3.85.1",
    "ostree": "2.5.0",
    "python": "3.19.0",
    "ansible": "0.28.0",
    "certguard": "3.85.1",
    "container": "2.26.1"
  }

QE

  • no updates
2 Likes
#132

2025-10-29

Overview

  • Katello Schedule

Pulp

Katello

  • https://github.com/pulp/pulp_rpm/issues/4178
    • Pulp needs to figure this out ASAP - bug, or bindings-issue
    • Pedro to work with iballou as needed
    • Options for katello:
      • We fix it
      • Katello monkeypatches the not-working version of pulp-rpm-client
      • Katello pins to a not-broken version of pulp-rpm-client (3.32.2)

QE

  • [INFO] href → PRN automation done
  • [ToDo] pulpcore 3.85 testing

2025-10-15

Overview

  • Katello Schedule

Pulp

Katello

QE

  • no updates - I can donate some time

Build Team

2025-10-01

Overview

  • Katello Schedule

Pulp

  • Pulpcore
  • RPM
    • LOTS of work on evr_t
      • Working to keep it in pgsql so Katello can reuse
    • Downgraded openapi vers to 3.0.1 to address ruby-bindings issue
  • Ansible
  • Pulp Container
  • OSTree
    • Is being used in upstream foreman/katello
    • flathub.org will require this
  • Pulp CLI
  • Pulp Replica
  • PulpCon 2025
    • CfP open til 31-OCT
    • Thanks iballou!

Katello

  • Pulp vulnerability tracking - any demo out there with pulp-services?
    • Katello could benefit from having vulnerabilities in container images (for example) shown for a Quay-like experience.
    • Ping bmbouters and/or hyagi
  • Pulp upgrade is going smoothly!
    • VCR have been run successfully
    • “Weird edge case” testing

QE