Katello/Pulp-Community Integration meeting minutes

Minutes
#43

2024-10-01 1000-1030 GMT-5

Attendees: @iballou forgot to record attendees but it was more or less the “usual crew”

Regrets: not recording attendees

Agenda:

  • Structured-APT PR is ready for a final review
    • Tested on an internal production system by quba42
    • Just requires small remaining changes
  • Prototyping transparent/rolling/simplified content views starting soon
  • New release of pulp-deb would be good before Katello upgrades
    • Pain-point: updating pulp-deb Pulp CLI compatability each time Katello packages pulp-deb is cumbersome.
      • Potential fix: loosen the boundaries for pulp-deb Pulp CLI compatability. This could mean breaking changes get in, but it also means packaging happens more smoothly and people can get to testing it sooner.
  • Previous AIs:

Action Items:

3 Likes
#44

2024-11-12 1000-1030 GMT-5

Attendees: ggainey, sjha, hstct, quba42, mbucher

Regrets: iballou

Agenda:

Action Items:

#45

2024-12-03 1000-1030 GMT-5

Attendees: ggainey, quba42, mbucher, dalley, pbrochado, sjha, hstct, qjames, mdellweg

Regrets:

Agenda:

Action Items:

1 Like
#46

2025-01-07 1000-1030 GMT-5

Attendees: sjha ggainey mbucher quba42 vsedmik hstct

Regrets:

Agenda:

Action Items:

1 Like
#47

2025-03-04 1000-1030 GMT-5

Attendees: ggainey, vsedmik, qjames, quba42, mbucher, hstct

Regrets: iballou

Agenda:

  • Previous AIs:
  • [m-bucher]: Katello; Org with trailing . (e.g. ACME Inc.) => {"base_path":["The provided base path contains forbidden characters."]}
    • container-related - ORG with trailing-dot
    • Pulp is complaining, should katello sanitize?
    • is this container-only? pulp-content-app?
    • atix asking their support to experiment w/ deb/rpm
    • qjames: katello may have already fixed in 4.15-container-push
    • mbucher: will try against nightly
    • this may be specifically a not-legal-container-path Thing
    • if pulp is being “overly restrictive” around base-path-legality, we’ll need an issue to get that addressed
  • [quba42] Still open: Handle URL params for AptRepoFiles by quba42 · Pull Request #3454 · candlepin/subscription-manager · GitHub
    • ignored for 5 months?
    • let’s try and get tthis reviewed/merged
    • ptoscano is prob a person to nag
  • [quba42] Need packaging help: Update rubygem-pulp_deb_client to 3.5.1 by quba42 · Pull Request #11793 · theforeman/foreman-packaging · GitHub
    (Known)Issue or already fixed?
    • skip for now - needs a complicated-packaging-dance to address
    • might happen as part of the ongoing katello-branch-and-rebase-to-new-pulp
  • [quba42]: rolling CV: Fixes #38048 - Add rolling content views by quba42 · Pull Request #11240 · Katello/katello · GitHub
    • Katello 4.16 has branched so we want to finalize ASAP.
    • Lots of review and rework activity happening.
    • Open issues:
      • Smart proxy sync not re-syncing rolling CVs in all cases (maybe fixed now)
      • Missing repo names for rolling CVs on the smart proxy view
      • Container push repos need to be disallowed
      • Why is ::Actions::Katello::ContentView::AddToEnvironment async during rolling CV creation?
      • Hammer PR
      • docs PR
    • work is in-progress, will report if/when/as there may be roadblocks
    • PR might get taken over by mbucher
  • Availabilty: quba42 on leave for 4 weeks starting next week, nag hstct for pulp_deb or mbucher for katello-deb issues
  • [quba42] Potential future rolling CV extension: “I don’t want to sync all of Library to smart proxy just to use my hand full of rolling CV repos”
    • Possibility 1: Call it something different for rolling CVs, e.g.: “RollingLibrary” or just “Rolling”.
      • I don’t want to introduce another new “special” name.
    • Possibility 2: Allow “promoting” rolling CVs to other LCENVs?
      • Easy enough to implement, but does not make a lot of semantic sense.
    • Possibility 3: Allow some kind of filtering by CV on the smart proxy sync?
      • Sounds hard and potentially disruptive.
    • Any other ideas?
    • qjames: let’s rule out 1 above
    • qjames: 2 “makes more sense” (imnsho)
    • vsedmik: autosync-after-promotion
    • vsedmik: option 2 - breaks rolling-content-view “concept”
    • vsedmik: what about org-scoping libraries?
    • quba42: customers prob won’t be happy w/ introduced complications of org-scoping
    • consensus: proposal-1 is Right Out. Discussion needs to continue
    • finish the existing PR, and then perhaps open the discussion to a wider audience

Action Items:

1 Like
#48

2025-04-01 1000-1030 GMT-5

Attendees: mbucher, sjha, ggainey

Regrets:

Agenda:

  • Previous AIs:
  • We do not do “April Fools!” jokes in this meeting! :slight_smile:
  • pulp_deb and domains
  • [m-bucher] container-image naming in katello revisited
    • see last mtg’s discussion
    • katello’s container-name-regex more permissive than Pulp’s
    • Pulp’s regex is different than OCI’s? - why?
    • current katello-PR should fix immediate issue
      • it’s OK if katello is stricter-than Pulp’s regex

Action Items:

1 Like
#49

2025-05-06 1000-1030 GMT-5

Attendees: quba42, mbucher, hstct, sjha, ggainey, vsedmik

Regrets:

Agenda:

Action Items:

  • quba42: open a UX issue RE “green task that didn’t actually DO anything is…confusing”
  • quba42: update rolling-content-view/environments issue w/ discussion about approach being taken
  • hstct: work w/ SUSE to fix their repo?
  • ggainey: extend mtg to remainder of 2025
  • ggainey to add minutes to Katello/Pulp-Community Integration meeting minutes
1 Like
#50

2025-06-03 1000-1030 GMT-5

Attendees: pbrochado, manisha, sjha, ggainey, mbucher, quba42, vsedmik, hstct

Regrets:

Agenda:

Action Items:

1 Like
#51

2025-07-01 1000-1030 GMT-5

Attendees: ggainey, mbucher, qjames, quba42

Regrets:

Agenda:

  • Previous AIs:
    • None
  • quba42: Update on “rolling environments”: https://github.com/Katello/katello/pull/11407
    • sync-to-proxy fix almost ready
    • needs more unit-tests to cover all the state-changes
    • quba42 will undraft and ping when he thinks it’s ready
    • katello pushing hard towards 4.18 dev-freeze - be aware
  • Q: ATIX customer issue w/ update-capsule-repository task “starts, task-finds-no-capsules, dynflow shows green and then ‘eventually’ foreman-task turns red”?!?
    • transient/not-reproduceable
    • have we seen this before? (alas, no)
    • no issue yet, just reported
  • katello-dev-server on a Mac VM (aarch64)
    • everything works except pulpcore, because RPMs only avail for x86_64
    • Pulp team doesn’t build RPMs, TheForeman Build Gang does
    • Can you point foreman/katello at an “external/containerized” Pulp?
    • discussion ensues
#52

2025-08-05 1000-1030 GMT-5

Attendees: quba42, sjha, mbucher, vsedmik, hstct, ggainey, pbrochado

Regrets:

Agenda:

Action Items:

2 Likes
#53

2025

2025-09-02 1000-1030 GMT-5

Attendees: pbrochad, Bernhard S, Jan B, Quirin, Samir, Tobias, Vladmir, ggainey

Regrets:

Agenda:

  • Previous AIs:
    • update to py3.11 as minimum (py3.9 EOLs in October)
    • core<3.100 (should be done by the time this mtg happens)
    • both accomplished via 1311
  • quba42: Thanks to everyone involved with rolling CVs! https://github.com/Katello/katello/pull/11407
  • quba42, iballou: Rolling CV tests for robottelo
  • quba42, mbucher: Small open PR: https://github.com/Katello/katello/pull/11481
  • quba42: Migrate all deb content to use structured APT for Katello 4.19?
    • Draft PR: https://github.com/Katello/katello/pull/11487
    • How sure can I be that “during upgrade rake tasks” will really be run in all cases?
    • Where can I add docs to recommend users enable structured APT BEFORE upgrading to Katello 4.19 (or whatever version this will land in?)
  • quba42: Incremental CV update (RHEL/rpm content with >100 repositories and at least one filter) => OOM death
    • 188G of RAM is not enough
    • Not exactly good practice but apparently there are “reasons to do things this way”
    • Incremental CV update to bring back a errata
    • Why should this be a performance killer?
      • Even if there are many repos presumably only one of them needs the errata packages/copy action?
      • Same for the filter? (Does the filter perhaps apply to every repo even if there are no relevant packages?)
    • dep-solve on or off?
      • because “on” will try to dep-solve against All The Things
      • the claim is “off”
    • what is the form of the exact filter being used?
      • how is the filter applied? (can be limited to single repo, but if not, is applied to all of them)
    • Any Thoughts?
      • check on depsolve flags
      • check on where filter is applied
      • who can ATIX talk to with questions on CV-Incr-Update path - sjha or iballou
  • quba42: HTTP proxy with SSL inspection. Host can reach the repo, Pulp fails with “certificate verify failed: unable to get local issuer certificate”. Worked with Katello 4.14/pulpcore 3.49, not working with Katello 4.16/pulpcore 3.63 (both Python 3.11). Any ideas?
    • can we add CA explicitly? will that help?
  • how are things going w/ upstream user balu?
1 Like
#54

(ggainey failed to post a few mtg-minutes - let’s catch up!)

2025-12-02 1000-1030 GMT-5

Attendees:

Regrets:

Agenda:

Action Items:

2025-11-04 1000-1030 GMT-5

Attendees: sjha, iballou, hhstct, pbrochado

Regrets: quba42, ggainey, vsedmik

Agenda:

Action Items:

2025-10-07 1000-1030 GMT-5

Attendees: ggainey, pbrochado, vsedmik, sjha, iballou, quba42

Regrets:

Agenda:

  • Previous AIs:
    • Update on Incremental CV update performance?
    • Update on HTTPS proxy woes?
  • quba42: I noticed there is still Pulp 2 stuff in the Katello code base. Is it safe to ignore/remove?
    • there exists a PR to yank all that out
    • please tell iballou it’s worth finishing :slight_smile:
  • PulpCon 2025!
  • hstct: We are looking to enable ACS support for debian in Katello. Hoping for some pointers what to look out for :slight_smile:
  • quba42: “Migrate all deb content to structured APT” (https://github.com/Katello/katello/pull/11487) nearly done.
    • What is the deadline for inclusion in Katello 4.19 (When does it branch?)
      • December-ish
    • Upgrade docs: I understand version specific upgrade docs are written after branching. How can I be “pinged” to provide those docs at the relevant point in time?
      • do docs and put them into nightly (will be carried into 4.19 branch automatically)
      • remove post-branching (if not needed after)
    • Regarding review: Has been reviewed and tested internally at ATIX. (And also released within orcharhino, i.e. “tested” in production; i.e. we are confident in the current state)
      • Essentially all changes are within repository.deb? type guards, so low likelyhood to impact other content types
      • One open thread regarding how talkative the migration task should be on systems that don’t have any deb content
      • Expectation: resolve that thread, add two more tests, then merge by ATIX maintainer.
  • Can/should we widen this to add other community contributors?
    • G-Research just added Pulp Manager
    • consensus: The focus of this mtg is “people who need to coordinate between pulp and katello”
  • iballou: does ATIX coordinate with EU OS?
    • https://eu-os.eu/
    • Jonas? met with iballou
    • maybe another contender for adding to this mtg
    • working w/ katello/foreman already

Action Items:

1 Like
#56

2026-01-06 1000-1030 GMT-5

Attendees: ggainey, samir, pbrochado

Regrets:

Agenda:

  • Previous AIs:
  • Welcome to 2026!
  • core/3.100 being released today
    • update plugin pulpcore UB to core<3.115 please
    • lots of Django5 discussion ensues

Action Items:

1 Like
#57

[There was no meeting in February for various out-of-office reasons]

2026-03-03 1000-1030 GMT-5

Attendees: auba42, mbucher, bsuttner, iballou, sjha, hstct, pbrochado, ggainey

Regrets:

Agenda:

  • Previous AIs:
  • (m-bucher) Katello-UI: is there a roadmap for migrating pages to React?
    • no “schedule”, but a list of pages-to-be-affected, prioritized
    • maybe open a community post with remaining work - get some help?
    • some changes will impact community-users more than others (e.g. page-merges)
    • AI: someone-katello to open a community discussion
  • (m-bucher) Katello-UI RfC: add Errata to ‘all newer’ ContentView-Versions
    Example:
    1. CV has versions 1(PROD), 2(Test), 3(Library)
    2. Erratum ERR1 is Applicable to all versions
    3. Host A on PROD will get ERR1 fixed by creating Incr.Update to Version 1 => 1.1
      CV-Version 2, 3 will not get Incr.Updates, if no Hosts are subscribed!
    4. when PROD moves to CV-version 2, all new hosts created will not have ERR1 fixed⚡
    • does this make sense as an RfC?
    • If so - where/how should the UI be changed?
    • Incremental-CV-Update definitely needs love - today users generally use hammer to accomplish Things
    • https://issues.redhat.com/browse/SAT-3304
    • AI: mbucher to open RfC “somewhere”
    • discussion about where this would happen ensues - implementation discussion will need to happen in the RfC
    • prob could stand to query community-users how this functionality could be approached? (host-centric vs repo/CV-centric)
  • [quba42]: Structured APT cleanup: https://github.com/Katello/katello/pull/11631
    • There is a known edge case that I would like to fix as a follow up task. A version of the issue existed in the past, but now it is a hard error during CV version creation, rather than silently letting you create a broken CV version. I would argue that is already an improvement.
  • [quba42]: Packaging (nearly?) ready for ACS for deb content: https://github.com/Katello/katello/pull/11567
    • Depends on pulp_deb 3.8.0, which is currently only available in “nightly” pulpcore-packaging. Can we add this to a Katello version that does not switch to a new pulpcore repo version?
  • [iballou] Looking at orphan-cleanup PR from bsuttner
#58

2026-04-07 1000-1030 GMT-5

Attendees: ggainey, pbrochado, sjha, iballou, quba42, mbucher

Regrets: hstct

Agenda:

  • Previous AIs:
  • Making sure you're not a bot! Do we want to pick this back up?
    • Missed discussion because mtg-organizer skipped a groove
  • don’t push the release-button
    • RE Quirin’s “why is the core/107 tag not pointing where it should?”
    • Pulp project-team is working to fix the release-process, stay tuned
  • triage-needed label in pulp_deb
    • is it useful for you at all?
      • sometimes, but not systematically
      • “no strong opinions”
    • AI: ggainey to take that to triage-mtg
  • [quba42, hstct on PTO] https://github.com/Katello/katello/pull/11567 Has already received significant review/nearing completion.
    • New development: New coderabbitai added its 2ct. Unsure about handling the new tool that is late to this particular party.
    • AI: iballou: will review coderabbit findings and see which if any really need to be addressed
  • [quba42] I have (almost certain) abandoned the approach in https://github.com/Katello/katello/pull/11631
    • However, the new approach requires a change in pulp_deb to land. The change is merged in pulp_deb, and I can release at any time. How close are we to new pulpcore version packaging?
      • DON’T RELEASE YET! (7-APR)
      • if we can fix release-process “soon”, pulp_deb would like a new pulp-deb-Y to be packaged as part of katello/4.21
  • We (Katello) are looking more into containerized Foreman, have others been looking?
    • heavy testing in process
    • ATIX has taken a “first light” look
    • why require Vagrant for dev-env?

Action Items:

#59

2026-05-05 1000-1030 GMT-5

Attendees: ggainey, iballou, sjha, pbrochado, quba62, mbucher

Regrets:

Agenda:

Action Items:

1 Like
#60

2026

2026-07-07 1000-1030 GMT-5

Attendees: ggainey, quba42, iballou, sjha, mbucher

Regrets:

Agenda:

  • Previous AIs:
  • [quba42] Hanging rolling CV tasks: Making sure you're not a bot!
    • Anyone already working on this?
    • My only idea for how to fix this, is to avoid the async task invocation via a big sync action restructuring. This feels disruptive/dangerous/hard. Anyone have any better ideas?
    • there exists a SAT-Jira. This can be a candidate for next sprint-planning (this week)
  • [quba42] Incremental CV publication performance (all content types)
    • Issue 1: No “exclude” functionality on the Pulp side
      • workflow is using (in rpm/deb) Advanced Copy, not repo-modify
    • Issue 2: Katello appears to be doing a lot of work on all repos in the CV not just the ones that need a change in the new incremental CV version. The result is poor performance when a CV contains many repositories.
    • more analysis, but it “feels like” we could optimize in both Pulp and Katello
  • [quba42] We are working on authentication for pulp_ansible git remotes. Katello does not appear to make use of pulp_ansible git remotes. Unless that changes, this is a pure Pulp topic.
  • Interesting research being done in pulp-domain/scalability-land : multi-database support

Action Items:

#61

2026-08-04 1000-1030 GMT-5

Attendees: quba42, mbucher, ggainey, jbostian, bsuttner, iballou, sjha, pbrochado

Regrets:

Agenda:

  • Previous AIs: None
  • Mtg overview/intro for observers from IBM CA (zopen team)
  • Whom do we talk to about updating the ATIX logo at Foreman?
  • PulpCon 2026 CfP is up!
  • Is the following set of observations intentional/by design?
    • CCV with a version promoted to “test” and “prod”
    • Applicable (not installable) errata for host in “test”
    • Installing the errata creates new incremental CV version to make the errata installable.
    • Expectation: Errata is only installable in “test” environment.
    • Observation: Errata is also installable in “prod” environment.
    • discussion ensues
      • iballou is “not surprised” due to some special logic re lifecycle envs
      • think about “short circuit” of “critical security updates”
      • maybe look back to originating PR? (from a Very Long Time Ago)
    • maybe quba42 file a redmine for discussion, if it is/becomes A Problem
    • start a community thread in theforeman
  • We have another report about slow incremental CV updates:
  • discussion around postgres 65K issue
    • talk to pbrochado/dalley
  • looking for any thoughts/feedback on recent optimization involving non-artifact metadata (e.g. for pullthru repos)
2 Likes
#62

2026-09-01 1000-1030 GMT-5

Attendees: ggainey, mbucher, pbrochado, iballou, sjha, quba42

Regrets:

Agenda:

  • Previous AIs:
  • Eyes on https://github.com/pulp/pulp_deb/pull/1506 please (65K Limit PR)
    • discussion ensues - comparing to pulp_deb 1499 (ATIX has that patch in prod already)
  • Pulp and EU Cybersecurity Act
  • Pulp CVE (not embargo’d) incoming
  • where is katello with next branch/new pulpcore version RPMing
  • Q: Is ATIX advertising containerized-foreman?
    • not yet (beyond “on the roadmap”)
    • katello/5.0 will be containerized (but no upgrade path)
    • katello/5.1 will have an upgrade path for katello/4.21
  • PulpCon CfP!
    • please consider adding your talk(s)!
1 Like
#63

2026-10-06 1000-1030 GMT-5

Attendees: ggainey, quba42, mbucher, sjha

Regrets:

Agenda:

  • Previous AIs:
  • Does this affect Katello? (Need to run pulpcore-manager container-repair-pull-through-distributions “before upgrading pulpcore”) Changelog - Pulp Project
    • get full low-level details from Gerrod if you want
    • katello is updating foreman-ctl to handle this upgrade scenario along with the container-repo-type-merge upgrade
  • pulp-replication, smart-proxies, and pulp_deb - discuss
    • pulp-replication does not support pulp_deb yet - should we panic?
    • this is currently an idea, not yet a Plan
    • do we even have a Jira for this somewhere?
  • [ATIX] We heard on the wind, that there are plans to add SBOM data to packages / or repositories to fulfill the Cyber Resilience act for June 2027. Do you know more?
    • required for “Champion” status
    • we’d like to do that “eventually”, but currently only required to do “Light” stewardship for regulatory obligations (and have completed the “Light” stewardship requirements)

Action Items: