Katello/Pulp-Community Integration meeting minutes

Minutes of the Pulp/Community Integration meetings.

Hackmd minutes: Pulpcore/katello/pulp_deb integration meeting - HackMD

Template:

## date 1000-1030
### Attendees: 
### Regrets:
### Agenda:
* AI review
* Agenda
  * agenda items here
### Action Items:
* AI: ggainey to schedule next meeting 
* AI: ggainey to add minutes to GH discussion-topic, send pointer pulp-dev@
1 Like

2021-06-01 1000-1030

Attendees: quba42, mbucher, ggainey, jsherrill, manisha

Regrets:

Agenda:

  • AI review
    • release-wiki updated? - DONE
    • VCR-review? - moved to AI
  • Taking stock: Latest (for the relevant pulpcore version) pulp_deb versions now available for Katello >= 4.0.1; Now just waiting on the Katello 3.18.4 release (3.18.3 is meant to be released this week?)
    • 3.18.3 is, in fact, imminent
  • Cherry Pick the following to Katello 3.18 (before 3.18.4 release)?
  • pulpcore Django incompatibility problem (Issue #8691: Pulp fails against django 2.2.21 - Pulp)
    • bmbouter is contact if backport-info required
  • Pulp 3 Smart Proxy modes/features
    • Pulp2/squid setup ATIX uses
    • Pulp3/katello - plans for non-syncing pulp passthru?
      • mirrored-metadata support for RPM makes sync way more efficient
      • that and on-demand
    • pulp3 lazy - doesn’t actually store RPMs at all
  • Manisha will be working on CV-filtering for deb in katello
    • katello new-UI work largely in place
    • Samir working on CV-UI-side
    • new UI originally aimed to 4.3 - 4.5 may be more likely
    • if ATIX work aimed at 4.2/4.3 - focus on old-UI
  • deb-vers-compare discussion
    • deb-vers are more exciting than rpm nevra compare
    • working to get compare-fn building in pulpcore-packaging, migration

Action Items:

  • VCR-review - AI for jsherrill and quba
  • AI: ggainey to schedule next meeting [DONE]
    • first-Tues for Jul/Aug/Sep, same timeslot
  • AI: ggainey to send minutes to pulp-dev@
    • experimenting w/ GitHub discussions - post there, point from pulp-dev@
1 Like

2021-07-13 1000-1030 GMT-5 (rescheduled from 6-JUL due to mtg conflicts)

Attendees: ggainey, mbucher, jsherrill

Regrets: quba42

Agenda:

Action Items:

2021-08-03 1000-1030 GMT-5

Attendees: ggainey, jsherrill, msinghal, mbucher, quba42

Regrets:

Agenda:

  • We need a final path forward for deb_version_compare postgres extension.
    • will need to be in pulp_deb #soon
    • pulp_rpm includes as a db-migration
    • if we do that in pulp_deb, how do we keep in sync w/ work being done in katello-rpm?
      • having it in two places has downsides
      • building in SCL is running into build-obstacles
      • on the plus-side - doing it as a migration makes handling external-dbs much simpler
    • net: pulp_deb will mirror what pulp_rpm is doing
  • Seen on Katello: “Updated to use mirrored metadata for library repo syncs & capsule syncs”
    Katello/Pulp3 Integration meeting · Discussion #4 · pulp/community · GitHub
    => How to ensure pulp_deb is not forgotten as “mirror mode” is adopted by Katello
    • upgraded to 3.14 gave katello ‘mirror mode’ almost incidentally
    • katello adjusted to “just use returned publication” if there is one
    • look at pulp_rpm, see if pulp_deb can make it work for smart-proxies
      • broken currently, according to community input
      • need to look into katello-changes and see if verbatim-publication can be modelled/used as pulp_rpm is?
  • How exactly is Katello handling the drop in python 3.6 and 3.7 support (with pulpcore 3.15?)
    • build-team will handle all the plugins at once
  • pulp_deb nightly CI is perennially broken due to infrastructure issues => Where do I raise this issue? Task #8642: Fix nightly tests - Debian Support - Pulp
    • will bring up on OpenFloor
  • Best strategy for plugin_template application to older release branches?
    • community-discussion provided an answer
  • PostgreSQL function for deb-version compare in Katello (new PR):
    Refs #25978 #27625 - Add deb_version_cmp DB-func by m-bucher · Pull Request #9507 · Katello/katello · GitHub</title
    • katello to merge, look into applicability-pr for deb

Action Items:

2021-09-14 1000-1030 GMT-5

Attendees: ggainey, quba42, mbucher, jsherrill

Regrets: msinghal

Agenda:

  • Smart Proxy sync development environment with forklift?
    • atix having some issues w/ forklift
    • need a smart-proxy sync env, community bug reports in this setup
    • “proxy cannot activate puppet” at foreman-install
    • combining centos7-katello-devel-stable w/ centos7-proxy-devel
    • who should atix work with?
    • make puppet-module-existence match on server/proxy
    • open a thread if we continue being unable to fix
    • error logs:
vagrant up centos7-proxy-devel
    2021-09-14 08:49:23 [ERROR ] [configure] Proxy centos7-proxy-devel.example.com has failed to load one or more features (Puppet), check /var/log/foreman-proxy/proxy.log for configuration errors
    2021-09-14 08:49:23 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[centos7-proxy-devel.example.com]/features: change from ["Container_Gateway", "Logs", "Pulpcore", "Puppet CA", "Registration", "Templates"] to ["Container_Gateway", "Logs", "Pulpcore", "Puppet", "Puppet CA", "Registration", "Templates"] failed: Proxy centos7-proxy-devel.example.com has failed to load one or more features (Puppet), check /var/log/foreman-proxy/proxy.log for configuration errors
    2021-09-14 08:49:24 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[centos7-proxy-devel.example.com]: Failed to call refresh: Proxy centos7-proxy-devel.example.com has failed to load one or more features (Puppet), check /var/log/foreman-proxy/proxy.log for configuration errors
    2021-09-14 08:49:24 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[centos7-proxy-devel.example.com]: Proxy centos7-proxy-devel.example.com has failed to load one or more features (Puppet), check /var/log/foreman-proxy/proxy.log for configuration errors
    2021-09-14 08:49:25 [NOTICE] [configure] System configuration has finished.

Action Items:

2021-10-12 1000-1030 GMT-5 (moved due to conflicts)

Attendees: msinghal, quba42, mbucher, jsherrill, ggainey

Regrets:

Agenda:

Action Items:

2021-11-02 1000-1030 GMT-5

Attendees: quba42, mbucher, jsherrill

Regrets: ggainey, msinghal

Agenda:

  • Promoting changed pulp-repo-settings into Katello after 2to3-migration (best-practise)
    pulp_deb-distribution/-release list (comma- to whitespace-separated)
    Options (using rake-task):
    • just change it in Katello(, Candlepin?)
    • get current settings from pulp3(-API) and sync them to Katello
      • rake task or db migration since its a pure db change would work
      • whatever suits us best, there is no strong upstream preference one way or another (in this case)
  • Advanced Sync Options: What underlying pulp feature do they use for pulp_rpm?
    • pulp_rpm uses optimize=false for the advanced sync in Pulp 3.
    • This is most likely plugin specific (not a pulpcore feature), so pulp_deb will just end up doing “its own thing”.
  • How best to find out what Pulp API calls a given Katello action actually triggers?
  • State of PRs:
    • got many deb related Katello PRs merged! Thanks to everyone!
    • Nearing completion: https://github.com/Katello/katello/pull/9634 (Oracle ULN related)
      • Will have one more look at the failing tests.
      • Set the release version on the Foreman issue for automatic cherry-picking into that release (post merge)
  • (How) should we (ATIX) be thinking about backports?
    • We are not anticipating any more 3.18, 4.0 (and probably 4.1) bugfix releases, so there is no need to backport to those versions unless some major reason to do so appears.

Action Items:

1 Like

2021-12-07 1000-1030 GMT-5

Attendees: ggainey, mbucher, quba42, jsherrill

Regrets:

Agenda:

I now created an upstream issue (Draft PR to follow) for the pulp_deb 2to3 migration bug mentioned: Issue #9616: deb migration not adding content to all relevant repo versions when multiple pulp2 units map onto the same pulp3 unit - Migration Plugin - Pulp

1 Like

2022-01-04 1000-1030 GMT-5

Attendees: ggainey, jsherrill, quba42

Regrets:

Agenda:

  • A failed Katello upgrade rake task does not appear to cause a failed foreman-installer run. Is this intentional or a bug?
    • there is a skipfailure=True
    • make sure it’s not set to true for the new task
  • Why do the packages for the 3.15 and 3.16 versions of the Index of /pulpcore repo have a tfm-pulpcore- prefix relative to prior versions of the same repo?
    • to avoid conflicts w/ SCL (which we’re using python-3.8 now which isn’t in EL7 base)
    • do the new packages use ‘obsoletes’ correctly?
      • we believe yes
  • Backport candidate: Debian Repo sync end with the error "This field may not be blank" - #10 by jeschero - Support - TheForeman
    • easily backportable
    • to what extent do ‘we’ (ie, ATIX) want to support backports?
    • 3.14 is katello/4.1 and /4.2, 3.16 is katello/4.3
  • ATIX has discovered need to build ruby-client-gems on feature branches
    • upstream-katello always uses pypi-released bindings
  • discussion opened in pulp-discourse

Action Items:

2022-02-01 1000-1030 GMT-5

Attendees: mbucher, quba42, jsherrill, ian

Regrets: ggainey

Agenda:

  • “verbatim” deb content smart proxy sync: https://github.com/Katello/katello/pull/9891
    • PR under review
  • Any news on on_demand/streamed download policy for smart proxy syncs? (What Katello version is this planned for?)
    • download policy already exists for capsule syncs
    • we don’t support streamed yet though, should be easy to add
  • Is it possible to have Katello run different smart proxy sync options depending on whether it is a Pulp 2 or Pulp 3 based smart proxy?
  • What is the Katello Agent deprecation timeline (in Katello versions)?
    • rex pull provider needs to be in place for ~2 releases before katello-agent will be removed. The earliest it would be dropped is ~4.6

Action Items:

1 Like

2022-03-01 1000-1030 GMT-5

Attendees: ggainey, jsherrill, bsutter, jlenz, sajha, jturel, iballou, quba42, mbucher

Regrets:

Agenda:

Action Items:

  • jturel to get together w/ quba42/bsuttner to gather/document SCA-related concerns, report back here next month
  • jturel to comment on 27524 and open redmine for error msg
  • ggainey to add sajha and jlenz to meeting invite
  • ggainey to add minutes to Katello/Pulp-Community Integration meeting minutes
2 Likes

2022-04-05 1000-1030 GMT-5

Attendees: iballou, quba42, mbucher, sajha, paji, ggainey

Regrets:

Agenda:

  • State of aiohttp(s) progress? E.g.: https://github.com/theforeman/pulpcore-packaging/pull/379
    • “can’t use https proxies w/ Pulp w/out aiohttp/3.8+, puthon3.8+”
    • [AI] iballou to inquire and let quba42 know, doc here as well
  • katello-host-tools service restarts:
  • When is the next breaking changes pulcore release? How is Katello approaching the changed pulpcore deprecation policy?
    • core/3.20 is allowed to have breaking changes. 3.19 to be released “really soon now” (today?)
    • katello: doesn’t want to get stuck on old versions, will pay attention to the proboem if/when/as it happens
  • Smart-proxy download policy streamed: https://github.com/Katello/katello/pull/10005
    • Merged!
  • 2to3 migration issue
    • large migration, Pulp3-orphan-cleanup ran in the middle of it (?!?!?)
    • is there any way this could happen in a “vanilla katello/3.18” install?
    • poss a cfg-issue (katello-installer shouldn’t allow this to happen)
  • ATIX has targetted customers in mind to test 2to3
  • Pulp moving to “releases on backport-branches” to once a week
    • ATIX mostly interested in RPM results

Action Items:

2022-05-03 1000-1030 GMT-4

Attendees: ggainey, quba42, mbucher, sjha, iballou, paji

Regrets:

Agenda:

  • Upgrading Katello nightly to pulp_deb 2.18.0 (including clients):
  • DB field size migration and backporting woes:
  • Debian Errata strategy regarding pulp3 & Katello 4.x
    • current draft Katello-only implementation reusing Katello::Erratum model
    • question1: should we add them to pulp_deb to more resemble yum-errata
    • question2: should we distinguish them more in Katello creating DebErratum and YumErratum models
    • feels possible to keep this in katello, make the inheritance/implementation hierarchy make more sense
    • maybe ATIX discuss internally, bring something more concrete to this meeting #eventually?
  • Discussing pulpcore-packaging minor version bump workflows.
    • and by packaged, we mean “get RPMs made”
    • how can ATIX make this request less ad-hoc?
    • current katello approach is similar to current ATIX - open pulp-packaging PR and then ping folk if ignored
    • new pulpcore-Y-release takes more work (since requires new directory)
1 Like