Katello/Pulp-Community Integration meeting minutes

Minutes
#1

Minutes of the Pulp/Community Integration meetings.

Hackmd minutes: Pulpcore/katello/pulp_deb integration meeting - HackMD

Template:

## date 1000-1030
### Attendees: 
### Regrets:
### Agenda:
* AI review
* Agenda
  * agenda items here
### Action Items:
* AI: ggainey to schedule next meeting 
* AI: ggainey to add minutes to GH discussion-topic, send pointer pulp-dev@
1 Like
#2

2021-06-01 1000-1030

Attendees: quba42, mbucher, ggainey, jsherrill, manisha

Regrets:

Agenda:

  • AI review
    • release-wiki updated? - DONE
    • VCR-review? - moved to AI
  • Taking stock: Latest (for the relevant pulpcore version) pulp_deb versions now available for Katello >= 4.0.1; Now just waiting on the Katello 3.18.4 release (3.18.3 is meant to be released this week?)
    • 3.18.3 is, in fact, imminent
  • Cherry Pick the following to Katello 3.18 (before 3.18.4 release)?
  • pulpcore Django incompatibility problem (Issue #8691: Pulp fails against django 2.2.21 - Pulp)
    • bmbouter is contact if backport-info required
  • Pulp 3 Smart Proxy modes/features
    • Pulp2/squid setup ATIX uses
    • Pulp3/katello - plans for non-syncing pulp passthru?
      • mirrored-metadata support for RPM makes sync way more efficient
      • that and on-demand
    • pulp3 lazy - doesn’t actually store RPMs at all
  • Manisha will be working on CV-filtering for deb in katello
    • katello new-UI work largely in place
    • Samir working on CV-UI-side
    • new UI originally aimed to 4.3 - 4.5 may be more likely
    • if ATIX work aimed at 4.2/4.3 - focus on old-UI
  • deb-vers-compare discussion
    • deb-vers are more exciting than rpm nevra compare
    • working to get compare-fn building in pulpcore-packaging, migration

Action Items:

  • VCR-review - AI for jsherrill and quba
  • AI: ggainey to schedule next meeting [DONE]
    • first-Tues for Jul/Aug/Sep, same timeslot
  • AI: ggainey to send minutes to pulp-dev@
    • experimenting w/ GitHub discussions - post there, point from pulp-dev@
1 Like
#3

2021-07-13 1000-1030 GMT-5 (rescheduled from 6-JUL due to mtg conflicts)

Attendees: ggainey, mbucher, jsherrill

Regrets: quba42

Agenda:

Action Items:

#4

2021-08-03 1000-1030 GMT-5

Attendees: ggainey, jsherrill, msinghal, mbucher, quba42

Regrets:

Agenda:

  • We need a final path forward for deb_version_compare postgres extension.
    • will need to be in pulp_deb #soon
    • pulp_rpm includes as a db-migration
    • if we do that in pulp_deb, how do we keep in sync w/ work being done in katello-rpm?
      • having it in two places has downsides
      • building in SCL is running into build-obstacles
      • on the plus-side - doing it as a migration makes handling external-dbs much simpler
    • net: pulp_deb will mirror what pulp_rpm is doing
  • Seen on Katello: “Updated to use mirrored metadata for library repo syncs & capsule syncs”
    Katello/Pulp3 Integration meeting ¡ Discussion #4 ¡ pulp/community ¡ GitHub
    => How to ensure pulp_deb is not forgotten as “mirror mode” is adopted by Katello
    • upgraded to 3.14 gave katello ‘mirror mode’ almost incidentally
    • katello adjusted to “just use returned publication” if there is one
    • look at pulp_rpm, see if pulp_deb can make it work for smart-proxies
      • broken currently, according to community input
      • need to look into katello-changes and see if verbatim-publication can be modelled/used as pulp_rpm is?
  • How exactly is Katello handling the drop in python 3.6 and 3.7 support (with pulpcore 3.15?)
    • build-team will handle all the plugins at once
  • pulp_deb nightly CI is perennially broken due to infrastructure issues => Where do I raise this issue? Task #8642: Fix nightly tests - Debian Support - Pulp
    • will bring up on OpenFloor
  • Best strategy for plugin_template application to older release branches?
    • community-discussion provided an answer
  • PostgreSQL function for deb-version compare in Katello (new PR):
    Refs #25978 #27625 - Add deb_version_cmp DB-func by m-bucher ¡ Pull Request #9507 ¡ Katello/katello ¡ GitHub</title
    • katello to merge, look into applicability-pr for deb

Action Items:

#5

2021-09-14 1000-1030 GMT-5

Attendees: ggainey, quba42, mbucher, jsherrill

Regrets: msinghal

Agenda:

  • Smart Proxy sync development environment with forklift?
    • atix having some issues w/ forklift
    • need a smart-proxy sync env, community bug reports in this setup
    • “proxy cannot activate puppet” at foreman-install
    • combining centos7-katello-devel-stable w/ centos7-proxy-devel
    • who should atix work with?
    • make puppet-module-existence match on server/proxy
    • open a thread if we continue being unable to fix
    • error logs:
vagrant up centos7-proxy-devel
    2021-09-14 08:49:23 [ERROR ] [configure] Proxy centos7-proxy-devel.example.com has failed to load one or more features (Puppet), check /var/log/foreman-proxy/proxy.log for configuration errors
    2021-09-14 08:49:23 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[centos7-proxy-devel.example.com]/features: change from ["Container_Gateway", "Logs", "Pulpcore", "Puppet CA", "Registration", "Templates"] to ["Container_Gateway", "Logs", "Pulpcore", "Puppet", "Puppet CA", "Registration", "Templates"] failed: Proxy centos7-proxy-devel.example.com has failed to load one or more features (Puppet), check /var/log/foreman-proxy/proxy.log for configuration errors
    2021-09-14 08:49:24 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[centos7-proxy-devel.example.com]: Failed to call refresh: Proxy centos7-proxy-devel.example.com has failed to load one or more features (Puppet), check /var/log/foreman-proxy/proxy.log for configuration errors
    2021-09-14 08:49:24 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[centos7-proxy-devel.example.com]: Proxy centos7-proxy-devel.example.com has failed to load one or more features (Puppet), check /var/log/foreman-proxy/proxy.log for configuration errors
    2021-09-14 08:49:25 [NOTICE] [configure] System configuration has finished.

Action Items:

#6

2021-10-12 1000-1030 GMT-5 (moved due to conflicts)

Attendees: msinghal, quba42, mbucher, jsherrill, ggainey

Regrets:

Agenda:

Action Items:

#7

2021-11-02 1000-1030 GMT-5

Attendees: quba42, mbucher, jsherrill

Regrets: ggainey, msinghal

Agenda:

  • Promoting changed pulp-repo-settings into Katello after 2to3-migration (best-practise)
    pulp_deb-distribution/-release list (comma- to whitespace-separated)
    Options (using rake-task):
    • just change it in Katello(, Candlepin?)
    • get current settings from pulp3(-API) and sync them to Katello
      • rake task or db migration since its a pure db change would work
      • whatever suits us best, there is no strong upstream preference one way or another (in this case)
  • Advanced Sync Options: What underlying pulp feature do they use for pulp_rpm?
    • pulp_rpm uses optimize=false for the advanced sync in Pulp 3.
    • This is most likely plugin specific (not a pulpcore feature), so pulp_deb will just end up doing “its own thing”.
  • How best to find out what Pulp API calls a given Katello action actually triggers?
  • State of PRs:
    • got many deb related Katello PRs merged! Thanks to everyone!
    • Nearing completion: https://github.com/Katello/katello/pull/9634 (Oracle ULN related)
      • Will have one more look at the failing tests.
      • Set the release version on the Foreman issue for automatic cherry-picking into that release (post merge)
  • (How) should we (ATIX) be thinking about backports?
    • We are not anticipating any more 3.18, 4.0 (and probably 4.1) bugfix releases, so there is no need to backport to those versions unless some major reason to do so appears.

Action Items:

1 Like
#8

2021-12-07 1000-1030 GMT-5

Attendees: ggainey, mbucher, quba42, jsherrill

Regrets:

Agenda:

#9

I now created an upstream issue (Draft PR to follow) for the pulp_deb 2to3 migration bug mentioned: Issue #9616: deb migration not adding content to all relevant repo versions when multiple pulp2 units map onto the same pulp3 unit - Migration Plugin - Pulp

1 Like
#10

2022-01-04 1000-1030 GMT-5

Attendees: ggainey, jsherrill, quba42

Regrets:

Agenda:

  • A failed Katello upgrade rake task does not appear to cause a failed foreman-installer run. Is this intentional or a bug?
    • there is a skipfailure=True
    • make sure it’s not set to true for the new task
  • Why do the packages for the 3.15 and 3.16 versions of the Index of /pulpcore repo have a tfm-pulpcore- prefix relative to prior versions of the same repo?
    • to avoid conflicts w/ SCL (which we’re using python-3.8 now which isn’t in EL7 base)
    • do the new packages use ‘obsoletes’ correctly?
      • we believe yes
  • Backport candidate: Debian Repo sync end with the error "This field may not be blank" - #10 by jeschero - Support - TheForeman
    • easily backportable
    • to what extent do ‘we’ (ie, ATIX) want to support backports?
    • 3.14 is katello/4.1 and /4.2, 3.16 is katello/4.3
  • ATIX has discovered need to build ruby-client-gems on feature branches
    • upstream-katello always uses pypi-released bindings
  • discussion opened in pulp-discourse

Action Items:

#11

2022-02-01 1000-1030 GMT-5

Attendees: mbucher, quba42, jsherrill, ian

Regrets: ggainey

Agenda:

  • “verbatim” deb content smart proxy sync: https://github.com/Katello/katello/pull/9891
    • PR under review
  • Any news on on_demand/streamed download policy for smart proxy syncs? (What Katello version is this planned for?)
    • download policy already exists for capsule syncs
    • we don’t support streamed yet though, should be easy to add
  • Is it possible to have Katello run different smart proxy sync options depending on whether it is a Pulp 2 or Pulp 3 based smart proxy?
  • What is the Katello Agent deprecation timeline (in Katello versions)?
    • rex pull provider needs to be in place for ~2 releases before katello-agent will be removed. The earliest it would be dropped is ~4.6

Action Items:

1 Like
#12

2022-03-01 1000-1030 GMT-5

Attendees: ggainey, jsherrill, bsutter, jlenz, sajha, jturel, iballou, quba42, mbucher

Regrets:

Agenda:

Action Items:

  • jturel to get together w/ quba42/bsuttner to gather/document SCA-related concerns, report back here next month
  • jturel to comment on 27524 and open redmine for error msg
  • ggainey to add sajha and jlenz to meeting invite
  • ggainey to add minutes to Katello/Pulp-Community Integration meeting minutes
2 Likes
#13

2022-04-05 1000-1030 GMT-5

Attendees: iballou, quba42, mbucher, sajha, paji, ggainey

Regrets:

Agenda:

  • State of aiohttp(s) progress? E.g.: https://github.com/theforeman/pulpcore-packaging/pull/379
    • “can’t use https proxies w/ Pulp w/out aiohttp/3.8+, puthon3.8+”
    • [AI] iballou to inquire and let quba42 know, doc here as well
  • katello-host-tools service restarts:
  • When is the next breaking changes pulcore release? How is Katello approaching the changed pulpcore deprecation policy?
    • core/3.20 is allowed to have breaking changes. 3.19 to be released “really soon now” (today?)
    • katello: doesn’t want to get stuck on old versions, will pay attention to the proboem if/when/as it happens
  • Smart-proxy download policy streamed: https://github.com/Katello/katello/pull/10005
    • Merged!
  • 2to3 migration issue
    • large migration, Pulp3-orphan-cleanup ran in the middle of it (?!?!?)
    • is there any way this could happen in a “vanilla katello/3.18” install?
    • poss a cfg-issue (katello-installer shouldn’t allow this to happen)
  • ATIX has targetted customers in mind to test 2to3
  • Pulp moving to “releases on backport-branches” to once a week
    • ATIX mostly interested in RPM results

Action Items:

#14

2022-05-03 1000-1030 GMT-4

Attendees: ggainey, quba42, mbucher, sjha, iballou, paji

Regrets:

Agenda:

  • Upgrading Katello nightly to pulp_deb 2.18.0 (including clients):
  • DB field size migration and backporting woes:
  • Debian Errata strategy regarding pulp3 & Katello 4.x
    • current draft Katello-only implementation reusing Katello::Erratum model
    • question1: should we add them to pulp_deb to more resemble yum-errata
    • question2: should we distinguish them more in Katello creating DebErratum and YumErratum models
    • feels possible to keep this in katello, make the inheritance/implementation hierarchy make more sense
    • maybe ATIX discuss internally, bring something more concrete to this meeting #eventually?
  • Discussing pulpcore-packaging minor version bump workflows.
    • and by packaged, we mean “get RPMs made”
    • how can ATIX make this request less ad-hoc?
    • current katello approach is similar to current ATIX - open pulp-packaging PR and then ping folk if ignored
    • new pulpcore-Y-release takes more work (since requires new directory)
1 Like
#15

2022-07-06 1000-1030 GMT-4

Attendees: ggainey, quba42, mbucher, hstct, iballou, sjha, paji

Regrets:

Agenda:

  • welcome to Tobias (hstct) from ATIX
  • orcharhino 6.0 is out https://orcharhino.com/orcharhino-6-0-0/
    • Uses Katello 4.1 with pulpcore 3.14
    • Reporting issues back to the community? (We have talked about this before)
      • Yes please.
      • discussion about when/whether to backport will be ongoing.
  • How will the breaking pulpcore 3.20 work?
    • release aimed for 14-JUN
  • Outlook: Improving pulp_deb sync performance
    • What has your experience been: Is performance a pure Pulp topic, or is it as much to do with how Katello uses Pulp?
      • almost all in the plugin itself
  • Tobias is rewriting tests to use pytest
    • ggainey cheers wildly
    • talk to dkliban/bmbouters
  • debian pkg-upload broken in katello master
    • ATIX nag incoming
    • one fix made, more probs discovered
  • katello removed reviewing deb-filters-PR from sprint
  • [paji] needs-restart PR stalled

Action Items:

1 Like
#16

2022-07-05 06-28 1000-1030 GMT-4 (rescheduled due to conflict)

Attendees: ggainey, iballou, quba42, hstct, msinghal

Regrets:

Agenda:

Action Items:

#17

2022-08-09 1000-1030 GMT-5 (rescheduled due to conflict)

Attendees: quba42, sjha, iballou, hstct, ggainey

Regrets:

Agenda:

Action Items:

  • RE backport/release discussion - iballou to make the public parts, public, in Discourse thread
  • RE HTTPS Proxy work - ggainey to look for any other discussion around this and include here [DONE]
  • RE pulp_deb and urlencoding content-URLS - quba42 to release pulp_deb w/ the fix
  • RE pulp_deb CV filtering - quba42 to talk to msinghal about presenting at katello-community-demo
  • ggainey to add minutes to Katello/Pulp-Community Integration meeting minutes
1 Like
#18

2022-09-06 1000-1030 GMT-5

Attendees: msinghal, quba42, iballou, sjha, ggainey, hstct, mbucher

Regrets:

Agenda:

  • Previous AIs
    • RE pulp_deb and urlencoding content-URLS - quba42 to release pulp_deb w/ the fix
  • How does Katello install Pulp? I believe the pulp_installer is just for pip based installs. Could I just install the RPMs and start the services? Is a from RPM Pulp installation (without installing all of Katello) somehow supported? Is it documented somewhere?
  • pulp_deb optimize sync will depend on pulpcore >= 3.21.0. Given the latest pulpcore-packaging is 3.18, when can we reasonably expect this to become available for Katello?
  • HTTPS proxy (again), will this be a “downstream only fix” (for older python versions): Pulpcore 3.21 release schedule - #4 by bmbouter
    • in a word - “yes” :slight_smile:
  • I would like to set FORCE_IGNORE_MISSING_PACKAGE_INDICES = True in /etc/pulp/settings.py for Katello installations. I imagine this would be some foreman-installer change, but how can I find the right place?
  • Import/Export feature doesn’t work in Katello but works fine with pulp_deb.
    • work directly w/ Partha Aji (paji@redhat.com, partha in matrix in #pulp-dev)
  • “regenerate repository metadata” was removed from UI (for all content types) because of problems with pulp_rpm “complete mirror” mode? Katello 4.3 Cannot Republish Repository Metadata or View Packages of a Repository - #4 by jeremylenz - Support - TheForeman
    • pulp_deb would like to see it made available for deb-repos
    • “Why does RPM ‘Win’?”
      • Partially a one-off emergency fix
      • katello would like to have it back as well
      • what katello PR removed this?
      • great discussion around what “republish” means for other content types
      • maybe remove UI IFF “complete mirroring” in use
  • Forklift issues.
    • what’s up w/ katello-devel?
    • centos7/8-katello-stable/unstable aren’t working (for a variety of reasons)
    • see foreman-dev matrix
      • hstct: anually reinstall webpack?
      • sjha: “bundle exec rails webpack:compile in foreman dir…”
      • sjha: immediately upgrade nodejs to 12 on dev-box
      • sjha: delete package-lock/node files in foreman/katello and reinstall?
      • iballou: stable boxes should “just work”?
#19

2022-10-04 1000-1030 GMT-5

Attendees: sjha, iballou, ggainey

Regrets:

Agenda:

#20

2022-11-02 0900-0930 GMT-5

Attendees: ggainey, iballou, quba42, mbucher, sjha, hstct

Regrets:

Agenda:

Action Items:

  • AI: [iballou] to look at SCA-thread, add debugging commentary if/as useful
  • AI: [ATIX] to start a thread on import/export with “katello” tag, maybe ping iballou/sjha/paji in #foreman-dev
  • AI: [ATIX] someone at ATIX will poke at issue #35362
  • ggainey to add minutes to Katello/Pulp-Community Integration meeting minutes