Pulpcore
RPM
Depsolving work ongoing
Retain package versions sync optimization work nearly finished
pulp_rpm/3.18 coming “soon” (for core/3.20 support)
Migration
EOL discussion still ongoing
Ansible
galaxy-ng/migration discussions in progress
Namespace model is moving into pulp-ansible
Pulp Container
OSTree
What’s the status of ostree support in katello?
Released in upstream katello
Will be downstream “soon”
Pulp CLI
Pulp team meetings in September?
12-SEP - 16-SEP in Raleigh
18-JUL - 22-JUL in Brno
Thoughts on https://issues.redhat.com/browse/SAT-8244 (Priority on pulp tasks)
katello/4.5 on core/3.18 “just about done”
Pulpcore
RPM
3.17.6 released
3.18 probably in the next 2 weeks
Working with Waldirio on some customer cases / BZs
Module obsoletes support - will probably need to be backported to 3.17
Will eventually show up in RHEL9
Migration
Discussion around RH IT and their enormous Sat6.9 and migration
Ansible
Under discussion: Expanding the “Ansible Roles” API to allow for uploads
Is this useful for katello? discussion/brainstorming in order
Currently not supported at all, not sure if it’s “on the roadmap”
Pulp Container
Prep for 3.20 compat release
OSTree
Pulp CLI
How was ACS with RHUI tested?
Q: How does one get access to test when inside AWS?
Correct certs and correct header to talk to RHUI
Mtg RE Sat-IT-setup, discussion ongoing
Will work together “if weirdness shows up”
Ansible client consumption of collections?
AI: sjha to set up call, sjha/bmbouters/newswangerd to connect on What’s Changing
See #pulp-ansible in Matrix
next-Y (snap 26) bz verification
prev-Z (snap 1) bz verification
Automation failure fixes
Automation for feature areas
Pulpcore
RPM
Modularity-obsoletes
Supported in rpm/3.18 and later
Needed whenever modules decide to implement it (not currently used in RHEL9) - have been told it is possible in RHEL9.1
Planned to be a backportable change, but will be downstream-only, need feedback on how to go about shipping this
Prob should be brought up to PMs
probably/possibly will affect katello-applicability?
Dalley to send doc to iballou
Orphan cleanup failure fix https://github.com/pulp/pulp_rpm/issues/2459
Focusing on tests and BZs
rpm/3.18 release coming #soon (ish)
Migration
EOL is picked 31 Dec 2022
Ansible
0.14 release pending (3.20 compatibility)
Pulp Container
OSTree
Adding support for s3 storage
Pulp CLI
next-Y (snap 26) bz verification
prev-Z (snap 1) bz verification
Automation failure fixes
Automation for feature areas
Pulpcore
Query of all changes from last week 3.20 and plugin-compat releases are all out
Planning to comment and close
No progress to report on last week’s BZs, alas
Note on releasing backports
Pulp is working to do backport-releases only on “downstream” requests
Should pulp “proactively” backport? Or only do so in response to downstream requirements?
RPM
Migration
Ansible
Adding role-import-from-github support
Pulp Container
OSTree
Pulp CLI
Orphan cleanup issue
Which pulp_rpm versions: rpm/3.14 and rpm/3.17 will be supported
Need a BZ for this to get it triaged and assigned to release(s)
Lots of discussion around releases/builds/process-expectations - fun!
Would be good to have current understanding written out
6.11 GA yesterday
6.10.7 GA due today
next-Y Snap 1 expected today
Waiting for next-Z (no date set for snap 1)
Automation failure fixes
Automation for feature areas
Katello Schedule
Katello 4.5 - core/3.18, rpm/3.17, container/2.10, file/1.10
Pulpcore
Query of all changes from last week Meeting scheduled to talk about backport policy tomorrow
Thanks to iballou for scheduling!
Add ppicka to mtg notice please
Not much to report for core this week
Ongoing convos from Discourse
RPM
[ppicka] module-obsoletes and EXD?
[ipanova] some conversation has happened, ppicka to reach out to dalley
Migration
Iballou/paji investigating a post-migration issue, very poss. All-katello
Questions RE hardlink-vs-copy
Ansible
Verifying signatures is changing, public keys will now be added to repositories
Does katello currently use collection-signatures? Sjha should know
Lots of work going on in pulp-ansible
Pulp Container
New z stream releases for 2.9 and 2.10
OSTree
Pulp CLI
Updated Sat-6.12-version-requirements above
Discussion RE docker/container and migration Fun
next-Z snap 1 bz verification
next-Y snap 1 bz verification
next-Y snap 2 is delayed till tomorrow
Automation failure and fixes for prev-Yand pre-next-Y branch
Automation for feature area
Pulpcore
Query of all changes from last week Bugfix of redirect URL rewrite that also was backported
Someone should summarize the outcome/agreement regarding the backporting strategy between Pulp/Katello/QE
RPM
Which version of Sat should the ACS go to?
Katello reports “very tech preview in katello/4.5, full support katello/4.7”
Support for module-obsoletes is approved
Testing being worked on
Prob needs to get into curr-Y - needs PM approval [dalley to nag]
Discussion ensues
Mirrored metadata would be fine, but publishing own will lose the tag
BZ work continues
Can we consider limits/rethinking usecases around on repo-copy?
libmodulmd/zabbix question raised by a customer case
Migration
BSON size-problem BZ - Hao has potential PR drafted to address
Ansible
Collection Version global uniqueness switching to sha256
Adding Collection download log
ggainey to update katello-version-matrix with pulp-ansible versions
Pulp Container
OSTree
Pulp CLI
New release #soon (3.20 compat) - 0.15
curr-Z snap 2 bz verification
next-Y snap 2 bz verification
Automation failure and fixes for curr-Y pre-next-Y branch
Automation for feature area
Pulpcore
RPM
Pavel finished the module obsoletes work
Working on BZs
Migration
Ansible
Pulp Container
OSTree
Pulp CLI
Display total size of a yum\docker\iso\file\custom repository in Satellite GUI after syncing them - https://issues.redhat.com/browse/SAT-2512
Processing metadata task should fail with an appropriate error message, whenever a task is stuck due to insufficient file system space. - https://issues.redhat.com/browse/SAT-2354
Can Pulp be configured to log outgoing requests for content downloads, for example, RPMs or metadata files? We’re trying to figure out how to verify that Alternate Content Sources are working.
No code currently, could possibly be added and backported
Alternatively: a man-in-the-middle proxy or firewall could be configured to log where requests are going
Should Katello use Pulpcore 3.20 or wait for 3.21?
Katello 4.7 stabilizes in ~ 3 months (November-ish). We estimate we need 1 month to upgrade Pulp versions (generous estimate). The delivery team needs about 2 weeks.
We can wait for Pulpcore 3.21, it should release around September
ACS issue: https://github.com/pulp/pulp_rpm/issues/2667
Figured it out, details are on the issue.
curr-Z snap 1 testing
next-Y snap 3 testing
Feature testing
Automation failure fixes
Waiting on static ouia-id for get into a build
Pulpcore
RPM
Multiple high prio BZs in flight
New releases - 3.14.17, 3.17.8
Don’t actually use these, need to release a small new patch
Another set of releases will come shortly afterwards (rpm/3.14.18, rpm/3.17.9)
From katello: make sure to set fixed-in correctly in BZs to get these picked up
Libmodulemd data repair
Fallout from having shipped wrong libmodulemd version
Repair-script will need to be called from katello utility (foreman-maintain? Rake task?)
Get katello example affected module/RPM
Migration
One PR being reviewed
Evaluating open 2to3 BZs for investigation/fixing
Ansible
Pulp Container
New releases 2.9.6 and 2.10.5
OSTree
Pulp CLI
Quick chat on 1993917 – Syncing repositories with https proxy set ends with warning Katello::Errors::Pulp3 Error Only http proxies are supported
2 cacerts are needed when talking to CDN via https proxy, one for proxy and one for cdn
The proxy cert is typically in the systems default CA trust store
Sounds like pulp2 requests library trusted it in 6.9 but pulp3 doesn’t seem to want to load it from the trust store and instead wants to be explicit on ca certificates to use.
If we have to stay explicit then it’ll be a lot of work from a katello perspective.
Let’s see if we can come up with approaches.
Paji doing a POC on the pulp-side, will report on progress
curr-Z snap 1 build is scheduled for Friday
next-Y snap 4 testing
Feature testing
Automation failure fixes
Waiting on static ouia-id for get into a build
ggainey
August 10, 2022, 4:36pm
#57
Pulpcore
Query of all changes from last week Paji’s HTTPS-Proxy PR under intense discussion
Pulp will reach out to paji to schedule
3.18.6 and 3.16.12 released and already picked up by bldteam
RPM
New releases out (3.14.18, 3.17.9)
Include dup-nevra BZs that are def affecting customers w/ smart-proxies
Continuing to bang on BZs
Working with EXD to reduce the size of their metadata
BZ filed due to CDN not updating “updated_date” on some republished advisories
This breaks several workflows
There are Unhappy Customers as a result
Migration
BSON size fix that has been merged into 0.11 and 0.12
Ansible
Pulp Container
OSTree
Pulp CLI
Creating Discourse thread for Pulpcore backporting strategy to loop-in the community
Katello and pulp should talk about the simultaneous in-person NA meetups
Chatting w/ Odilon RE katello-4.7/AH syncing up on core/3.21
curr-Z snap 1 bz verification
4 pulp BZs showed up unexpectedly, being verified
Sign-off needs to happen by 26-AUG
next-Y snap 5 testing
Snap 6 on schedule for today (from email)
Feature testing
Automation failure fixes
Waiting on static ouia-id for get into a build
1 Like
ggainey
August 17, 2022, 7:51pm
#58
Pulpcore
Query of all changes from last week 2 core/3.18 releases to update ruby-clients for faraday
Dkliban working w/ ewoud/zhunting
All the plugins also getting releases to match (in order to make installation work)
Hoping to get this fun resolved today
Need to meet w/ partha RE https proxy
Discussion w/ Galaxy RE “shared remote configurations”
RPM
Bug fixed for syncing SUSE repos (SLES)
Continuing to work on BZs, Foreman discourse reports
Ppicka working on missing-metadata w/ modules (reported by users)
New releases out for 3.14 and 3.17
dalley taking over as Satellite Liason
fao89 joining the RPM miniteam!
Migration
Hao’s PR to address 2to3 query
Ansible
Pulp Container
OSTree
Pulp CLI
What’s up with Faraday? (NameError: uninitialized constant Faraday::FilePart → 0.17.5)
Faraday needs to be 0.16 (or 1.0.1, <2.0) to have the feature needed
Should prob be able to install/pin faraday >=1.0 <2.0, on dev/test boxes
https-proxy update
Foreman fix exists that katello can take advantage of (we think)
Paji to finish investigation before calling mtg w/ pulp-team
Iballou demoing at devconf - hope to see new user feedback for us!
Paji demoing fs-exporter work to Support team on 25-AUG
prev-Z snap 1 bz verification
Sign-off needs to happen by 26-AUG
Still a half-dozen or so left to verify
next-Y snap 6.1 testing
Helping Grant https://bugzilla.redhat.com/show_bug.cgi?id=2062526
Feature testing
Automation failure fixes
Waiting on static ouia-id to get into a build
“This katello sprint” - or at least #soon
ggainey
August 24, 2022, 6:36pm
#59
August 24, 2022
Overview
Pulp
Pulpcore
Query of all changes from last week Katello: what things must be in core for 4.7?
Fs-exporter gaps (e.g., incrementals) not planned for 4.7
Is there a date in mind for “latest a core-release can be out for katello to use in 4.7”?
AI: paji to review with katello team and get back to us
RPM
Migration
“BSON too large” issues being addressed
Ansible
CollectionVersion upload is getting refactored for better consistency
https://github.com/pulp/pulp_ansible/pull/1176/ Upload fields changes: removed: name, namespace, version; added: artifact, upload, expected_[sha256, name, namespace, version]
AI: paji/sjha to catch up w/ katello-ansible-gang to make sure they’re aware
Pulp-ansible 0.13.4 was released
Pulp Container
Ipanova is getting out of the door releases with bugfixes starting with 2.8+
There is a new feature release 2.14 compat with 3.20
OSTree
Pulp CLI
Katello
https proxy discussion continues
QE
curr-Z snap 2 bz verification
Sign-off this Friday, 26-AUG
next-Y snap 7 testing
Helping Grant 2062526 – Another deadlock issue when syncing repos with high concurrency
Fix is under-review and will be backported
Feature testing
Automation failure fixes
Waiting on static ouia-id to get into a build
CV OUIA-ID PR already merged, waiting for it to be in a build
Pending - Host details page OUIA-ID PR
Question: how long is “too long” to publish a CV where we have added 1 erratum and incrementally updating (ie, we’re doing depsolving)
ggainey
September 1, 2022, 7:33pm
#60
August 31, 2022
Overview
Pulp
Pulpcore
RPM
Migration
Ansible
Pulp Container
OSTree
Pulp CLI
Katello
QE
next-Z snap 1 bz verification
next-Y snap 7 testing
Testathon is happening at the moment
Feature testing
Automation failure fixes
Waiting on static ouia-id to get into a build
CV OUIA-ID PR already merged, waiting for it to be in a build
Pending - Host details page OUIA-ID PR
Question: how long is “too long” to publish a CV where we have added 1 erratum and incrementally updating (ie, we’re doing depsolving)
Answer: 10-15m for rhel8_bos + rhel8_app for both incremental update and publishing cv
ggainey
September 7, 2022, 9:00pm
#61
September 07, 2022
Overview
Pulp
Pulpcore
RPM
3.18 blockers in the process of being merged
Still minor cleanup to finish up pre-release
Considering some changes that would allow pulp_rpm 3.18 to be used alongside pulpcore 3.18
Currently, as mentioned last week, it contains a feature that relies on pulpcore 3.19
Coordination going on with certain BZs / customer cases, hotfixes
Migration
Ansible
Pulp Container
OSTree
Pulp CLI
Katello
Testathon in-progress, so changes are on-hold for two weeks
Waiting patiently for core/3.21
RDU Tean meetups next week - should coord w/ pulp RDU meetup
QE
next-Z snap 1 bz verification underway
next-Y snap 9 bz verification
Feature testing
Automation failure fixes
Waiting on static ouia-id to get into a build
Some missing OUIA-ID for cv, sent BZ back to dev
Pending - Host details page OUIA-ID PR
Question: how long is “too long” to publish a CV where we have added 1 erratum and incrementally updating (ie, we’re doing depsolving)
Rhel8_baseos + Rhel8_appstream
Created a cv with a filter of before June 30th
Publish and did incremental update
Incremental update time: 10min 15 sec
Publishing cv after inc update: 10min 15 sec
Created a filter that includes up to July 31st
Publish and did incremental update
Incremental update time: 10min 15 sec
Publishing cv after inc update: 10min 15 sec
rhel7
Created a cv with a filter of before June 30th
Publish and did incremental update
Incremental update time: 9min 9 sec
Publishing cv after inc update: 13min 23 sec
Created a filter that includes up to July 31st
Publish and did incremental update
Incremental update time: 8min 49 sec
Publishing cv after inc update: 18min 35 sec
[Iballou] It would be interesting to see the breakdown between “pulp time” and “katello time” for this
Katello-indexing should be much faster now, would be good to know if that isn’t the case
Note: depSolving is not selected for these runs, but iirc, doing filtering and publishing does the depsolving?
In general usage, it does not - katello only asks Pulp to depsolve, if the user asks for it explicitly
Incrementals depsolve by default, so they do
ggainey
September 21, 2022, 5:09pm
#62
Pulpcore
RPM
3.18.2 is out, I had a preliminary agreement from Sat that they would be OK with having this in 6.12 - 3.18 is just a generally a small bugfix release that includes some migrations
3 still aligned to sat-next that aren’t complete, 1 in danger of not making the cutoff
Migration
New release 0.17 , that doesn’t affect katello (plugin-eol-dates)
Ansible
0.15 releasing today
Requires core/3.21
Bugfixes and 3 new features
Pulp Container
Pulp-container 2.9 and 2.10 is out with bugfixes for sat-next
OSTree
Pulp CLI
Filling out “tasks” support, to help ppl doing problem-determination
core/3.21 RPMs “almost done”
This will be for katello/4.7
Question: what needs to happen for https://github.com/pulp/pulpcore/pull/3188 ?
Multiple issues, needs multiple CHANGES/ entries
Discussion ensued around multiple-commits or not
next-Z snap 2 bz verification underway
next-Y snap 11 bz verification
Feature testing
Automation failure fixes
Waiting on static ouia-id to get into a build
ggainey
October 6, 2022, 6:08pm
#63
Pulpcore
RPM
Migration
Ansible
Pulp Container
OSTree
Pulp CLI
next-Z snap 2 bz verification underway
Sign off this Friday 9/30
next-Y snap 12 bz verification
Expecting 2 more snaps
ggainey to work w/ ltran to test potential deadlock fixes in ‘real’ environments
Feature testing & automation
Automation failure fixes
Ouia-id for both cv and host details are in
Any missing IDs will prob be in later releases
ggainey
October 6, 2022, 6:09pm
#64
Pulpcore
RPM
Migration
Ansible
Pulp Container
OSTree
Pulp CLI
next-next-Z is scoped
next-Y snap 13 bz verification
Feature testing & automation
Automation failure fixes
Ltran last day is 10/7
Lai to tell ggainey who the right QE contact will be for this mtg
ggainey
October 12, 2022, 3:59pm
#65
Pulpcore
RPM
3.18.7 is out. Fixed a migration
Note that most of the 3.18.z releases have empty changelogs due to the only changes being progressively loosening some dependency requirements for the build team
Pulp workflow discussion - “prio-list”
Migration
Ansible
Pulp Container
OSTree
Pulp CLI
Um…who will be covering Pulp/this mtg going forward?
Katello, Pulp Sat Liason to work w/ QE mgt to get definite answer
ggainey
October 19, 2022, 4:03pm
#66
Pulpcore
Query of all changes from last week Pulpcore 3.21.1 released last night
Paji’s fs-export BZ fix, see changelog for other fixed issues
Pulp_file client released last night (to unblock 3.21)
RPM
Progress being made on various BZs
Progress being made on potentially dropping the libmodulemd dependency
Discussion about dependency solving
Not sure if this is pulp or hammer:
Migration
Ansible
Pulp Container
OSTree
Pulp CLI
Oct 26 mtg
Conflicts w/ another important mtg - ok to skip next week for us?
We will skip the 26-OCT mtg
1 Like
ggainey
November 2, 2022, 4:10pm
#67
Pulpcore
RPM
Continuing to work on errata issues
Progress being made on concurrent repo syncs
Some discussion about deprecating default profiles upstream
Thinking about how that impacts us as we’ve just added a related feature (to show profile metadata)
Lots of BZ cleanup happening (yay dralley!)
Migration
QE testing for 2to3-migration-version release
ggainey working w/ vsedmik
Ansible
Migration fix updating Collection search_vector:
Pulp Container
OSTree
Pulp CLI
AWS RHUI – What is missing in Katello for basic support?
Only the two special headers?
https://github.com/pulp/pulp_container/issues/1122
Updated client-libraries can’t talk to old-smart-proxy
E.g., 3.21-client can’t talk back to 3.18-pulp
AI: sjha to pull a group together to talk through this complicated issue
Reminder – does Pulp always copy an erratum if all of its packages are copied too?
Telemetry changes are merged and will be off by default in katello 4.7
QE
3 BZs to be verified for 2to3-migration-version (1 FailedQA but fix PR is already up (thanks ggainey), 2 under test)
ggainey
November 16, 2022, 4:53pm
#68
Pulpcore
RPM
Domains-in-rpm planning started
BZs in progress
Migration
0.11.3 released for last 6.9.10 zstream
Ansible
Pulp Container
OSTree
Pulp CLI
Conflicting Meetings make moderators Sad
“Fun” simultaneity BZ logged (poss a dup?)
Oh look - shared-distttree-subrepo problem
Already in main and rpm/3.18
Backporting to rpm/3.17 requires a diff patch - need advance warning
ggainey
November 30, 2022, 4:48pm
#69
Pulpcore
RPM
More progress on dropping the libmodulemd dependency
Lots of BZs being escalated by support, trying to investigate
ppicka will be leaving RH, RPM Team somewhat undercapacity for a bit
Migration
Ansible
Pulp Container
OSTree
Pulp CLI
Pulpcore 3.18 branch upgraded pulp-rpm to 3.18.9 (From 3.18.7)
Client bindings are also at 3.18.9
Escalation around “container-stuff on smart-proxies”
Build Team
Pulpcore
Query of all changes from last week Pulpcore 3.22.0 scheduled for Nov 30, but will likely slip by one week
All tests converted to Pytest and build team should be able to run them on the RPMs they build.
RPM
Considering allowing users to skip treeinfo file. How is katello using the skip types?
Migration
Ansible
Pulp Container
OSTree
Pulp CLI
Pulp_deb monkey patch in the works. Should make it to 4.7 GA.
ggainey
December 7, 2022, 5:31pm
#70
Pulpcore
RPM
Primarily working on escalated BZs
Fedora discussions around XML-modulemd are finishing up
Ansible
Pulp Container
OSTree
Pulp CLI
Question RE reproducing prob involving Azure
Discussion RE deadlock-testing
