Pulp/Galaxy integration meeting

Minutes
#6

May 16, 2022

Action Items

Priority List

  1. synclist removal (Andrew + Adrian)
    • on schedule to deploy tomorrow
  2. Collection signing (Bruno)
    • waiting on API rerooting
  3. API rerouting (released pulpcore 3.18) (Andrew)
    • open PR enabling this, waiting on QE review
  4. Container signing (Bruno)
    • On the queue to work on after LDAP integration task.
  5. Transition galaxy_ng to Roles (Brian M/David)
    • Are there restrictions on prefixing roles?
      • locked roles: yes, user def roles: no
  6. Private cloud storage (Matthias + Andrew)

In the priority Queue

  • Logo sync / namespaces
  • PoC: Repo Management - waiting on PMs requirements
  • community galaxy replacement

Pulp

  • Pulp_ansible
    • need to pick an LTS branch for pulp_ansible that is compat w/ pulpcore==3.18
      • pulp-ansible 0.13.0 would align well with katello
  • Pulp_container
    • 2.1.3 is released and compat with django 2.2.28
    • 2.12.1 is released - it fixes installation dep conflicts between ansible and container plugins
  • Pulpcore
    • declaring pulpcore 3.18 as LTS
    • downloader improvements SIG underway

Galaxy

Open PRs

#7

May 23rd

Action Items

  • [andrew] invite Tanner to next pulp galaxy mtg to give update on community galaxy replacement
  • [davidn] schedule namespace meeting
  • [gerrod] collaborate on bugs filed by Andrew

Priority List

  1. synclist removal (Andrew + Adrian)
  2. Collection signing (Bruno)
    • waiting on API rerouting
  3. API rerouting (released pulpcore 3.18) (Andrew)
  4. Container signing (Bruno?)
    • becoming higher prioritiy
  5. Transition galaxy_ng to Roles (Brian M/David)
  6. Private cloud storage (Matthias + Andrew)
    • fixed some things, and will give more complete update next week

In the priority Queue

  • Logo sync / namespaces (higher prio)
  • PoC: Repo Management - to implement collection content type (low-medium prio)
  • community galaxy replacement

Pulp

  • Pulp_ansible
    • waiting on changed from gerrod to release as backports
  • Pulp_container
    • no updates
  • Pulpcore
    • LTS convo ongoing

Galaxy

  • should we re-route pulp apis on all galaxy ng deployments and advertise the pulp apis under the galaxy api root?
GET /api/galaxy/
{
    "available_versions": {
        "v3": "v3/",
        "pulp/api/v3": "pulp/api/v3"
    },
}

Open PRs

#8

June 6, 2022

Action Items

[fabricio] to release pulp_ansible 0.7.z and 0.10.z
[tanya] to schedule pulp-cli meeting

Priority List

  1. synclist removal (Andrew + Adrian)
  2. Collection signing (Bruno)
    • deploy in progress + PR up
    • need to learn how to use rpmsign to create signatures
  3. Container signing (Bruno!)
    • will start next week after LDAP work
  4. Transition galaxy_ng to Roles (Brian M/David)
  5. Private cloud storage (Matthias + Andrew)
    • skip this week so mdellweg can confirm this is done next week

In the priority Queue

  • Logo sync / namespaces (higher prio)
  • PoC: Repo Management - to implement collection content type (low-medium prio)
  • community galaxy replacement

Pulp

  • Pulp_ansible
  • Pulp_container
    • RBAC docs
    • uses queryset_scoping from core
  • Pulpcore
    • 3.20 on track to release on June 14th, all plugins will need to release with pulpcore>=3.20,<3.25
    • Working on simpmlifying CI
      • simplifying how we install

Galaxy

  • update on community galaxy replacement

Open PRs

#9

June 13, 2022

Action Items

Priority List

  1. synclist removal (Andrew + Adrian)
    • deployed to c.rh.c and no longer using the curate task
  2. Collection signing (Bruno)
    • deploy in progress + PR up
    • need to learn how to use rpmsign to create signatures
  3. Container signing (Bruno!)
    • will start this week after LDAP work
  4. Transition galaxy_ng to Roles (Brian M/David)
  5. Private cloud storage (Matthias + Andrew)
    • Waiting on pulpcore==3.20 to release
    • Waiting on pulp_ansible compat release with pulpcore>=3.20,<3.25
  6. galaxy.ansible.com replacement
    • PoC available
    • bmbouter, davidN, and jtanner

In the priority Queue

  • Logo sync / namespaces (higher prio)
  • PoC: Repo Management - to implement collection content type (low-medium prio)

Pulp

  • Pulp_ansible
    • Upcoming y-release for 3.20 compat pulpcore>=3.20,<3.25
  • Pulp_container
    • Upcoming y-release for 3.20 compat pulpcore>=3.20,<3.25
    • adding import/export for container-push repos
  • Pulpcore

Galaxy

#10

June 27th

Action Items

Priority List

  1. Collection signing (Bruno)

  2. Container signing (Bruno!)

    • onboarding to container signing podman/skopeo in progress

  3. Transition galaxy_ng to RBAC Roles (Brian M/David)

    • waiting on pulpcore 3.20 and plugins compat with it

  4. galaxy.ansible.com replacement

    • PoC available
    • bmbouter, davidN, and jtanner

In the priority Queue

  • Logo sync / namespaces (higher prio)
  • PoC: Repo Management - to implement collection content type (low-medium prio)

Pulp

Galaxy

July 11, 2022

Action Items

[bruno] write a jira ticket for verification changes

Priority List

  1. Collection signing (Bruno)
  2. Container signing (Bruno!)
    • onboarding to container signing podman/skopeo in progress
  3. Transition galaxy_ng to RBAC Roles (Brian M/David)
    • Waiting for cloud deployment. Date TBD
  4. galaxy.ansible.com replacement
    • PoC available
    • bmbouter, davidN, and jtanner
  5. Logo sync / namespaces (higher prio)
    • Namespaces are for RBAC only
    • Logos will be represented elsewhere

In the priority Queue

  • PoC: Repo Management - to implement collection content type (low-medium prio)

Pulp

Galaxy

Open PRs

#11

July 18th

Action Items

Priority List

  1. Collection signing (Bruno)
  2. Container signing (Bruno!)
    • QE has a test plan
  3. Transition galaxy_ng to RBAC Roles (Brian M/David)
    • Waiting for cloud deployment.
  4. Logo sync / namespaces (higher prio)

In the priority Queue

  • PoC: Repo Management - to implement collection content type (low-medium prio)
    • started to work on UI
    • some issues to be addressed on AH side
    • Mockups:

Pulp

Galaxy

Open PRs

#12

July 25th

Action Items

Priority List

  1. Collection signing (Bruno)
  2. Container signing (Bruno!)
    • need to upgrade pulp-container to 2.11 and that needs rbac
  3. Transition galaxy_ng to RBAC Roles (Brian M/David)
    • prioritizing RBAC work first
  4. Logo sync / namespaces (higher prio)

In the priority Queue

  • PoC: Repo Management - to implement collection content type (low-medium prio)
    • started to work on UI
    • some issues to be addressed on AH side
    • Mockups:

Pulp

Galaxy

  • mostly the focus is on RBAC

Open PRs

#13

August 1st

Priority List

  1. Collection signing (Bruno)
  2. Container signing (Bruno!)
    • waiting on the RBAC branch to be merged
  3. Transition galaxy_ng to RBAC Roles (Brian M/David)
    • rebasing of the feature branch to get in sync with master
  4. Logo sync / namespaces (higher prio)
    • no updates

In the priority Queue

  • PoC: Repo Management - to implement collection content type (low-medium prio)
    • Architectural planning

Pulp

Galaxy

  • RBAC is worked towards merging
  • Deploying collection signing to CRC planned shortly after RBAC
#14

August 8st

Action Items

  • [mdellweg] david, andrew, tanner, matthias, gerrod, brian? - meeting for sha256 uniqueness

Priority List

  1. Collection signing (Bruno)
    • no pending items, some UI issues/improvements in progress
  2. Container signing (Bruno!)
    • waiting on the RBAC branch to be merged
    • will submit a PR against master including dev env changes
  3. Transition galaxy_ng to RBAC Roles (Brian M/David)
    • almost there, hope to merge PR tomorrow
  4. Move Ansible Namespaces to pulp ansible (higher prio)
    • there is a WIP PR in pulp-ansible for Namespace model

In the priority Queue

  • PoC: Repo Management - to implement collection content type (low-medium prio)
    • Architectural planning - Ina/Matthias reviewed the doc

Pulp

Galaxy

  • RBAC is worked towards merging
  • Deploying collection signing to CRC planned shortly after RBAC

Open PRs

#15

August 15th

Action Items

  • [rtfm] discuss the state of RBAC and content isolation and distill a common goal for plugins

Priority List

  1. Collection signing (Bruno)
    • no pending items, some UI issues/improvements in progress
    • c.rh.c soon but blocked on RBAC
  2. Container signing (Bruno!)
    • waiting on the RBAC branch to be merged
    • will submit a PR against master including dev env changes
  3. Transition galaxy_ng to RBAC Roles (Brian M/David)
    • RBAC branch merged
    • to be tested in stage
  4. Move Ansible Namespaces to pulp ansible (higher prio)
    • there is a WIP PR in pulp-ansible for Namespace model
    • working on tests

In the priority Queue

  • PoC: Repo Management - to implement collection content type (low-medium prio)
    • Architectural planning - Ina/Matthias reviewed the doc

Pulp

Galaxy

  • RBAC is merged
  • Deploying collection signing to CRC planned shortly after RBAC is deployed to CRC
#16

August 22nd

Action Items

Priority List

In the priority Queue

Pulp

  • Pulp_ansible
    • revisit renegotiate coownership
      • overall supportive
      • encourage PR reviews
    • Can we rebuild the upload workflow with Pulp facilities like SingleArtifactContentUploadViewSet, deferred_validate, …?
      • currently pulp_ansible is reinventing all the stuff
      • https://github.com/pulp/pulp_ansible/pull/1176
      • pulp_ansible has 7 APIs for uploading collections across 5 viewsets, which can we remove? (from the generated docs):
        • /ansible/collections/ - viewsets.CollectionUploadViewSet
        • /pulp/api/v3/content/ansible/collection_versions/ - viewsets.CollectionVersionViewSet
        • /pulp_ansible/galaxy/{base_path}/api/v2/collections/ - galaxy.views.GalaxyCollectionView
        • (deprecated) /pulp_ansible/galaxy/{base_path}/api/v3/artifacts/collections/ - galaxy.v3.views.LegacyCollectionUploadViewSet
        • (deprecated) /pulp_ansible/galaxy/default/api/v3/artifacts/collections/ - galaxy.v3.views.LegacyCollectionUploadViewSet
        • /pulp_ansible/galaxy/{path}/api/v3/plugin/ansible/content/{base_path}/collections/artifacts/ - galaxy.v3.views.CollectionUploadViewSet
        • /pulp_ansible/galaxy/default/api/v3/plugin/ansible/content/{base_path}/collections/artifacts/ - galaxy.v3.views.CollectionUploadViewSet
      • We are not properly testing all these APIs in our tests
  • Pulp_container
  • Pulpcore

Galaxy

  • Any updates on the Namespace work?
  • Download log LGTM with (minor fixes)

Open PRs

  • want to talk, link a PR to the agenda
#17

September 26th

Action Items

  • need list of “fast response branches”

Priority List

In the priority Queue

Pulp

  • Can we make a list of “fast response branches”?

Galaxy

Open PRs

  • want to talk, link a PR to the agenda
#18

Oct 10, 2022

Action Items

Priority List

  1. Repo managment
  2. Namespace sync

In the priority Queue

Pulp

  • Pulp_ansible
    • Can the OCI-env run galaxy_ng and the tests
    • some intererst from two upstream user groups to use pulp_ansible and/or galaxy_ng
  • Pulp_container
  • Pulpcore

Galaxy

#19

Oct 24th

Action Items

Priority List

  1. Repo managment
  2. Namespace sync

In the priority Queue

Pulp

  • Pulp_ansible
    • bmbouter to focus on hosted content and step away from working on pulp_ansible
      • going to focus on pulp being deployed with the RPM content type
      • will stop attending our meetings
      • please work with mdellweg as the pulp_ansible lead and ipanova as the pulp_container lead
      • thank you for all the collaboration
    • https://github.com/pulp/pulp_ansible/pull/1252
  • Pulp_container
  • Pulpcore

Galaxy

  • need to add to the UI option to sign specific tags in pulp-container (Execution env). Now the sign button signs whole repo ( all tags)
#20

Oct 31st

Action Items

Priority List

  1. Repo managment
  2. Namespace sync

In the priority Queue

Pulp

  • Pulp_ansible
  • Pulp_container
  • Pulpcore

Galaxy

#21

Nov 14th

Action Items

[gerrod] release pulp_ansible after “lockless import” is merged.
[davidn] file an issue for “private” on ansible repositories.

Priority List

  1. Repo managment
  2. Namespace sync

In the priority Queue

Pulp

  • Pulp_ansible
    • Lockless import is blocked by how to run tests.
    • Want a release after it.
  • Pulp_container
  • Pulpcore

Galaxy

#22

Nov 21

Action Items

[gerrod] release pulp_ansible after “lockless import” is merged.
[davidn] file an issue for “private” on ansible repositories.
[mdellweg] find and prioritize the pulp_ansible RBAC issue.

Priority List

  1. Repo managment
  2. Namespace sync

In the priority Queue

Pulp

Galaxy

  • object permissions for repositories and remotes
    • Looks like the time has come to add RBAC to pulp_ansible.
#23

Feb 20

Action Items

[dnewswanger] file an issue for pulp_ansible copy.

Pulp

  • Pulp_ansible
    • cross repo search performance requiring a new approach.
    • RBAC is close to be ready for final review.
    • Associated Content must be managed separately with modify.
      • A copy (that takes a source repo) could carry associated parts.
    • Namespace api seems to introduce some breaking changes.
      • e.g. image upload vs url
      • API’s may be coexisting for a while
  • Pulp_container
  • Pulpcore

Galaxy

#24

Feb 27

Action Items

Pulp

Galaxy

Mar 06

Action Items

Pulp

  • Pulp_ansible
  • Pulp_container
  • Pulpcore
    • Django 4.2 is going to be released in April (pulpcore v3.25)

Galaxy

Mar 13

Action Items

Pulp

  • Pulp_ansible
  • Pulp_container
  • Pulpcore

Galaxy

  • (event conflicts with galaxy team meetings thanks to DST now)
  • alikins leaving red hat
  • reviews, reviews, reviews
    • RBAC
    • Collection search
    • more to come?
  • Thoughts on https://github.com/pulp/pulp_ansible/pull/1393?
    • API endpoint for copying/moving collections to multiple repos

March 20th

Pulp

Galaxy

April 27th

Action Items

Pulp

Galaxy

  • [awc] Is it still the future plan to release pulp* frequently as-needed, perhaps weekly, when pulp and galaxy have active development? (to avoid galaxy_ng using pulp dev branches)
    • pulpcore bugfix releases are out every week given that there are fixes to roll out

May 15

Action Items

Pulp

  • Pulp_ansible
    • 0.17.1 released last week
  • Pulp_container
  • [lmjachky] hassle around the keycloak integration bug one day before dev-freeze (https://github.com/pulp/pulp_container/issues/1254)
    - Is there anything that can be done in the future to avoid similar situations? Discuss with David N.
    - keycloak not present in the CI pipeline at the moment (just ldap)
  • Pulpcore
    • every tuesday new Y (if possible)/X release
    • released pulpcore 3.25 (breaking change release)
      • comes with django 4.2
      • all plugins need comptability migrations (the release for pulp_file and pulp_certguard already handled)
      • next breaking release will be 3.40

Galaxy

  • pulp_ansible 0.17.1 unblocked the performance/investigation work

May, 22nd

Action Items

Pulp

  • Pulp_ansible
  • Pulp_container
    • Planning to release 2.15 soon compatible with 3.25.
  • Pulpcore
    • 3.26 released

Galaxy

  • Beta Galaxy to Production
    • Discovered some bottlenecks
      • Namespace meta data query maxing out db, will first need namespace sync to be resolved
  • https://issues.redhat.com/browse/AAH-2262
    • reupload of collections that got removed is impossible until cleaned up by orphan cleanup
    • adding a checksum may be too dangerous
      • We need to understand the implications better.
#25

June, 05

Action Items

Pulp

Galaxy

  • [awcrosby] For galaxy-authored PRs that are higher priority & need pulp reviews, what is best way to mark/ask?
    • Keep chasing people in chat channels.
    • Add short information about the change and why they are a priority.
  • [awcrosby] What is best way to handle backport PRs… pulp reviewer, galaxy reviewer, patchback pr w/ no CI issues
    • We use patchback.
    • A single review is needed to merge; if everything goes smoothly, the original author can do that.
  • Galaxy to production