Pulpcore Meeting Minutes

Minutes
#28

December 7, 2021

  • [ipanova] Are we going to participate in the upcoming conferences? Devconf and Fosdem. Do we plan any booths or just talks/workshops
    • dkliban will submit a talk to FOSDEM on using the single container
  • [ipanova] Story #9603: Reclaim disk space without providing a list of repositories - Pulp reclaim disk space for whole pulp, katello would benefit from this
    • The proposed solution looks good
  • [bmbouter] interested in adding an additional commiter
  • [bmbouter] forming telemetry working group in Jan
  • [mdellweg] Do we want to make 3.20 a remove deprecations only release?
    • let’s see what happens
    • probably not

Action Items

#29

January 4th, 2022

Action Items

  • [all] take a look at memory use feature proposal Reducing Memory Usage during Sync
  • [dkliban] Talk to downstream to (re-)define Bugzilla interaction
  • [bmbouter] Organize public, recorded talk on the new pytest fixture facilities
#30

January 11th, 2022

Pending AIs

  • [all] take a look at memory use feature proposal Reducing Memory Usage during Sync
  • [dkliban] Talk to downstream to (re-)define Bugzilla interaction
  • [ipanova] write an issue about not using SET_NULL for dependent resources
  • [ggainey] reach out to katello for tasking system upgrade
#31

January 18th, 2022

  • pulpcore is on github issues now! (BZs updated)
    • we need to rethink triage, planning, Epics, Milestones …
    • ggainey has been looking at github-projects to see if we can use it
      • github projects - not quite rdy for primetime
      • write scripts to take advantage of GH API?
      • label-driven queries
      • only need “current sprint” label
      • PRs-merged/issues-closed within date-range
    • epics are an epic pain right now
      • maybe use projects?
      • or milestones? (maybe not)
    • Example query for issue-triage
  • Discuss memory use proposal Reducing Memory Usage during Sync
    • Want to look at basic PoC soon
  • Downloader logic is convoluted (and broken to some extend)
    • can we schedule the custom-file-object for removal in 3.20?
    • [x9c4] file a PR and discuss improvements there
  • pulpcore deprecation warns for the use of the old orphan interface
    • It’s a valid test that keeps the old interface running and it’s going nowhere in the percievable universe
    • we could try to distiguish rest and plugin api deprecations
    • we don’t warn the user about the use of the old interface
    • https://github.com/pulp/pulpcore/pull/2100
    • [x9c4] remove the deprecation logging and make sure the deprecation is in the api.docs
    • [bmbouter] keep track of user facing logging
  • pulpcore is getting parallel test running

Pending AIs

  • [all] take a look at memory use feature proposal Reducing Memory Usage during Sync
  • [dkliban] Talk to downstream to (re-)define Bugzilla interaction
  • [ipanova] write an issue about not using SET_NULL for dependent resources
  • [ggainey] reach out to katello for tasking system upgrade
    • talkedf w/ katello at last integration mtg
    • ‘best’ option: downstream-only change to task-migration to trunc old-task-info at 6.10 upgrade time
    • make sure build-team is ok with this approach
    • upstream: we could update the existing migration and cherrypick back
      • backport to 3.14/15/16
    • AI: [ggainey] open upstream issue for this
    • [x9c4] file a PR and discuss improvements there
      • for removal of custom-file-object from downloaders
    • [bmbouter] keep track of user facing logging
#32

January 25th, 2022

1 Like
#33

February 1, 2022

  • Discuss memory use proposal Reducing Memory Usage during Sync
  • Github issues process adjustment suggestions
  • Changelog cherry-picking problem
    • No longer using backport issues. Changelog commit removes files from the CHANGES directory. These files need to be present on ‘main’ branch until the changes are released in the latest release.
    • We need to figure out Something - because this is the last, larget, manual step blocking releases
    • Maybe each previous-branch-changelog is a separate file?
      • How?!?

February 8, 2022

#34

February 15, 2022

  • rerooting update
    • pulp-cli and pulpcore is working well
    • pulp_installer to get some updates to its reverse proxy configs
    • plan is to test in one of the jobs
  • review periodic tasks discussion (aka TaskSchedule)
  • plugin removal
    • blocked by CI weirdness around service names in ci container
    • We fix the ci base container to forget all workarounds for the old tasking system
#35

Pending AIs

  • [dkliban] work on changelog cherry-picking automation
  • [bmbouter] PoC of memory reduction in Stages API
  • [matthias] write a post on discourse to start download refactor SIG

March 1st, 2022

#36

Pending AIs

March 8th

  • memory PoC in-progress
    • one feature tries to log the relative memory usage of declarative versus everything else
    • adds a new setting TASK_MEMORY_LIMIT
      • Attempts to reduce batching and put() calls at first stage
  • pending openapi schema changes to improve usability
  • update on pytest unittest issue
  • we will need to release 3.15.5 once QE test the patches
    • [daniel] will take care
#37

Pending AIs

[bmbouter] open proposal issue to remove queues
[fabricio/dennis] ask katello if they want ruby bindings docs
[brian/matthias] open django prefetch bug

March 22

March 15th

  • memory PoC concluded
    • adjusting batch size is effective at reducing memory
    • dynamically adjusting it doesn’t keep it within a “target amount”
    • it runs extremely slow 10x plus
    • memory is never returned to the OS, the pages mostly just stay allocated and later
    • new memory charting debugging feature added:
    • proposal: to expose the batch size to users, the batch size is likely the single best adjustment to reduce memory
    • we need each plugin to also look at their own first-stage code to see if it can be memory optimized
    • proposal: https://github.com/pulp/pulpcore/issues/2069#issuecomment-1066853598
  • maintainability policy
    • how far should we keep ensuring old branches are working?
    • single container currently does latest 3 y-stream
    • pulp-cli tests with 5 y-streams
#38

Pending AIs

[bmbouter] Go through possible close query Issues · pulp/pulpcore · GitHub

March 29

#39

Pending AIs

[someone] open issue to audit and establish an upper bound for our requirements.txt deps

April 5th

#40

Pending AIs

[fabricio] audit and establish an upper bound for our requirements.txt deps

April 12th

#41

Pending AIs

  • [bmbouter] talk about plugin compatibility policy for 3.20+ on discourse

April 19th

#42

Pending AIs

[matthias] find a slot for downloaders SIG kick off meeting (ipanova, dalley, bmbouter)
[grant] will open ticket with legals re:logo licensing

April 26th

  • Discussion around list APIs for Exporters and Remotes
  • Downloader SIG: need to schedule kick-off meeting
  • Release rotation - time to rotate?
    • Brian will take care of 3.20 release
    • what to do with the blogposts on the pulpproject.org? Probably we will drop those and have only discourse release announcements
    • need to fix deps requirements( upper bound) on older releases and main branch before 3.20 release
  • decision on compatibility release?
    • if 3.25 then it wil be 9-10 months roughly until next breaking change. 3.25 it is
  • Is the Pulp logo available under Creative Commons, CC0, CC-BY, or CC-BY-SA licenses?
    • our logo is unlicensed
    • we need to open a ticket with legals
  • Streaming support needs a little more work around content ranges and caching; test will probably a new ci scenario with sftp storage
  • triage security alerts?
    • yes, add them next to the PR review ( from the next meeting)
#43

May 3rd

#44

May 10th

  • Look through remaining maybe-finished issues
  • z-releases
  • 3.20 tentative release date? first go/no-go?
  • status of django 2.2 in pulpcore 3.7 / 3.14
    • updating core/3.7 to last 2.2LTS, will require a change in (at least) matching pulp_container version
    • need to evaluate other plugins (Required PR: ?)
    • need a plan for who/when/testing
  • bmbouter and dkliban are updating pulp_file tests to use pytest, merging them with pulpcore
#45

May 24th

#46

May 31st

#47

Pending AIs

[dkliban] release pulpcore 3.19.1
[bmbouter] to propose majority use of nightly tests
[bmbouter] to highlight UI component creation on discourse

June 7th

  • telemetry pr in-review
  • CI Update
    • Functional tests to be run inside the container instead of the host
      • Installation is a nightmere today
      • More similar to dev-environment
  • Reducing CI runtime:
    • moving most tests to nightly
  • Collaboration with david N on using galaxy NG generic React + Patternfly components for a future Pulp UI