Multi-host Deployment Without Kubernetes (pulpcore 3.95.3)

Support
#1

Hi all,

We cannot run Kubernetes in production, but we do have external PostgreSQL, Redis, and nginx available.

For pulpcore 3.95.3, is it supported to run multiple instances of:

  • pulp-api
  • pulp-content
  • pulp-worker

(using the pulp-minimal image) across multiple hosts, with nginx/load balancer in front of the API and content services?

In this setup, I’d like to confirm:

  1. Is this a supported HA / multi-host deployment model?
  2. Is pulpcore-resource-manager still required in 3.95.3, or has it been fully removed and no longer needed?

Thanks in advance for the clarification.

#2

Hey @bli111,

Yep any of the latest versions of the pulp-minimal images should support HA deployments. The current latest version as of February 20 is 3.103.4. For each pulp-minimal image you launch it with the command for the instance you want to run, e.g. pulp-api, pulp-content, pulp-worker, allowing you to scale Pulp however you desire. Each container instance should be given access to the DB, storage, and your settings to work properly. How you network the containers is up to you, however all of them need access to the internet (if you want to do syncs/on-demand/pull-through-caching), but only the content and api instances need to be routable to the nginx/load balancer. We have an example for podman/docker compose here: https://github.com/pulp/pulp-oci-images/blob/latest/images/compose/compose.yml

As for the pulpcore-resource-manager that has been removed long ago (I can’t even remember when we removed it). Everything is now done through the DB. Dispatching tasks and managing pulp-workers is now done through postgres singals and locks. The tasking system is now much more performant and reliable, allowing you to scale up many more workers with the limiting factor being the speed of your DB instance.

1 Like
#3

Thanks for the clarification — that answers our questions.

We’re currently running 3.95.3, since our internal DB platform only supports PostgreSQL 13. We’ll proceed with scaling pulp-api, pulp-content, and pulp-worker across multiple hosts using the shared DB and storage.

Appreciate the guidance.