Matching Ubuntu Repository Directory Structure in pulp_deb

Support
#1

I’ve been working with pulp_deb to mirror upstream Ubuntu repositories and noticed a difference in the directory structure.

Upstream Ubuntu repos typically look like this:

dists/<release>/

But in Pulp, since each distribution requires a unique base_path, the resulting structure ends up being something like:

<release>/dists/<release>/
e.g. noble/dists/noble, bionic/dists/bionic

This adds an extra level compared to the upstream layout.

My question is: Is there a way (or future plan) to mirror the exact upstream directory structure (dists/<release>) in Pulp distributions? Or is the current base_path constraint the expected and only supported approach?

Thanks!

#2

I am not sure I understand the question. The general form of an APT repository structure is:

<URL>/<repository_base_path>/dists/<distribution>/<release_files>`

This is true of upstream Ubuntu, e.g.:

http://archive.ubuntu.com/ubuntu/dists/bionic/InRelease
<URL> = http://archive.ubuntu.com/
<repository_base_path> = ubuntu
<distribution> = bionic

Are you asking if <repository_base_path> can be the empty string in Pulp? If so, the answer is no. Otherwise we could only provide a single repository on the Pulp content app. There are no plans to change the Pulp requirement to have a non empty base path.

Am I missing something?

#3

Thanks for clarifying. Let me show what I meant with two possible repo structures:

Structure 1 (current layout I was testing):

<URL>/
├── noble/
│   └── dists/
│       └── noble/
│           └── <Release files>
├── focal/
│   └── dists/
│       └── focal/
│           └── <Release files>
└── bionic/
    └── dists/
        └── bionic/
            └── <Release files>

Structure 2 (with ubuntu as repository base path):

<URL>/
└── ubuntu/
    └── dists/
        ├── noble/
        │   └── <Release files>
        ├── focal/
        │   └── <Release files>
        └── bionic/
            └── <Release files>

So my question was really about whether Structure 1 (where <repository_base_path> is effectively the distribution name itself, which has to be unique per repo) is supported in Pulp, or if only Structure 2 (with a shared non-empty <repository_base_path>, e.g. ubuntu) is valid.

#4

It looks like after mirroring upstream, the repo structure in Pulp always follows the pattern:

<base_path>/dists/<distribution>

For example:

<base_path>/dists/noble

So <base_path> is mandatory and uniq, and <distribution> sits under dists as expected.

#5

So in case this is still unclear and also for future reference: Both Strucutre 1 and Structure 2 are possible. To get Structure 1 you would create three different Pulp repositories with three different base paths (“noble”, “focal”, “bionic”) that contain one distribution each. Structure 2 would be one Pulp repository with base path “ubuntu” and containing 3 distributions (“noble”, “focal”, “bionic”) within it. In other words, and assuming you are getting your content via sync, you would be synchronizing multiple distributions from the upstream Ubuntu repository into just one Pulp repository.

2 Likes
#6

Thanks for clarifying the difference between Structure 1 and Structure 2.

I went ahead and tested syncing multiple distributions (each with its own remote) into a single Pulp repository, and the resulting layout does match the upstream Ubuntu structure. That’s exactly what we’re aiming for since we’d like to maintain a one-to-one mirror of the upstream directory layout.

Just to confirm: when combining multiple distributions into a single repository like this, we can still use the mirror and optimize sync options as usual, correct?

#7

Then I have one more clarification for you: You can do this using a single remote. A remote stores a list of distributions, components, and architectures it should sync. If you are using Pulp CLI to create the remote, you can simply supply the command with multiple --distribution= flags. For example, you can do something like:

REMOTE_OPTIONS='--url=https://fixtures.pulpproject.org/debian/ --distribution=ragnarok --distribution=ginnungagap --architecture=ppc64 --architecture=armeb --policy=on_demand'
pulp deb remote create --name='my_test_remote' ${REMOTE_OPTIONS}

I suppose you could also use separate remotes and sync them into a single repo one after the next using policy “additive”. I believe this should end up with the same result, but I have not done a lot of testing, because we always assumed a single remote with multiple distributions would be used.