Create snapshot of OS distributions

Support
#1

Problem/ context :
I would like to create snapshot of rpm but also deb repositories
I have create differents reposiroties for each OS to distribute packages on the company.
Now we take an example with pulp-deb for ubuntu repos.

I would like to generate each 3 months a snapshot of a distribution to be able all ubuntu servers with the same versions of patchs

I have generated a script to do create repos

POETRYRUN='poetry run'
REPOID='ext-ubuntu_bionic-dockerce_amd64'

echo "# Create repository for ${REPOID} #" 
$POETRYRUN pulp deb repository create --name=${REPOID} | jq -C

echo "# Create remote with all the necessary params for ${REPOID} #"
$POETRYRUN pulp deb remote create --name ${REPOID} ${PARAM_REMOTE_L[${ind}]} ${DEFAULT_REMOTE_OPTIONS[@]} | jq -C

echo "# Sync the remote with the repo for ${REPOID} #"
$POETRYRUN pulp deb repository sync --name ${REPOID} --remote ${REPOID} --mirror | jq -C

echo "# Create metadatas for ${REPOID} #"
$POETRYRUN pulp deb publication --type apt create --repository "${REPOID}" | jq -C

echo "# Publish on the website for ${REPOID} #"
$POETRYRUN pulp deb distribution create --name "${REPOID}" --base-path "${REPOID}" --repository "${REPOID}" | jq -C

Now I would like to know how to generate my snapshot ?
If I have well understood, I have to create a new publication base on the same repository and create a new distribution based on this publication.
I execute that commands :

$POETRYRUN pulp deb publication --type apt create --repository "${REPOID}" 
# I get the pulp_href in the output 
$POETRYRUN pulp deb distribution create --name "${REPOID}_Snap" --base-path "${REPOID}_Snap" --publication "/pulp/api/v3/publications/deb/apt/018fa564-1dad-7759-a8b2-2
1ba27026329/"

Is it the good way to generate this snapshot. The goal is to have no change in this new snapshot because I will do every 3 days an update od the main repo:

# Update every 3 days of the main repo ( and keep the snapshot as initial )
echo "# Sync the remote with the repo for ${REPOID} #"
$POETRYRUN pulp deb repository sync --name ${REPOID} --remote ${REPOID}

echo "# Create metadatas for ${REPOID} #"
$POETRYRUN pulp deb publication create --repository "${REPOID}"

Pulpcore + plugin version:
“deb”: “3.2.0”,
“rpm”: “3.25.2”,
“core”: “3.50.2”

Operating system - distribution and version:
k8s

I have read to documentation ( Promotion — Pulp Project 3.54.0 documentation ) but I would like to be sure because I don’t find examples to follow.

Thanks in advance

#2

You don’t even have to re-create a second publication. You can take the publication currently published at --base-path "${REPOID}" and use it for your new snapshot distribution using some different --base-path. You could consider naming the --base-path for your snapshot something like ${REPO_ID}_${DATE_OF_SNAPSHOT}. Keeping that snapshot publication for ever (and not deleting the distribution served there) is entirely up to you.

Every publication is a snapshot. It will never change. You can delete it or stop serving it at a given base path, but not change it.

Even if you accidentally delete a publication you wanted to keep around, you can re-create a new publication from the same repository version (assuming that is still known and still exists) and it will contain the same content as before.

In summary: Repository versions, and Publications are all immutable (snapshots) once created. They can be deleted, but they cannot be modified. The only thing that changes is what publication is served in what distribution (base path), but if and when you change that is entirely up to you. You can use features like auto publish to automatically create new publications and serve them whenever a new repo version is created, or you can micromanage everything yourself.

2 Likes
#4

Thank you.
Finally and with your help, I have found the solution.

For the community, it is what I do :

CreateRepo() 
{ 
for ind in $(seq -s ' ' 0 $((${#REPOID_L[*]}-1)) )
do
    echo "# Create remote with all the necessary params for ${REPOID_L[${ind}]} #"
    $POETRYRUN pulp deb remote create --name ${REPOID_L[${ind}]} \
    ${PARAM_REMOTE_L[${ind}]} \
    ${DEFAULT_REMOTE_OPTIONS[@]} | jq -C
    check

    echo "# Create repository for ${REPOID_L[${ind}]} repo and remote #" 
    $POETRYRUN pulp deb repository create --name=${REPOID_L[${ind}]} \
    --description ${REPOID_L[${ind}]} \
    --remote ${REPOID_L[${ind}]} | jq -C
    check

    echo "# Sync ${REPOID_L[${ind}]} repo #"
    $POETRYRUN pulp deb repository sync --name ${REPOID_L[${ind}]} | jq -C
    check

    echo "# Create publication (snapshot) for ${REPOID_L[${ind}]} repo#"
    PULP_HREF_CURR=$($POETRYRUN pulp deb publication --type apt create --repository "${REPOID_L[${ind}]}" | jq -r .pulp_href)
    check

    echo "# Create a distribution attached to the last reference for ${REPOID_L[${ind}]}_Snap repo #"
    $POETRYRUN pulp deb distribution create --name "${REPOID_L[${ind}]}_Snap" \
    --base-path "${REPOID_L[${ind}]}_Snap" --publication ${PULP_HREF_CURR} | jq -C
    check

    echo "# Create a distribution attached to the last reference for ${REPOID_L[${ind}]} repo #"
    $POETRYRUN pulp deb distribution create --name "${REPOID_L[${ind}]}" \
    --base-path "${REPOID_L[${ind}]}" --publication ${PULP_HREF_CURR} | jq -C
    check

done
}

Thanks.

2 Likes