Best way to setup an autosync

Support
#1

Hi guys,
I would like to know which way do you suggest to trigger the sync of a remote.
Right now I set up a crontab that executes the usual cli command, but it doesn’t look like the cleanest approach.
For reference:
pulp rpm repository sync --name <repo-name>

I was thinking about creating a task that triggers periodically the sync, but I couldn’t find any reference of how to do it.
Thank you in advance!

1 Like
#2

There are a lot of ways users control when/how they want Pulp to do things, so in Pulp3 we got out of the business of adding our own; Pulp doesn’t have any “intrinsic” recurring-task support. Invoking pulp-cli via cron is def a valid approach. We also have pointers to rundeck and dist-cron : Scheduling Tasks — Pulp Project 3.23.1 documentation

4 Likes
#3

I must admit that the question for scheduled syncs comes up over and over again. So Pulp may eventually adopt such a feature.
But also if you like (and if we are talking >= a dozen repositories), i think you can come up with a shellscript that filters repositories by a “sync: nightly” label (and runs as part of cron.nightly). That way you can easily put the control over which repos are subject to nightly sync back into the Pulp API.
Thinking about weekly, monthly now? You get the idea…

2 Likes
#4

The sequence that has happened historically with management systems and scheduling goes something along these lines:

  • “Scheduling tools are too complicated - can you just build in a ‘repeating task’ that runs at a specific time, until I decide to cancel it?”
  • “I don’t want to specify it in Zulu time, I want to specify it in my time zone”
  • “What do you mean, the server is running in Madagascar?”
  • “Oh - it needs to only run on Fridays”
  • “Actually, it needs to not-run if the previous run failed”
  • “It needs to run/not-run when the following things are currently-true about the system”
  • “Why did DST mess up my runs?”
  • “Don’t run on holidays”
  • “Why did Easter mess up my runs?”
  • “Why can’t thiis just implement cron’s interface?”

Eventually, every time a “simple” scheduling function is built into a management tool, it turns into cron. Eventually. After a lot of RFEs, and bugs.

“cron” and friends have decades of “how to schedule” experience, so it’s better to take advantage of them, than take advantage of a low-budget version built into the tool :slight_smile:

3 Likes
#5

One thing I just want to mention: if you don’t have auto-publish set up, that command will only sync new content into pulp. So you may want to consider calling publish after or setting up autopublish.

3 Likes