Pulp file sync slow

Support
#1

Problem:
Pulp file sync is extremely slow. It takes about the same time as pulp content upload while using the default 1MB chunk size

Expected outcome:
Faster sync between master /slaves
Ability perhaps to specify a network rate at sync or remote repository definition

Pulpcore version:
3.17.3

Pulp plugins installed and their versions:
{
“component”: “rpm”,
“version”: “3.17.1”
},
{
“component”: “file”,
“version”: “1.10.1”
},
{
“component”: “ansible”,
“version”: “0.11.1”
}

Operating system - distribution and version:
RHEL 7.9

Other relevant data:

#2

Could you provide a little more detail?

  • How big is the repo / the files in the repo
  • Is the “master” repo synced with a policy of “immediate” or “on_demand”
  • What are the network traffic stats on the master & replica pulp machines
#3

A:

  • the repo is about 13GB in size
  • the sync policy is set to immediate
  • the network traffic doesnt seem to be beyond normal usage, using scp we are able to transfer all the files contained in the repo in matter of around 5 to 6 minutes

the replication takes:
real 69m37.250s
user 2m18.949s
sys 0m32.900s

#4

How many files? Is it mostly big files, or a large number of small files?

#5

It is a single large file at this point.