Warnings During Ubuntu Repo Sync with pulp_deb – ContentLengthError

Support
#1

While syncing an Ubuntu repo with pulp_deb, I’m seeing a lot of warning messages like the following:

Not enough data to satisfy content length header.'>)
pulp [49d4ebbf591444d6b6f949dc66cfd9a0]: backoff:INFO: Backing off download_wrapper(...) for 0.3s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed: <ContentLengthError: 400, message='Not enough data to satisfy content length header.'>)
Backing off download_wrapper(...) for 0.7s (aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed: <ContentLengthError: 400, message='Not enough data to satisfy content length header.'>)

The sync eventually completes, but the logs fill up with these retries.

  • What exactly do these warnings mean?
  • Is this expected when syncing from Ubuntu upstream mirrors, or does it indicate something wrong with the remote server/network?
  • Is there anything we can configure in pulp_deb to reduce or avoid these warnings (e.g., retries, mirror selection)?

Thanks in advance

#2

I have not seen these warnings before. It appears to be happening during download, which means it can be difficult to pin down what is broken (is it something pulp_deb passes to pulpcore, is it the downloader in pulpcore, is it the python library that performs the download, or is something broken with the server you are syncing from, or maybe there is something weird happening in the network infrastructure).

Can you provide the exact remote and any sync options you are using, so we can try to reproduce this?

1 Like
#3

You can configure some rate limits (like number of concurrent downloads) on the remote. Maybe your Pulp trying to pull all content is considered too greedy on the upstream server and you fall victim to rate limiting on their side.