[Solved] Pulpv3 registry as remote = 406 Not Acceptable

Problem:
I managed to config remote registries in pulpv3, like a harbor and dockerhub. However I am unable to use a pulp registry (pulpportatil) as a remote for another pulpv3 (proteopulp). I get:

Error: Task /pulp/api/v3/tasks/018a89ff-8e04-77b8-99b0-7fe8a80ed6e1/ failed: '406, message='Not Acceptable', url=URL('http://pulpportatil:7897/v2/project/image/tags/list')'

And I can even pull images from the proteopulp podman image:

proteopulp# podman exec -it pulp bash
[root@933ac2957f9b /]# podman pull pulpportatil:7897/project/image:0.1.0

Notice both pulps are insecure (no tls). But I used a reverse proxy to add tls to the remote pulp with the same result. But AFAIK this shouldn’t trigger the 406

Expected outcome:
I can use a pulp-registry as a remote for another pulp-registry

Pulpcore version:
Many, but last week I stuck with 3.28

Pulp plugins installed and their versions:
docker.io/pulp/pulp:3.28.0
pulp-cli 0.21.2
squeezer 0.0.13

Operating system - distribution and version:
Redhat 8.6 /podman 4.0.2

Other relevant data:
The full verbose log is:

 {"pulp_href":"/pulp/api/v3/tasks/018a89ff-8e04-77b8-99b0-7fe8a80ed6e1/","pulp_created":"2023-09-12T15:25:41.253239Z","state":"failed","name":"pulp_container.app.tasks.synchronize.synchronize","logging_cid":"539ff80f72a94d53a98b69f192807792","created_by":"/pulp/api/v3/users/1/","started_at":"2023-09-12T15:25:41.313716Z","finished_at":"2023-09-12T15:25:41.495505Z","error":{"traceback":"  File \"/usr/local/lib/python3.8/site-packages/pulpcore/tasking/tasks.py\", line 66, in _execute_task\n    result = func(*args, **kwargs)\n  File \"/usr/local/lib/python3.8/site-packages/pulp_container/app/tasks/synchronize.py\", line 41, in synchronize\n    return dv.create()\n  File \"/usr/local/lib/python3.8/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 161, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/lib64/python3.8/asyncio/base_events.py\", line 616, in run_until_complete\n    return future.result()\n  File \"/usr/local/lib/python3.8/site-packages/pulpcore/plugin/stages/api.py\", line 220, in create_pipeline\n    await asyncio.gather(*futures)\n  File \"/usr/local/lib/python3.8/site-packages/pulpcore/plugin/stages/api.py\", line 41, in __call__\n    await self.run()\n  File \"/usr/local/lib/python3.8/site-packages/pulp_container/app/tasks/sync_stages.py\", line 138, in run\n    tag_list = await self.get_paginated_tag_list(tag_list_url, repo_name)\n  File \"/usr/local/lib/python3.8/site-packages/pulp_container/app/tasks/sync_stages.py\", line 335, in get_paginated_tag_list\n    await list_downloader.run(extra_data={\"repo_name\": repo_name})\n  File \"/usr/local/lib/python3.8/site-packages/pulpcore/download/http.py\", line 273, in run\n    return await download_wrapper()\n  File \"/usr/local/lib/python3.8/site-packages/backoff/_async.py\", line 151, in retry\n    ret = await target(*args, **kwargs)\n  File \"/usr/local/lib/python3.8/site-packages/pulpcore/download/http.py\", line 258, in download_wrapper\n    return await self._run(extra_data=extra_data)\n  File \"/usr/local/lib/python3.8/site-packages/pulp_container/app/downloaders.py\", line 75, in _run\n    response.raise_for_status()\n  File \"/usr/local/lib64/python3.8/site-packages/aiohttp/client_reqrep.py\", line 1005, in raise_for_status\n    raise ClientResponseError(\n","description":"406, message='Not Acceptable', url=URL('http://pulpportatil:7897/v2/hiperion/utm/tags/list')"},"worker":"/pulp/api/v3/workers/018a89b4-c671-7e47-a05f-778c535f8872/","parent_task":null,"child_tasks":[],"task_group":null,"progress_reports":[{"message":"Downloading Artifacts","code":"sync.downloading.artifacts","state":"canceled","total":null,"done":0,"suffix":null},{"message":"Associating Content","code":"associating.content","state":"canceled","total":null,"done":0,"suffix":null},{"message":"Downloading tag list","code":"sync.downloading.tag_list","state":"failed","total":1,"done":0,"suffix":null}],"created_resources":[],"reserved_resources_record":["/pulp/api/v3/repositories/container/container/018a89bd-2da0-7c69-b550-b1f664e178ae/","shared:/pulp/api/v3/remotes/container/container/018a89bd-22a9-76b9-a932-43b19bba9e20/"]}
Started background task /pulp/api/v3/tasks/018a89ff-8e04-77b8-99b0-7fe8a80ed6e1/
Error: Task /pulp/api/v3/tasks/018a89ff-8e04-77b8-99b0-7fe8a80ed6e1/ failed: '406, message='Not Acceptable', url=URL('http://pulpportatil:7897/v2/project/image/tags/list')'

Thanks for reaching out!

Which pulp_container version are you using right now? I assume you are experiencing this bug that was fixed in the past: https://github.com/pulp/pulp_container/issues/1329. We can backport this fix.

1 Like

Thanks to you Lubos, that was exactly my problem. There is no need to backport it, I can update to docker/pulp:3.34 .

Best regards,

Felix

2 Likes