Pulpcore.app.tasks.repository.repair_all_artifacts: Task process died unexpectedly with exitcode 1

Support
#1

Hi All

Please help identifying the problem.

Problem:

Cleanup keeps failing.

https --auth admin:passwd POST https://ita01.group.net:30082/pulp/api/v3/repair/

Pulp-minimal:3.35

Expected outcome:

Cleanup to succeed.

Pulpcore version:

  "versions": {
    "deb": "3.0.0",
    "rpm": "3.22.4",
    "core": "3.35.0",
    "file": "1.14.5",
    "maven": "0.6.0",
    "ostree": "2.1.3",
    "python": "3.10.0",
    "ansible": "0.20.0",
    "certguard": "1.6.5",
    "container": "2.16.2"
  }

Pulp plugins installed and their versions:

Operating system - distribution and version:

Containers runs on SLES15SP5

Other relevant data:

Task 1

pulp task show --href /pulp/api/v3/tasks/018af592-9339-741c-992c-8cce730b3528/

{
  "pulp_href": "/pulp/api/v3/tasks/018af592-9339-741c-992c-8cce730b3528/",
  "pulp_created": "2023-10-03T14:45:38.496468+02:00",
  "state": "failed",
  "name": "pulpcore.app.tasks.repository.repair_all_artifacts",
  "logging_cid": "ec33e45ab2dd4067bd86f28611f35071",
  "created_by": "/pulp/api/v3/users/1/",
  "started_at": "2023-10-03T14:45:38.697973+02:00",
  "finished_at": "2023-10-03T21:13:26.450606+02:00",
  "error": {
    "reason": "Task process died unexpectedly with exitcode 1."
  },
  "worker": "/pulp/api/v3/workers/018af58f-c9f9-79df-86db-ad6a062e296d/",
  "parent_task": null,
  "child_tasks": [],
  "task_group": null,
  "progress_reports": [
    {
      "message": "Identify missing units",
      "code": "repair.missing",
      "state": "completed",
      "total": null,
      "done": 0,
      "suffix": null
    },
    {
      "message": "Identify corrupted units",
      "code": "repair.corrupted",
      "state": "completed",
      "total": null,
      "done": 0,
      "suffix": null
    },
    {
      "message": "Repair corrupted units",
      "code": "repair.repaired",
      "state": "completed",
      "total": null,
      "done": 0,
      "suffix": null
    }
  ],
  "created_resources": [],
  "reserved_resources_record": [
    "/api/v3/repair/",
    "shared:/pulp/api/v3/domains/01897284-ff96-78ee-8ff8-9dcf388d08eb/"
  ]
}

2023-10-03T14:42:16.747514147+02:00 stderr F + /usr/bin/wait_on_postgres.py
2023-10-03T14:42:20.611980015+02:00 stdout F Waiting on postgresql to start...
2023-10-03T14:42:20.611980015+02:00 stdout F Postgres started.
2023-10-03T14:42:20.750398770+02:00 stderr F + /usr/bin/wait_on_database_migrations.sh
2023-10-03T14:42:20.752791055+02:00 stdout F Checking for database migrations
2023-10-03T14:42:29.302394540+02:00 stdout F Database migrated!
2023-10-03T14:42:29.302859836+02:00 stderr F + export PATH=/usr/local/bin:/usr/bin/
2023-10-03T14:42:29.302859836+02:00 stderr F + PATH=/usr/local/bin:/usr/bin/
2023-10-03T14:42:29.302895415+02:00 stderr F + exec pulpcore-worker
2023-10-03T14:42:35.832077860+02:00 stderr F pulp [None]: pulpcore.tasking.entrypoint:INFO: Starting distributed type worker
2023-10-03T14:42:35.924086104+02:00 stderr F pulp [None]: pulpcore.tasking.worker:INFO: New worker '1@fed12e36e8aa' discovered
2023-10-03T14:42:36.014747471+02:00 stderr F pulp [None]: pulpcore.tasking.worker:INFO: Cleaning up task 018af4f0-d40b-7a97-a2e1-cae6690a6a16 and marking as failed. Reason: Worker has gone missing.
2023-10-03T14:45:38.705740225+02:00 stderr F pulp [ec33e45ab2dd4067bd86f28611f35071]: pulpcore.tasking.tasks:INFO: Starting task 018af592-9339-741c-992c-8cce730b3528
2023-10-03T14:45:38.706093780+02:00 stderr F pulp [ec33e45ab2dd4067bd86f28611f35071]: pulpcore.app.tasks.repository:INFO: Repairing artifacts.'
2023-10-03T21:13:26.407610951+02:00 stderr F Process Process-1:
2023-10-03T21:13:26.407610951+02:00 stderr F Traceback (most recent call last):
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
2023-10-03T21:13:26.407610951+02:00 stderr F     return self.cursor.execute(sql, params)
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/psycopg/cursor.py", line 737, in execute
2023-10-03T21:13:26.407610951+02:00 stderr F     raise ex.with_traceback(None)
2023-10-03T21:13:26.407610951+02:00 stderr F psycopg.OperationalError: consuming input failed: EOF detected
2023-10-03T21:13:26.407610951+02:00 stderr F 
2023-10-03T21:13:26.407610951+02:00 stderr F The above exception was the direct cause of the following exception:
2023-10-03T21:13:26.407610951+02:00 stderr F 
2023-10-03T21:13:26.407610951+02:00 stderr F Traceback (most recent call last):
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/lib64/python3.8/multiprocessing/process.py", line 315, in _bootstrap
2023-10-03T21:13:26.407610951+02:00 stderr F     self.run()
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/lib64/python3.8/multiprocessing/process.py", line 108, in run
2023-10-03T21:13:26.407610951+02:00 stderr F     self._target(*self._args, **self._kwargs)
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/pulpcore/tasking/worker.py", line 496, in _perform_task
2023-10-03T21:13:26.407610951+02:00 stderr F     execute_task(task)
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/pulpcore/tasking/tasks.py", line 48, in execute_task
2023-10-03T21:13:26.407610951+02:00 stderr F     contextvars.copy_context().run(_execute_task, task)
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/pulpcore/tasking/tasks.py", line 76, in _execute_task
2023-10-03T21:13:26.407610951+02:00 stderr F     task.set_completed()
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/pulpcore/app/models/task.py", line 175, in set_completed
2023-10-03T21:13:26.407610951+02:00 stderr F     rows = Task.objects.filter(pk=self.pk, state=TASK_STATES.RUNNING).update(
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 1206, in update
2023-10-03T21:13:26.407610951+02:00 stderr F     rows = query.get_compiler(self.db).execute_sql(CURSOR)
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1984, in execute_sql
2023-10-03T21:13:26.407610951+02:00 stderr F     cursor = super().execute_sql(result_type)
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
2023-10-03T21:13:26.407610951+02:00 stderr F     cursor.execute(sql, params)
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute
2023-10-03T21:13:26.407610951+02:00 stderr F     return self._execute_with_wrappers(
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
2023-10-03T21:13:26.407610951+02:00 stderr F     return executor(sql, params, many, context)
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
2023-10-03T21:13:26.407610951+02:00 stderr F     return self.cursor.execute(sql, params)
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 91, in __exit__
2023-10-03T21:13:26.407610951+02:00 stderr F     raise dj_exc_value.with_traceback(traceback) from exc_value
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
2023-10-03T21:13:26.407610951+02:00 stderr F     return self.cursor.execute(sql, params)
2023-10-03T21:13:26.407610951+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/psycopg/cursor.py", line 737, in execute
2023-10-03T21:13:26.407610951+02:00 stderr F     raise ex.with_traceback(None)
2023-10-03T21:13:26.407610951+02:00 stderr F django.db.utils.OperationalError: consuming input failed: EOF detected
2023-10-03T21:13:26.427985879+02:00 stderr F pulp [None]: pulpcore.tasking.worker:WARNING: Task process for 018af592-9339-741c-992c-8cce730b3528 exited with non zero exitcode 1.
2023-10-03T21:13:26.441036039+02:00 stderr F pulp [None]: pulpcore.tasking.worker:INFO: Cleaning up task 018af592-9339-741c-992c-8cce730b3528 and marking as failed. Reason: Task process died unexpectedly with exitcode 1.
2023-10-04T04:05:52.436712601+02:00 stderr F pulp [0fc145d73e984678852e2508b4cd32d3]: pulpcore.tasking.tasks:INFO: Starting task 018af86f-34f1-7c27-8ebb-77fd384e0852
2023-10-04T04:05:52.451443468+02:00 stderr F pulp [0fc145d73e984678852e2508b4cd32d3]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLES12-SP5-Pool remote=remote_upstream-SLES12-SP5-Pool

Task 2

pulp task show --uuid 018af947-8d72-779c-9fc6-7276114e745e

{
  "pulp_href": "/pulp/api/v3/tasks/018af947-8d72-779c-9fc6-7276114e745e/",
  "pulp_created": "2023-10-04T08:02:10.712482+02:00",
  "state": "failed",
  "name": "pulpcore.app.tasks.repository.repair_all_artifacts",
  "logging_cid": "d18b434379844fc9bcab7b5cc38c8784",
  "created_by": "/pulp/api/v3/users/1/",
  "started_at": "2023-10-04T08:02:11.065654+02:00",
  "finished_at": "2023-10-04T15:16:05.857742+02:00",
  "error": {
    "reason": "Task process died unexpectedly with exitcode 1."
  },
  "worker": "/pulp/api/v3/workers/018af58f-c9f9-79df-86db-ad6a062e296d/",
  "parent_task": null,
  "child_tasks": [],
  "task_group": null,
  "progress_reports": [
    {
      "message": "Identify missing units",
      "code": "repair.missing",
      "state": "completed",
      "total": null,
      "done": 0,
      "suffix": null
    },
    {
      "message": "Identify corrupted units",
      "code": "repair.corrupted",
      "state": "completed",
      "total": null,
      "done": 0,
      "suffix": null
    },
    {
      "message": "Repair corrupted units",
      "code": "repair.repaired",
      "state": "completed",
      "total": null,
      "done": 0,
      "suffix": null
    }
  ],
  "created_resources": [],
  "reserved_resources_record": [
    "/api/v3/repair/",
    "shared:/pulp/api/v3/domains/01897284-ff96-78ee-8ff8-9dcf388d08eb/"
  ]
}


2023-10-04T07:58:16.779250845+02:00 stderr F pulp [d91ed6554c004e29b464c9fde5f54451]: pulpcore.tasking.tasks:INFO: Task completed 018af8e9-9518-76d5-95a5-2d48aa9950bc
2023-10-04T07:58:16.923536821+02:00 stderr F pulp [9cc10983e95b427282de716b82f7c33d]: pulpcore.tasking.tasks:INFO: Starting task 018af8e9-9c3e-7c80-bb1e-fa898290f0d6
2023-10-04T07:58:19.526500590+02:00 stderr F pulp [9cc10983e95b427282de716b82f7c33d]: pulpcore.tasking.tasks:INFO: Task completed 018af8e9-9c3e-7c80-bb1e-fa898290f0d6
2023-10-04T07:58:19.708167927+02:00 stderr F pulp [640ae4ef2fce457baf1b00463db23b5c]: pulpcore.tasking.tasks:INFO: Starting task 018af8e9-a30d-77a7-a3f2-075051ca2784
2023-10-04T08:01:41.081997570+02:00 stderr F pulp [640ae4ef2fce457baf1b00463db23b5c]: pulpcore.tasking.tasks:INFO: Task completed 018af8e9-a30d-77a7-a3f2-075051ca2784
2023-10-04T08:02:11.072955901+02:00 stderr F pulp [d18b434379844fc9bcab7b5cc38c8784]: pulpcore.tasking.tasks:INFO: Starting task 018af947-8d72-779c-9fc6-7276114e745e
2023-10-04T08:02:11.073307087+02:00 stderr F pulp [d18b434379844fc9bcab7b5cc38c8784]: pulpcore.app.tasks.repository:INFO: Repairing artifacts.'
2023-10-04T15:16:05.806204684+02:00 stderr F Process Process-88:
2023-10-04T15:16:05.806204684+02:00 stderr F Traceback (most recent call last):
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
2023-10-04T15:16:05.806204684+02:00 stderr F     return self.cursor.execute(sql, params)
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/psycopg/cursor.py", line 737, in execute
2023-10-04T15:16:05.806204684+02:00 stderr F     raise ex.with_traceback(None)
2023-10-04T15:16:05.806204684+02:00 stderr F psycopg.OperationalError: consuming input failed: EOF detected
2023-10-04T15:16:05.806204684+02:00 stderr F 
2023-10-04T15:16:05.806204684+02:00 stderr F The above exception was the direct cause of the following exception:
2023-10-04T15:16:05.806204684+02:00 stderr F 
2023-10-04T15:16:05.806204684+02:00 stderr F Traceback (most recent call last):
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/lib64/python3.8/multiprocessing/process.py", line 315, in _bootstrap
2023-10-04T15:16:05.806204684+02:00 stderr F     self.run()
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/lib64/python3.8/multiprocessing/process.py", line 108, in run
2023-10-04T15:16:05.806204684+02:00 stderr F     self._target(*self._args, **self._kwargs)
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/pulpcore/tasking/worker.py", line 496, in _perform_task
2023-10-04T15:16:05.806204684+02:00 stderr F     execute_task(task)
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/pulpcore/tasking/tasks.py", line 48, in execute_task
2023-10-04T15:16:05.806204684+02:00 stderr F     contextvars.copy_context().run(_execute_task, task)
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/pulpcore/tasking/tasks.py", line 76, in _execute_task
2023-10-04T15:16:05.806204684+02:00 stderr F     task.set_completed()
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/pulpcore/app/models/task.py", line 175, in set_completed
2023-10-04T15:16:05.806204684+02:00 stderr F     rows = Task.objects.filter(pk=self.pk, state=TASK_STATES.RUNNING).update(
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 1206, in update
2023-10-04T15:16:05.806204684+02:00 stderr F     rows = query.get_compiler(self.db).execute_sql(CURSOR)
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1984, in execute_sql
2023-10-04T15:16:05.806204684+02:00 stderr F     cursor = super().execute_sql(result_type)
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
2023-10-04T15:16:05.806204684+02:00 stderr F     cursor.execute(sql, params)
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute
2023-10-04T15:16:05.806204684+02:00 stderr F     return self._execute_with_wrappers(
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
2023-10-04T15:16:05.806204684+02:00 stderr F     return executor(sql, params, many, context)
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
2023-10-04T15:16:05.806204684+02:00 stderr F     return self.cursor.execute(sql, params)
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 91, in __exit__
2023-10-04T15:16:05.806204684+02:00 stderr F     raise dj_exc_value.with_traceback(traceback) from exc_value
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 89, in _execute
2023-10-04T15:16:05.806204684+02:00 stderr F     return self.cursor.execute(sql, params)
2023-10-04T15:16:05.806204684+02:00 stderr F   File "/usr/local/lib/python3.8/site-packages/psycopg/cursor.py", line 737, in execute
2023-10-04T15:16:05.806204684+02:00 stderr F     raise ex.with_traceback(None)
2023-10-04T15:16:05.806204684+02:00 stderr F django.db.utils.OperationalError: consuming input failed: EOF detected
2023-10-04T15:16:05.831377744+02:00 stderr F pulp [None]: pulpcore.tasking.worker:WARNING: Task process for 018af947-8d72-779c-9fc6-7276114e745e exited with non zero exitcode 1.
2023-10-04T15:16:05.844915163+02:00 stderr F pulp [None]: pulpcore.tasking.worker:INFO: Cleaning up task 018af947-8d72-779c-9fc6-7276114e745e and marking as failed. Reason: Task process died unexpectedly with exitcode 1.

Thank you.

#2

Looking at the traceback it seems that the task ran perfectly, but then failed to be set as complete, task.set_completed(). Most likely the database query timed-out resulting in the error. Can you try other tasks to see if they are also experiencing the same problem, i.e. syncing, publishing, creating a distribution.

#3

Thank you for looking at this.

We currently create four repos to a latest available rpm. (bundle)
This updates from SLES smt repos daily and creates over a hundred Pulp repos creating publications and distributions.
From that we create the four latest repo rpm versions with their publications and distributions.
Using API copy. (saving space and time)

This all works without a problem.
The only error we see is the repair not completing.

Added below is one of the current two pulp-minimal:worker container image’s output. (shortened)
But still long.

Thank you.

2023-10-03T14:42:40.443697904+02:00 stderr F + /usr/bin/wait_on_postgres.py
2023-10-03T14:42:44.095392192+02:00 stdout F Waiting on postgresql to start...
2023-10-03T14:42:44.095392192+02:00 stdout F Postgres started.
2023-10-03T14:42:44.224082272+02:00 stderr F + /usr/bin/wait_on_database_migrations.sh
2023-10-03T14:42:44.226470454+02:00 stdout F Checking for database migrations
2023-10-03T14:42:52.719382053+02:00 stdout F Database migrated!
2023-10-03T14:42:52.719841872+02:00 stderr F + export PATH=/usr/local/bin:/usr/bin/
2023-10-03T14:42:52.719841872+02:00 stderr F + PATH=/usr/local/bin:/usr/bin/
2023-10-03T14:42:52.719883138+02:00 stderr F + exec pulpcore-worker
2023-10-03T14:42:59.122614784+02:00 stderr F pulp [None]: pulpcore.tasking.entrypoint:INFO: Starting distributed type worker
2023-10-03T14:42:59.202880413+02:00 stderr F pulp [None]: pulpcore.tasking.worker:INFO: New worker '1@c1420756b4d8' discovered
2023-10-04T04:05:56.558849322+02:00 stderr F pulp [36bff05ec2bc46d19e4c47c5b9cc22ef]: pulpcore.tasking.tasks:INFO: Starting task 018af86f-451a-7985-b2cf-746f19a74725
2023-10-04T04:05:56.573828019+02:00 stderr F pulp [36bff05ec2bc46d19e4c47c5b9cc22ef]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLES12-SP5-Updates remote=remote_upstream-SLES12-SP5-Updates
2023-10-04T04:11:17.267662706+02:00 stderr F pulp [None]: pulpcore.tasking.worker:WARNING: Task process for 018af86f-451a-7985-b2cf-746f19a74725 exited with non zero exitcode -9.
2023-10-04T04:11:17.275350994+02:00 stderr F pulp [None]: pulpcore.tasking.worker:INFO: Cleaning up task 018af86f-451a-7985-b2cf-746f19a74725 and marking as failed. Reason: Killed by signal 9.
2023-10-04T04:11:27.228070306+02:00 stderr F pulp [082d356d361f445d8dd9cbcc69277cb9]: pulpcore.tasking.tasks:INFO: Starting task 018af874-50cd-74ba-9e84-8834d2fbf90b
2023-10-04T04:11:27.242731646+02:00 stderr F pulp [082d356d361f445d8dd9cbcc69277cb9]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLE-HA12-SP5-Updates remote=remote_upstream-SLE-HA12-SP5-Updates
2023-10-04T04:11:27.327202676+02:00 stderr F pulp [082d356d361f445d8dd9cbcc69277cb9]: pulpcore.tasking.tasks:INFO: Task completed 018af874-50cd-74ba-9e84-8834d2fbf90b
2023-10-04T04:16:55.061561750+02:00 stderr F pulp [a55470cd5683417fa678af529563c2ec]: pulpcore.tasking.tasks:INFO: Starting task 018af879-5165-79d4-84da-e8198cbef279
2023-10-04T04:16:55.076356515+02:00 stderr F pulp [a55470cd5683417fa678af529563c2ec]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLE-12-SP5-SAP-Updates remote=remote_upstream-SLE-12-SP5-SAP-Updates
2023-10-04T04:16:55.154628973+02:00 stderr F pulp [a55470cd5683417fa678af529563c2ec]: pulpcore.tasking.tasks:INFO: Task completed 018af879-5165-79d4-84da-e8198cbef279
2023-10-04T04:17:03.428054044+02:00 stderr F pulp [5b73d94121384030a9584815f7da868b]: pulpcore.tasking.tasks:INFO: Starting task 018af879-71fb-7c93-b6a6-16f311917270
2023-10-04T04:17:03.443220176+02:00 stderr F pulp [5b73d94121384030a9584815f7da868b]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLE-Module-Containers12-Updates remote=remote_upstream-SLE-Module-Containers12-Updates

repository=upstream-SUSE-PackageHub-15-SP4-Pool remote=remote_upstream-SUSE-PackageHub-15-SP4-Pool
2023-10-04T05:22:44.112997668+02:00 stderr F pulp [cddfd15769cb4e12a1de93672e6815f1]: pulpcore.tasking.tasks:INFO: Task completed 018af8b5-92f3-7896-8715-dd4b5179c9b6
2023-10-04T05:22:48.356276884+02:00 stderr F pulp [0b4d2654e723433c874ff6240667e247]: pulpcore.tasking.tasks:INFO: Starting task 018af8b5-a37f-7075-b3d2-36da79f2480b
2023-10-04T05:22:48.371663937+02:00 stderr F pulp [0b4d2654e723433c874ff6240667e247]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SUSE-PackageHub-15-SP4-Backports-Pool remote=remote_upstream-SUSE-PackageHub-15-SP4-Backports-Pool
2023-10-04T05:22:48.453633365+02:00 stderr F pulp [0b4d2654e723433c874ff6240667e247]: pulpcore.tasking.tasks:INFO: Task completed 018af8b5-a37f-7075-b3d2-36da79f2480b
2023-10-04T05:24:09.545984870+02:00 stderr F pulp [9d881ec178ad4368859dbef695c3aa12]: pulpcore.tasking.tasks:INFO: Starting task 018af8b6-e104-71bd-8b9c-2581762ae32e
2023-10-04T05:24:09.567598530+02:00 stderr F pulp [9d881ec178ad4368859dbef695c3aa12]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=upstream-SLE-Module-Desktop-Applications15-SP4-Updates, version=11
2023-10-04T05:24:18.107636071+02:00 stderr F pulp [9d881ec178ad4368859dbef695c3aa12]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018af8b6-e1e0-7581-bcf0-b4a3e0f04bbf created
2
2023-10-04T05:25:15.220766586+02:00 stderr F pulp [dc98041482954b2ca673e4ba280c3e0a]: pulpcore.tasking.tasks:INFO: Task completed 018af8b7-d070-75a5-b675-d283eb56a456
2023-10-04T05:25:29.236175534+02:00 stderr F pulp [cc5a22415f804ba1a137dd233f82d4de]: pulpcore.tasking.tasks:INFO: Starting task 018af8b8-1850-7997-8251-17697ede5105
2023-10-04T05:25:29.259602364+02:00 stderr F pulp [cc5a22415f804ba1a137dd233f82d4de]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=upstream-SLE-Module-Server-Applications15-SP4-Updates, version=6
2023-10-04T05:25:39.382643415+02:00 stderr F pulp [cc5a22415f804ba1a137dd233f82d4de]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018af8b8-192c-7cd2-b7d7-88096c507474 created

2023-10-04T05:26:04.438928175+02:00 stderr F pulp [7e854d6b3f9d4a50a0eadc58e39956ab]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018af8b8-8ada-70e5-ae48-ff2249fb44be created
2023-10-04T05:26:04.456235212+02:00 stderr F pulp [7e854d6b3f9d4a50a0eadc58e39956ab]: pulpcore.tasking.tasks:INFO: Task completed 018af8b8-89b7-7332-af69-313e19c4a420
2023-10-04T05:33:18.725687371+02:00 stderr F pulp [4ba6e590fbab4280b547a7263b47cf30]: pulpcore.tasking.tasks:INFO: Starting task 018af8bf-4257-7c3e-b114-54b3784ab852
2023-10-04T05:33:18.746658611+02:00 stderr F pulp [4ba6e590fbab4280b547a7263b47cf30]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=active-P-2023Q04.00-GRP-SLES15-SP4-latest, version=933
2023-10-04T05:37:04.046090731+02:00 stderr F pulp [4ba6e590fbab4280b547a7263b47cf30]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018af8bf-431b-7fc8-8501-4c5e1e3ab52d created

2023-10-04T06:12:19.535843406+02:00 stderr F pulp [835b2293ad884bc3b2281f2d7bc58d79]: pulpcore.tasking.tasks:INFO: Task completed 018af8e2-ee54-7fb3-b655-b1a3656ebc89
2023-10-04T06:12:31.883012765+02:00 stderr F pulp [91a6c12728e245c6a3d090c72a398a48]: pulpcore.tasking.tasks:INFO: Starting task 018af8e3-2a61-7381-a1a1-58f0eede2525
2023-10-04T06:12:31.897730990+02:00 stderr F pulp [91a6c12728e245c6a3d090c72a398a48]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLE-Module-Server-Applications15-SP5-Pool remote=remote_upstream-SLE-Module-Server-Applications15-SP5-Pool
2023-10-04T06:12:31.976022669+02:00 stderr F pulp [91a6c12728e245c6a3d090c72a398a48]: pulpcore.tasking.tasks:INFO: Task completed 018af8e3-2a61-7381-a1a1-58f0eede2525
2023-10-04T06:12:44.416127291+02:00 stderr F pulp [a8f581ebee8b4931b1e476afdc2d5721]: pulpcore.tasking.tasks:INFO: Starting task 018af8e3-5b47-76b9-8c83-ef667561c6f8

2023-10-04T06:13:13.753016020+02:00 stderr F pulp [fe38a641f7454a8995c3bdaac52bf3e5]: pulpcore.tasking.tasks:INFO: Task completed 018af8e3-cd8a-742a-b956-d5606ec73568
2023-10-04T06:13:22.228058255+02:00 stderr F pulp [dc6f7c1d1dc04cc6bc84c1b9ae0135c5]: pulpcore.tasking.tasks:INFO: Starting task 018af8e3-eec0-703d-8f41-8bf9fff45324
2023-10-04T06:13:22.242749112+02:00 stderr F pulp [dc6f7c1d1dc04cc6bc84c1b9ae0135c5]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SUSE-PackageHub-15-SP5-Pool remote=remote_upstream-SUSE-PackageHub-15-SP5-Pool
2023-10-04T06:13:22.320407812+02:00 stderr F pulp [dc6f7c1d1dc04cc6bc84c1b9ae0135c5]: pulpcore.tasking.tasks:INFO: Task completed 018af8e3-eec0-703d-8f41-8bf9fff45324
2023-10-04T06:14:04.498684254+02:00 stderr F pulp [47763d7f0b034988888769c112bc8af0]: pulpcore.tasking.tasks:INFO: Starting task 018af8e4-940f-7e16-9c60-0a08764e963b
2023-10-04T06:14:04.520734548+02:00 stderr F pulp [47763d7f0b034988888769c112bc8af0]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=upstream-SLE-Module-Containers15-SP5-Updates, version=5
2023-10-04T06:14:09.748006530+02:00 stderr F pulp [47763d7f0b034988888769c112bc8af0]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018af8e4-94e9-7a56-9ee0-86341ba1ac80 created
2023-10-04T06:14:09.767333566+02:00 stderr F pulp [47763d7f0b034988888769c112bc8af0]: pulpcore.tasking.tasks:INFO: Task completed 018af8e4-940f-7e16-9c60-0a08764e963b
2023-10-04T06:14:41.048680702+02:00 stderr F pulp [1776e063e1af4898b619997099c41cca]: pulpcore.tasking.tasks:INFO: Starting task 018af8e5-22f8-71fe-ae7d-3ed7cc46bf3a
2023-10-04T06:14:41.067292197+02:00 stderr F pulp [1776e063e1af4898b619997099c41cca]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=upstream-SLE-Module-Packagehub-Subpackages15-SP5-Updates, version=11
2023-10-04T06:14:48.533541905+02:00 stderr F pulp [1776e063e1af4898b619997099c41cca]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018af8e5-23ab-7009-b479-e8857588ac96 created
2023-10-04T06:14:48.550170621+02:00 stderr F pulp [1776e063e1af4898b619997099c41cca]: pulpcore.tasking.tasks:INFO: Task completed 018af8e5-22f8-71fe-ae7d-3ed7cc46bf3a
2023-10-04T07:04:46.585007960+02:00 stderr F pulp [f8bf725de64343db8ab56c952dc874fa]: pulpcore.tasking.tasks:INFO: Task completed 018af8e9-26e2-7a31-87a3-f2861f3aa11f


2023-10-04T07:07:00.053521141+02:00 stderr F pulp [830200ee150b443795b90867488a021f]: pulpcore.tasking.tasks:INFO: Task completed 018af915-0777-723a-bd14-6bfab4316e44
2023-10-04T07:07:04.118854583+02:00 stderr F pulp [1042b2cac84a4238adcb66110e50e153]: pulpcore.tasking.tasks:INFO: Starting task 018af915-17cc-77fe-82fb-d39e8eef2fda
2023-10-04T07:07:04.133525509+02:00 stderr F pulp [1042b2cac84a4238adcb66110e50e153]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLE-Module-Web-Scripting12-Updates remote=remote_upstream-SLE-Module-Web-Scripting12-Updates
2023-10-04T07:07:04.214172324+02:00 stderr F pulp [1042b2cac84a4238adcb66110e50e153]: pulpcore.tasking.tasks:INFO: Task completed 018af915-17cc-77fe-82fb-d39e8eef2fda
2023-10-04T07:07:50.722822468+02:00 stderr F pulp [268cecd5db5944dc8763a272d94e4110]: pulpcore.tasking.tasks:INFO: Starting task 018af915-cdd6-79f0-9ce8-255e5e2060cc
2023-10-04T07:07:50.744622706+02:00 stderr F pulp [268cecd5db5944dc8763a272d94e4110]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=upstream-SLE-Module-Web-Scripting12-Updates, version=2
2023-10-04T07:08:16.581013845+02:00 stderr F pulp [268cecd5db5944dc8763a272d94e4110]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018af915-cf59-7d6b-ae83-94060d8e87d1 created


---------- TRUNCATED: Click "tail" to view the bottom of the log ----------


2023-10-05T04:26:45.643426979+02:00 stderr F pulp [3a06521f3f1c449da7de7b7257e6e9a6]: pulpcore.tasking.tasks:INFO: Task completed 018afda8-afed-7f9e-bb6f-f698c0df22f2
2023-10-05T04:26:49.789530243+02:00 stderr F pulp [fd997040178446798b1c553baf6c3303]: pulpcore.tasking.tasks:INFO: Starting task 018afda8-c086-78ef-868b-b5ed91b2d7fc
2023-10-05T04:26:49.805328042+02:00 stderr F pulp [fd997040178446798b1c553baf6c3303]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLE-Module-Legacy12-Updates remote=remote_upstream-SLE-Module-Legacy12-Updates
2023-10-05T04:26:49.887917577+02:00 stderr F pulp [fd997040178446798b1c553baf6c3303]: pulpcore.tasking.tasks:INFO: Task completed 018afda8-c086-78ef-868b-b5ed91b2d7fc
2023-10-05T04:30:20.630897482+02:00 stderr F pulp [ebf6eb5662864a33bf87753f79d42599]: pulpcore.tasking.tasks:INFO: Starting task 018afdab-f825-70e8-8cce-17f2d7cff9ab
2023-10-05T04:30:20.651084147+02:00 stderr F pulp [ebf6eb5662864a33bf87753f79d42599]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=upstream-SLE-SDK12-SP5-Updates, version=12
2023-10-05T04:30:54.279876415+02:00 stderr F pulp [ebf6eb5662864a33bf87753f79d42599]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018afdab-f8eb-77c6-a4fd-4b50cab094c4 created
2023-10-05T04:30:54.300969989+02:00 stderr F pulp [ebf6eb5662864a33bf87753f79d42599]: pulpcore.tasking.tasks:INFO: Task completed 018afdab-f825-70e8-8cce-17f2d7cff9ab
22023-10-05T05:08:07.785824186+02:00 stderr F pulp [459cdb65677041a683d6fbdb312ec0b7]: pulpcore.tasking.tasks:INFO: Starting task 018afdce-9044-78d7-8a64-b3da1b261d90
2023-10-05T05:08:07.800388699+02:00 stderr F pulp [459cdb65677041a683d6fbdb312ec0b7]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLE-Module-Basesystem15-SP4-Pool remote=remote_upstream-SLE-Module-Basesystem15-SP4-Pool


2023-10-05T05:22:57.907211197+02:00 stderr F pulp [f692e1f1a0864d57a9e1a588ab345e23]: pulpcore.tasking.tasks:INFO: Task completed 018afddc-24db-78e3-bb56-3e9da65aec5f
2023-10-05T05:23:06.264606856+02:00 stderr F pulp [3ea12b75dc244ba8a580fa2f0d29b1fd]: pulpcore.tasking.tasks:INFO: Starting task 018afddc-45f1-7a63-9a65-f5e5cd1a6a62
2023-10-05T05:23:06.279640026+02:00 stderr F pulp [3ea12b75dc244ba8a580fa2f0d29b1fd]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SUSE-PackageHub-15-SP4-Backports-Pool remote=remote_upstream-SUSE-PackageHub-15-SP4-Backports-Pool
2023-10-05T05:23:06.353838575+02:00 stderr F pulp [3ea12b75dc244ba8a580fa2f0d29b1fd]: pulpcore.tasking.tasks:INFO: Task completed 018afddc-45f1-7a63-9a65-f5e5cd1a6a62
2023-10-05T05:23:16.562393341+02:00 stderr F pulp [0b1471f3d4f24e329f7987e60f55a27d]: pulpcore.tasking.tasks:INFO: Starting task 018afddc-6daa-7eab-b034-889e9a4755fa
2023-10-05T05:23:16.583228667+02:00 stderr F pulp [0b1471f3d4f24e329f7987e60f55a27d]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=upstream-SLE-Module-Basesystem15-SP4-Updates, version=16
2023-10-05T05:24:04.313728086+02:00 stderr F pulp [0b1471f3d4f24e329f7987e60f55a27d]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018afddc-6ee7-7fe0-9f2d-c03b68a5dcd9 created

2023-10-05T05:25:17.628843906+02:00 stderr F pulp [fed8223a328b4c9a91d3485924fb84d6]: pulpcore.tasking.tasks:INFO: Task completed 018afdde-2da0-79f8-8fa3-2f3c9608b3c1
2023-10-05T05:26:16.924152362+02:00 stderr F pulp [fda7ab2f7304462193b87a65936207d4]: pulpcore.tasking.tasks:INFO: Starting task 018afddf-2eb7-7e95-b381-f0cc3e1e2b7d
2023-10-05T05:26:16.946401952+02:00 stderr F pulp [fda7ab2f7304462193b87a65936207d4]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=upstream-SLE-Product-HA15-SP4-Updates, version=7
2023-10-05T05:26:23.261958613+02:00 stderr F pulp [fda7ab2f7304462193b87a65936207d4]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018afddf-2f73-7c3f-81fd-16045c3f9b5a created
2023-10-05T05:26:23.281874533+02:00 stderr F pulp [fda7ab2f7304462193b87a65936207d4]: pulpcore.tasking.tasks:INFO: Task completed 018afddf-2eb7-7e95-b381-f0cc3e1e2b7d
2023-10-05T06:08:09.603206566+02:00 stderr F pulp [c69b11e188034244b9a42303497d0c4e]: pulpcore.tasking.tasks:INFO: Starting task 018afe05-85ce-7b13-af37-cd4a75eb515f
2023-10-05T06:08:09.618870209+02:00 stderr F pulp [c69b11e188034244b9a42303497d0c4e]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLE-Module-Basesystem15-SP5-Updates remote=remote_upstream-SLE-Module-Basesystem15-SP5-Updates

2023-10-05T06:14:25.689982572+02:00 stderr F pulp [72a357c14a8148119969a04ab5c3e6e8]: pulpcore.tasking.tasks:INFO: Task completed 018afe0b-4220-7bc2-b40b-8274463aeb4f
2023-10-05T06:14:29.725074797+02:00 stderr F pulp [61f487bb1db74c0bb32ae4ab4d1c5111]: pulpcore.tasking.tasks:INFO: Starting task 018afe0b-5243-7ff9-9b63-191d55e90a96
2023-10-05T06:14:29.741028124+02:00 stderr F pulp [61f487bb1db74c0bb32ae4ab4d1c5111]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLE-Product-SLES_SAP15-SP5-Updates remote=remote_upstream-SLE-Product-SLES_SAP15-SP5-Updates
2023-10-05T06:14:29.823514457+02:00 stderr F pulp [61f487bb1db74c0bb32ae4ab4d1c5111]: pulpcore.tasking.tasks:INFO: Task completed 018afe0b-5243-7ff9-9b63-191d55e90a96
2023-10-05T06:14:47.989669047+02:00 stderr F pulp [d032db647704474398055f7771e61753]: pulpcore.tasking.tasks:INFO: Starting task 018afe0b-99ec-70f4-a3c4-f89e4217cb45
2023-10-05T06:14:48.014048930+02:00 stderr F pulp [d032db647704474398055f7771e61753]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=upstream-SLE-Module-Basesystem15-SP5-Updates, version=18
2023-10-05T06:15:07.345913056+02:00 stderr F pulp [d032db647704474398055f7771e61753]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018afe0b-9ace-759d-ba0b-d42104604baa created
2023-10-05T06:15:07.369292550+02:00 stderr F pulp [d032db647704474398055f7771e61753]: pulpcore.tasking.tasks:INFO: Task completed 018afe0b-99ec-70f4-a3c4-f89e4217cb45

2023-10-05T06:16:01.077534236+02:00 stderr F pulp [6d7e46e30dff4d0fb45a95ee7c3910cd]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018afe0c-a13e-7676-adfc-8c568144fbc6 created
2023-10-05T06:16:01.099093852+02:00 stderr F pulp [6d7e46e30dff4d0fb45a95ee7c3910cd]: pulpcore.tasking.tasks:INFO: Task completed 018afe0c-a076-7cbe-8859-a0ab104a8ade
2023-10-05T06:16:08.460746088+02:00 stderr F pulp [f23dffbe3fb24b7899160b83547e36bd]: pulpcore.tasking.tasks:INFO: Starting task 018afe0c-d3fe-7e6e-9f20-29e81456e51f
2023-10-05T06:16:08.484157962+02:00 stderr F pulp [f23dffbe3fb24b7899160b83547e36bd]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=upstream-SLE-Module-Packagehub-Subpackages15-SP5-Updates, version=12
2023-10-05T06:16:16.210539983+02:00 stderr F pulp [f23dffbe3fb24b7899160b83547e36bd]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018afe0c-d524-7145-be38-66bcc1563490 created
2023-10-05T06:16:16.232128678+02:00 stderr F pulp [f23dffbe3fb24b7899160b83547e36bd]: pulpcore.tasking.tasks:INFO: Task completed 018afe0c-d3fe-7e6e-9f20-29e81456e51f
2023-10-05T06:16:55.265228668+02:00 stderr F pulp [7f9911601ab246cdae57c1aa7227d39c]: pulpcore.tasking.tasks:INFO: Starting task 018afe0d-8b38-7783-b681-7a982519e09e
2023-10-05T06:16:55.299935718+02:00 stderr F pulp [7f9911601ab246cdae57c1aa7227d39c]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=upstream-SLE-Product-HA15-SP5-Updates, version=6
2023-10-05T06:16:59.898450101+02:00 stderr F pulp [7f9911601ab246cdae57c1aa7227d39c]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018afe0d-8c04-7253-8789-bc97edf89950 created
2023-10-05T06:16:59.916468169+02:00 stderr F pulp [7f9911601ab246cdae57c1aa7227d39c]: pulpcore.tasking.tasks:INFO: Task completed 018afe0d-8b38-7783-b681-7a982519e09e
2023-10-05T07:07:07.750301166+02:00 stderr F pulp [930797ef8b3f4a78af0d7aba07a1018c]: pulpcore.tasking.tasks:INFO: Starting task 018afe3b-81ff-72c2-a85d-4ed33a83d3a1
2023-10-05T07:07:07.765309931+02:00 stderr F pulp [930797ef8b3f4a78af0d7aba07a1018c]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLE-Module-Web-Scripting12-Pool remote=remote_upstream-SLE-Module-Web-Scripting12-Pool
2023-10-05T07:07:07.847451590+02:00 stderr F pulp [930797ef8b3f4a78af0d7aba07a1018c]: pulpcore.tasking.tasks:INFO: Task completed 018afe3b-81ff-72c2-a85d-4ed33a83d3a1
2023-10-05T07:07:12.014034510+02:00 stderr F pulp [5f5b0d6e541b4d00ae6efcf6c1b2f092]: pulpcore.tasking.tasks:INFO: Starting task 018afe3b-92b4-7f9e-8e08-58a7c30f2121
2023-10-05T07:07:12.030957109+02:00 stderr F pulp [5f5b0d6e541b4d00ae6efcf6c1b2f092]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=upstream-SLE-Module-Web-Scripting12-Updates remote=remote_upstream-SLE-Module-Web-Scripting12-Updates
2023-10-05T07:07:12.113591547+02:00 stderr F pulp [5f5b0d6e541b4d00ae6efcf6c1b2f092]: pulpcore.tasking.tasks:INFO: Task completed 018afe3b-92b4-7f9e-8e08-58a7c30f2121
2023-10-05T07:13:10.275864801+02:00 stderr F pulp [bf748a4c88c5436aa226822d5378c6de]: pulpcore.tasking.tasks:INFO: Starting task 018afe3e-1d72-7593-8f1b-989e80e4c8e1
2023-10-05T07:13:35.476818385+02:00 stderr F pulp [bf748a4c88c5436aa226822d5378c6de]: pulpcore.tasking.tasks:INFO: Task completed 018afe3e-1d72-7593-8f1b-989e80e4c8e1
2023-10-05T07:13:42.844722900+02:00 stderr F pulp [bf4e74cf7a7d44e19ede58bdff7c9a0e]: pulpcore.tasking.tasks:INFO: Starting task 018afe41-88d0-7ff4-b831-89555aeb169a
2023-10-05T07:13:42.867087361+02:00 stderr F pulp [bf4e74cf7a7d44e19ede58bdff7c9a0e]: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=active-P-2023Q04.00-GRP-SLES12-SP4-latest, version=714
2023-10-05T07:17:01.063535325+02:00 stderr F pulp [bf4e74cf7a7d44e19ede58bdff7c9a0e]: pulp_rpm.app.tasks.publishing:INFO: Publication: 018afe41-8ad3-771e-9a85-9a6e89576047 created
2023-10-05T07:17:01.088756271+02:00 stderr F pulp [bf4e74cf7a7d44e19ede58bdff7c9a0e]: pulpcore.tasking.tasks:INFO: Task completed 018afe41-88d0-7ff4-b831-89555aeb169a
#4

I noticed that the repair task reports you posted both took 5+ hours to complete. Do you have any other tasks that have run for as long as that? I’m wondering if something is happening to the worker after such a long runtime.

How often do you want to run the repair artifacts task? Looking at the reports, both times the task didn’t find any corrupt/missing artifacts. If you want to run it often I would suggest running the task on the repository version, e.g. http POST :/pulp/api/v3/repositories/rpm/rpm/{repo_pk}/versions/{version}/repair/. This way it will run the repair task on a smaller subset of artifacts and should be faster to complete. Also, running it on the repository version should give us more clues as to where the problem is.

1 Like
#5

Hello gerrod

Sorry for the delayed answer.

Ran this : /pulp/api/v3/repositories/rpm/rpm/018a5aa4-7696-7276-b137-581dc537f5d3/versions/10/repair/

Took 1.5 hours on this one, successful.

There are ± 105 of them.

I’ll see how to iterate through them all.

Thanks for the support.