Worker stucks at running state

Support
#1

Hi everyone, i have been deploying Pulp using containers in ECS, but when i scale the workers to sync the channels. Sometimes, the task of a worker keeps running forever.

Problem:
Tasks stuck at running forever
Expected outcome:
All subtasks are in failed state.

Pulpcore version:
3.27.0

Pulp plugins installed and their versions:
pulp-container 2.15.0
pulp-rpm 3.21.0

Operating system: distribution and version:
Amazon Linux 2

Other relevant data:
This is the logs of a worker container i got from Docker logs. As we can see, the task was running, but suddenly, it stopped and only sent the heartbeat from then on.

05e811b9-2410-48ec-a77b-c851bc22f8ed
05e811b9-2410-48ec-a77b-c851bc22f8ed2040×820 296 KB

#2

This mighth be a symptom of Pulp sync gets stuck forever after specific connection issues · Issue #5439 · pulp/pulpcore · GitHub. @dralley , what do you think?

If it is this issue - we’re looking into the problem but don’t have a solution yet.

On a side-note - core/3.27 last had a release a year ago, and isn’t a branch we backport to. core/3.28 and core/3.39 are much better choices, assuming you can’t “just use latest”.

#3

Thanks for your reply. That could be it. Because when i sync the channel alone, it could sync successfully.