ERROR: duplicate key value violates unique constraint

Support
#1

Hello

Our pulp server has been running for one year. Recently, I see the following error in logs:

Aug 17 18:13:48.314 xx.xx.xx dockerd[4016612]: 2025-08-17 16:13:48.314 UTC [1599671] ERROR:  duplicate key value violates unique constraint "core_artifact_sha512_pulp_domain_id_5e5f2eb0_uniq"
Aug 17 18:13:48.314 xx.xx.xx dockerd[4016612]: 2025-08-17 16:13:48.314 UTC [1599671] DETAIL:  Key (sha512, pulp_domain_id)=(6b8812c75141760124324499c12f8ec7155d9132de075fa488076085ef6453cd294e4a27c6f4718d8ec9a492c7f57307d40c85e745551b8e34d4b30fc608c74d, 019077ef-4160-79b1-be2f-dd6954576070) already exists.
Aug 17 18:13:48.314 xx.xx.xx dockerd[4016612]: 2025-08-17 16:13:48.314 UTC [1599671] STATEMENT:  INSERT INTO "core_artifact" ("pulp_id", "pulp_created", "pulp_last_updated", "file", "size", "md5", "sha1", "sha224", "sha256", "sha384", "sha512", "timestamp_of_interest", "pulp_domain_id") VALUES ('0198b8cef798710395dff745f6202dda'::uuid, '2025-08-17 16:13:48.313114+00:00'::timestamptz, '2025-08-17 16:13:48.313131+00:00'::timestamptz, 'artifact/f1/cfba619390b75081ce335678f03d5231c9ce5860af5f5d5968272b020450ce', 1248116, NULL, '18264785fca92ff07f8a2677ec8404edfbe0ace7', '12f79c0665815ea22294b0dbd660cec6ef345acf87cf0235581c0ab8', 'f1cfba619390b75081ce335678f03d5231c9ce5860af5f5d5968272b020450ce', '75df8de15801e7ce49f9189b49206f7bcba03c4430f44693e3614e19d2639e38c9aca775247ed5b6ca87687b15f13566', '6b8812c75141760124324499c12f8ec7155d9132de075fa488076085ef6453cd294e4a27c6f4718d8ec9a492c7f57307d40c85e745551b8e34d4b30fc608c74d', '2025-08-17 16:13:48.313788+00:00'::timestamptz, '019077ef416079b1be2fdd6954576070'::uuid)

Is it something critical? what should I do?

Our pulp instance status are:

Summary

{
“versions”: [
{
“component”: “core”,
“version”: “3.80.1”,
“package”: “pulpcore”,
“module”: “pulpcore.app”,
“domain_compatible”: true
},
{
“component”: “ansible”,
“version”: “0.26.0”,
“package”: “pulp-ansible”,
“module”: “pulp_ansible.app”,
“domain_compatible”: false
},
{
“component”: “container”,
“version”: “2.25.1”,
“package”: “pulp-container”,
“module”: “pulp_container.app”,
“domain_compatible”: true
},
{
“component”: “deb”,
“version”: “3.5.2”,
“package”: “pulp_deb”,
“module”: “pulp_deb.app”,
“domain_compatible”: false
},
{
“component”: “maven”,
“version”: “0.10.1”,
“package”: “pulp-maven”,
“module”: “pulp_maven.app”,
“domain_compatible”: true
},
{
“component”: “ostree”,
“version”: “2.4.8”,
“package”: “pulp-ostree”,
“module”: “pulp_ostree.app”,
“domain_compatible”: true
},
{
“component”: “python”,
“version”: “3.16.0”,
“package”: “pulp-python”,
“module”: “pulp_python.app”,
“domain_compatible”: true
},
{
“component”: “rpm”,
“version”: “3.30.1”,
“package”: “pulp-rpm”,
“module”: “pulp_rpm.app”,
“domain_compatible”: true
},
{
“component”: “certguard”,
“version”: “3.80.1”,
“package”: “pulpcore”,
“module”: “pulp_certguard.app”,
“domain_compatible”: true
},
{
“component”: “file”,
“version”: “3.80.1”,
“package”: “pulpcore”,
“module”: “pulp_file.app”,
“domain_compatible”: true
}
],
“online_workers”: [
{
“pulp_href”: “/pulp/api/v3/workers/0198cd10-ea13-714c-8fd5-8f6cf06f085d/”,
“prn”: “prn:core.worker:0198cd10-ea13-714c-8fd5-8f6cf06f085d”,
“pulp_created”: “2025-08-21T14:38:14.548350Z”,
“pulp_last_updated”: “2025-08-21T14:38:14.548372Z”,
“name”: “1@2741f204b08b”,
“last_heartbeat”: “2025-08-21T15:28:14.996943Z”,
“versions”: {
“deb”: “3.5.2”,
“rpm”: “3.30.1”,
“core”: “3.80.1”,
“file”: “3.80.1”,
“maven”: “0.10.1”,
“ostree”: “2.4.8”,
“python”: “3.16.0”,
“ansible”: “0.26.0”,
“certguard”: “3.80.1”,
“container”: “2.25.1”
},
“current_task”: null
},
{
“pulp_href”: “/pulp/api/v3/workers/0198cd10-eab8-7d23-9a48-2feda753bc5e/”,
“prn”: “prn:core.worker:0198cd10-eab8-7d23-9a48-2feda753bc5e”,
“pulp_created”: “2025-08-21T14:38:14.713798Z”,
“pulp_last_updated”: “2025-08-21T14:38:14.713820Z”,
“name”: “1@ffa2e40b355b”,
“last_heartbeat”: “2025-08-21T15:28:14.994007Z”,
“versions”: {
“deb”: “3.5.2”,
“rpm”: “3.30.1”,
“core”: “3.80.1”,
“file”: “3.80.1”,
“maven”: “0.10.1”,
“ostree”: “2.4.8”,
“python”: “3.16.0”,
“ansible”: “0.26.0”,
“certguard”: “3.80.1”,
“container”: “2.25.1”
},
“current_task”: null
}
],
“online_api_apps”: [
{
“name”: “14@pulp-api”,
“last_heartbeat”: “2025-08-21T15:28:15.107847Z”,
“versions”: {
“deb”: “3.5.2”,
“rpm”: “3.30.1”,
“core”: “3.80.1”,
“file”: “3.80.1”,
“maven”: “0.10.1”,
“ostree”: “2.4.8”,
“python”: “3.16.0”,
“ansible”: “0.26.0”,
“certguard”: “3.80.1”,
“container”: “2.25.1”
}
},
{
“name”: “15@pulp-api”,
“last_heartbeat”: “2025-08-21T15:28:15.105643Z”,
“versions”: {
“deb”: “3.5.2”,
“rpm”: “3.30.1”,
“core”: “3.80.1”,
“file”: “3.80.1”,
“maven”: “0.10.1”,
“ostree”: “2.4.8”,
“python”: “3.16.0”,
“ansible”: “0.26.0”,
“certguard”: “3.80.1”,
“container”: “2.25.1”
}
}
],
“online_content_apps”: [
{
“name”: “16@pulp-content”,
“last_heartbeat”: “2025-08-21T15:28:12.884913Z”,
“versions”: {
“deb”: “3.5.2”,
“rpm”: “3.30.1”,
“core”: “3.80.1”,
“file”: “3.80.1”,
“maven”: “0.10.1”,
“ostree”: “2.4.8”,
“python”: “3.16.0”,
“ansible”: “0.26.0”,
“certguard”: “3.80.1”,
“container”: “2.25.1”
}
},
{
“name”: “18@pulp-content”,
“last_heartbeat”: “2025-08-21T15:28:12.966624Z”,
“versions”: {
“deb”: “3.5.2”,
“rpm”: “3.30.1”,
“core”: “3.80.1”,
“file”: “3.80.1”,
“maven”: “0.10.1”,
“ostree”: “2.4.8”,
“python”: “3.16.0”,
“ansible”: “0.26.0”,
“certguard”: “3.80.1”,
“container”: “2.25.1”
}
}
],
“database_connection”: {
“connected”: true
},
“redis_connection”: {
“connected”: false
},
“storage”: {
“total”: 4036463951872,
“used”: 3295962320896,
“free”: 740501630976
},
“content_settings”: {
“content_origin”: “http://pulp_content:24816”,
“content_path_prefix”: “/pulp/content/”
},
“domain_enabled”: false
}

Thanks in advance for help

#2

Hard to tell without more info. Also, I think the error message in your title vs the logs you shared are different. One is about Artifacts and the other is about the ContentAppStatus (the content worker section of the pulp status output). Typically if there is a duplicate in artifacts on upload or sync we will try to reuse the existing artifact. Are you seeing the duplicate error message when performing a sync/upload/download? As for a duplicate ContentAppStatus error it would most likely happen when Pulp loses track of a content worker, it misses too many heartbeats, but then it comes back online and tries to resave with its old name. As long as the content woker isn’t getting stuck in a boot reloop this error shouldn’t be critical. Maybe check your logs to see the frequency of workers going missing, might be indicative of straining resources, e.g. not enough memory for the workers or too much load on the DB.

#3

My bad for the title.
The tile should be like the log messages.
ERROR: duplicate key value violates unique constraint “core_artifact_sha512_pulp_domain_
id_5e5f2eb0_uniq”.
we do sync once a day, these messages happens during the sync everytime since three days.
How to check content worker stucked or not? or how to see the frequence of worker going missing?

#4

Another question is. today we try to test another image of pulp, and we got the error:

2025-08-21 13:19:13.252 UTC [94] ERROR: duplicate key value violates unique constraint “core_contentappstatus_name_key”
pulp [None]: pulpcore.content:ERROR: A content app with name 16@pulp-content already exists in the database.
2025-08-21 13:19:13.357 UTC [96] ERROR: duplicate key value violates unique constraint “core_contentappstatus_name_key”
pulp [None]: pulpcore.content:ERROR: A content app with name 18@pulp-content already exists in the database.
[2025-08-21 13:19:13 +0000] [1] [ERROR] Worker (pid:16) exited with code 3
[2025-08-21 13:19:13 +0000] [1] [ERROR] Worker (pid:18) was sent SIGTERM!
[2025-08-21 13:19:13 +0000] [1] [ERROR] Shutting down: Master
[2025-08-21 13:19:13 +0000] [1] [ERROR] Reason: Worker failed to boot.
2025-08-21 13:19:18.173 UTC [116] ERROR: duplicate key value violates unique constraint “core_apiappstatus_name_key”
pulp [None]: pulpcore.app.entrypoint:ERROR: An API app with name 14@pulp-api already exists in the database.
2025-08-21 13:19:18.188 UTC [117] ERROR: duplicate key value violates unique constraint “core_apiappstatus_name_key”
pulp [None]: pulpcore.app.entrypoint:ERROR: An API app with name 15@pulp-api already exists in the database.
[2025-08-21 13:19:18 +0000] [1] [ERROR] Worker (pid:14) exited with code 3
[2025-08-21 13:19:18 +0000] [1] [ERROR] Worker (pid:15) was sent SIGTERM!
[2025-08-21 13:19:18 +0000] [1] [ERROR] Shutting down: Master
[2025-08-21 13:19:18 +0000] [1] [ERROR] Reason: Worker failed to boot.
2025-08-21 13:19:21.396 UTC [123] ERROR: duplicate key value violates unique constraint “core_contentappstatus_name_key”
pulp [None]: pulpcore.content:ERROR: A content app with name 16@pulp-content already exists in the database.
2025-08-21 13:19:21.433 UTC [124] ERROR: duplicate key value violates unique constraint “core_contentappstatus_name_key”
pulp [None]: pulpcore.content:ERROR: A content app with name 18@pulp-content already exists in the database.
[2025-08-21 13:19:21 +0000] [1] [ERROR] Worker (pid:16) exited with code 3
[2025-08-21 13:19:21 +0000] [1] [ERROR] Worker (pid:18) was sent SIGTERM!
[2025-08-21 13:19:22 +0000] [1] [ERROR] Shutting down: Master
[2025-08-21 13:19:22 +0000] [1] [ERROR] Reason: Worker failed to boot.

It is due to this error, I notice the previous error of ERROR: duplicate key value violates unique constraint "core_artifact_sha512_xxxx.

Maybe the new image failed because of the core artifact sha512 xxx
duplicate problem?
If not, what should I trouble shooting these errors?

#5

We identified this issue of the crashing working: https://github.com/pulp/pulpcore/issues/6880 and should have a fix for it soon.

2 Likes
#6

Hello @gerrod , thanks for the news.

I realized that what I am reporting, are two different problems. They are not related to each other.

First problem: the ERROR: duplicate key value violates unique constraint “core_artifact_sha512_pulp_domain_id_5e5f2eb0_uniq”., in fact, has been there long time ago, several months ago.

Second problem, is when we wanted to update pulp and its plugins, it is core_apiappstatus_name_key” duplicated, message like: ERROR: An API app with name 14@pulp-api , et ERROR: A content app with name 16@pulp-content .

For the first problem, no sure what to to. I have not found it is due to which repo or rpm package. Suppose it doesn’t impact pulp service ?

For the 2nd problem, thanks for the issue fixing. I also read that, these can be recreated, so we can delete the duplicated in DB, eg:

DELETE FROM core_contentappstatus WHERE name LIKE '%@pulp-content';
DELETE FROM core_apiappstatus WHERE name LIKE '%@pulp-api';

Is it true? Is it safe to do? I have not tested yet.