Pulp compose signing_key_service" didn't complete successfully

xm1234567 · January 5, 2024, 3:12pm

Hello
Problem:
I tired docker compose up as describe here https://github.com/pulp/pulp-oci-images/blob/latest/images/compose/README.md

I see:

compose-signing_key_service-1 | /usr/bin/add_signing_service.sh: line 6: PULP_SIGNING_KEY_FINGERPRINT: unbound variable
service “signing_key_service” didn’t complete successfully: exit 1

detail here:

Summary

[xxxx@xxxx compose]$ docker compose up
[+] Running 13/13
Network compose_default Created 0.1s
Container compose-redis-1 Created 0.1s
Container compose-postgres-1 Created 0.1s
Container compose-set_init_password_service-1 Created 0.3s
Container compose-migration_service-1 Created 0.3s
Container compose-pulp_worker-2 Created 0.2s
Container compose-pulp_worker-1 Created 0.3s
Container compose-signing_key_service-1 Created 0.3s
Container compose-pulp_content-2 Created 0.2s
Container compose-pulp_content-1 Created 0.3s
Container compose-pulp_api-2 Created 0.2s
Container compose-pulp_api-1 Created 0.2s
Container compose-pulp_web-1 Created 0.1s
Attaching to compose-migration_service-1, compose-postgres-1, compose-pulp_api-1, compose-pulp_api-2, compose-pulp_content-1, compose-pulp_content-2, compose-pulp_web-1, compose-pulp_worker-1, compose-pulp_worker-2, compose-redis-1, compose-set_init_password_service-1, compose-signing_key_service-1
compose-redis-1 | 1:C 05 Jan 2024 15:05:07.398 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see vm.max_map_count growing steadily when vm.overcommit_memory is 2 · Issue #1328 · jemalloc/jemalloc · GitHub. To fix this issue add ‘vm.overcommit_memory = 1’ to /etc/sysctl.conf and then reboot or run the command ‘sysctl vm.overcommit_memory=1’ for this to take effect.
compose-redis-1 | 1:C 05 Jan 2024 15:05:07.398 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
compose-redis-1 | 1:C 05 Jan 2024 15:05:07.398 * Redis version=7.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
compose-redis-1 | 1:C 05 Jan 2024 15:05:07.398 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
compose-redis-1 | 1:M 05 Jan 2024 15:05:07.399 * monotonic clock: POSIX clock_gettime
compose-redis-1 | 1:M 05 Jan 2024 15:05:07.400 * Running mode=standalone, port=6379.
compose-redis-1 | 1:M 05 Jan 2024 15:05:07.400 * Server initialized
compose-redis-1 | 1:M 05 Jan 2024 15:05:07.401 * Loading RDB produced by version 7.2.3
compose-redis-1 | 1:M 05 Jan 2024 15:05:07.401 * RDB age 11 seconds
compose-redis-1 | 1:M 05 Jan 2024 15:05:07.401 * RDB memory usage when created 0.85 Mb
compose-redis-1 | 1:M 05 Jan 2024 15:05:07.401 * Done loading RDB, keys loaded: 0, keys expired: 0.
compose-redis-1 | 1:M 05 Jan 2024 15:05:07.401 * DB loaded from disk: 0.000 seconds
compose-redis-1 | 1:M 05 Jan 2024 15:05:07.401 * Ready to accept connections tcp
compose-postgres-1 |
compose-postgres-1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
compose-postgres-1 |
compose-postgres-1 | 2024-01-05 15:05:07.477 UTC [1] LOG: starting PostgreSQL 13.13 (Debian 13.13-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
compose-postgres-1 | 2024-01-05 15:05:07.478 UTC [1] LOG: listening on IPv4 address “0.0.0.0”, port 5432
compose-postgres-1 | 2024-01-05 15:05:07.478 UTC [1] LOG: listening on IPv6 address “::”, port 5432
compose-postgres-1 | 2024-01-05 15:05:07.507 UTC [1] LOG: listening on Unix socket “/var/run/postgresql/.s.PGSQL.5432”
compose-postgres-1 | 2024-01-05 15:05:07.524 UTC [28] LOG: database system was shut down at 2024-01-05 15:04:56 UTC
compose-postgres-1 | 2024-01-05 15:05:07.543 UTC [1] LOG: database system is ready to accept connections
compose-set_init_password_service-1 | Waiting on postgresql to start…
compose-set_init_password_service-1 | Postgres started.
compose-set_init_password_service-1 | Checking for database migrations
compose-set_init_password_service-1 | Database migrated!
compose-migration_service-1 | Operations to perform:
compose-migration_service-1 | Apply all migrations: ansible, auth, certguard, container, contenttypes, core, deb, file, maven, ostree, python, rpm, sessions
compose-migration_service-1 | Running migrations:
compose-migration_service-1 | No migrations to apply.
compose-migration_service-1 exited with code 0
compose-pulp_content-1 | Waiting on postgresql to start…
compose-pulp_content-1 | Postgres started.
compose-signing_key_service-1 | Waiting on postgresql to start…
compose-signing_key_service-1 | Postgres started.
compose-pulp_content-1 | Checking for database migrations
compose-pulp_worker-1 | Waiting on postgresql to start…
compose-pulp_worker-1 | Postgres started.
compose-pulp_worker-1 | Checking for database migrations
compose-signing_key_service-1 | Checking for database migrations
compose-pulp_content-2 | Waiting on postgresql to start…
compose-pulp_content-2 | Postgres started.
compose-pulp_worker-2 | Waiting on postgresql to start…
compose-pulp_worker-2 | Postgres started.
compose-pulp_content-2 | Checking for database migrations
compose-set_init_password_service-1 exited with code 0
compose-pulp_worker-2 | Checking for database migrations
compose-pulp_content-1 | Database migrated!
compose-pulp_content-1 | /usr/local/bin/pulpcore-content
compose-pulp_worker-1 | Database migrated!
compose-signing_key_service-1 | Database migrated!
compose-signing_key_service-1 | /usr/bin/add_signing_service.sh: line 6: PULP_SIGNING_KEY_FINGERPRINT: unbound variable
compose-signing_key_service-1 exited with code 1
compose-pulp_worker-2 | Database migrated!
compose-pulp_content-2 | Database migrated!
compose-pulp_content-2 | /usr/local/bin/pulpcore-content
service “signing_key_service” didn’t complete successfully: exit 1

[xxx@xxx compose]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dc7256cf0b34 pulp/pulp-minimal:latest “pulp-content” 2 minutes ago Up About a minute (healthy) compose-pulp_content-1
f0adb25f45b8 pulp/pulp-minimal:latest “pulp-worker” 2 minutes ago Up About a minute compose-pulp_worker-1
758bbf7d5f2a pulp/pulp-minimal:latest “pulp-content” 2 minutes ago Up About a minute (healthy) compose-pulp_content-2
f420a6e02fe3 pulp/pulp-minimal:latest “pulp-worker” 2 minutes ago Up About a minute compose-pulp_worker-2
d558c0c54cc4 redis:latest “docker-entrypoint.s…” 2 minutes ago Up 2 minutes (healthy) 6379/tcp compose-redis-1
7dafd5149e3e postgres:13 “docker-entrypoint.s…” 2 minutes ago Up 2 minutes (healthy) 0.0.0.0:5432->5432/tcp, :::5432->5432/tcp compose-postgres-1

Then I tried with docker compose restart, and seems all ok now.

Summary

[xxx@xxx compose]$ docker compose restart
[+] Restarting 12/12
Container compose-pulp_content-2 Started 6.3s
Container compose-signing_key_service-1 Started 2.2s
Container compose-redis-1 Started 2.7s
Container compose-pulp_worker-2 Started 3.0s
Container compose-pulp_web-1 Started 1.1s
Container compose-pulp_api-1 Started 2.0s
Container compose-pulp_api-2 Started 1.9s
Container compose-pulp_content-1 Started 5.3s
Container compose-pulp_worker-1 Started 3.1s
Container compose-postgres-1 Started 2.8s
Container compose-migration_service-1 Started 0.8s
Container compose-set_init_password_service-1 Started 1.6s
[xxx@xxxx compose]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
286042e39259 pulp/pulp-web:latest “container-entrypoin…” 3 minutes ago Up 25 seconds 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 8443/tcp compose-pulp_web-1
a5cf667ad92a pulp/pulp-minimal:latest “pulp-api” 3 minutes ago Up 24 seconds (healthy) compose-pulp_api-2
40e3fede366e pulp/pulp-minimal:latest “pulp-api” 3 minutes ago Up 24 seconds (healthy) compose-pulp_api-1
dc7256cf0b34 pulp/pulp-minimal:latest “pulp-content” 3 minutes ago Up 21 seconds (healthy) compose-pulp_content-1
f0adb25f45b8 pulp/pulp-minimal:latest “pulp-worker” 3 minutes ago Up 23 seconds compose-pulp_worker-1
758bbf7d5f2a pulp/pulp-minimal:latest “pulp-content” 3 minutes ago Up 20 seconds (health: starting) compose-pulp_content-2
f420a6e02fe3 pulp/pulp-minimal:latest “pulp-worker” 3 minutes ago Up 23 seconds compose-pulp_worker-2
d558c0c54cc4 redis:latest “docker-entrypoint.s…” 3 minutes ago Up 24 seconds (healthy) 6379/tcp compose-redis-1
7dafd5149e3e postgres:13 “docker-entrypoint.s…” 3 minutes ago Up 24 seconds (healthy) 0.0.0.0:5432->5432/tcp, :::5432->5432/tcp compose-postgres-1

Is there something wrong? Or I can ignore this error?

rfguimaraes · January 9, 2024, 7:56am

I am having the exact same issue and from what I could gather, it is necessary to setup the signing keys (GPG keys). I still did not have time to try adapting the instructions here though: Metadata Signing — Pulp Project 3.43.1 documentation.

mikedep333 · January 9, 2024, 4:50pm

Hey, I apologize for the issues with our docs. I am writing up a navication issue right now.

You need to create the key 1st:

github.com

pulp/pulp-oci-images/blob/latest/docs/signing_script.md#creating-a-gpg-key


# Adding Signing Services


> :information_source: Content Signing is in tech-preview and may change in backwards incompatible ways in future releases.

It is possible to sign Pulp's metadata so that users can verify the authenticity of an object.  
This is done by enabling the *Signing Services* feature. The steps to enable it are:

* [create a gpg key](#creating-a-gpg-key)
* [create the signing script](#creating-the-collection-signing-script)
* [create the signing services](#creating-the-signing-services)


See pulpcore documentation for details on ***Content Signing***: https://docs.pulpproject.org/pulpcore/workflows/signed-metadata.html#metadata-signing  
See pulp_container documentation for details on ***Container Image Signing***: https://docs.pulpproject.org/pulp_container/workflows/sign-images.html


## Creating a gpg key

This file has been truncated. show original

xm1234567 · January 10, 2024, 1:12pm

Hello @rfguimaraes @mikedep333
Thanks for information.
In in add_signing_service.sh , it is for collection sign and container sign. I am using pulp_rpm mainly, so signing key service doesn’t bother much to my use case for the moment.

I still try to understand more about signing_key_service, the example in the doc shows to create the gpg key inside pulp container, which I suppose it is the pulp-all-in-one-container way. It means to do the signing service after the container created. Do I understand it correctly?

So another question is, how it is happening in compose way?
Is it something like:

podman-compose up -d,
then choose which container to do create gpg key?
then maye podman-compose restart signing_key_service?

xm1234567 · January 10, 2024, 1:48pm

@mikedep333
Do you have time to have a look of my another ticket, please?

In the pulp compose way, the web service is on port 8080:8080, is it possible to have https ?
I tried to use the nginx conf from pulp-all-in-one-container as the compose/assets/nginx/nginx.conf.template, and with self certifcate under compose/assets/certs, but not working.

Thanks in advance!

xm1234567 · January 11, 2024, 9:13pm

found how to do compose.yml + https finally, related info is updated here: How to use https in pulp compose way

xm1234567 · January 12, 2024, 10:01am

I understand better the signing service. Since I don’t need it for the moment, a workaround is to give PULP_SIGNING_KEY_FINGERPRINT an empty value:

  signing_key_service:
    image: "pulp/pulp-minimal:latest"
    environment:
      PULP_SIGNING_KEY_FINGERPRINT: ''