How to add sha1 checksum in k8s deployment

nicokruger · March 11, 2025, 12:13pm

Hi guys,

Problem:
I am trying to add elasticsearch repo and i am getting an error that the allowed checksum (sha1) is not included. I thought i could add it in the instance yaml like this

    pulp_settings:
       ...
      allowed_content_checksums:
        - sha1
        - md5
        - sha224
        - sha256
        - sha384
        - sha512

but then pods dont start… any way i can include sha1 ?

Expected outcome:

Pulpcore version:

Pulp plugins installed and their versions:

Operating system - distribution and version:

Other relevant data:
sha1 error on worker:
pulp [27678ef7cec4471988819ca3b05cec9d]: pulpcore.tasking.tasks:INFO: Task[pulp_rpm.app.tasks.synchronizing.synchronize] 019584d0-36ab-70d3-b533-f82999ff4324 failed (UnsupportedDigestValidationError: Content at the URL ‘https://artifacts.elastic.co/packages/8.x/yum/repodata/222b3ae8783a48bfd6520d6b53671fa6fca1a442-other.xml.gz’ does not contain at least one trusted hasher which is specified in the ‘ALLOWED_CONTENT_CHECKSUMS’ setting ([‘sha512’, ‘sha384’, ‘sha256’, ‘sha224’]). The downloader expected one of the following hashers: {‘sha1’}) in domain: default

error when trying to enable via instance yaml:
Waiting on postgresql to start…
Checking postgres host pulp-mpe-database-svc
Checking postgres port 5432
Postgres started!
Checking for database migrations
Traceback (most recent call last):
File “/usr/local/bin/pulpcore-manager”, line 8, in
sys.exit(manage())
File “/usr/local/lib/python3.9/site-packages/pulpcore/app/manage.py”, line 11, in manage
execute_from_command_line(sys.argv)
File “/usr/local/lib/python3.9/site-packages/django/core/management/init.py”, line 442, in execute_from_command_line
utility.execute()
File “/usr/local/lib/python3.9/site-packages/django/core/management/init.py”, line 416, in execute
django.setup()
File “/usr/local/lib/python3.9/site-packages/django/init.py”, line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File “/usr/local/lib/python3.9/site-packages/django/apps/registry.py”, line 116, in populate
app_config.import_models()
File “/usr/local/lib/python3.9/site-packages/django/apps/config.py”, line 269, in import_models
self.models_module = import_module(models_module_name)
File “/usr/lib64/python3.9/importlib/init.py”, line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 1030, in _gcd_import
File “”, line 1007, in _find_and_load
File “”, line 986, in _find_and_load_unlocked
File “”, line 680, in _load_unlocked
File “”, line 850, in exec_module
File “”, line 228, in _call_with_frames_removed
File “/usr/local/lib/python3.9/site-packages/pulpcore/app/models/init.py”, line 34, in
from .exporter import (
File “/usr/local/lib/python3.9/site-packages/pulpcore/app/models/exporter.py”, line 11, in
from pulpcore.app.models.repository import Repository
File “/usr/local/lib/python3.9/site-packages/pulpcore/app/models/repository.py”, line 33, in
from pulpcore.cache import Cache
File “/usr/local/lib/python3.9/site-packages/pulpcore/cache/init.py”, line 1, in
from .cache import (
File “/usr/local/lib/python3.9/site-packages/pulpcore/cache/cache.py”, line 17, in
from pulpcore.app.settings import settings
File “/usr/local/lib/python3.9/site-packages/pulpcore/app/settings.py”, line 618, in
raise e
File “/usr/local/lib/python3.9/site-packages/pulpcore/app/settings.py”, line 577, in
raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: There have been identified artifacts missing checksum ‘sha1’. Run ‘pulpcore-manager handle-artifact-checksums’ first to populate missing artifact checksums.
Database migrated!
Traceback (most recent call last):
File “/usr/local/bin/pulpcore-worker”, line 5, in
from pulpcore.tasking.entrypoint import worker
File “/usr/local/lib/python3.9/site-packages/pulpcore/tasking/entrypoint.py”, line 9, in
django.setup()
File “/usr/local/lib/python3.9/site-packages/django/init.py”, line 19, in setup
configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File “/usr/local/lib/python3.9/site-packages/django/conf/init.py”, line 102, in getattr
self._setup(name)
File “/usr/local/lib/python3.9/site-packages/django/conf/init.py”, line 89, in _setup
self._wrapped = Settings(settings_module)
File “/usr/local/lib/python3.9/site-packages/django/conf/init.py”, line 217, in init
mod = importlib.import_module(self.SETTINGS_MODULE)
File “/usr/lib64/python3.9/importlib/init.py”, line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “/usr/local/lib/python3.9/site-packages/pulpcore/app/settings.py”, line 618, in
raise e
File “/usr/local/lib/python3.9/site-packages/pulpcore/app/settings.py”, line 577, in
raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: There have been identified artifacts missing checksum ‘sha1’. Run ‘pulpcore-manager handle-artifact-checksums’ first to populate missing artifact checksums.

nicokruger · March 11, 2025, 12:15pm

this is the default instance i am using thats working…

apiVersion: repo-manager.pulpproject.org/v1beta2
kind: Pulp
metadata:
  name: pulp-mpe
spec:
  # if this is commented out then the instance will use emptydir temp storage IE get wiped after restart
  # but still good for testing without any pre-configured storage
  pvc: pulp-pvc-core
  database:
    pvc: pulp-pvc-db
  cache:
    pvc: pulp-pvc-cache
  # file_storage_storage_class: local-storage
  # file_storage_size: 20G
  # file_storage_access_mode: ReadWriteOnce
  # database:
  #   postgres_storage_class: local-storage
  ipv6_disabled: true
  pulp_settings:
    content_origin: "https://<%= @config['pulp']['host'] %>"
    ansible_api_hostname: "https://<%= @config['pulp']['host'] %>"
    token_server: "https://<%= @config['pulp']['host'] %>/token/"
    api_root: "/pulp/"
    allowed_export_paths:
      - /tmp
    allowed_import_paths:
      - /tmp
  api:
    strategy:
      type: Recreate
    replicas: 2
    resource_requirements:
      requests:
        cpu: "0.5"
        memory: 256Mi
      limits:
        cpu: "1"
        memory: 1Gi
  cache:
    strategy:
      type: Recreate
    enabled: true
  content:
    strategy:
      type: Recreate
    replicas: 2
    resource_requirements:
      requests:
        cpu: 150m
        memory: 256Mi
      limits:
        cpu: "1"
        memory: 1Gi
  worker:
    replicas: 2
    resource_requirements:
      requests:
        cpu: "1"
        memory: 2Gi
      limits:
        cpu: "4"
        memory: 10Gi

hyagi · March 12, 2025, 11:44am

Hi @nicokruger,

Some pulp plugins do not support (and/or recommend using) sha1, so we tried to avoid its usage when installing Pulp through pulp-operator.
From the stack trace provided, it seems like there are already some stored artifacts, but they are missing the sha1:

 django.core.exceptions.ImproperlyConfigured: There have been identified artifacts missing checksum ‘sha1’. Run ‘pulpcore-manager handle-artifact-checksums’ first to populate missing artifact checksums.

We can try to fix this error by following the steps from How can I manually run a database migration?, but instead of running the pulpcore-manager migrate command, we would need to run pulpcore-manager handle-artifact-checksums, for example:

$ kubectl exec deployments/pulp-api pulpcore-manager handle-artifact-checksums

note: this is a step that the operator would normally run automatically by creating a k8s Job, but maybe because of a bug it didn’t happen.

nicokruger · March 20, 2025, 12:36pm

Seems its a bug

I see the issue was logged already

github.com/pulp/pulp-operator

Can't enable sha224 with sha1

opened 02:12PM - 15 Jan 25 UTC

apampurin

Issue Triage-Needed

**Version** pulp-operator:v1.0.0-beta.4 pulp-operator:v1.0.0-beta.5 **Descr…ibe the bug** I'm creating rpm repositiries for rhel9 and rhel7. I successfully uploaded rhel9 repos. Then, I tried to upload rhel7 repo which is required to add sha1 shecksum in pulp CR which I've done as follow: ```yaml spec: allowed_content_checksums: - sha1 ``` After that I found that some already uploaded artifacts with sha224 were removing, so I stopped pulp-rpm-update-content-checksums pod. I added sha224 and other to allowed_content_checksums like: ```yaml spec: allowed_content_checksums: - sha224 - sha256 - sha512 - sha1 - sha384 ``` But handle-artifact-checksums job was not started. I've got error in pulp-operator-controller-manager logs: ``` 2025-01-15T13:39:44Z INFO controller/controller.go:217 Starting workers {"controller": "pulprestore", "controllerGroup": "repo-manager.pulpproject.org", "controllerKind": "PulpRestore", "worker count": 1} 2025-01-15T13:39:44Z WARN repo_manager/precheck.go:276 Checksum sha1 is deprecated by some Pulp plugins, it is not recommended using it in production. 2025-01-15T13:39:44Z ERROR repo_manager/precheck.go:271 Checksum sha224 is not valid! github.com/pulp/pulp-operator/controllers/repo_manager.checkAllowedContentChecksums /workspace/controllers/repo_manager/precheck.go:271 github.com/pulp/pulp-operator/controllers/repo_manager.prechecks /workspace/controllers/repo_manager/precheck.go:80 github.com/pulp/pulp-operator/controllers/repo_manager.(*RepoManagerReconciler).Reconcile /workspace/controllers/repo_manager/controller.go:100 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.3/pkg/internal/controller/controller.go:116 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.3/pkg/internal/controller/controller.go:303 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.3/pkg/internal/controller/controller.go:263 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.3/pkg/internal/controller/controller.go:224 ``` I can return sha224 with removing allowed_content_checksums from pulp CR, and artifacts will be restored. But how I can activate sha1 and sha224? I see related conversation in https://github.com/pulp/pulp-operator/issues/1060, but I can't found answer to question above.