Problem with modules in distributed rpm repositories

Support
#1

Problem:
Hi Folks,
we have problems with rpm repos containing module information.
Since some time we are not able to sync “Redhat 8 appstream” repo anymore.

Or in detail:
If we try to do yum update on a sytem attached to the repos delivered from pulp3,
we see dependency problems because yum tries to install the latest perl version (V5.32) although the default module perl V5.26 is enabled.

the command “yum module list” does not produce errors. I see modules and their versions and status as if everything was fine…

If i attach to or older pulp2 repo server everything works fine also…
If i attach the system to redhat online repos everythings works as expected.

Does anyone have a idea whats wrong here?

Thanks for help
regards
Andi

Expected outcome:
yum should be able to honor the modules as designed

Pulpcore version:
“versions”: [
{
“component”: “core”,
“version”: “3.23.3”,
“package”: “pulpcore”,
“domain_compatible”: true
},
],
Pulp plugins installed and their versions:
{
“component”: “rpm”,
“version”: “3.20.0”,
“package”: “pulp-rpm”,
“domain_compatible”: false
}

Operating system - distribution and version:
RHEL 9

Other relevant data:
It seems the problem was introduced with pulp-rpm module 3.19.5
since my test pulp instance running pulp-rm 3.19.4 does not cause this issues…

And what i can see if i check the content of the repodata directory:
On newer pulp-rpm version the File -modules.yaml is not compressed anymore!
earlier version allways contain a file -modules.yaml.gz

I allready tried to update pulp-rpm since i saw in the changelog there were changes i the modulemd code.
But this did not help at all…

#2

Hi @Andi2301 - can you grab the current modules.yaml from your appstream repo and post it somewhere I can look at it?

Also - in the course of upgrading, did you ever see an error like the one described here : 2213515 – Upgrade to 6.14 fails on rpm.0048_artifacts_dependencies_fix migration with TypeError: 'NoneType' object is not subscriptable ?

#3

Hi,
thanks for your quick reply.

  1. during update i did not notice any errors. and i could not ifnd anything in the logs.
    But i have to admit i still use pulp_installer playbook for my updates, and there its hard to see any erros at all…

  2. unfortunately, i cannot provide the modules.yaml file anymore. Since i never saw this behaviour with pulp-rpm 3.19.4 which i used for some weeks now, i decided to downgrade the module again.
    So right now i’m back to
    pulp-core 3.23.3
    pulp-rpm 3.19.4

And since there i could not see this problems anymore…

But i’m a little confused. Do you know when exactly the files in repodata directory are generated?
From my understanding during creation of the repository publications.
And with downgrading/upgrading the pulp-rpm module these files should not be re-created.
Am i right with this ?

Thanks for your help and for clarification

Greetz
Andi

1 Like
#4

There’s too many things that could be going on here, I’m afraid. The ansible-installer isn’t supported past core/3.22, crossing y-boundaries can include data-migrations, we don’t (yet) guarantee that all migrations are reversible and idempotent (altho we’re working hard on that), and there have been bugs in the rpm/3.18 rpm/3.19 releases specifically with modularity migrations. (I just released a 3.19 yesterday with a fix to a modularity-related-migration, in fact)

If you could show me an example of modules.yaml.gz from your current working installation, I’ll try to pry up some time to start from where you are, import R8-appstrm, and then upgrade, and See What Happens

#5

Hi,
so bad to hear that pulp_installer is not supported anymore. Means i have to figure out whats the correct way to do updates now… i wish there was a rpm repo available with stable releases of all components.
this would make my life easier…

anyway. how could i provide the modules.yaml file ? unfortunately i dont have web or ftp server available for that…
And this board dont have upload function.
Any idea?

#6

RE RPMs - theforeman project builds RPMs to support their releases, checkout out (for example) Index of /pulpcore/3.22

You won’t get recent-changes unless/until foreman needs them: I believe core/3.28 and friends is being worked on now.

RE modules.yaml - unzip and throw the text into https://paste.centos.org/ , it’s just a large-ish YAML file - nothing in there is sensitive to your setup, but do skim the file just to be sure

#7

I will add this about these RPMs: Since they are mainly intended to be used within Katello installations, where they are installed via the foreman-installer (which uses Puppet for its backend) there is very little documentation on how to actually install and run them. Just running dnf install won’t run DB migrations, or create systemd unit files, or required configuration files.

So I’d argue the only easy way to use them is if you know enough Puppet to use the Puppet module also used by Foreman installer: https://github.com/theforeman/puppet-pulpcore

If anyone knows more about this than I do, please do tell!

1 Like
#8

We have run into a similar problem after upgrading from 3.17 to 3.22.

While running the migrate command we did see the NoneType object is not subscriptable error, but from 0045_modulemd_fields.py. Unfortunately we did not get this in our test environment and since we needed to get the prod environment up and running we hacked around this in the migration script. Everything seemed to work afterwards but now when publishing new repoversions we get bad module metadata.

The modules.yaml is missing data compared to what I see in pulp db ( rpm.modulemd), but all the modules in the db are missing profiles which I guess might be why they are not present in the generated metadata file.

Is there any way to fix the modulemd metadata?

#9

So I think I managed to solve my problem by applying the bugfix from https://github.com/pulp/pulp_rpm/issues/3241 and running the database migration.
After that I synced our rhel9-appstream-repo and then the modulemd-metadata looked a lot better (that is, no more empty profiles: dicts).

1 Like