Is dependency solving supposed to be deterministic?

I am a Satellite/katello user who identified that pulp-3 dependency solving does not work in a deterministic way. As I am aware of some limitations of libsolv, I am asking: is that nondeterminism expected behaviour or not?

More technically: having a system with a given repo synced and copying a given set of packages to another repo, should dependency solving add the same set of packages, regardless of what system I execute it?

Particular example (as minimal as I was able to prepare, so bit artificial):

  • have a RHEL8 x86_64 BaseOS repository
  • create a new repo and copy there - with dependency solving enabled:
    • RPMs without an errata
    • errata (and their content) issued no newer than 2020-01-01
  • that use case corresponds to katello’s Content View with filters “add RPMs without errata” and “add errata issued no newer than 2020-01-01” and with dep.solving enabled
  • the copy itself copies 3394 packages
  • then, dependency solver adds some more packages

And here the problem comes: on two identical pulp servers with identical synced repo content, I get different outcome. E.g. on one pulp server, I always get 3571 packages, repeatedly - but on another server, I get (repeatedly) 3577 packages.

First and main question: is that nondeterminism expected or allowed?

Second, I see some subtle weirdness in libsolv reasoning when inspecting /var/tmp/pulp/*/depsolving_summary.txt - aren’t these bugs alone?

In particular, one of extra package added by the dep.solving on just one pulp server was python3-hawkey-0:0.55.0-7.el8.x86_64 explained via:

    Pulp Content unit 'e5ee4d30-aa2b-4458-87e8-a889fae9e72a' from repo 'Red_Hat_Enterprise_Linux_8_for_x86_64_-_BaseOS_RPMs_8-1144: version=1'
    Reason: SOLVER_REASON_RESOLVE - The package was installed to fulfill package dependencies.
        SOLVER_RULE_PKG_REQUIRES - Similar to SOLVER_RULE_PKG_NOTHING_PROVIDES_DEP, but in this case some packages provided the dependency but none of them could be installed due to other dependency issues.
            Because package 'kpatch-dnf-0:0.4-3.el8.noarch' requires 'python3-hawkey'

kpatch-dnf was not in the copied packages, it was also added by the libsolv. BUT this package was added on both pulp servers - so why one server triggered "oh, I additionally need python3-hawkey" step but not the other pulp server?

Further, when checking why kpatch-dnf was added by libsolv, I got confusing reasoning from one pulp server:

    Pulp Content unit '80b88ce4-79c9-41f4-abbd-2365dfdbddcf' from repo 'Red_Hat_Enterprise_Linux_8_for_x86_64_-_BaseOS_RPMs_8-15544: version=1'
    Reason: SOLVER_REASON_UNIT_RULE - The package was installed/erased/kept because of a unit rule, i.e. a rule where all literals but one were false.
        SOLVER_RULE_JOB - Job rules implement the job given to the solver.
            Because package 'ledmon-0:0.96-2.el8.x86_64' requires 'iucode_rev(fname:intel/06-55-04;cpuid:00050654;pf_mask:0xb7;segment:"Server";codename:"Skylake";stepping:"U0";pf_model:0xb7)'

But ledmon-0.96-2.el8.x86_64 does not require that iucode_rev capability, and kpatch-dnf package does not provide that capability - so why the reasoning…? (I see no reason of a dependency between those two packages at all)

Expected outcome:

  1. Should be dependency solving deterministic (not only on one pulpserver, but also “server-wide”)?
  2. Why python3-hawkey was added on one pulp server but not on the other?
  3. Why the incorrect reasoning of the kpatch-dnf <-> ledmon dependency?

Pulpcore version:

Pulp plugins installed and their versions:
a typical Satellite 6.12 deployment, but the problem is generic (observed on older versions also)

Operating system - distribution and version:

Other relevant data:
I can provide anything upon request, like:

  • list of packages added without dep.solving
  • list of packages added by depsolving
  • depsolving_summary.txt
  • or even access to either reproducer system

Just any data is too big to copy&paste here.

1 Like

Dependency solving is supposed to be deterministic. In fact, if it wasn’t, then dnf --debugdata would be completely useless because it would be impossible to generate libsolv testcases for bug reports.

1 Like

Right… Thanks for the answer. As I was more thinking on the topic, esp. on the misleading/wrong reasoning “package was added due to this dependency of that package”, I feel the problem is rather on pulp side communicating with libsolv. Anyway, I will raise a bug for it for some proper / more deep investigation.