CI/CD: Removing unecessary "bindings" runner

Development
#1

Several years ago, when Pulp3 first started shipping bindings, we didn’t use them in the tests directly. Instead to “test the bindings” we made a “bindings” runner with each repo containing a test script for the python bindings and one for the ruby bindings. You can see those for pulpcore for example here and here.

Now the functional tests themselves are expected to directly test by using the Python bindings themselves, so the “bindings” runner is redundant at this point. I propose we delete this to simplify the CI maintenance and reduce the minute runtime of the CI.

The Ruby bindings would no longer be tested, but arguably no one is writing ruby into these test files anyway. Additionally any problems we have between Python and Ruby bindings are an openapigenerator problem, not a Pulp problem, so we shouldn’t be focused on testing their project (my take).

1 Like
#2

we ensure the basic workflow works on Ruby, and we do have pulp problems with ruby bindings, bad serializer fields for example

#3

How can I learn more about these issues? I’m not clear on how something Pulp does can affect Ruby and not Python bindings.

#4

Interestingly I learned that even though this is a CI feature available to all plugins, the only place I can find it in use is with pulpcore. So really this is a pulpcore specific issue.

#5

I believe this one was the latest:

#6

Thanks for sharing that example; it looks like it was detected not by the CI though, but by Ian’s usage of Katello. To me, that example highlights that the check we have isn’t worth keeping because it wasn’t able to detect that problem anyway.

We really need the CI minutes back and the complexity of the plugin template to be reduced, so I’d still like to remove this runner entirely if that’s ok.

#7

I am in favor of removing the separate job that tests bindings.

The job was initially supposed to help us identify any unintentional changes to our OpenAPI schema. Such changes are reflected in the generated bindings. Now that we have a lot of tests that use the Python bindings, we are able to detect them using the functional tests from the other CI jobs. The script that we use to test the Ruby bindings covers a very small portion of the API and has not detected problems in a long time.

1 Like
#8

I agree, but I was just showing how pulp code can affect ruby and not python bindings.
And I believe this fall in the same case of upgrade tests, the ideal would have more tests, but as we won’t probably add these tests, it is better to remove

#9

These PRs are available here: