RFC: Reunite the library that drives Pulp CLI and Squeezer

:tangerine: Pulp CLI and :tropical_drink: Squeezer are client tools to Pulp that depend on talking efficiently to its REST API.
While using the same approach to parse and use the server provided API docs on the fly, they are on completely separate code bases. The reasons on the one hand are, the :tangerine: CLI is Python3 only, it depends on requests for http communication and uses py3 style type annotations. On the other hand, the :tropical_drink: Squeezer collection is built with dependencies limited to the Ansible distribution and required to run with multiple python interpreters including 2.7.
The desire to consolidate the backends arises from the realization that I am maintaining basically the same thing twice. Features like e.g. the version dependent workarounds would benefit directly.
However, merging these two code bases will definitely compromise some of these requirements:

  1. Squeezer can only be used with python >= 3.6 (on the target node)
  2. Squeezer will either depend on pulp-cli or a new factored out shared library (in both cases an external requirement)
  3. pulp-cli contexts need to be independent of click (not too hard and a good idea anyway)

Please tell us if any of these are a blocker for your specific use cases of Squeezer.

Maybe the first two of these points can be addressed by vendoring the “library parts” of Pulp CLI as module_utils in Squeezer, but that would be a rather intricate piece of hackery and the external dependency on requests would remain.

4 Likes

For us, python3.6+ is not a problem - python2 has been EOL for some time now.

The lack of external dependencies in squeezer is nice, and makes it easy to pick up and use. Knowing exactly where to install python packages such that they can be used by ansible modules is non-trivial (I am fighting with this problem in github CI as we speak), and depends on things like the remote interpreter, connection plugin, etc. Still, it’s a common requirement for other modules, so while it might present a barrier to entry, it’s not a full blocker.

3 Likes

Sounds great! Python3.6+ is fine for us.

The AWX project has a very similar architecture and tech stack, maybe it’s a good place for inspiration…

I just had a quick look at the code, and for me, it looks like they use some common codebase (awxkit python package)

2 Likes

Personally I am all for a common library for squeezer and pulp-cli. I take a pretty dim view of this Ansible world idea that the correct way to solve real world complexities is to define them out of existence (“We just won’t have any external dependencies in Ansible, and we will support every python interpreter for ever!”).

1 Like

It’s been a month, and i received overall positive feedback to the proposed change. So i’ll take it that we should go forward with the effort. If anyone has real concerns with these changes, please speak up now or forever hold your peace.

We need a name for the library to emerge. Please post your suggestions. I’ll start:

  • pulpglue
  • pulpkit
  • pulpkitt
  • pulp-openapi-abstraction
1 Like

Some thoughts in my brain. It may be obvious that I tend to concise/terse names :slight_smile:

  • pulp-oapi-lib
  • pulp-oapi-shim
  • pulp-oapi-base
  • pulpcomms

or the classic

  • pulp-oapi-utils

(No that last is not serious - ‘utils’ long ago stopped being meaningful…)

1 Like

I am not super attached to any one name, but I would suggest using word separators. Unfortunately Pulp already has repos using a mix of - and _, but I think pulpcore is currently the only one that does not have any word separation past the initial “pulp” prefix, and I think it should stay that way. :wink:

I think pulp-openapi-lib is very descriptive, while pulp-glue is still fairly descriptive and more fun.

P.S.: Since everyone is adding at least one non-serious suggestion, how about: pulp-peel

1 Like

To give some progress on this old (slow moving) topic, there is a new PR moving in hopefully the right direction:

1 Like

In case anyone is interested, this is my current PoC/design PR:

1 Like

I just started using the CLI as a library in pulp-replica. I chose to use the CLI instead of the bindings because I want to make sure that pulp-replica can communicate with older/newer versions of Pulp. It would be great if I didn’t have to install all of the CLI (which includes click) to be able to use the relevant parts of it.

2 Likes

With pulp-replica on the horizon, we are going to act on this soon. We realized that we need to keep the current plugin architecture to support custom pulp plugins in client tooling.
The current plan is to keep the number of git repositories as is, and start shipping two python packages from it (1. a pulp-glue component, 2. the pulp-cli parts). This way, we can still make use of the current test coverage.

3 Likes

Yet another PR in the process.

2 Likes

The split is done. A release is on the way. Now the time for reuniting has begun.

2 Likes

The first version (0.0.14) of squeezer (partly) based on pulp_glue is released.

https://galaxy.ansible.com/ui/repo/published/pulp/squeezer/

1 Like