Pulp BASE URL not accessible

[AJJA]

Problem:

Expected outcome:
Access to pulp distribution base_URL

Pulpcore version:
3.29.1

Operating system - distribution and version:
Pulp3 through Pulp Operator 1.0.0-alpha.8 ( latest version in the GitHub repo ) deployed in Kubernetes v1.25.6

Other relevant data:
Uploaded a file to a file repository. Created a publication according to the documentation. Also created a distribution. Publish and Host — Pulp File 1.14.3 documentation

I cannot access the base_url. I have deployed everything in Kubernetes. Exposed pulp-web-svc 24880 through port forwarding, still I cannot reach the base_url. Any suggestions ? I want to get through the content of the file repository.

[hyagi]

Hi @AJJA

From the screenshot provided it seems like you are trying to access your Service from a “remote location” (host outside of your k8s cluster).
First, double-check if the Endpoints are defined:

kubectl -nsystem get ep pulp-web-svc

you should see the list of pulp-web pods, for example:

$ kubectl get ep pulp-web-svc
NAME                   ENDPOINTS           AGE
pulp-web-svc           10.123.0.123:8080   108s

If you can see the pulp-web pod(s) address(es) listed as endpoints the next step is to expose the service.

There are many ways to do it and some depend on the k8s “type”. For example, if it is an OpenShift cluster an easy way to expose is through Routes, if it is an EKS you can define the Service as type Load Balancer, if you have an IngressController provisioned you can also configure the operator to deploy the Ingress resource, etc.
Please, check Exposing Pulp - Pulp Operator for more information on each one of these possible configurations.

If you are in a rush or just want to test from your local machine, you can also try to use the kubectl port-forward command:

$ kubectl port-forward svc/<name of svc>  <localhost port>:<pulp-web svc port>

for example:

$ kubectl -nsystem port-forward svc/pulp-web-svc 8080:24880
$ curl localhost:8080/pulp/api/v3/status/
{"versions":[{"component":"core","version":"3.29.0","package":"pulpcore" ...

[AJJA]

Hello

Thank you for the fast response.

I can already access pulp-web. I created an ingress resource as in https://github.com/pulp/pulp-operator/blob/1.0.0-alpha.8/docs/ingress.yaml and deployed everything.

So I am able to do everything but access “http://pulp-web-svc.system.svc.cluster.local:24880/”.

I created a file repository. According to the documentation I needed to create a publication for that, and then a distribution in order to access whatever I uploaded to that file repo. Maybe the next screenshot helps. Any other suggestion ?

Maybe I can play around with the settings.py and put another value to CONTENT_ORIGIN variable ?

Or add another entry to my ingress resource file like

      - backend:
          service:
            name: pulp-web-svc
            port:
              name: foo-24880
        path: /foo

My ingress looks like this at the moment.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: pulp
  namespace: system
spec:
  tls:
    - hosts:
      - xxxxx
  rules:
  - host: xxxxx
    http:
      paths:
      - backend:
          service:
            name: pulp-content-svc
            port:
              name: content-24816
        path: /pulp/content/
        pathType: Prefix
      - backend:
          service:
            name: pulp-api-svc
            port:
              name: api-24817
        path: /pulp/api/v3/
        pathType: Prefix
      - backend:
          service:
            name: pulp-api-svc
            port:
              name: api-24817
        path: /
        pathType: Prefix
      - backend:
          service:
            name: pulp-api-svc
            port:
              name: api-24817
        path: /auth/login/
        pathType: Prefix
      - backend:
          service:
            name: pulp-api-svc
            port:
              name: api-24817
        path: /pulp_ansible/galaxy/
        pathType: Prefix
      - backend:
          service:
            name: pulp-content-svc
            port:
              name: content-24816
        path: /pulp/container/
        pathType: Prefix
      - backend:
          service:
            name: pulp-api-svc
            port:
              name: api-24817
        path: /token/
        pathType: Prefix
      - backend:
          service:
            name: pulp-api-svc
            port:
              name: api-24817
        path: /pypi/
        pathType: Prefix

[hyagi]

Thank you for the explanation! now I understand why you are trying to use the Service hostname (which is not resolved from outside of k8s clusters).

Since you already have an ingress resource and can also access pulp-web remotely the next question is: why your base_url is still pointing to the service? (maybe it is a bug)

Let me try to write down my train of thought so that we can think/investigate this issue together:

• you mentioned that you created the Ingress resource using the sample from our repo
• in this case, I guess you are not configuring Pulp CR with ingress_type: ingress
• the operator’s logic to set CONTENT_ORIGIN depends on the ingress_type
  https://github.com/pulp/pulp-operator/blob/78eb8f750ddf1c975e3833bc31c99a15a5089d78/controllers/repo_manager/utils.go#L845-L864
• considering that your Pulp CR has no ingress_type definition (or didn’t define it as route or ingress), the CONTENT_ORIGIN received the Service hostname

if the above assumptions are right, we can try the workaround you mentioned (overwrite the CONTENT_ORIGIN in settings.py):

kubectl -nsystem patch pulp pulp --type merge -p '{"spec": { "pulp_settings": { "content_origin": "http://<your ingress host>" }}}'

[AJJA]

All of your assumptions are right. I am using an Ingress resource and I am not configuring Pulp CR with ingress_type.

Didn’t work at this time. I am getting:

403: Access not authenticated

[hyagi]

hum… ok, considering that you received an HTTP response from Pulp API, it seems that now we have remote communication with Pulp.

To get a better understanding of this “new error” we will need more information.
Please try to provide as much detail/information as you can:

• What command did you run to get this error? (I guess you were trying to download protected content)
• Were you following any specific documentation? (If so, please share the link with us so we can try to reproduce the error)
• What IngressController do you have?
• Did you configure your Ingress to pass the client certificate? (Reverse Proxy Config — Pulp certguard Support 1.6.5 documentation)
• Have you tried to download a non-protected content?
  • If so, did it work?
  • If you didn’t, can you please try it as a test so we can validate if the error also happens in this scenario

Sorry for asking so many questions, but the more information we have the less I need to guess and the troubleshooting can be more assertive.

[AJJA]

I was trying to access the wrong URL. So if I go to the specific distribution instance and try to access the base_url that doesn’t work. I think it’s named after the pod. This is in Python Distribution instance.

If I go one step before to Python Distribution List and again try to access the base_url, doesn’t work again.

If I go one step before to List Distribution List then I get the right base_url. The one we patched in the previous steps and I get access to the content I uploaded to the python repository.

So the workaround we did works for now. If I have anything else (like an issue,if something not working as expected) I will come back and send you all the info you needed.

Thank you for all the help.