Unable to sync redhat repos

bkwfanena · December 7, 2021, 5:17am

Problem:
I’ve managed to sucessfully create an remote repo for redhat 7/8 using the certificates provided by redhat, however I am unable to run a sync… I get the below error…

Started background task /pulp/api/v3/tasks/f15222fa-5924-4cbe-b7fa-c8ccbad1b860/
Error: Task /pulp/api/v3/tasks/f15222fa-5924-4cbe-b7fa-c8ccbad1b860/ failed: ‘no start line: cadata does not contain a certificate (_ssl.c:4161)’

here is the repo output
{
“pulp_href”: “/pulp/api/v3/remotes/rpm/rpm/019e4c89-e7c2-444b-a0c7-0b4ea05d1247/”,
“pulp_created”: “2021-12-07T02:25:44.544901Z”,
“name”: “rhel-7-x86_64”,
“url”: “https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os”,
“ca_cert”: “/etc/pki/rhel/rhel_ssl_ca.crt”,
“client_cert”: “/etc/pki/rhel/rhel_ssl.crt”,
“tls_validation”: false,
“proxy_url”: null,
“pulp_labels”: {},
“pulp_last_updated”: “2021-12-07T02:25:44.544923Z”,
“download_concurrency”: 1000,
“max_retries”: null,
“policy”: “immediate”,
“total_timeout”: null,
“connect_timeout”: 5.0,
“sock_connect_timeout”: null,
“sock_read_timeout”: null,
“headers”: null,
“rate_limit”: null,
“sles_auth_token”: null
}

Expected outcome:
Should be able to sync redhat repos.
Pulpcore version:
latest 3.16

Pulp plugins installed and their versions:
pulp-rpm: latest
pulp-file: latest
Operating system - distribution and version:
Redhat server 7

Other relevant data:

Please let me know if you need further logs and how to collect them.
Thanks

bkwfanena · December 8, 2021, 3:05am

Hello… anyone?

ggainey · December 8, 2021, 3:01pm

Sorry, just saw this - ca_cert and client_cert need to be the content of the cert-file, not the filename. Here’s an example of me creating a RH-CDN-remote via httpie:

github.com

ggainey/pulp_startup/blob/main/7537_fips/all_sync_test.sh#L309

    
      
          
          
for r in ${!REMOTES[@]}; do 
            echo ">>>>> " TESTING [${REMOTES[$r]}] INTO [${NAMES[$r]}];
            # create repo
            REPO_HREF=$(http POST :/pulp/api/v3/repositories/rpm/rpm/ name=${NAMES[$r]} | jq -r '.pulp_href')
            echo "repo_href : " $REPO_HREF
            if [ -z "$REPO_HREF" ]; then echo ">>>>> " FAILED REPO; continue; fi
          
          
  # create remote
            if  [[ ${NAMES[$r]} == cdn* ]] ; then
              REMOTE_HREF=$(http POST :/pulp/api/v3/remotes/rpm/rpm/ name=${NAMES[$r]} url=${REMOTES[$r]} policy='on_demand' download_concurrency=5 client_cert=@/home/vagrant/devel/pulp_startup/CDN_cert/cdn.crt client_key=@/home/vagrant/devel/pulp_startup/CDN_cert/cdn.key ca_cert=@/home/vagrant/devel/pulp_startup/CDN_cert/redhat-uep.pem | jq -r '.pulp_href')
            else 
              REMOTE_HREF=$(http POST :/pulp/api/v3/remotes/rpm/rpm/ name=${NAMES[$r]} url=${REMOTES[$r]}  policy='on_demand' download_concurrency=5 | jq -r '.pulp_href')
            fi
            ###REMOTE_HREF=$(http :/pulp/api/v3/remotes/rpm/rpm/ | jq -r ".results[] | select(.name == \"${NAMES[$r]}\") | .pulp_href")
            echo "remote_href : " $REMOTE_HREF
            if [ -z "$REMOTE_HREF" ]; then echo FAILED REMOTE; continue; fi
          
          
  # sync repo using that remote
            TASK_URL=$(http POST :$REPO_HREF'sync/' remote=$REMOTE_HREF | jq -r '.task')
            echo "Task url : " $TASK_URL

Here’s me using the CLI, and taking advantage of having env-variables set up containing the content of my cert-files:

github.com

ggainey/pulp_startup/blob/main/sync_test/multios.sh#L31-L32

    
      
          pulp rpm remote create --name ${NAMES[$n]} --url ${REPOS[$n]} --policy immediate \
          --ca-cert "${CDN_CA_CERT}" --client-key "${CDN_CLIENT_KEY}" --client-cert "${CDN_CLIENT_CERT}" | jq .pulp_href

Hopefully this helps!

G

ggainey · December 8, 2021, 3:07pm

Example w/ output:

(pulp) [vagrant@pulp2-nightly-pulp3-source-centos7 pulp_startup]$ pulp rpm remote create \
    --name r6 --url https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os 
    --policy on_demand   
    --ca-cert "$CDN_CA_CERT" 
    --client-key "$CDN_CLIENT_KEY" 
    --client-cert "$CDN_CLIENT_CERT"
{
  "pulp_href": "/pulp/api/v3/remotes/rpm/rpm/04b345ad-e49f-41c6-8ecd-47155ed22e11/",
  "pulp_created": "2021-12-08T15:02:19.551657Z",
  "name": "r6",
  "url": "https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os",
  "ca_cert": "-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----...-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----...-----BEGIN CERTIFICATE-----...----END CERTIFICATE-----",
  "client_cert": "-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----",
  "tls_validation": true,
  "proxy_url": null,
  "pulp_labels": {},
  "pulp_last_updated": "2021-12-08T15:02:19.551673Z",
  "download_concurrency": null,
  "max_retries": null,
  "policy": "on_demand",
  "total_timeout": null,
  "connect_timeout": null,
  "sock_connect_timeout": null,
  "sock_read_timeout": null,
  "headers": null,
  "rate_limit": null,
  "sles_auth_token": null
}

bkwfanena · December 9, 2021, 12:44pm

Thanks
That fixed the problem, would be nice to have this in the doco somewhere…

ipanova · December 9, 2021, 2:56pm

@bkwfanena are you willing to open doc issue?

ggainey · December 9, 2021, 6:46pm

We have one - Task #9149: As a Pulp user, I want an end-to-end tutorial with all the steps to get an RPM into Pulp - RPM Support - Pulp

Note the assignee

mcorr · December 13, 2021, 10:18am

Is there any chance of getting a status update on this ticket?
I think it might be helpful to understand the blockers and pieces that need to be put in place before this work can be completed.
As this was one of the requests in our survey this year, I am particularly interested in monitoring the status of this ticket.

ggainey · December 13, 2021, 6:46pm

Added an update to 9149 , Mel!

Andi2301 · April 5, 2023, 12:46pm

Hi,
newer version of pulp-cli support the useage of certificates directly using the @sign in front of the cert filename.

–client-cert @/var/tmp/mycertificate.pem

Easy to forget

LG
A