Exporter filesystem does not display content

Support
#1

Problem:
hi! After allowing the export of paths and successfully creating the filesystem exporter repository, the content is not displayed in pulp/content. The repository itself does not appear either. Updating the repository also happens without errors, but also does not bring any results. Do I need to publish and distribute for this type of repository? As I understand it, the content that is in the folder that we have allowed to export to should be displayed as a result. settings.py
Expected outcome:
displaying content in the repository
Pulpcore version:
pulpcore 3.66.0
Pulp plugins installed and their versions:

Operating system - distribution and version:
Alma linux 9 5.14.0-427.40.1.el9_4.x86_64

Other relevant data:

#2

Not entirely sure, i understand your setup, but import export creates repositories (or versions thereof). You still need to publish and distribute them for clients to consume them.

#3

you got it right, I created filesystem exports. Now I’m trying to publish it, passing it as the publication address http://hostname:8080/pulp/api/v3/publications/file/file /, And a link to filesystem exports as the repository url http://hostname:8080/pulp/api/v3/exporters/core/filesystem/01936979-987f-7175-bd0d-de139f6faa17 / , but for some reason pulp refuses to eat it and returns me “Invalid hyperlink - Incorrect URL match” although both links are alive, I can click on them

#4

Please have a look here:
https://pulpproject.org/pulp_file/docs/user/guides/publish-host/#create-a-publication-manually

I’m not even convinced filesystem export is the thing you wanted here.

#5

maybe I’m really doing something wrong… as a result, I need to get a file repository where I can upload files from the file system, and not synchronize with an external repository)

#6

Yes that should go without exporting.
You can either sync from the local file system or just use pulp file content upload to get the files in.

#7

well, I don’t find anything like “synchronize data from the local file system” in the API documentation… please tell me

#8

If you are talking about “a bunch of files” i’d go with Upload Content - Pulp Project .

If it’s like an immense amount of files in a local directory, you control, then you can do sync by providing a file:// url to the remote. But you’d need to create a manifest file in that directory first.

#9

Yes, apparently this is what is needed. However, I’m trying to do this through the API. I don’t quite understand what I should specify in the relative_path parameter. That is, I understand, of course, but I do not understand what this path should be specified for. I tried to specify the whole folder, I tried a specific file, but both ways the error is an incorrect URL parameter

#10

Perhaps it will become a little clearer this way. I have a Pulp docker container deployed on my server. It already has rpm and deb repositories, but I also need to be able to upload files to the repository file type. To do this, I created a new repository with the file type. The files themselves that need to be uploaded are located outside the container on the server itself. I am trying to download it through the API, not through the CLI. Maybe after creating a repository, I should first publish and distribute, and only after that try to upload files there? Then everything seems logical, and as a path I need to specify not the path to the catalog, but the path to distribution

#11

relative_path is the name (including a relative path), the file should assume within the repository.
So if you upload a file specifying the relative path to be foo/bar.txt and you publish the repository with a distribution that has base_path="snap", then you can expect to download that file from https://<server>/pulp/content/snap/foo/bar.txt. It does not matter how that file was named on your disc.

#12

That’s understandable, thank you! I don’t quite understand how to specify the path to the file itself, no matter what I write in “file”: I get the error “The submitted data was not a file. Check the encoding type on the form”

#13

I think, when sending a file, you need to use multipart encoding.
Did you manage to use the cli? Adding -vvv directly after pulp can tell you a little about the actual api calls performed.

#14

no, the command line could not be used. Inside the container itself, as well as outside, on the local machine itself, the pulp command returns command not found. And as for the path, should I specify the path to the file based on the container, or should I specify the path relative to the physical server?

#15

The pulp cli is using the API. Given network access, you can run it from everywhere. And it might even help you learn more about the API.
But anyway, relative_path is where you want to see the file be published relative to the base_path.
The multipart part where you upload it to should be named “file” I think.

#16

Hi, it’s me, but under a different account, so my personal account stopped working under the old one… I looked at the output of the vvv CLI. In fact, a lot has become clear! Thank you very much. The only thing I don’t understand is where the “artifact” line comes from: “/pulp/api/v3/artifacts/01939274-b1fc-7fbd-95c9-11345a55df05/”, the value 01939274-b1fc-7fbd-95c9-11345a55df05/", is it generated by pulp itself?

1 Like
#17

Artifacts are de-duplicated and stored based on their sha256 checksum. That value is the checksum of the artifact.

#18

Yes, that is the artificial primary key in the database (it looks the same for all other models).

But also an artifact should be seen as implementation detail. We are trying to steer usage away from them and ultimately hide that detail in what is to become pulp4 soon.

#19

then it is unclear how this can be used through the API. If you do not specify Artifact, then the file does not load and an error occurs, but in order to specify it, you need to take it from somewhere. it turns out to be a vicious circle)

#20

If you look a the upload workflow once again there is not mention of an artifact anywhere:

https://pulpproject.org/pulp_file/docs/user/guides/upload/