Will the values in settings.py merge with the defauts?

Support
#1

I add additional settings in settings.py when I test token and keycloak as described Keycloak — Pulp Project 3.48.0 documentation.
What I did:

  • run a container basic, get the defaults conf about
    INSTALLED_APPS AUTHENTICATION_BACKENDS TEMPLATES , by dyconf list , then
  • add these additional lines

For exemple: the part about INSTALLED_APPS becomes:

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.postgres',
    'import_export',
    'django_filters',
    'django_guid',
    'drf_spectacular',
    'rest_framework',
    'pulpcore.app',
    'pulp_ansible.app.PulpAnsiblePluginAppConfig',
    'pulp_certguard.app.PulpCertGuardPluginAppConfig',
    'pulp_container.app.PulpContainerPluginAppConfig',
    'pulp_deb.app.PulpDebPluginAppConfig',
    'pulp_gem.app.PulpGemPluginAppConfig',
    'pulp_maven.app.PulpMavenPluginAppConfig',
    'pulp_ostree.app.PulpOstreePluginAppConfig',
    'pulp_python.app.PulpPythonPluginAppConfig',
    'pulp_rpm.app.PulpRpmPluginAppConfig',
    'pulp_file.app.PulpFilePluginAppConfig',
    'storages',
    'django_readonly_field',
    'social_django',
    'rest_framework.authtoken'
    ]

My question is: if I just add

INSTALLED_APPS = [
    'social_django',
]

in my settings , will it be deep merger with the default settings? or I have to put the complete list
in my settings.py?

#2

Hello @xm1234567 ,
You example will replace all INSTALLED_APP, but you can use this to enable merging:

INSTALLED_APPS = [
    'social_django',
    'dynaconf_merge',
]

Using that merge token, Dynaconf will append social_django to the end of the list.
You can check other merging strategies here, if you need to.

Let me know if that helps!

3 Likes
#3

@pedro-psb sorry for late reply, thank you for answering my question :slight_smile:

Great! the list can be merged with dynaconf_merge easily!

the doc says The dynaconf_merge and @merge functionalities work only for the first level keys, it will not merge subdicts or nested lists (yet). ref here . So to add the additional setting as belows, i have to put all settings, since context_processors is not the first level keys. just to want to confirm this one :slight_smile:

TEMPLATES = [
    {
        ...
        'OPTIONS': {
            ...
            'context_processors': [
                ...
                'social_django.context_processors.backends',
                'social_django.context_processors.login_redirect',
                ...
            ]
        }
    }
]
1 Like
#4

So to add the additional setting as belows, i have to put all settings, since context_processors is not the first level keys. just to want to confirm this one :slight_smile:

Yes, currently you have to copy the content.

You actually can merge nested dicts with the dunder separator method. For list we have the limitation that we don’t support index access with this syntax.

There is a PR that should be merged in 3.3.0 that will probably add support for doing this:

# note the 0 for accessing the TEMPLATES first item.
TEMPLATES__0__OPTIONS__context_processor = [
    'social_django.context_processors.backends',
    'social_django.context_processors.backends',
    'dynaconf_merge',
]
1 Like
#5

just to add my example of Dunder merging for memory.
The part needed to merge is under DEFAULT_AUTHENTICATION_CLASSES , the new authentication way rest_framework.authentication.TokenAuthentication:

REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': [
        'rest_framework.authentication.TokenAuthentication',  # <-- the part to add for token creation
    ],
}

I did like this with dunder merge:

#For token creation
REST_FRAMEWORK__DEFAULT_AUTHENTICATION_CLASSES = ['rest_framework.authentication.TokenAuthentication','dynaconf_merge']

Note the __ for dunder merging.

1 Like
#6

Are you sure this should work ?

Sorry if I post in an old Thread, but I cannot create new Topics yet unfortunately :frowning_face:.

I was just trying to get it working with OIDC instead of Keycloak, but I first had to rebuild the Docker Container to install the social-auth-core and social-auth-core-django Packages.

However using this Syntax caused an error related to ANOTHER Key (TEMPLATES[0].APP_DIRS), where Django complained about a get() Method not being valid on str Object.

#7

Can you please elaborate where/what/how you did dyconf list and in which Docker Container you did that ?

dyconf doesn’t seem to be installed in any of them.

Trying to use get_debug_info from the Dynaconf Package in settings.py just causes an Error saying that the Module/Package dynaconf.utils.inspect_settings is not installed:

from dynaconf.utils.inspect_settings import get_debug_info
get_debug_info(settings.DYNACONF, verbosity=0, key="data")

I’m stuck :frowning_face:.

#8

Hey @luckylinux ,
That dynaconf feature of merging at a given index (the TEMPLATES example) is not yet released. In the TEMPLATES case, you should copy the full default value into your user settings.py and edit it as you please.

As for dynaconf list, dynaconf is a python dependency for pulpcore, so it should be present in any container instance where you have a pulp component (api, content or worker). If you are able to run pulpcore-manager you should be able to run dynaconf list. What are you using in your deployment? Single-process (compose) or multi-process images?

And btw, dynaconf.utils.inspect_settings module does not exist. You can do from dynaconf import inspect_settings, but I’m not sure you need that. And even so, you should probably use dynaconf inspect (cli).

#9

Hi @pedro-psb,

Thank you for the quick Reply.

What do you mean that dynaconf Feature hasn’t been released ?

The PR that you mentioned above was merged more than 1 Year Ago:

I’m using Docker Compose Images (approx. 7 different Containers IIRC).

Where should I be able to print the current Settings so that I can merge that manually into the TEMPLATES Dictionary in settings.py ?

I was hoping that there was a way to just call dyconf list just inside settings.py but I guess I need to yet again rebuild the Container, just like I had to do to install the social-auth-core and social-auth-core-django Packages.

#10

What do you mean that dynaconf Feature hasn’t been released ?

The PR was merged more than 1 Year Ago:

Yes, unfortunetely it’s not released.

Where should I be able to print the current Settings so that I can merge that manually into the TEMPLATES Dictionary in settings.py ?

I recommend replacing the TEMPLATES settings entirely. You can do it by editing the settings.py (I don’t think you need to rebuild for editing the settings, right?) or even via environment variables. E.g (passing to the containers):

PULP_TEMPLATES="@json [{"APP_DIRS": true, "BACKEND": "django.template.backends.django.DjangoTemplates", "DIRS": ["<override/with/correct/path>"], "OPTIONS": {"context_processors": ["django.template.context_processors.debug", "django.template.context_processors.request", "django.contrib.auth.context_processors.auth", "django.contrib.messages.context_processors.messages", "<extra-processor>"]}}]"

Let me know if this works for you.

I was hoping that there was a way to just call dyconf list just inside settings.py

Fwiw, dynaconf list is a cli command. Anyway, there was supposed to have a way to do this (use the previous settings context) using the hook feature, but it’s not working as expected in my development environment. I’ll have a closer look at this.

2 Likes
#11

No, settings.py is the one Thing that works out of the Box, I just bind-mount that inside the Container, that’s easy :smiley:.

The social-auth-core and social-auth-app-django were the Reasons for the rebuild. I find it weird that most Plugins aren’t installed out of the Box. That kinda defeats the purpose of a Docker Container somewhat :frowning_face:.

My point about TEMPLATES is: how do I get the CURRENT (default) Value of TEMPLATES ?

Replacing the value in settings.py is NOT the Issue.

The Issue is knowing what to put in SETTINGS together with the Customization that I need to do. For that I need to know which other (default) Settings to put there :frowning:.

Furthermore, what would be <override/with/correct/path> in your Example ?

1 Like
#12

The Issue is knowing what to put in SETTINGS together with the Customization that I need to do. For that I need to know which other (default) Settings to put there :frowning:.

If it’s ok to hardcode the defaults + your edits (instead of the dynamic merging we wanted to do), you can run dynaconf get {key} or (dynaconf list) to know what the default values are. Which brings us back to the issue of using that.

So assuming you can ssh into a container instance with a running Pulp and run pulpcore-manager, are you able to run dynaconf list? If not, what error do you get?

Furthermore, what would be <override/with/correct/path> in your Example ?

Here are how we define our defaults in pulpcore. It’s not 100% reliable to use this file, since plugins may add things to some of those settings and some settings are not hardcoded. See:

I find it weird that most Plugins aren’t installed out of the Box

You mean django plugins, like social-auth-core? I believe the reason is to not bloat the image, but that’s just my thoughts.

2 Likes