Features in tech preview

Development
#1

As per the discussion in the issue https://github.com/pulp/pulpcore/issues/3429, we have concluded that more inputs on the respective topic are needed.

A couple of months ago, we decided that telemetry or analytics would help us to gather relevant data that could drive our decision-making strategy towards developing new features (Proposal: Analytics). Today, we again discuss how to define the state of features in tech preview; yet, without relevant data.

Right now, it is not definite whether we should first try to write a facility that will enable us to track the use of features in tech preview and respond to the analytics appropriately, e.g., by adjusting the workforce needed to support the features in question. On the other hand, we do not want to be stuck in a loop where we wait for the analytics to tell us whether the feature is used while users wait for us until we let them know whether the feature is production ready.

Therefore, we need to find a consensus on the definition of tech preview since multiple features are still marked as tech preview even though they are already used in production and their API has not changed over the years.

A question to the audience (e.g., users or developers) is: How do you view features in tech preview? Are they safe to be used? Do you think we (Pulp developers) should strive first to understand the actual use of the features in tech preview before marking them production ready?

I propose marking almost all of the features in tech preview (e.g., ACS, import/export, Redis caching, RBAC) to be production ready for now. Then, we can start implementing a facility that will help us to track the use of the existing and upcoming features (e.g., Domains) with higher priority.

2 Likes
Katello/Pulp3 Integration meeting
Pulpcore Meeting Minutes
Pulpcore Meeting Minutes
#2

As a user, I think I might be slightly more reluctant to use a feature if it’s in tech preview. But if I really need to use it, then I will use it.

The fact that Pulp uses semver and doesn’t really do major releases, I’d probably lean towards keeping features in tech preview for as long as possible (or until you have production data). I’m using some parts of Pulp/pulp_deb now that have bad designs that are probably unchangeable due to semver and it’s a suboptimal situation.

So all that is to say, I don’t have a strong opinion other than that tech preview benefits users as well as developers.

#3

One other thing to mention: I know from having worked on Pulp that “tech preview” means that a feature could have backwards incompatible changes in the future. It does not mean that a feature is not supported. So that makes me more confident in using a tech preview feature. Not sure if other users have the same view though.

#4

Morning all

For me, I see tech preview as almost akin to a beta release for a particular feature set.

This allows developers to extend the reach of a feature to those that “only run production code” in production, but for one reason or another are willing to test a particular feature.
Maybe they are already in contact with developers to address some corner cases that they have hit.

I would very much expect “tech preview” to be under active development/testing.

An example of where I would have adopted something in tech preview would be the switch to the new tasking system (if it was done that way) several years ago.

  1. I was actively working with developers to address an issue.
  2. If we didn’t try the new tasking system (early) then we’d have had to put the whole pulp3 role out on indefinite hold until it transitioned from tech preview.

The longer something sat in a tech preview state, the lower the confidence I would have in it, and the less likely I am to adopt it (though with pulp, I’d be more likely to reach out to the Dev’s and ask what’s going on).

1 Like
#5

The thing with “under active development/testing” is that we add the flag while developing, so that box is checked. But with time development fizzles away and now, just not receiving issues for the feature is a bad reason to mark it “production ready”. It can mean one of “there are no bugs” and “no one is using it”.
So keeping it in “tech-preview” is really the preloaded excuse for eventually changing or removing it again.

1 Like
#6

Tech preview means to me in general that a feature should not be used in production unless you are prepared to clean your system when a bug is hit. It also could mean that the developers could drop the feature at any time because it is unfinished. As I’ll get to below, I don’t use this reasoning with Pulp typically, however.

From a stakeholder’s standpoint (Katello), I would like to be kept up-to-date about which features are tech-preview and which aren’t. I would also be happy to give feedback about how we use features, especially if they are still in tech-preview. So yes, I think Pulp should first understand what the current state of the feature is to decide if should be marked production ready.

This quote from @davidd is how I personally treat Pulp tech preview features. It would be nice if Pulp gave a solid definition for what tech preview means to them.

I also however consider that tech preview means it can be later removed. If a feature has a sufficient amount of users, perhaps it’s time to consider moving it out of tech preview. Breaking changes and removal both could then frustrate the community. If it’s well-adopted, perhaps that means it’s good as-is!

Any feature can evolve, but I don’t think that means it needs to be marked as tech-preview. With the current tech-preview behavior, perhaps that means some developer has a future plan for it. Rather than mark it as tech-preview, perhaps there should be a document with the future plans for that feature.

So, as a way forward, I would think perhaps Pulp should be stricter in marking things tech preview. Mark them as such if you truly believe the features should not be used in production. It’s okay if a feature has a bug or two, so keep it only to things that are truly incomplete. That’s what we do in Katello at least.

1 Like
#7

Thanks for the inputs so far.

From the comments above, it is notable that features in tech preview are less prone to be used by real users. To escape from the infinite loop I have described previously, we should focus on marking the features as production ready as soon as possible.

Also, to my understanding, the users usually participate in discussions about the development and usage of a feature as the work progresses since they stand behind the initial effort. Based on that, we should always have direct feedback from the users immediately. This leads me to question whether the analytics/telemetry will ever be useful for us in that matter.

1 Like
#8

After today’s pulpcore meeting, it is notable that we have decided to mark most of the features in tech preview as production ready. If someone is against this change, please, raise your concerns with respect to a particular feature as soon as possible. Do not forget to state why the feature should not be marked as production ready.

The list of features in question is in the section “Features no longer in tech preview”:

1 Like
#9

The deadline for raising any concern is Feb 19 (23:59).

2 Likes