Proposal: Analytics

Development
#21

I’m cancelling tomorrow, once more. The CI has required a lot of attention this week, which is what took the time. I’m planning to host on May 5th at the regular time and place.

#22

I’m cancelling tomorrow, once more. The CI has required a lot of attention this week again. I believe we’re almost through the challenging CI changes we’ve been making. I’m planning to host on May 12th at the regular time and place.

#23

The good news is I’ve started on the cloudflare telemetry pulpcore aiohttp data posting. The bad news is it needs more work before it would be productive for us to meet, so I’m cancelling for tomorrow. I’m hoping that given I’m into the work now we can meet next week.

#24

We met today to go over the PoC which consists of this new repo and this pulpcore PR. See the the minutes from today on the details.

I’m going to revise the telemetry pulpcore PR to only submit data if it’s a dev install, and that will start showing up in the dev area of the PoC. This will let us safely merge the PR, let us see how stuff goes with dev boxes reporting.

We’ll meet next week to look at the revised graphs, and hopefully the pulpcore PR will be merged by then also.

Any feedback is welcome.

#25

We’re going to cancel for today and plan to meet next week June 9th. The action items I’ve not completed and plan to complete by next week include:

  • making the X.Y versions graph.
  • The telemetry PR is still in review to be merged.

Here’s what has been accomplished since last week’s meeting:

  • Shared the project details with Katello so they can think about what they want to see happen with this technology. They are reviewing and posting on the Katello community site also.
  • The telemetry PR had some review and revision
  • Adjusted the telemetry PR to pulpcore to only ever post for dev installation
  • Refactored the telemetry PR to keep it more organized
  • Preliminary approval from RH to request a pay-for billing for cloudflare, but I wanted to wait a bit to see how far we can get with the unpaid version.
#26

We’re going to cancel once more today. We’re in the same state as last week. Let’s plan to meet next week June 16th at the normal time.

#27

We met today for a short check-in. You can find the minutes here. We identified a variety of action items for me to work on over the next few weeks. I’m going to cancel next week’s meeting to give more time for me to work on the AIs.

The next meeting then will be at the usual time (see thread above) on June 30th.

#28

Hi all, I’m going to cancel this week’s meeting because of other things taking the priority, and also next week’s meeting because I’ll be on PTO. We’ll plan to meet next on July 14th.

FYI, for some reason the summarization stopped working so I’m going to fix that before we go too much further.

#29

2022-07-14

Attendees: bmbouters, dkliban, ipanova, ppicka, ggainey

  • Current State
  • PROBLEMS
    • summarization isn’t working, investigation isn’t getting us past whatever the problem is
    • server-side-code pagination isn’t working
    • DNS for analytics-pulpproject-org to be analytics.pulpproject.org would require all pulpproject.org be handed over to cloudflare
      • reverse-proxy is possible, POC works but is…suboptimal
  • OSCI asking why we’re not just running this on their openshift instance/platform
    • This is a fine question!
  • PROPOSAL
  • discussion ensues
    • reliability/availability? visibility into admin/monitoring?
      • health probe/autorestart-pod should work
  • proposal: openapi work to auto-generate client/server side of this
    • makes available to other projects who might want to do this
#30

I’m still waiting on the infra from osci.io, so I’ll cancel for tomorrow. Some of it’s in place already, but some of it isn’t so it needs another week.

#31

I’m cancelling tomorrow because the full-PoC isn’t ready and our next step is a demo. I’m also cancelling for the week of Aug 1 because I’ll be on PTO. Our next meeting will be Aug 11th.

There has been good progress in the past week though:

#32

Here are the minutes from today’s call. It was mostly a demo. Next week I hope to have the fully-compete demo done and merged to the dev site.

2022-08-11

Attendees: ggainey, dkliban, ppicka, bmbouters, ipanova, wibbit

Agenda

  • discussion around https://github.com/pulp/pulpcore/pull/3032
    • def a good idea, prob want this backported to 3.20
  • progress update
    • lots of progress being made, not baked yet
    • lots of interaction w/ duck@osci
      • analytics.pulpproject.org has 2 branches, main and dev
      • auto-deploys to 2 diff OSCI deployments
      • both use LetsEncrypt TLS
      • web-process pod, posstgres backend
      • django-admin enabled for superuser controls
    • modification to how payloads are defined
      • consolidates client and server definitions of payload
      • using Google’s “Protocol Buffer” approach (q.v.)
      • what about version mismatches?
        • ProtocolBuffer is Opinionated - follow their requirements
    • next steps
      • charting
      • summaries
        • manage.py cmd, to be called by openshift cron every 24 hrs
      • data expiry

AIs

  • bmbouter hoping for a tech demo next mtg

Links

#33

2022-08-18

Attendees: ggainey, dkliban, bmbouters, ipanova, ppicka

Prev AIs

Agenda

  • progress made on finalizing POC
    • demo time!
    • proposal: have “summarizer” delete old content (rather than replace)
    • proposal: have “summarizer” only delete data older-than some window (2 weeks?)

AIs

#34

The draft PR that would introduce the TELEMETRY setting and cause end-user systems to submit anonymous telemetry info to https://analytics.pulpproject.org/ when merged is available: https://github.com/pulp/pulpcore/pull/3116

I’m adding discussion of this PR into tomorrow’s pulpcore meeting agenda here: Pulpcore team meeting - HackMD

#35

2022-08-25

Attendees: ppicka, ggainey, bmbouter

Agenda

#36

With the 3.21 release, telemetry should start arriving and summarizing at https://analytics.pulpproject.org/ The site is ready, so I think we’re all set.

I’m going to cancel meetings until Oct 13th. In between now and then I’m going to work on the following todos:

  • Add plugin documentation on the processes and checklists this group currently has in hackmds
  • Add documentation on how to create the local dev environment
  • Move the telemetry protobuf source and built binary to its own repo and publish on PyPI.
  • Take down the cloudflare PoC

When we reconvene I hope we can look at adding additional metrics.

Thank you to everyone who has participated and given feedback so far!

#37

I’m cancelling the Oct 13th meeting, and we’ll resume on the Oct 20th meeting. At the Oct 20th meeting I’m planning to bring a proposal to collect the postgresl database version as a new telemetry metric. This would be the metric my “Adding a Telemetry Metric Together – Live Coding” talk at pulpcon will add. I’ll post the database metric proposal here ahead of the Oct 20th meeting when its available.

#38

Here’s the agenda for Thursday’s meeting:

  • Here’s a proposal to collect, summarize, and visualize postgresql version which would be a new metric. This is going to be the “live coding” part that I do at Pulpcon to add it.

  • Here’s a new set of graphs to look at accepting from @mdellweg

  • FYI lots of new docs here including importing data from the production site

  • Should we be limiting summaries to only systems with at least 2 checkins?

  • Proposal: Add a “summarization” and “visualization” sections to the “proposal template”

#39

The raw minutes are here but here’s a summary:

Our next few meetings are being cancelled due to conflicts. Our next meeting will be Nov 17th.

#40

I’m cancelling today’s working group and only providing an update here instead. Our next call will be Dec 1st. Here are some updates:

  • We will be merging the postgresql data collection into pulpcore 3.22, those PRs are here and here.
  • Here is the video from pulpcon outlining how to add a telemetry metric.
  • I believe it’s widely agreed that we need to rename “telemetry” to “analytics” so let’s do that I’ll make updates today.
  • We formally added mdellweg and dkliban and myself as the 3 maintainers of the analytics site. They have openshift access, merge access, and django admin access to both sites

Next steps:

  • We need to move the protobuf definition to a new repo.
  • Add additional docs in pulpcore on how to add an analytics proposal
  • Make the analytics site a profile in oci_env, that issue is here.
  • Wait for more analytics proposals to be made.