@ngkountas Yes, I have add you and engineers from the Metrics Platform team as reviewers for that.

@ngkountas I have submitted a patch for stream configuration and registration. As per the discussion with MP team yesterday, please change the stream name in instrumentation to mediawiki.product_metrics.mint_for_readers

Update: @ngkountas and I met with the Metrics Platform team (thank you @Sfaci for the walkthrough). Here is a summary

We will not be using Sqoop approach, please T366869#9891978.

We will not be using sqoop approach, please T366869#9891978.

This is blocking at least three tasks for me: T362615, T367016 & T366869#9891978 (all of which need to access MariaDB from Airflow).

Listing down all the metrics for future reference (priority can be discussed and changed)

Update fom my meeting with Pau today: a daily update will be fine to start with, we can make it more frequent if needed later.

A monitoring dashboard has been setup at: https://superset.wikimedia.org/superset/dashboard/p/xn8BnnLBzYy/

For the record: @JAllemandou and I met yesterday. We decided it is best to hold off on these sqoop tasks, as sqoop might be deprecated sooner or later. The suggested approach is the to create an Airflow job that mimics sqoop i.e. gather and load raw data as it is from MariaDB into Data Lake, and then another job to do the necessary aggregations. I will come to back this once we materialize the proof of concept for running Airflow jobs based on MariaDB with T362615.

I did some initial exploration of AM's reverts on testwiki (9 edits to be exact) at https://nbviewer.org/urls/gitlab.wikimedia.org/kcvelaga/automoderator-measurement/-/raw/main/pilot_analysis/testwiki_activity.ipynb/%3Fref_type%3Dheads

@fnegri thanks for the update!

@fnegri I am curious about the status of this task, and especially the subsequent step for Superset to be able to access ToolsDB as I have a couple of use cases for dashboards dependent on that.

@JAllemandou - I misread your question, please ignore my previous comment.

@JEbe-WMF Thanks for working on the initial analysis. The following improvements would be helpful:

@CMyrick-WMF I reviewed the notebook, this is great work! I have some feedback, mostly style related.

Thanks for the review @JAllemandou

The events are being logged without any errors.

I have added a sheet based on the latest mw_history snapshot (May 2024).

@srishakatux for a another task (T366044), I have listed all the key metrics that we currently track across three various reports and dashboards.

Given my current bandwidth, I am not sure if I can get to this before the last week of June (especially as it needs investigation on data availability). I am unassigning myself for now.

Additional context from meeting with @JWheeler-WMF

Here is the spreadsheet with engagement status (edits within the preceding 30, 90 and 180 days to 30 April 2024*) of CWS participants from 2021 to 2023.

@Pginer-WMF I have identified the events mentioned in the task description as priorities for the next set of events to be instrumented. Please review and if you think something else should be prioritized to understand user interactions, we can discuss.

@Pginer-WMF is right.

What do you think of the changes?

@Pginer-WMF I have consolidated all the metrics we are currently tracking across dashboards / reports and are good to have for the first version. Additionally, from this search, I have listed the open tickets and if we can addressed those with the first version of the dashboard. Please review and share your thoughts.