(Go: >> BACK << -|- >> HOME <<)

Page MenuHomePhabricator

Measure network latency of page load requests (iOS)
Closed, ResolvedPublic

Description

From the description in T322142

One of the consequences of the deprecation of RestBase can potentially be increased latency times for loading article content. In order to measure the difference in latency after switching off RestBase, we must first measure a baseline of latency in our current configuration.

The MobileWikiAppSessions schema contains a field called pageLoadLatency, which is intended precisely for this purpose. It represents an average latency for all page requests made by the user during the current session (session = continuous usage of the app, ending after 30 minutes idle)

(Note that the current values in this field should not be relied upon, since this schema might be no longer wired correctly in our code.)

Event Timeline

Seddon triaged this task as High priority.Nov 7 2022, 5:14 PM

MobileWikiAppSessions is located here, in the legacy system. I couldn't find any use of this in the iOS app, nor any record that we ever used it, but we do use MobileWikiAppiOSSessions. It logs different data, but maybe we can add a pageLoadLatency field to that schema?

@Seddon @SNowick_WMF Let me know what y'all think, thanks!

@Tsevener I think we should use MobileWikiAppiOSSessions but can we first verify how it's measuring sessions (ie no sampling data, is it every user, every session, etc.)? I just want to be sure it's working before we add more columns.

Only other question is if we are measuring pageLoadLatency will the count be for the first page load event or will we track multiple page load events for each user? I think either is fine but if we're sending multiple events per user per session we will need to use the same session_id and possibly consider limiting how many events and/or sample throttle the data. Lmk what you think.

@SNowick_WMF I did some digging and we only log session_start and session_end events (both with an unknown category and null label, just FYI).

  • session_start is logged upon app launch, app foreground, and when the "send usage reports" toggle is turned on in Settings.
  • session_end is logged upon app background and when the "send usage reports" toggle is turned off in Settings. I notice it tries to log when the app is terminated, but I suspect maybe there's no time for it to save the event to the local database before termination, because I never see the session_end even logged upon relaunch in that case.

We aren't rate limiting it, but we can easily add one on a schema-wide level (which is probably not what you want). The other schema description for pageLoadLatency suggests that we take the average (""Average latency, in milliseconds, of loading articles from the network and rendering them."). So we could keep an ongoing list of the latency times for each article they visited, then average them up and send as a part of the session_start and session_end events. Copying @cooltey - is this how y'all plan on doing it on Android?

Alternatively, we could set this event up as a new schema, and we send it for every article that loads. If you wanted rate limiting, I think that would be the way to do it without affecting when the session_start and session_end events are logged.

Hi @Tsevener

We have a PR of updating the code for the load latency measurement, and we use Schema:MobileWikiAppSessions
https://github.com/wikimedia/apps-android-wikipedia/pull/3675

For the current stage, we are planning to log the page/mobile_html first, which logs the start time before making the request and the end time after making the request.
https://github.com/wikimedia/apps-android-wikipedia/pull/3675/files#diff-412b85c8c440bf68c3341ef5ff5d7c6d934eae7f68a38b9093c499e105e08df2R49-R56

And we'll do the sampling and then send the average latency to the pageLoadLatency field.
https://github.com/wikimedia/apps-android-wikipedia/pull/3675/files#diff-0de18585f3eb62d3ac6fe9bd2fbec5799f7a950d456ef889bcdf988fe3a6bf46R89

Thanks @cooltey! Out of curiousity, when is the MobileWikiAppSessions event logged in the Android app? I'm hoping it's mostly similar to when iOS logs our session_end action for MobileWikiAppiOSSessions.

So far iOS's plan is to add some new fields to https://meta.wikimedia.org/wiki/Schema:MobileWikiAppiOSSessions: pageLoadLatencyMin, pageLoadLatencyMax, and pageLoadLatencyAverage (not required, integers), which get logged whenever the session_end action is logged. pageLoadLatencyAverage will be the same as the pageLoadLatency field in MobileWikiAppSessions. We're also grabbing the min and max extremes for extra data.

The Android logs the event here:
https://github.com/wikimedia/apps-android-wikipedia/blob/04903c96b74e8517ac46cd5eccf82b9cd49758ce/app/src/main/java/org/wikipedia/analytics/SessionFunnel.kt#L32-L45

Which is based on the last touch time and compares it with the session time, and then sends the event to the database.

The touchSession() function will be triggered in different places, such as the funnel object created, app resumed, page viewed, and back button (virtual) pressed, it will set the sessionData.lastTouchTime and only send the event if the session is detected time out.

@cooltey cool, sounds like it's generally the same idea behind this event then. Thanks for explaining!

PR for page load times: https://github.com/wikimedia/wikipedia-ios/pull/4412

@SNowick_WMF - to match the pattern of the other properties, I have the client-side sending the new values in snake case (page_load_latency_min, page_load_latency_max, page_load_latency_average`). I can update the schema whenever you're ready. I don't have an exact date but we'll probably release in ~1.5 - 2 weeks from now.

Tsevener moved this task from Doing to Needs Code Review on the ios-app-v7.0.1 board.

Note: legacy schema was updated with new latency measurement properties. The new revision ID is 24181799.

Validating we are seeing values in new columns page_load_latency_min, page_load_latency_max and page_load_latency_average for events tracked in MobileWikiAppiOSSessions with revision 24181799.

The min date for these values is 2022-12-07, will wait until we have accumulated more data, events counts are still ~150 daily Data

iOS Daily Averages (with min and max) Data

Android Daily Averages (with min and max) Data

SNowick_WMF lowered the priority of this task from High to Medium.Jan 24 2023, 6:08 PM
SNowick_WMF moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.
JTannerWMF subscribed.

I think Shay is working on this now right?

Note: Android metrics are being re-populated after a change in code to improve our latency measuring. Will re-post in 2 weeks.

@SNowick_WMF Cassandra backing storage was turned off for PT Wiki on March 9th. Can we get a breakdown of average values by wiki, grouped by day from around this time? It would be useful to see if there has been a jump for PT Wiki. Thanks!