(Go: >> BACK << -|- >> HOME <<)

Page MenuHomePhabricator

Spike: Investigate WWT Expansion [4 hours]
Closed, ResolvedPublicSpike

Description

As a CommTech team member, I want to know the various options we may have to expand accessibility of the WWT tool, so that we can determine what future potential the project may hold.

Relevant Resources:

Acceptance Criteria:

  • Investigate ability to allow WWT in other language wikis
  • Investigate ability to make WWT a gadget
  • Contact Fabian at WikiWho to see if they have any plans in the pipeline or if they can assist in the expansion of the API's reach
  • Keep in mind that certain product and UX considerations will also need to be considered if we expand accessibility

Other stuff to think about (but not necessarily requiring an investigation, as it requires WhoColor API changes):

  • Ability to allow WWT in Wikipedia namespace (does WhoColor have any data analysis on this?)
  • Ability to use WWT on external links, references, and other content not included

Event Timeline

ifried renamed this task from Spike: Investigate WWT Expansion to Spike: Investigate WWT Expansion [4 hours].Jan 16 2020, 7:05 PM
ifried updated the task description. (Show Details)
Restricted Application changed the subtype of this task from "Task" to "Spike". · View Herald TranscriptJan 16 2020, 7:07 PM
ifried updated the task description. (Show Details)

There is another option we should add to the investigation, @MusikAnimal -- let's also check into making WWT into a MediaWiki extension.

I think that would mitigate some concerns we have (no spreading around of code, more control on behavior and where it is installed, having it available on all browsers, etc) *but* would also require some checking into whether this is feasible -- for example, we didn't test the extension on browsers we know it won't operate on (like Safari and Opera... is there code that won't work there? etc) and also I think there aren't many extensions that use es6 and run a build step. Would that be a problem? (not sure?) and maybe a consideration for the time it'll take to run a security check.

All the above are just the top of my head. I think it's worth at least looking into the option of an extension and see whether it's feasible, and whether it's better than what we currently have and/or a gadget.

let's also check into making WWT into a MediaWiki extension.

It was my impression that services with external dependencies don't go in MediaWiki. Is that true? It is something I would be concerned about, indeed along with security review and general slowness/strictness associated with the production pipeline. I envisioned gadget deployment would be like MoreMenu or HotCat, where the code lives in one place and individual wikis who want it as a gadget will import it from Meta.

It was my impression that services with external dependencies don't go in MediaWiki. Is that true?

This is one of the questions we can answer as part of this investigation. Examples for or against this theory would be good to discover.

ifried updated the task description. (Show Details)

Adding new languages

According to the maintainers, adding new languages is mainly an issue of lack of manpower. We cannot be granted access to the WikiWho servers, and it seems there is no system for publicly contributing to their server configuration. I am told this is being investigated more, in hopes we can at least partially assist with adding new languages via pull requests on GitHub. This would still require a fair amount of commitment on our end, as we'd need to acquaint ourselves with their stack, and possibly do a lot of testing to ensure accuracy of the data. Overall, it seems that unless there is a Wikimedia-funded grant of sorts, it is unlikely the WikiWho team will be able to work on this independently.

The WikiWho algorithm is open-sourced, so we could instead host it and the data ourselves. I expect this to be an unrealistic amount of work for us in the short-term, however. It would be a slow process involving multiple teams and possibly acquiring new hardware.

Adding support for other namespaces

Similar to above. This is possible but requires manpower and/or funding that the WikiWho team doesn't have. I suspect this will be more time-consuming than adding new languages.

Making a WWT gadget or extension

There are two options here -- either a "global" gadget, or a MediaWiki extension. In both cases we will need to do some rudimentary browser testing. There are a few things in our code that don't work in IE 11, some of which are easily fixable. We aren't required to support IE 11 either (even as an extension), and I think it's fine if we don't so long as it's clearly explained in the documentation. It's possible there are compatibility problems with Safari too, but other modern browsers should be okay.

Global gadget

We were doing this before, and it seemed to work, so I think there's no technical barrier to restoring it.

The system could mimic MoreMenu or HotCat, where the code lives in one place on Meta and the applicable wikis will import it into their local gadget. Translations live as separate JS files, and the appropriate one is loaded based on user's interface language, falling back to English. In the Grunt build step for the gadget, we'd generate these JS files from the JSON similar to what we're doing now. We can use a simple Node script to sync the files with the wiki.

Advantages:

  • Easiest and quickest way to get WWT to non-Chrome/Firefox users. We don't need security review or any work involved with getting it into production, and we had a working system of gadgetizing that we can revive.
  • Easily QA-able.

Disadvantages:

  • Code could in theory be forked and the "global" version maintained by us isn't what is seen on the relevant wiki. This is low-risk I think, since other long-standing gadgets use this same system without issue.
  • The code that lives on-wiki will be the transpiled version of the ES6+ code, meaning it might be hard to read. I don't think this is a huge risk either, since we can write in the comments to refer to the original source code for reviewing and proposing changes.
  • Requires manual deployment, but this should be much more simple than deploying the browser extension.
MediaWiki extension

Advantages:

  • Allows us (and those with +2 rights) to have complete control of where the tool lives (e.g. only the applicable wikis) and the codebase itself.
  • i18n system is already in place (don't need to load all languages or write code to tell it what language to load).
  • We don't need to worry about the Content-Security-Policy headers (T135963) blocking users from using the tool (presumably they'll have to manually whitelist our API proxy, see T208188).

Disadvantages:

  • We possibly will need a separate code repository just for the extension (?)
  • Requires security review, along with time and resources to set up the extension and get it deployed to production.
  • Possibly no way to QA before deploying to production, since there is no WikiWho data for wikis on the beta cluster.
  • Currently, our use of ES6+ may not be acceptable for MediaWiki. I think we can just override the linting rules with our own, but this is probably discouraged.
  • Introduces a dependency on an external service in MediaWiki. I don't think this is strictly disallowed, but it doesn't feel right to me, personally.
ifried updated the task description. (Show Details)
ifried updated the task description. (Show Details)

Note: I've added some notes at the bottom, based on user feedback on the tool (just for the sake of record-keeping). However, they are not a part of the requirements of the investigation.

After some thinking, I unfortunately think that the extension route might actually not be feasible because of the external API request (I don't think we allow for any of that in extensions in production) but either way, a question -- @MusikAnimal

Is it possible to add some code to a gadget to make sure it either only run from the "central" place we host it at, or at least notifies the user (in console if nothing else) where it is sourced from? My concern with a gadget is losing control over the code in a way that will make it extremely difficult to maintain it and answer bugs. Eager users installing it on their wikis (or as a user script) and not having proper bug fixes, or, worse, trying to enable it on wikis that don't support it and having troubles we have to run after with potentially not knowing where they're from. Is it possible to be a bit limiting on a global gadget?

My second point you already talk about, which are things we will need to make sure support other browsers (IE11, Edge, Safari, Opera, etc). I don't think it's a huge piece of work but we shouldn't forget it. I think we have a couple of CSS rules that may need some vendor extension and, as you point out, definitely an extended browser test.

Is it possible to add some code to a gadget to make sure it either only run from the "central" place we host it at, or at least notifies the user (in console if nothing else) where it is sourced from? My concern with a gadget is losing control over the code in a way that will make it extremely difficult to maintain it and answer bugs. Eager users installing it on their wikis (or as a user script) and not having proper bug fixes, or, worse, trying to enable it on wikis that don't support it and having troubles we have to run after with potentially not knowing where they're from. Is it possible to be a bit limiting on a global gadget?

Hmm I'm not sure. Off of the top of my head I don't know how we'd do that.

What usually seems to happen is someone will look at say English Wikipedia's gadget (or wherever they discovered the gadget) and copy it to their wiki. In this case the code would be something like mw.loader.load('https://meta.wikimedia.org/w/load.php?modules=ext.gadget.WhoWroteThat');, so they would have actually done it correctly. It may not work for that language, of course, so we could show a message in the UI/console that the language isn't supported. As long as we link to the documentation in gadget descriptions, and in a comment at the top of the code itself, I don't think we'll have many problems. If we suspect there are unmaintained forks, we could find them with https://tools.wmflabs.org/global-search.

We don't need to worry about the Content-Security-Policy headers (T135963) blocking users from using the tool (presumably they'll have to manually whitelist our API proxy, see T208188).

Just to clarify, for extensions, there's various hooks that can be used to whitelist specific sources. The Special:CSPException opt out is only meant to be used by users when they enable gadgets. That said, typically calling out to external services from the JS side from an extension is not allowed in Wikimedia contexts for privacy reasons (among others).

Per standup I'm going to mark this as resolved. @ifried said she got the information she needed.