Barely a week after Wikimedia’s volunteer-driven email response team helped Wikipedia score number one in the American Consumer Satisfaction Index’s “Internet and Social Media,” survey, the software being used to make this happen got its first upgrade in four years. This happened thanks in large part to a generous donation of time and service by Martin Edenhofer’s consulting firm Znuny GmbH.

Emails to Wikipedia and the Wikimedia Foundation’s ten other free knowledge projects are processed by free software called the Open-Ticket Request System or “OTRS.” OTRS was developed in 2001 by Martin Edenhofer and has been used by the Wikimedia Foundation since volunteers began responding to emails in 2004. Until this week, the email response team used an OTRS system that had not been upgraded since 2009, running version 2.4. The sheer volume of emails Wikimedia has received  (nearly a million emails are stored from the last 9 years!) made the feasibility of an upgrade seem to be an impossible task.

Commenting on a bug that had been filed in 2010 to request a software upgrade, Martin Edenhofer himself offered his OTRS consulting company to assist with the huge task of the upgrade as a donation to the Wikimedia Foundation. Fast forward to a little over a year later and all the pieces were put in to place by the Wikimedia Foundation’s Maggie Dennis. Znuny’s Marcel Fratczak and the Wikimedia Foundation’s Jeff Green then spent several long days upgrading Wikimedia’s OTRS system to the latest version (3.2.9) and migrated the database to a new server.

Thanks to this great upgrade to the tool we use for responding to emails from the general public, the Wikimedia volunteers who process emails tirelessly behind the scenes have the ability to work much more efficiently and effectively to make sure that the high quality of personal customer service continues.

Philippe Beaudette
Director, Community Advocacy, Wikimedia Foundation

Major news in July include:

• Giving more editors an easy-to-use editing interface (the VisualEditor) on several Wikipedias
• Improving language support on our sites via summer interns’ projects and easier configuration options, and asking for help translating the VisualEditor interface
• Enabling users to edit our sites from mobile devices, like phones and tablets, and announcing a future user experience bootcamp focusing on mobile editing
• Finishing our transition from keeping source code in Subversion to storing it in Git
• Launching a Wikipedia Zero partnership with Aircel, giving mobile subscribers in India the potential to access Wikipedia at no data cost
• Updating the Wikimedia movement on how we intend to protect our users’ privacy with HTTPS
• Signing a contract with longtime MediaWiki contributors to manage MediaWiki releases for the open source community
• Explaining how we find and gather software problems and deliver the fixes to users

Note: We’re also providing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge.

(more…)

Wikimania 2013 Hong Kong

Wikimania, the largest gathering of the Wikimedia world, is just around the corner. In less than a week, hundreds of volunteers will be gathering in Hong Kong to share stories and talk about grand plans for the year ahead. The Language Engineering team of the Wikimedia Foundation also plans to join in! The team has been working consistently on improving internationalization support in MediaWiki tools and in Wikimedia projects. At Wikimania 2013, the team will be present to talk and exchange ideas about these projects.

Team members will be presenting technology sessions, organizing a translation sprint and showcasing at the DevCamp. At the Translation sprint workshop, participants will get a quick introduction to using the Translate extension on Wikimedia wikis and translatewiki.net. The Translate extension provides MediaWiki with essential features needed to do translation work. It can be used to translate simple content, wiki user interfaces and system messages.

In an interesting design session, our team’s interaction designer Pau Giner will be presenting on the challenges and complexities of designing language-conscious user interfaces. In the talk titled Improving the user experience of language tools, he will look at two significant projects developed by the team: the Universal Language Selector (ULS) and Translate UX as case studies.

Team engineers Amir Aharoni and Niklas Laxström will discuss multilingual feature enhancement possibilities for handling multimedia content meta-information on Wikimedia Commons. Multilingual Wikimedia Commons – What can we do about it? To find an answer, they will touch upon the advanced features in Translate, ULS and other tools that can be used to translate images, templates, descriptions and other graphics metadata to make Commons a truly multilingual Wiki.

A technical session, MediaWiki i18n getting data-driven and world-reusable, on improvements in MediaWiki internationalization (i18n) will be presented by Santhosh Thottingal and Niklas Laxström. They will be talking about the preference of using data-driven approach in place of custom code which helps code maintainability across multiple i18n frameworks. An example is data usage from the Unicode Common Locale Data Repository (CLDR). They will be speaking about the challenges and benefits of maintaining two code bases MediaWiki JavaScript i18n extension and the jquery.i18n library for different developer audiences.

Language team members will be part of panel discussions covering WMF Agile practices and at the Ask the Developers session. Come meet us at any of these sessions and bring your toughest language software questions along!

Complete list of Language Engineering sessions.

Runa Bhattacharjee
Outreach and QA coordinator, Language Engineering, Wikimedia Foundation

This post is available in 2 languages:

中文 • English

English

The Wikimedia Foundation believes strongly in protecting the privacy of its readers and editors. Recent leaks of the NSA’s XKeyscore program have prompted our community members to push for the use of HTTPS by default for the Wikimedia projects. Thankfully, this is already a project that was being considered for this year’s official roadmap and it has been on our unofficial roadmap since native HTTPS was enabled.

Our current architecture cannot handle HTTPS by default, but we’ve been incrementally making changes to make it possible. Since we appear to be specifically targeted by XKeyscore, we’ll be speeding up these efforts. Here’s our current internal roadmap:

1. Redirect to HTTPS for log-in, and keep logged-in users on HTTPS. This change is scheduled to be deployed on August 21, at 16:00 UTC.
2. Expand the HTTPS infrastructure: Move the SSL terminators directly onto the frontend varnish caches, and expand the frontend caching clusters as necessitated by increased load.
3. Put in engineering effort to more properly distribute our SSL load across the frontend caches. In our current architecture, we’re using a source hashing based load balancer to allow for SSL session resumption. We’ll switch to an SSL terminator that supports a distributed SSL cache, or we’ll add one to our current solution. Doing so will allow us to switch to a weighted round-robin load balancer and will result in a more efficient SSL cache.
4. Starting with smaller projects, slowly soft-enable HTTPS for anonymous users by default, gradually moving toward soft-enabling it on the larger projects as well. By soft-enable we mean changing our rel=canonical links in the head section of our pages to point to the HTTPS version of pages, rather than the HTTP versions. This will cause search engines to return HTTPS results, rather than HTTP results.
5. Consider enabling perfect forward secrecy. Enabling perfect forward secrecy is only useful if we also eliminate the threat of traffic analysis of HTTPS, which can be used to detect a user’s browsing activity, even when using HTTPS.
6. Consider doing a hard-enable of HTTPS. By hard-enable we mean force redirecting users from HTTP pages to the HTTPS versions of those pages. A number of countries, China being the largest example, completely block HTTPS to Wikimedia projects, so doing a hard-enable of HTTPS would probably block large numbers of users from accessing our projects at all. Because of this, we feel this action would probably do more harm than good, but we’ll continue to evaluate our options here.
7. Consider enabling HTTP Strict Transport Security (HSTS) to protect against SSL-stripping man-in-the-middle attacks. Implementing HSTS could also lead to our projects being inaccessible for large numbers of users as it forces a browser to use HTTPS. If a country blocks HTTPS, then every user in the country that received an HSTS header would effectively be blocked from the projects.

Currently we don’t have time frames associated with any change other than redirecting logged-in users to HTTPS, but we will be making time frames internally and will update this post at that point.

Until HTTPS is enabled by default, we urge privacy-conscious users to use HTTPS Everywhere or Tor [1].

Ryan Lane
Operations Engineer, Wikimedia Foundation

[1]: There are restrictions with Tor; see Wikipedia’s information on this.

(more…)

The VisualEditor beta release is being gradually rolled out to all Wikipedia editors in all languages. This is one the most exciting developments in the history of Wikipedia, because it will make editing the site accessible to the general public, rather than just to the people who have the patience to learn Wikipedia’s arcane markup language.

To make this accessibility really complete, however, the VisualEditor’s user interface needs to be completely translated to all the languages in which there is a Wikipedia. Its interface includes over a hundred new strings, and if they aren’t translated, they will appear in a foreign language on that Wikipedia (i.e. English text on Polish Wikipedia).

Take a look at the translation statistics for the VisualEditor. As you can see, the translation to a lot of important languages is far from complete or entirely absent: Arabic, Portuguese, Hindi, Swahili, Hungarian, Bulgarian, Tagalog, Urdu, Lithuanian, and many others. If you know a language in that list and the translation to it is not at 100 percent, please click the language name and complete the translation. (You’ll have to create an account at translatewiki.net, if you don’t have one already.)

The article “Vilnius” in the Lithuanian Wikipedia, being edited in the VisualEditor. Note that most of the buttons are written in Lithuanian, but the buttons on the toolbar are in English: “Edit source”, “Page settings”, “Cancel”, “Save page”, “Paragraph”. These buttons weren’t translated yet, so they are unusable for people who don’t know English.

Even if the translation to your language is currently complete, please check your language’s page every few days—the VisualEditor beta is in very active development, the messages to translate are updated literally every day, and you want your language to be at 100 percent all the time.

This is also an opportunity to thank the hundreds of translatewiki.net contributors, who work quietly, but persistently, and make MediaWiki and its extensions into one of the most thoroughly localized pieces of software ever.

If you haven’t joined the translatewiki.net community yet, you are very welcome!

Amir E. Aharoni
Software Engineer, Language Engineering team, Wikimedia Foundation

At a Cooper boot camp

The Wikimedia Foundation Product & Design team is excited to announce that Wikipedia mobile editing will be the case study for an upcoming Cooper UX bootcamp.

Cooper, a design and strategy firm based in San Francisco, is organizing a four-day intensive workshop to teach user experience methodologies to designers, developers and product managers who have product design experience but want hone their skills further.

The bootcamp will focus on effective collaboration, product ecosystems, design research methodologies, creating empathetic personas, storyboarding and designing for engagement rather than just features. Specifically, students will look at making mobile content contribution more approachable, intuitive and less reliant on traditional input methods like typing, as well as exploring other areas of interest.

Previous UX bootcamps have worked with the Edible Schoolyard Project, Marketplace Money, and Fair Trade USA, among others. We’re looking forward to the ideas that are generated from having a great group of bright minds in one place, focusing on complex real-world problems that our users face every day.

The class is built around hands-on activities conducted in pairs and small teams, and will be held in Petaluma, CA, from September 17-20, 2013.

More information about Cooper UX bootcamps is available on their site.

Jared Zimmerman
Director of User Experience, Wikimedia Foundation

(more…)

MediaWiki, the software that powers the Wikimedia movement sites, is a remarkable piece of engineering. Not only does it support the very specific use cases of the various projects (Wikipedia, Wikimedia Commons, Wikisource, etc), but it is also used by many other individuals and organizations spanning the entire range of institutional size, from the biggest multinational firm to the smallest boutique site. Those third-parties (users outside Wikimedia) are an important part of the MediaWiki ecosystem, as they provide added testing and development time to the project.

Today, the Wikimedia Foundation is pleased to announce that we have contracted with two long-time members of the MediaWiki community–Mark Hershberger and Markus Glaser–to manage and drive forward third-party focused releases of MediaWiki.

Over a month ago, the Wikimedia Foundation sent out a request for proposals (RFP) to help us fill an important and underserved role in the development of MediaWiki. Two very solid proposals were produced, the community weighed in with questions and comments, and there was an open IRC office hour with the authors and interested community members. The Wikimedia Foundation is pleased with the outcome of this RFP and excited to begin this new chapter in the life of MediaWiki.

Mark and Markus bring a wealth of knowledge to this endeavor, as they are both MediaWiki contractors helping others set up, use, and do unique customizations of MediaWiki on a daily basis. They certainly know what third-party users of MediaWiki want.

“We are excited to work with the Foundation to enable the community of developers to respond in a more agile way to third-party MediaWiki users,” Mark said about the opportunity. ”Together, we will develop the next generation of MediaWiki software and build strategic and lasting relationships with Open Source organizations and third-party wikis.”

Mark and Markus will be working on various things, especially leading the efforts to make new tarball releases of the software, to improve the continuous integration infrastructure, to shepherd through changes to extension maintenance, and to collaborate with others as they add documentation for third-party users. All of their progress will be documented on the MediaWiki wiki for others to follow along and help.

Please join me in congratulating Mark and Marcus. We’re very excited to see what they will accomplish!

Greg Grossmeier
Release Manager, Wikimedia Foundation

Have you ever looked up something quickly on Wikipedia on your phone, noticed a small mistake, and wished you could fix it on the spot? Or maybe you didn’t realize that you could contribute? Now you can help keep Wikipedia and its sister projects up-to-date and accurate when you’re on the go by editing from your phone.

Wikipedia’s quality content is built by ordinary people all over the world watching and editing articles every day. Anyone with a computer can edit, but with over 15% of our users accessing Wikipedia on mobile devices and growing, the Wikimedia Foundation had to do more to let anyone with Internet access contribute to the sum of all human knowledge. For this reason, we’ve just released a new feature: editing for mobile.

As a trial, the first version of mobile editing requires a Wikimedia account. To get started, look for the pencil icon. If you don’t have an account, don’t worry, it’s quick and easy to create one on desktop or mobile! Once you’re logged in, you can use the pencil next to any section to make changes to that section.

For our first release, our primary goal was to create a fast, intuitive editing experience for new users and experienced editors alike, while still sticking with markup editing for now. We started simple so we could observe our users’ needs and expectations.

We’ve already seen an encouraging number of users try out editing on our experimental beta site, where we first built the feature. Now that it’s available for all users, we hope to learn more about the kinds of edits people make on mobile and build more advanced features, including possible VisualEditor integration, in future releases.

We spent a lot of time testing the new mobile editor and trying to make it work on as many devices as possible, but this is still the first release. We would love to hear your feedback, any suggestions you have, and any problems you encounter, so we can continue to improve this feature. Tell us what you think on Twitter @WikimediaMobile.

If you love Wikipedia and have an Internet-enabled phone, give mobile editing a try. If you’ve never edited Wikipedia before, now is a great time to start – it’s quicker and easier than ever before. Help us make Wikipedia and its new mobile editor better!

Juliusz Gonera
Software engineer, mobile web

Wikipedia Zero is now available in India

It is our mission to provide free access to knowledge for everyone in the world. It’s only fitting then that today we announced our first Wikipedia Zero partnership launch in India, the world’s second most populous country with over 1.2 billion people.  Our new partnership with Aircel will give 60 million mobile subscribers in India the potential to access Wikipedia at no data cost, bringing the program’s global partnership footprint to 470 million subscribers.

While mobile penetration in India is over 70 percent (867 million subscribers), the total Internet audience of India is only 77 million people (Comscore, June 2013), roughly the same as Japan (73 million, Comscore June 2013) – a country with 1/10th the population.  Infrastructural and economic barriers in India, where income per capita is 1/30th that of Japan, have led to this divide in information access.  However, the proliferation of mobile – and programs like Wikipedia Zero – will change that. Already, India has passed Japan to become the third largest smartphone market in the world.

The challenge in enabling knowledge access in India is not just about distribution and cost, though; it’s also about language. India has no national language, but there are 22 recognized official languages in the country. Many Indians are not only accessing the internet for the first time on mobile, but also non-English content is becoming accessible for the first time via mobile.

Hindi Wikipedia, for example, currently has 22.1 percent of page views globally coming from mobile compared to 17.3 percent for all other languages ^.[1]  We hope to further catalyze this transformation as Wikipedia users on Aircel can access English, Hindi, Tamil or any of the other 17 Indic language Wikipedias without being charged data fees ^.[2]

To meet our commitment to bringing free knowledge to everyone in the world, we need to break down the barriers that prevent access. With Wikipedia Zero officially available today on Aircel in India, we’re one step closer to that objective.

Amit Kapoor
Senior Manager, Mobile Partnerships, Wikimedia Foundation

Notes

1. From stats.wikimedia.org, June 2013: Hindi Wikipedia page views (2.1M mobile /9.5M all); Wikipedia worldwide (3,672M mobile/21,229M all)
2. A phone or browser must have the capability to render an Indic font in order to access non-English versions.
   

Wikipedia isn’t written just by humans! Bots have made great contributions to Wikipedia and other wikis like Wikidata (where bots have made over 90 percent of the edits so far). “Bots” are automated editing programs that can do anything from archiving discussions to reverting vandalism and  creating articles. Some bots even patrol new pages and report to Wikimedians.

How can you operate a bot? There are several ways and frameworks to run a bot, but the most popular is Pywikipediabot. It’s written in the Python programming language and has been in use since 2003. Pywikipediabot contains scripts for moving categories, creating articles, checking new images, working with Wikidata items and many other tasks. Besides the existing scripts, you can also create your own bot using generic scripts and classes (like “Page,” which handles Wikipedia pages in general).

Pywikipedia is now joining many other Wikimedia-related software tools and taking a big step forward by changing its version control system from SVN to Git. After July 26, developers will be able to easily submit their patches directly to Gerrit to help maintain the code.

If you’re already a bot operator, you should know that after that date, SVN checkouts won’t be updated and you’ll need to switch to git; we’re providing a manual to help with the process.

If you’re interested in working with us, there are several help pages in different languages. You can also contact us through our mailing list and the IRC channel.

Amir Sarabadani (User:Ladsgroup), editor on the Persian Wikipedia and pywikipedia developer