All the other things we changed in addition to Wikipedia’s look

This week my team reached an important milestone in our biggest project — the deployment of a new look for English Wikipedia. This was the most significant change to the desktop site in over a decade.

However, far more has changed than just appearance, and in this blog post, I hope to shed some light on what we’ve achieved and the people that did it. [1]

Since the first patchset back in January 2020, the redesign of Wikipedia’s desktop site has led to various improvements to our codebases and tooling.

I’m sure many would smirk at the idea a redesign could take three whole years, rather than three months, but much of this time has been consulting with various partners and refactoring the code that has evolved over twenty years. We’ve followed an agile approach to development, incrementally building out a grand vision set by our designers. Our first iteration, to our Basque language Wikipedia, was in fact deployed as far back as July 2020. We’ve built responsibly, intentionally, and strategically, thinking holistically about the entire behemoth that is our codebase. When I look back at our release timeline at all the things we did as a means to an end of this project, it’s incredible to see what’s been achieved over three years.

We became better at disagreeing

I’ve noticed huge improvements in the way Wikimedia Foundation works through internal impasses. As an engineer, I am so grateful that this project had technical program managers supporting us throughout our daily rituals and helping us overcome any obstacle that was thrown our way. Whether it was throwing people in a room together or just asking the right questions, or negotiating conflicting strongly held opinions, we found solutions to make big impactful changes whether it was using Vue.js or growing trust with each other. Among others, thank you to Jazmin, Lani, Suman, Nat, and Daniel for all your help here.

We worked effectively with our editing community

We’ve strengthened our relationship with our editing communities.

Although we deployed to English Wikipedia in January 2023, the new look and feel has been the default for many of our other projects, in far less complete form for a much longer time.

We owe a tremendous amount to our pilot communities and their feedback as we’ve iteratively built out the design. I am astounded by the quality and respectful feedback many volunteers were able to give us, and by the lengths our team has gone to describe our motivations and goals. For example, the limited width improvement for reading has been a controversial feature ever since we introduced it, and when we realized there was no accessible documentation for why something which had been of obvious importance to us, we wrote documentation ourselves. My work colleagues have organized and executed various office hours, some of which I’ve attended, which often although only scheduled for thirty minutes have massively overrun due to the enthusiasm and engaging conversation we’ve had. Kudos to Alex, Szymon, Olga, Phuong, Martin, Letizia, Sherry, and all the volunteers with their feedback that helped us tweak the new design.

We improved our frontend development practices

Our codebase was very much a backend-first piece of software, and working with frontend code historically has been a little lackluster, with a lack of support for modern JavaScript, a lack of industry-standard build tools, no frontend framework, and lockin to a myriad of less-desirable libraries such as JQuery.UI, moment and homegrown libraries.

One big challenge, we’ve faced historically has been deciding on which of the many JavaScript frameworks to use in our stack. This had a thorny decision to make, but with good management, and a process for discussion and discourse, we agreed that we should use Vue.js going forward for frontend development. We also found ways to support the writing of ES6 code without tooling, and our designers have been working with teams across the oranization to build a design system toolkit.

Since our initial pilot experiment of Vue.js, which is used in the new search experience, our toolkit has grown, incorporating views from a variety of teams and projects. Thanks go to Eric, Anne, Volker, Roan, Nick, Jan, Stephen, Alex, Barbara, Nat, and many others for trailblazing this tremendous effort.

We made improvements to backward compatibility

As I wrote previously in 2018, when we update the design of our site, old versions tend to stay around for a very long time. This traditionally hasn’t come for free, and historically often small things, such as markup changes, and removing CSS, require changes in multiple versions that can result in dozens of patches.

We’ve had a huge over-reliance on markup, where changing the HTML structure or an ID broke key integrations, such as unstyled menu items. To allow us to make the big changes we had planned, and reduce the burden on developers, we created a shared data and presentation layer using templates. This resulted in a 20% decrease in code maintained by my team.

To counter this we added APIs and tightened contracts where code integrates. For example, we now have stable methods for modifying the page subtitle and adding items to menus with icons in a consistent manner.

We’ve broken features many times during the project and I appreciate the patience and forgiveness of developers as we’ve tried to prevent that from happening again. Thank you, Leon, Ed, Roan, Ammarpad, Clare, Bernard, Mo, and Bartosz for all your help and patience here.

We improved language support

Language support in the new version is far more prominent than it historically has been and appears beside the article title. Making this feature more prominent invited considerable feedback from our community and required working closely with our language team, and distracting them from their own projects. Thanks to Santhosh, Pau, and the language team for working closely with us!

We simplified APIs and removed code paths

As we approached what looked like straightforward changes, such as adding items to menus, we met an unnecessarily high level of complexity. In the case of menus, there were over 5 different APIs for code to add an item to a menu. We simplified a lot of that for our own sanity. Thanks to Ammarpad, Clare, and Mo for your efforts here.

We retained a good level of accessibility

The old desktop look and feel actually enjoyed very good accessibility so we wanted to keep this standard going into the new redesign. We were lucky to benefit from not one but three engineers who specialized in this field and kept us accountable throughout the project and were fortunate enough to receive lots of feedback from users and experts in the field. Kudos in particular to Bernard, Jan, and Volker who’ve all been wonderful, and to the managers that have enabled them. Bernard in particular organized some very useful feedback from the American Foundation for the Blind.

We built with performance in mind

We’ve improved our performance. We’ve removed unused CSS and limited CSS to pages that need it and we’ve standardized thumbnail traffic. We’ve monitored big changes such as the switch to Vue.js and a new API for search. Thanks in particular to Volker, Nick, and Timo.

We restructured our Wikitext markup language

For the new table of contents experience, we needed to restructure Wikitext. In the old version, the table of contents was part of the parsed article HTML content rather than metadata that could be rendered elsewhere. This change in particular was a tricky change to make while maintaining the older style version. This hopefully serves as a blueprint for a major restructuring of article HTML in the future. Thanks to the content transform team — in particular Subbu, C Scott, and James.

We improved third-party support

Our software is run by various websites that are not Wikipedia and many of these look different from Wikipedia, using plugins known as skins. When we began the project various skins for our third parties were broken, unmaintained, or more complex than necessary due to limitations in the existing architecture and shared similar challenges to the ones that were blocking our goal. Auditing these helped us predict challenges we would face. There are now 101 skins that our 3rd parties can use with much better support and we have better systems in place for backward compatibility. You can explore these beautiful creations at the Skins lab and even make your own. Thanks to all the skin developers for their patience, engagment and insight: Mainframe, Alistair to name a few.

We started to monitor visual regressions

About a year into the project we were noticing lots of visual regressions sneaking into our releases, which slowed down our overall velocity.

To counter this we created a visual regression framework and incorporated it into our deployment process. This tool is now starting to be used outside the team. Kudos to Suman and Nick for most of the work here.

For more information on this, see Visual Testing: Building A More Robust Wikipedia Interface By Spotting the Differences. [2]

We started to log JavaScript errors

At the start of the project, we had no JavaScript error logging. We now do. Thanks to Jason L and Daniel for your support here.

We improved the mobile experience

While we didn’t quite get as far as we would have liked with this. The new desktop skin is more responsive than the old one. Mostly this has been an issue of prioritization — we have a perfectly capable mobile site right now, but the choice is important, and we’re keen to at least have a viable skin.

Meanwhile, our mobile traffic grows, so we’ve kept one eye open at all times. We’ve brought the technology stack for the mobile site closer to the desktop site and vice-versa. Features previously absent on the mobile site, are now available to use on mobile and some of the best ideas on mobile have been merged with the desktop versions. There are new projects over the next year that look at improving our diff pages.

Thanks in particular to Bernard, Jan, Bartosz, and Jason S here.

… but we’ve still got work to do

Importantly our work is not done. There are several imperfections that we had to deprioritize or delay. In particular:

• The new desktop skin is not as responsive as we’d like it to be. This wasn’t prioritized as we have a mobile site and much of the challenge here lies in working with our content creators to make article content responsive (in particular large tables).
• The new skin is not as performant as it could be. We’re currently shipping more than 2kb of CSS than we need to, for example, due to the way we roll out features, and overall the site’s critical CSS is a little larger than before. The flip side of this is added functionality that should make the site more useful, and a design that degrades on older browsers
• The font size is smaller than we’d like it to be because changing it at this point would create significant work for our editors.
• There are still remaining issues with accessibility on the site. We’ve received consultancy and believe we have identified those issues and continue to work towards fixing them.
• Several blogposts about our work are still being written. [3]

I look forward to improving these imperfections and more. Most importantly because of the last three years, I feel we have a much better foundation to work from now. The next improvement to the desktop experience will surely not take another decade.

Footnotes

[1] Apologies if I have missed out any of the village. If I have, please let me know and I’ll happily and apologetically credit you in this post.

[2] This was updated from the original blog post on 19th January to add a link to Nick’s blog post.

[3] I will be updating this post with links to these so please check back in a week or so!

This blog post is available on my personal blog as well as on Medium.com.

Thanks Alex for the last minute technical writing support!