Archive for August, 2009
… few hundred megabits at a time
Posted by Domas Mituzas in wikimedia on August 28th, 2009
We’re sitting here in Wikimania Underground, where our “hacking days” at Buenos Aires happen. Seeing each other face to face can allow us to discuss much faster how we should approach some of possible changes, that could make the site faster for everyone.
We eliminated quite a few web response headers (which cannot be compressed, due to how HTTP works), especially some of large ones we are using inside the cluster to achieve better caching, or debugging information – causing few hundred megabit savings (it is difficult to know exact numbers, due to the nature of caching).
Also, we’re experimenting with trade offs in content compression – by choosing more expensive compression methods, we decrease size of transmitted pages by up to 15%, though doubling compression costs on our side. We still think that we may end up doing different levels of compression for different types of content (something what will be efficiently cached from anonymous users will have way higher relative wins).
Of course, we will have reduced bandwidth bills, probably more than the additional hardware to cover the change would cost us in resources.
Presentations from Wikimania and More
Many folks do not know, but we actually try to upload and make available all our presentations. Presently, you can see a list of them on our Wikitech wiki. You can follow this link to see them all.
Keep checking back, because the conference isn’t finished!!
Wikimania talk videos
Yesterday’s tech talks from Wikimania are online at our temporary video file staging location (Ogg Theora format). They should appear on Commons soon. :)
Update: Some of the movies have encoding problems; reencoded versions should be reposted within a couple days. Sorry!
Code, code, code away home
Hacking was cut short a bit on our first day at Wikimania yesterday due to troubles with the conference facility network :( but we did get a lot done:
- Some basic specs and programming interface for a configuration database have been hashed out — once implemented, this’ll give us infrastructure to start phasing out the more fragile parts of LocalSettings.php editing and making it much easier to manage both multi-wiki sites and one-off installs!
- Bunch of folks hashed out some details on getting our mapping servers set up and online
- Lots of general code review and poking!
Feel free to join us during the day — project page on Wikimania wiki and #wikimania-codeathon on IRC!
Codeathon now!
The fun is starting! Get your code on…
- Wikimania_Codeathon/Projects
- #wikimania-codeathon on irc.freenode.net
Weekly Wiki Tech Update: Pre-Wikimania edition
What happened?
A few highlights from the last week…
- The first version of Wikimedia’s official Wikipedia Mobile app for iPhone has hit the iTunes app store (free download). As with all Wikimedia software, it’s open source and we welcome patches and bug reports! (Unfortunately there’s a known problem with this release which prevents installation on first-gen iPod Touches — just use the mobile web site for now, which shares the same backend.)
- Test wikis with Flagged Revisions and ReaderFeedback configurations have been set up to shake down UI and workflow before we prepare to deploy these extensions on English Wikipedia in the coming weeks. The test sites have been populated with featured articles, and should be getting some decent front pages soon. ;)
- A push of new donation buttons to English Wikipedia to test response rates has been delayed until we’ve got more of our techs in one place again.
- Mark is completing performance testing of SSD-based Squid proxy servers.
- We encountered failures on ms2, one of our text storage servers, which has required some behind-the-scenes running about.
- Ariel and River are bringing media storage replication between our Tampa master and Amsterdam off-site copy back up to date after cleaning up most of the base configuration.
- There’s been a lot of talk on-list in the last couple weeks about testing infrastructure, with various people poking at the parser tests and the other half-done test suites. This is a happy thing and I hope to see more solid tests going — and more automated reports into CodeReview like the parser tests!
The week ahead…
Come with us if you want to code!
It’s Wikimania week in Buenos Aires, and the Wikimania Codeathon starts this Tuesday at 10am (15:00 UTC). If you can’t be there in person, join us online in #wikimania-codeathon on FreeNode.
What’s going to happen? We can’t say for sure ahead of time, but here’s a few of my favorites I hope to work on:
- Deploying updated MediaWiki code to Wikimedia sites!
- Exploring the power of jQuery and Michael Dale’s advanced media features in JS2 mode!
- Pushing LocalisationUpdate to fast-track the work of our tireless translators!
- Driving plans for the Usability Initiative’s new UI work!
- Bringing forth the power of Flagged Revisions!
- Awesome times with OpenStreetMap testing!
- Hashing out plans for a MediaWiki configuration database… down with LocalSettings.php once and for all!
- Figuring out how to start Selenium-based MediaWiki testing!
- …YOUR project?
Now put on yer codin’ legs and get typing!
Wikimania Codeathon in Buenos Aires next week
Wikimania’s classic “Hacking Days” event is back, and better than ever as the Wikimania Codeathon will be open throughout the entire conference this year.
Based on the success of April’s Developer Meet-up in Berlin, we’re starting with an “unconference”-style planning session to let attendees break out into common working groups 10am Tuesday, August 25 (note — this is the day before the main conference begins). The coding room will remain open throughout the rest of the conference, so folks can pop in and out between other sessions.
We’ll be in Room F at the Centro Cultural, which should be nicely spacious. Non-developers are welcome during the conference if you just need a quiet place to sit and check on your sockpuppet accounts. ;)
There’ll be a wrap-up presentation in Room 3 at 14:00 Friday, August 28 to give folks a chance to do mini-talks on what they’ve been working on.
If you’re planning to attend, either in person or virtually via IRC, please add yourself to the list.
Squashing the Bugs
Posted by Trevor Parscal in mediawiki, wikimedia on August 18th, 2009
Fresh off the whiteboard of the Wikipedia Usability Initiative
Over 40,000 users are now participating in the beta testing of new features for Wikipedia, and other Wikimedia projects. These users have been helping the Wikipedia Usability Initiative to find bugs, of which we are doing our best to squash quickly.
If you are a Firefox 3 user, your experience with the new features has most likely been without issue, but we have found some older browsers such as Internet Explorer 6 to be less stable, especially for projects which use a right-to-left language. After collecting information on browser compatibility and evaluating each browser’s capabilities against the technologies we are using to achieve a better editing experience, we’ve designed a map of which browsers support the new editing tools well enough to ensure a quality user experience, and which do not. Where users’ browsers do not support the new editing tools the old editing tools are provided instead.
While we hope that we can extend support to some of the currently unsupported browsers, we are focusing our efforts on developing a richer browsing and editing experience for users of more modern browsers. Thank you to everyone who has been participated so far in our beta testing process!
– Trevor Parscal (trevor @ wikimedia.org)
Software Developer, Wikimedia Foundation
Weekly Wiki Tech Update
Got your coffee on? Good!
Let’s quickly review last week’s excitement and what we’ve got planned for this coming week…
Looking back:
- Ops magnate Mark Bergsma has completed his thesis and is able to come up for air again. Welcome back Mark and congratulations!
- Mark has been testing performance of Squid proxies backed by solid-state disk drives… so far so good!
- Ariel has continued background cleanup of our media fileservers to prevent recurrence of the performance problems we saw last month.
- The Wikipedia Usability Initative team has rolled out their opt-in beta program to get more people trying out the updated site theme and edit toolbar.
- We’ve announced plans to form and hire a new Wikimedia CTO position to free up Brion for more development time as Senior Software Architect
- Flagged Revisions extension is being tested on Help and Manual pages for www.mediawiki.org
Coming this week:
- The Wikimedia Foundation board election finishes tonight at midnight UTC! Go vote while you can…
- A testing configuration for Flagged Revisions on English Wikipedia will be deployed on a test site soon!
- Planning to bring the site up to date with development trunk by the end of the week…
- Tomasz is in progress getting our data dumps mirrored to Amazon’s public data sets
Upcoming events:
- August 25-28: The annual Wikimania conference in Buenos Aires will include a code-a-thon developer sprint — join us in person or via IRC!
$cto[] = clone $brion;
Back in 2005, Wikimedia brought me on as the Foundation’s first paid employee after two years leading MediaWiki development as a volunteer. Naturally as the only member of the tech staff, I started at the top: Chief Technology Officer.
In the 4 years since, we’ve gone from one tech employee to a dozen, from 50 servers to 350, from upstart novelty to established web juggernaut.
As our operations and our staff have grown over the years, so have my responsibilities. Beefing up our tech staff is in some ways just like adding servers to our data center — we can get more things done with less task switching, but scaling still has its overhead.
With the increase in administrative and organizational duties, I’ve been less and less able to devote time to the part of the job that’s nearest and dearest to me: working with our volunteer developer community and end users — Wikimedians and other MediaWiki users alike — who have bugs, patches, features, ideas, complaints, hopes and dreams that need attention.
The last thing I want to be is a bottleneck that prevents our users from getting what they need, or our open source developers from being able to participate effectively!
Multicore brain upgrades aren’t yet available, so to keep us running at top speed I’ve suggested, and gotten Sue & Erik’s blessing on, splitting out the components of my current CTO role into two separate positions:
As Senior Software Architect, I…
- maintain the MediaWiki development roadmap
- give timely feedback and review on feature ideas, patches and commits
- ensure that end-users and bug reporters are treated respectfully and that their needs are met
- get developers & users involved and talking at local and worldwide events as well as online
- represent the “face of the developers” interacting with our user community (both Wikimedians and third-party MediaWiki users)
As Chief Technology Officer, I…
- set high-level strategic priorities with the rest of WMF
- handle administrative management for the Wikimedia Foundation’s technical department & internal IT
- budgeting
- vendor relations & purchase approval
- hiring & personnel details
- work with the fundraising side of WMF to seek out and make use of potential resources:
- grants for projects we need
- in-kind donations of infrastructure
- sharing development work with like-minded orgs
- ensure that the operations team has what they need to address current and predictable future site needs
- ensure that the developers have what they need and are coding smoothly
- plan and implement internship programs and volunteer dev events both on-site and elsewhere
I’ll continue to act in both roles until we’ve found a satisfactory candidate to fill the CTO position (full job description will go up soon), at which point I’ll be freed up to concentrate on being a full-time Senior Software Architect. (Yes, I’ll review your patch!)
I will of course continue to work closely with our eventual CTO… the idea is to find someone who’ll make the decisions I would have wanted to if I only had time. ;)
– brion vibber (brion @ wikimedia.org)
CTO and Senior Software Architect, Wikimedia Foundation
San Francisco
Try the usability beta!
Have you noticed the “Try Beta” link on the top of Wikimedia project sites? The usability team is proud to introduce the new skin, Vector, and the enhanced toolbar. Well, they have been available from user preferences over a month now, but we wanted to reach out to anonymous users. Please check it out and let us know your thought, if you haven’t tried already. We deployed a bunch of bug fixes since the release on July 1st. Right-to-left languages are fully supported, and you will see integrated special characters in the tool bar. If you are a administrator of Wikimedia project sites, we would like to ask you a favor. Please consider removing the special character menu below the editing dialogue box in edit page. As special characters are easily accessible from the toolbar, it will be good to reduce the duplication to make the page simplified. The special character in the toolbar has generic universal set. If you find missing character sets or have a request, please let us know through the project page or file an enhancement request via bugzilla under User Interface.
Arigato!
Naoko Komura
Usability Initiative
"Try Beta" screenshot
Special Characters screenshot
Amusing Mobile Feedback
Posted by Hampton Catlin in wikimedia on August 3rd, 2009
“We have just launched this site… please send us your feedback!” When we added these words to the top of 1/5th of the mobile pages, we began a barrage of feedback to the mobile list. So much so that, we had to make a second list just for this kind of feedback. That certainly wasn’t unexpected and my fiancee, Michael Lintorn, has been fantastic about responding. He’s spending his summer answering about 150 messages to the list a day, ensuring that everyone knows that we are actually human and making sure we get all the weird bugs taken care of.
What I didn’t expect, was how many people absolutely, totally misunderstand the link. I would change the link to be more clear, but I’m not really sure how I can make it more clear without it just being a list of caveats the average person wouldn’t read.
However, the silver lining here is that we get a whole bunch of really hilarious messages where people are obviously not understanding A) What Wikipedia is, or B) What planet we are on.
Enjoy.
Hey I’m looking for a pic of the rood inverse with the inscriptons around it, if you can e-mail it to me I would really appreciate it. Thanks guys and keep up the good work
Oh, yes sir! We’ll get right on that.
Hey razor.I was just watching some boxing on tv and I thought about you.do you remember me from the time you bought your rv at independence rv in florida.my name is ### and I was the one who got your rv ready.as I remember I think you came back and traded 2 or 3 times in a row.you asked me if I liked working on rv’s and I made the joke. I guess it beats getting the crap beat out of you for a living.haha.any how I hope that large family of yours is well and your invention works out.take care
Razor, if you are out there. Please contact this person. He misses you.
This is great can you tell me where I can buy the whole series
Oh, its on Aisle 3.
Can’t find def for bafoone
Try another spelling.
Love your show. My wife & I watch every weekday.
Thanks for being a great great great great great American!
Yes, in my spare time I’m *am* a TV producer. Its one of the many hats we all wear at Wikimedia.
Best of luck this Friday.
Bob the window cleaner (just a normal guy like you)
Us regular guys have to stick together. And thanks for the goodluck wish. I could use it!
Why did he get Fired now will watch only ###LOCAL#STATION### news
Local news is the WORST. Politics, politics, politics.
I was looking up some health related stuff and would have preferred to
have a way of not being forced to see it. For example, when you go to
the page about jaundice, you imediately see a picture which looks
creepy when it fills up the small screen…
I do agree that all gross diseases should only have pretty people in the example pictures.
I would like to know more about the high sugar content caused by
cortisone injections and whether or not it’s reversable by diet. The
cortisone is for arthritis which many people suffer from. I was hoping
to get a more detailed info pack for this with diet tips. I only had
three large doses but it has raised my sugar very high and I want to
counteract it. Would be glad if some of this was on site as I know you
did mention cortisone. However the site was a bit involved and I could
not really decide into wwhich category I was fitting!!! Thanks though
for a generally helpful site
Our medical staff will be back to you with this advice shortly.
Anyhow, I hope some of these made you smile!