Fun with Subtitles
Posted by Michael Dale in wikimedia on November 9th, 2009
You can select your subtitle language
We have just finished up the Multimedia Usability Project Meeting here in France. I am sure there will be more general wrap up coverage of the meeting shortly… but I wanted to share a hack we worked here.
For a long time people have requested subtitle support for mediaWiki embed videos and at this meeting we were finally able to sit down and hack up an initial solution. The system works by putting the srt files into the wiki so people can collaborate on translation and contribution. The naming scheme is “TimedText” followed by the file name followed by the language code followed by .srt . We include a basic editor to upload srt files and switch between between languages. We presently display the subtitles on the side of the video but they should make there way below the video soon. There are lots of supporting projects to work on if anyone is interested ;)
How do I try it out
To try it out install the mwEmbed gadget and then visit either video linked to in this post. Hopefully we can produce some more documentation soon :)
Some quick ideas for enhancements ( I am sure you can think of some too) :
- A translation interface maybe borrowing from the techniques used on translatewiki.net
- A simple search of subtitles from the current video
- A more complex search system for subtitles across all videos
- Timed metadata a-la-metavid
subtitles on commons using the mwEmbed gadget
Localization Update: As mentioned in the comments we are still missing some of the localizations msgs. They should be making their way in there soon, along with some other updates.
Usability Beta Enhancements
New watch/unwatch and cascading tabs
Thanks for trying out the usability beta. Close to 280,000 people tried out the usability beta and about 220,000 people continue using it. One of the most-frequently reported frustration about the beta was that (1) watch/unwatch was hidden under the drop-down menu and (2) tabs overlaps when the browser is scaled down or the resolution is low. Tab issues were more significant for language communities whose words tend to be long such as German. We are also simplifying the search box to simplify the interface and in order to minimize the overlapping side-effect. Those new features are currently staged on the prototype wikis in English, German, Japanese, Arabic, Serbian, Russian, and Sinhala. You need to log-in in order to see watch/unwatch tabs. The screenshot on the right is the shot when the browser window is scaled down to 640×480. “View History” is moved to drop down menu in order to keep “Read” and “Edit” tabs visible. Let us know your feedback here.
- Naoko
Update on November 18th: Thank you for your support and feedback. These features are now available in the usability beta across all Wikimedia projects.
Wikipedia Mobile October Update
Posted by Hampton Catlin in mobile, software on October 12th, 2009
We have a lot of cool stuff that got pushed out today. I’m really excited to tell you about it. First of all, Derk-Jan Hartman (github: hartman) has been hard at work bringing us NetFront support. NetFront is the HTML rendering backend to many devices. Here is a list of the devices that Derk’s hard work has given us blessed access to.
- Most SonyEricsson Phones
- Nintendo Wii
- Sony PS3
- Sony PSP
He’s also working on Opera Mini support and has done a lot of awesome refactors on the code. Fixing many an embarrassing lines of code for me. Its great to see other people chipping into the project and I know for me personally, its a big inspiration to see people giving their time to help this project out!
Now, on to a small tweak that we made to the subcategory expansion system. Its a really small change, but should make using the app even easier. If you want to expand a subcategory, clicking on the title of the category gets the job done. No more having to aim your finger at the “Show” button. Obviously, it also hides if you do it when the section is visible. Its these types of changes that are my favourite. Someone might not even notice it, but it should make their usage of the site a litttttle bit easier.
We’ve also done various and sundry internal changes. We got a patch from Jacques Crocker to use Bundler to manage our Gems. Getting the capistrano part right on that was a bit difficult and so we had a little downtime today while I was working out those kinks. But, from now on, it should be much, much easier to get a local copy of Wikimedia Mobile up and running on your system.
We’ve also expanded the number of supported languages recently. All of this is thanks to the tireless work of Niklas Laxström and others from the TranslateWiki project. They were very helpful in building an API for us to import new languages. I wrote a couple rake tasks and Ruby bits to make it all go. So, now all you have to do is type `rake lang:import` and it will download the freshest strings you’ve ever parsed.
Here is an updated list of all of the languages we support: af, ak, ar, az, bg, bn, br, bs, ca, cy, de, dsb, el, en, eo, es, eu, fi, fr, gl, gv, he, hi, hr, hsb, hu, ia, id, is, it, ja, ka, km, kn, ko, ksh, kw, lb, li, lv, mg, mk, mt, nl, no, oc, pl, pms, ps, pt, ro, ru, sah, sh, sk, sl, su, sv, te, th, tr, uk, vec, vi, wo, and xal. I really have no idea what most of those languages are, but I’m also super happy to support them. And, I’d like to welcome the newest member to our class: Akan (ak), comes to us from Ghana! I know, cool right? Technically, its not a language but an Ethnologue. (Must stop myself from reading too much about this on Wikipedia… and finish this post)
Well folks, that’s it for the moment. Now that I’ve completed my move to another continent you should be seeing a lot more updates coming through on the mobile site. I’m really excited about where we are going and I’m also excited to reveal the results of our survey in the next couple weeks. Some really interesting and exciting things in there!
New Media Features Gadget
Posted by Michael Dale in wikimedia on October 10th, 2009
I would like to announce that some of the new media features are now available in gadget form on Wikimedia Commons and the English Wikipedia. These include a new ogg player, the add media wizard, and firefogg upload support. I hope having these components in gadget form will enable some more testing and feedback :)
Getting Started to enable these components you must turn on the mwEmbed gadget. You can turn it on by visiting your preferences page. Once you enable the gadget you should shift reload to ensure you have a fresh copy of the JavaScript. (note you will need to enable the gadget for each wiki you want to test (ie both for wikimedia commons and Wikipedia). Once enabled you can check out the following features: Read the rest of this entry »
Click Tracking on Edit Toolbar deployed
After many a hearty SQL battle, we finally have click tracking deployed on the wikimedia projects!
Data on button usage
What’s being tracked?
Which buttons are clicked on the toolbar during editing
What information is being recorded?
The button clicked, the time of the click, total edit count of the user clicking, and edit count for the last 1, 3, 6 months
What information is NOT being recorded?
Individually identifiable information of any sort (eg who exactly clicked what) and anything that would violate our privacy policy in general
Why?
As we revamp the UI, rather than randomly throwing buttons up there we think are pretty (we think they’re all pretty), we thought we’d put buttons up and features that people actually use. Novel, right?
What about the edit history and stuff?
We figure the way a novice editor uses the toolbar is different form a ‘power’ editor, and that there’s probably some gradation in between. Is there? Well, that’s what we hope to find out…
–Nimish
Image renaming fix
Fixed an ugly internal caching bug which could break renamed image redirects on Commons being accesses from other non-English sites. Hopefully that’s most of those solved now. :)
SVG in Wikipedia and Wikimedia Commons
Slides for my talk at SVG Open available for download as PDF or Keynote source. (I can make my test corpus available as well — let me know if interested!)
– brion
Babaco is ready for tasting
Preview of the second set of usability features, Babaco, are available through user preferences. The first feature is the article outline in the right hand side of the editing area. The outline updates itself in real-time as you type the headers in the articles and provides links which when clicked will jump you to the start of each section in the article. The second feature is the assisted way to insert links and tables. Instead of inserting wiki syntax, a dialogue box pops up when you hit the link icon in the toolbar. For internal links, link suggest features offers auto-complete options and validate the existence of articles. A click of the table icon in the toolbar will also assist define the number of rows and columns without modifying table rows and columns in the wiki syntax. Thirdly, new search and replace dialogue is added to the advanced toolbar. These features are released in the stealth mode, which means users need to turn them on by going to user preferences. To enable these features, please go to “My Preferences”, select ‘Editing’ tab and enable the features listed under ‘Experimental features’. We, the usability team, is still refining look and feel, but we wanted to invite active users to start using these features and provide us feedback.
Known Issues: Accuracy of cursor positions using the article outline feature (aka Navigable Table of Contents) degrades in long articles and significant so if you use Firefox. We are also still working on the display issue of NTOC in Firefox2 on Vista. Bug 20669
The release details and discussion can be found in the Babaco discussion page of the usability project wiki. Looking forward to receiving your feedback.
- Naoko
Navigable Table of Contents
Inserting link using link dialogue
SVG Open
Posted by brion in open-source, wikimedia on October 2nd, 2009
I’m hanging out down in Mountain View for the SVG Open conference this weekend, to speak a bit on how we use and plan to use SVG at Wikimedia and to get up to date on the state of the art. I’ll post my full talk slides on Sunday after my talk…
One of the most exciting new developments in the SVG world right now is svgweb, a very cool tool which brings high-quality SVG rendering support — including full support for the SVG DOM and interactivity — to any browser that supports Flash. This essentially fills the “SVG gap” for most Internet Explorer users, which opens up a huge world of possibilities for both interactive content and tools for building, editing, and localizing SVG-based diagrams, charts, maps, etc right in the browser.
Google web standards evangelist Brad Neuberg gave a great talk about the background of how something like svgweb was needed and showed some great demos, including a quick preview of an inline SVG pan-and-zoom tool for Wikipedia / Wikimedia Commons; we’ll have some even funner demos based on that Sunday!
Also saw a good talk from Sam Ruby on some of the gotchas in the current state of HTML vs XHTML vs HTML5 and how SVG is (or isn’t) supported in various profiles and various browsers. Most interesting was his proposal to rethink how we deal with markup validators in the webdev world — right now most validators give you a lot of errors about things that don’t really make a difference (font vs style?), but freely ignore problematic but “legitimate” structures (say, unclosed list items).
MediaWiki’s new discussion system in testing on Wikimedia Labs
Posted by andrew in mediawiki, software, summer of code, wikimedia on October 1st, 2009
I’m very excited to announce that LiquidThreads, the next-generation discussion system that I’ve spent the last few months developing for the Wikimedia Foundation, is now in beta testing on liquidthreads.labs.wikimedia.org.
Sample of the LiquidThreads interface
Wikimedia XML data sets released on Amazon Public Data Sets
For our community members that do analysis on Wikimedia project data, I’m happy to announce the release of our XML snapshots within Amazon Public Data Sets.
http://aws.typepad.com/aws/2009/09/new-public-data-set-wikipedia-xml-data.html
To those curious about why this happened … earlier this year I had gotten approached multiple times from researchers and community members wanting to parse our data but frustrated at the costs and time of doing it on their own infrastructure.
Many of them were already familiar with academic computational clusters and were wondering if there were any similar solutions available for them to do large scale processing. The tool sever was one option but sometimes it didn’t provide the level of flexibility that they needed and/or their projects were too computationally intensive to run alongside other tasks.
Instead I mentioned that I had already been thinking of pushing our data sets in the Amazon cloud as they had a wealth of the infrastructure in place and had an infrastructure that our users who were familiar with.
Fast forward to now, we have our first release ready to be worked on and I’m sure that we will hear back about new and exiting discoveries that our communities make.
This isn’t the first Wikipedia data set to exist in Public Data Sets but it will be the first that Wikimedia is committing to supporting on a regular release cycle. Amazon will be picking it up every month and retaining copies for at least three months.
I’m excited to see the stats of how many people use it.
–tomasz
Supporting translatewiki.net
Translatewiki.net is a core part of the MediaWiki ecosystem. While not a Wikimedia Foundation project, it’s used by hundreds of volunteers to improve the localization of MediaWiki and its extensions, alongside other open source projects, which has led to MediaWiki being one of the most internationalized software packages available.
We’re very pleased to be able to recognize the incredible volunteer efforts behind translatewiki.net at least in a small way. Starting tomorrow, Siebrand Mazeland will be able to devote one day a week to the support of the project on a contract with the Wikimedia Foundation. We’ve identified core priorities for the next year as an increase in the number of volunteer developers supporting the translatewiki.net infrastructure, and the number of volunteer translators working on localization for the most widely spoken languages.
Welcome, Siebrand, and a big thank you to the entire translatewiki.net community for their work. :-)
Erik Moeller
Deputy Director, Wikimedia Foundation
FlaggedRevs test wiki awaits you!
Apparently due to some miscommunications, a lot of people didn’t realize that the FlaggedRevs labs test wiki has been active and waiting for people to poke at it for a month, since just before Wikimania!
We need interested people to be get up as local administrators to try out the the per-page stabilization settings (accessed via the ‘protect’ tab); by default most pages do not activate FlaggedRevs in the configuration we’re testing for English Wikipedia.
I’ve added a couple quick notes to this affect on the main page.
Update: We’re collecting some folks to be bureaucrats and help set up more test admins so we can get things going quick!
Announce: Brion moving to StatusNet
I’d like to share some exciting news with you all… After four awesome years working for the Wikimedia Foundation full-time, next month I’m going to be starting a new position at StatusNet, leading development on the open-source microblogging system which powers identi.ca and other sites.
I’ve been contributing to StatusNet (formerly Laconica) as a user, bug reporter, and patch submitter since 2008, and I’m really excited at the opportunity to get more involved in the project at this key time as we gear up for a 1.0 release, hosted services, and support offerings.
StatusNet was born in the same free-culture and free-software community that brought me to Wikipedia; many of you probably already know founder Evan Prodromou from his longtime work in the wiki community, launching the awesome Wikitravel and helping out with MediaWiki development on various fronts. The “big idea” driving StatusNet is rebalancing power in the modern social web — pushing data portability and open protocols to protect your autonomy from siloed proprietary services… People need the ability to control their own presence on the web instead of hoping Facebook or Twitter always treat you the way you want.
This does unfortunately mean that I’ll have less time for MediaWiki as I’ll be leaving my position as Wikimedia CTO sooner than originally anticipated, but that doesn’t mean I’m leaving the Wikimedia community or MediaWiki development!
Just as I was in the MediaWiki development community before Wikimedia hired me, you’ll all see me in the same IRC channels and on the same mailing lists… I know this is also a busy time with our fundraiser coming up and lots of cool ongoing developments, so to help ease the transition I’ve worked out a commitment to come into the WMF office one day a week through the end of December to make sure all our tech staff has a chance to pick my brain as we smooth out the code review processes and make sure things are as well documented as I like to think they are. ;)
We’ve got a great tech team here at Wikimedia, and we’ve done so much with so little over the last few years. A lot of really good work is going on now, modernizing both our infrastructure and our user interface… I have every confidence that Wikipedia and friends will continue to thrive!
I’ll start full-time at StatusNet on October 12. My key priorities until then are getting some of our key software rollouts going, supporting the Usability Initiative’s next scheduled update and getting a useful but minimally-disruptive Flagged Revisions configuration going on English Wikipedia. I’m also hoping to make further improvements to our code review process, based on my experience with our recent big updates as well as the git-based workflow we’re using at StatusNet — I’ve got a lot of great ideas for improving the CodeReview extension…
Erik Moeller will be the primary point of contact for WMF tech management issues starting October 12, until the new CTO is hired. I’ll support the hiring process as much as I can, and we’re hoping to have a candidate in the door by the end of the year.
– brion vibber (brion @ wikimedia.org)
CTO, Wikimedia Foundation
San Francisco
Update: Evan’s announce is up on the StatusNet blog.
Theora 1.1 Released
Posted by Michael Dale in wikimedia on September 28th, 2009
Theora 1.1 has been released
Theora 1.1 has been released. This release reflects the efforts of xiph.org developers who over the past year have done incredible work to greatly improve the core free codec video library. This effort has been support by Mozilla Foundation, Red Hat and others. These improvements include:
Chris Blizzard from Mozilla has a detailed blog post about the improvements and what they mean for open video creators, distributors and viewers. Wikimedia foundation makes exclusive use free/open formats and has been a long time supporter of ogg theora and makes use of the free codec in its websites. Wikimedia also helped organize some improvements by administrating a theora improvement grant from Mozilla earlier this year.
Also a new version of ffmpeg2thora has been released using this new codebase making it easy to take advantage of these new features. If you would like to give the new encoder a spin you try it out with the web based firefogg encoding app.
LocalisationUpdate in testing
Ok, we still need to complete automation of update runs for LocalisationUpdate, but it seems to be working!
It’s not the most glamorous of extensions, but you can see here an updated message (”Den här sidan” where the current deployed message file says “Denna sidan har”)!
– brion
Server Donation Entry Period Ending
Just to let folks know, we have had quite a large interest in our donation of some of our decommissioned servers. In fact, I have way too many emails!
So to be fair, rather than just stop today, we will stop accepting submissions for this next Monday, September 28th. That means if you want your proposal/request in the running, you have to have it emailed to servers@wikimedia.org by Midnight GMT this coming Sunday, Sept. 27th.
For ease of reference, here is a copy of the post from the start of this process:
It is that time again. We have approx 35 servers to donate to a good home. These are servers that Wikimedia has used on the projects for 3+ years, so they are out of warranty and just not fast enough for us to keep using on the cluster.
The servers will go out to homes for folks who are willing to pay for the freight. They are as follows:
- Dual CPU 2.5 GHz AMD
- 3-4GB RAM Each
- Most have 80 GB or larger HDD
Disclaimers: The Wikimedia Foundation does not guarantee the operation or use of these servers in any shape or form. They are old, some may have dying fans, bad hdd sectors, and the like. Servers have been wiped of information, and they ran through that, but no promises on function!
If you would like to receive some of these servers for your NONPROFIT use, please email servers@wikimedia.org. Please include in your email how you will be using the servers, and the address they would be shipped to. We will review all requests and try to fairly pick out where they go. (Selection process may be refined, but it also may just include throwing darts at a board to break up ties.)
Additions: Due to request, the servers are indeed located in Tampa, FL USA. Zip code 33602 for shipping purposes. This means that if you are international, shipping this hardware is really not cost effective for you. If you want to be in the running still, and are comfortable with personally handing all customs, duties, export, and tax issues, go ahead and email us.
Correction: Dates were off.
LocalisationUpdate and ProofreadPage second try tomorrow; Collection too
Updates are coming back for LocalisationUpdate and ProofreadPage extensions, which we tested then pulled Tuesday due to performance problems.
Roan’s redone LU to store the message updates in serialized files instead of the database, which we can sync locally to web servers and should perform much better; I’ll also do a more gradual test rollout so we can scale it back more gracefully if we have problems again.
ThomasV has fixed up some bad queries in ProofreadPage, and it should be ready to go again; the updated version has much more advanced index support and looks pretty spiffy. :)
And if we’ve got time between other things, I’ll roll out the updated Collection as well — this makes big improvements to the UI for building a multi-page book, and leaves the sidebar much less cluttered when you’re not using it.
– brion
Commonist, CommonsHelper fixed
I’ve put in a fix for uploader tools and bots that have been broken since the last update — these bot tools didn’t expect the upload form to change, so they don’t pass in new required fields such as the edit token which was added to the form in the latest update.
Since the edit token isn’t actually required for web uploads (it’s a protection against a class of attacks which, as it happens, can’t forge file uploads) I’ve relaxed the check. I’ve confirmed that it fixes Commonist and have a report that CommonsHelper is also fixed.
Most other bots and tools that were affected are probably also fixed; please test them and let us know if anything’s still broken!
LocalisationUpdate deployment delayed
Problem #1 — causes huge increase in CPU load on web servers
Problem #2 — completely kills THE ENTIRE SITE when you DISABLE it because serialized LUDependency objects in the message cache can’t be reinstantiated.
Sigh… Why can’t life be easy :)
Update 2009-09-23 16:30: Roan is starting work on replacing the database storage layer with a flat-file which should be more efficient for our use case.
Feature deployment updates
Quick note for those who have been asking — we’re starting to maintain a list of feature & extension deployments that we’re rolling out in the very near future and their status on the Wikitech wiki.
Note that some things like the English Wikipedia FlaggedRevs deployment aren’t on there yet; we’ll start prepping something for these in the next round in a couple weeks.
– brion
Help us make Commons work better
It’s come to my attention that Commons has had some more problems than we saw on the other wikis… I want to make sure we get that cleaned up!
Andrew has just deployed some more fixes which should eliminate the ‘file already exists on Commons’ bug. It would be a huge help if folks who’ve been reporting problems can go through the lists on the Village Pump and collect just the problems that are still current:
- Uploads via Commonist — fixed or do we still have problems?
- (Who’s the best person to contact about debugging or developing fixes for Commonist?)
- Missing description page immediately after edit — is this still happening?
- Anything else?
Please add info about confirmed problems or fixes at the Commons Village Pump. Thanks to everybody — your feedback is important; we need to know which bugs are affecting you guys the most!
– brion
Update 2009-09-22 23:49 UTC: I’ve fixed the problem with Commonist by relaxing the edit token check in the form handler (this is safe for uploads via the web, which can’t be injected in a CSRF attack; we still enforce it for upload-by-URL). This probably fixes most of the other client-side upload tools and bots as well — please test!
File renaming enabled for admins
Renaming of uploaded files has been disabled for the last couple of months pending software updates; as of today it is now re-enabled for admins on all Wikimedia sites.
Local admins can now rename files as well as deleting them; this makes it easier to track history for images and other media that need to be moved to another name.
– brion
Update 00:49 UTC 22 Sep: We’ve tracked down a bug which at least on en.wikipedia would cause the redirected old name to stop working for 24 hours. Now fixed; notes for cause and workaround on enwp Village Pump.
Beta edit toolbar disabled temporarily
Due to compatibility problems on Internet Explorer after yesterday’s code update, I’ve temporarily disabled the Usability Initiative’s beta advanced toolbar. If you’ve had it enabled, you’ll just get the regular old edit toolbar until we re-enable it.
Hopefully we should have this resolved within a day or so, and it’ll be back on for all our happy testers!
– brion
Update Sep 18 14:13 UTC: the bug has been fixed, and the toolbar has been reenabled.
Babaco Preview
Screenshot of Navigable Table of Contents
As brion’s earlier post stated, the new set of usability features (nickname: Babaco)are integrate into the source code but disabled until these features are stable. New features include, navigable table of contents and content generation dialogues for links, tables, search and replace. These features are currently staged on the usability prototype environment, and you are welcome to check them out and encouraged to push them hard to find bugs. :-)
We are also hoping to integrate add-media-wizard created by Michael Dale in Babaco. While API to extend the toolbar is being worked on, media import functionality can be played at the usability sandbox environment. Click the “Edit” tab and click an image gallery icon in the toolbar. You will see a nice media import function there. Feel free to create an article and import image from Commons using this neat feature.
Browser compatibility matrix for Babaco is found here. As always, share your thoughts and experience through the usability wiki. Feedback on Babaco goes this discussion page.
- Naoko