Archive for category open-source
SVG Open
Posted by brion in open-source, wikimedia on October 2nd, 2009
I’m hanging out down in Mountain View for the SVG Open conference this weekend, to speak a bit on how we use and plan to use SVG at Wikimedia and to get up to date on the state of the art. I’ll post my full talk slides on Sunday after my talk…
One of the most exciting new developments in the SVG world right now is svgweb, a very cool tool which brings high-quality SVG rendering support — including full support for the SVG DOM and interactivity — to any browser that supports Flash. This essentially fills the “SVG gap” for most Internet Explorer users, which opens up a huge world of possibilities for both interactive content and tools for building, editing, and localizing SVG-based diagrams, charts, maps, etc right in the browser.
Google web standards evangelist Brad Neuberg gave a great talk about the background of how something like svgweb was needed and showed some great demos, including a quick preview of an inline SVG pan-and-zoom tool for Wikipedia / Wikimedia Commons; we’ll have some even funner demos based on that Sunday!
Also saw a good talk from Sam Ruby on some of the gotchas in the current state of HTML vs XHTML vs HTML5 and how SVG is (or isn’t) supported in various profiles and various browsers. Most interesting was his proposal to rethink how we deal with markup validators in the webdev world — right now most validators give you a lot of errors about things that don’t really make a difference (font vs style?), but freely ignore problematic but “legitimate” structures (say, unclosed list items).
SVG for all… with Flash?
Posted by brion in mediawiki, open-source on July 29th, 2009
For several years, we’ve supported uploading SVG vector images to Wikimedia sites… with the limitation that they would be rendered to static PNG raster images when actually used inline.
This gives our editors great flexibility in editing, customizing, and translating maps and diagrams using cross-platform free tools like Inkscape, but we’re missing out on some of the big potential in SVG — high-quality scaling for zoomed displays and printing, and animation and scripted interactivity.
In large part we can blame Internet Explorer — the most widely used browser has never supported SVG graphics natively, and Adobe isn’t even maintaining their plug-in anymore! With the majority of users cut out, we’ve had little incentive to move forward with new capabilities that would be closed to most visitors.
But that may be changing, thanks to… Flash??
svgweb implements a highly capable SVG renderer in JavaScript and Flash, bringing high-quality, scriptable SVG support to the ~95% of web users who have either Flash or a naitvely SVG-capable browser.
I love to see Flash’s near-ubiquity used for good — implementing support for modern, open web standards on older and less capable browsers.
One of the chief drivers of the project is Google open standards evangelist Brad Neuberg; we had a great talk today along with Trevor on our Usability team and Michael of Metavid/Kaltura/video awesomeness, and we’re all very excited at the possibilities.
We’re going to see if we can whip together some basic integration in time to show at the SVG Open conference in October, starting with a basic zoom-and-pan view for SVG images which can make use of native or emulated SVG support.
Future ideas that have us really excited include:
- Live previewing of parameterized images at insert time (localized text, highlighted map segments, charts, etc)
- On-web basic vector image editing? Sometimes you just need to make an adjustment and installing Inkscape is kind of heavyweight.
Pure SVG + Javascript should be able to provide for selecting, moving, adding, and altering objects, which we could then save back to a new version of the file… svgweb’s powerful scripting support should be able to extend this to Internet Explorer users too!
Use of SVG originals inline in article pages is more dependent on file size issues. We have a lot of files that are just plain huge, especially detailed maps, and the SVG version ends up being a lot slower to download and display.
A project which can help with that is Scour, a tool to optimize SVG source by stripping out unneeded verbosity and rearranging style bits to keep size down.
With further work to strip out detail that will never be visible, a filter like this could let us produce output files that are more suitable for on-screen viewing while still scaling up nicely on zoomed displays and printed output.
OER Search Discovery – not just another TLA
Posted by brion in mediawiki, open-source, software on July 10th, 2009
I’ve spent today in sunny Cambridge, MA attending the OER Search Discovery 2009 workshop at Harvard’s Berkman Center. But what’s it all about?
First off, what’s OER?
Open Educational Resources are a litle tough to really define to everyone’s satisfaction, but we can defer the details. :) We’re generally talking about pedagogical materials (something that could be put to use in the classroom to teach students) available under some sort of open content license.
Secondly, what’s OER search?
Creative Commons’ ccLearn project has put together DiscoverEd, a prototype search engine which includes some relevant metadata (subject matter, language, target age range, license) as well as metadata about which collection of resource links it came from. This is rather clever, allowing teachers or students to limit their searches to what’s relevant as well as what’s trusted.
Third, what’s OER search discovery?
Traditionally, most electronic educational resource collections have been walled silos. Even if the materials themselves are open and redistributable, the collections’ searches are separate, and often there’s been confusion over the openness of the metadata as well which has held back federated searches on a larger scale.
With major search engines like Google and Yahoo now starting to index metadata embedded in web pages (RDFa and/or microformats) and make them available for searches, this a great time to start pushing more active and integrated semantic search data. (Note: here we’re talking about metadata about the actual materials, not about the subject of the materials. That’s a matter for another day!) Content creators — if enabled by content management tool developers — can start actually getting some concrete benefit from embedding semantic data into their web sites. These’ll be picked up by the general search crawlers, but will also be available to targeted repositories collecting links and metadata about educational materials on the web.
How can we benefit?
There are two sides of this which Wikimedia can work at:
- On the content creation side, we can provide more ways to add useful metadata to our pages, making it easier for teachers and students searching through educatinoal-themed portals to find them. MediaWiki already provides basic language and license information, but projects like WikiBooks and Wikiversity (as well as other MediaWiki users like WikiEducator) could definitely benefit from a consistent way to specify the subject and target audience of lesson modules.
- On the consumer side, we want to be able to find and use free/open media resources from elsewhere on the web to supplement the ones we already have on Wikimedia Commons. The in-development Add Media Wizard can currently search and fetch from a few hardcoded repositories like Archive.org and Flickr, but editors could benefit a lot from having either broader (whole internet search like Google Images with license limits) or narrower sources (a particular educational resource repository desired by a given site or community).
How can we help?
- We’ll want to find a good, clean, maintainable, and easy to use way for wiki page authors to add resource metadata to their pages, which can be exposed to spiders and repository crawlers. RDFa vs microformats vs XHTML vs HTML 5 needs some resolution on the output format, but more interesting is making sure we have a clean user interface/workflow in the edit window without cluttering up the wiki markup.
- If/when folks standardize on a search query format as well, we can make it absurdly easy to add specific repositories to the MediaWiki media picker. In the meantime, we can target some whole-net search engines that index license and subject metadata such as Yahoo’s SearchMonkey, which will provide relevant indexing of web sites which have provided for metadata autodiscovery with embedded RDFa etc.
- We might also think about acting as a repository ourself — Wikipedia and our sister projects are full of references to excellent resources both online and off. Can we record what we know about them and make that searchable internally and externally?
Folks at the workshop are also hoping we can agitate for similar moves in other tools… I know I would benefit from a free media picker for WordPress!
First usability release, Acai, is now available.
Posted by Naoko in mediawiki, open-source, software, wikimedia on July 2nd, 2009
The first usability release, Acai, hit Wikipedia and sister projects this afternoon. The new skin, Vector, and the enhanced toolbar can be turned on from the user preference under “Appearance” and “Editing”. Search result page now has a new layout with less daunting information. Vector is only available for left-to-right languages at a moment due to IE6 incompatibility. However, the enhanced toolbar can be selected from all languages and the new search result page is enabled globally. We could not roll out two features we had planned. First, warning messages for unsaved changes when a user switches away from the edit tab did not work properly thus they are disabled. So please be careful when you switch away from the edit tab. Secondly importing language specific configuration for special characters were not graceful, so we disabled special character function from the toolbar. We are working on the fixes and plan to roll them out as soon as we have stable solutions. The usability project wiki has Vector and the new toolbar as a default, so if you prefer to check them out without changing your preferences it is a good place to visit first. Let us know what you think. We would love to hear from you.
Best,
Naoko
Wikimedia Mobile is Officially Launched
Posted by Hampton Catlin in open-source, software on June 30th, 2009
iPhone Version in English
After spending about 6 months in alpha-beta-development-maybe-kind-live mode, we have recently moved Wikipedia Mobile over to a new fast and sexy server. With this new server, we’ve reached the point in development where we can call this baby “launched”!
When I was brought on board at Wikimedia, I was tasked with endowing Wikimedia with a compelling mobile offering. From the beginning, we knew we were going to focus on “fully featured” smart phones. These phones are taking more and more of the market and we believe they will have an easy majority-share in a couple years. The goal is to build for the future.
At the moment, the Mobile site supports iPhone, Kindle, Android, and Palm Pre. And we fully support both English and German. There are other working languages, but they haven’t been fully translated yet. Our goal is to grow slowly and do it really well. We are starting out simple with limited support in order to test the usability and the platform’s stability. So far, things are looking good.
During the beta test period, we’ve served around 10,000,000 pages. You can view the hourly stats here (updated every hour on the hour). And with this new test server, we should be able to do more.
Based off of requests from Google and the Palm Pre folks… and with what just makes sense. We are doing default mobile redirects. That is, if you open a wikipedia link on a supported mobile device, then you get redirected automatically to the mobile gateway. If you click the “View this page on main Wikipedia” then we disable that redirect with a cookie. This way, the 99% of people using mobile devices to read Wikipedia on-the-go have a seemless experience. And, the 1% who like to edit on their mobile device can use their browser to view the main site and do all the fancy things that they like doing. We suspect an initial outcry from the editors that use their mobile devices, but hope that will calm down. We’ve had very good feedback from the 99% and so we can’t forget those folks. If anyone has any suggestions on how to make this easier for the 1% who are editing while mobile, we’d love to hear from you.
If you want live updates about the Mobile site then you can follow WikimediaMobile on Twitter. Also, if you know any Ruby, you can grab the source code via git from Github and helpout! Feel free to contact me via email with any questions.
Also, special thanks to Nic Williams and Ryan Bigg from Mocra for help with the Ruby 1.9 transition and thanks to Yahuda Katz for help with the XML parsing layer and for all his work on the Merb framework.
Firefox 3.5 brings native open video support
Posted by brion in open-source, wikimedia on June 30th, 2009
Congralutations are in order for our friends and comrades-in-arms at Mozilla: they’ve released version 3.5 of their open-source Firefox browser today.
Aside from major improvements to speed and memory usage, one of the updates that has got us most excited at Wikimedia is the support for HTML 5’s native <video> and <audio> elements.
What does this mean? Well in short, it means that Firefox 3.5 is the best browser to run video and audio clips from Wikimedia Commons on!
A few months more down the line, we’ll start being able to integrate support for our inline video sequencer, which’ll make it easy to extract snippets of a longer video and combine them — entirely using open-source, non-patent-encumbered web standards. This makes heavy use of the new HTML 5 multimedia support; while at first editing will be limited to Firefox 3.5 users, other browsers are continuing to improve and adopt the same support.
Blog Downtime
Posted by RobH in open-source, software, wikimedia on June 29th, 2009
I am sure that many folks noticed that on the morning of 2009-06-26, techblog.wikimedia.org and blog.wikimedia.org went down. It turns out that some of the parts of our Wordpress installations were compromised. I do not want to get in to a direct show and tell of what they did, but hopefully we have hardened the installation to the point that it will not occur again.
This is why the blogs exist on their own server, so when things like this happen we can minimize the impact. The blogs are both up and running now, along with the other services that were affected. All but techblog was back online before Friday was over, techblog lagged behind until today. (As techblog was the point of exploit, we got everything else back up first.) Other affected services were the Open Conference Systems site for Wikimania 2009, as well as our survey software. Both of those were back online ASAP after the incident and the rest followed after.
Of course, it was hard to get this information out to folks when the blogs were down! It goes to show how easily using the blogs to get info out has been, since without it we had to scramble to get the information out of other channels.
Thanks to everyone who assisted in the restoration, and also thanks to everyone for their patience while the system was fixed.
Wikimedia & FourKitchens support CiviCRM development
Posted by tomasz in open-source, wikimedia on June 9th, 2009
Here at Wikimedia we’ve been avidly using CiviCRM for over two years now. Over that period we’ve seen it grow and mature as a platform for fundraising, contact tracking & mailings and have been wanting to make the platform evolve even more. Together with Civi community, we’ve worked to organize the early release of the CiviReport architecture for the 2.2 branch. Thanks go to the core Civi team for doing the backport and FourKitchens for contributing a wealth of new reports for us. You can read a full write up of the release at the CiviCRM blog.
For those of our readers who are interested in CiviCRM and are in the Bay Area, we’ve also started to organize regular user meetups. The first one had a great turn out and we’d love for both developers and users of CiviCRM to attend the next one on August 4th at 6pm.
The Wikipedia Usability Initiative is still hiring.
Posted by Naoko in mediawiki, open-source, software, wikimedia on May 20th, 2009
The Wikipedia Usability Initiative has extended the application deadline for the Software Developer position till May 30th. We are recruiting two candidates for this position. Both local applicants to the San Francisco Bay Area and remote applicants are encouraged to apply. Please help spread the word.
http://wikimediafoundation.org/wiki/Job_openings/Software_Developer_(project)
Naoko Komura
Wikipedia Usability Initiative
Google Summer of Code student projects accepted!
Posted by brion in open-source, summer of code on April 22nd, 2009
Reposting the announce from Roan’s wikitech-l mailing list post:
Yesterday, the selection of GSoC projects was officially announced. For MediaWiki, the following projects have been accepted:
- Niklas Laxström (Nikerabbit), mentored by Siebrand, will be working on improving localization and internationalization in MediaWiki, as well as improving the Translate extension used on translatewiki.net
- Zhe Wu, mentored by Aryeh Gregor (Simetrical), will be building a thumbnailing daemon, so image manipulation won’t have to happen on the Apache servers any more
- Jeroen de Dauw, mentored by Yaron Koren, will be improving the Semantic Layers extension and merging it into the Semantic Google Maps extension
- Gerardo Antonio Cabero, mentored by Michael Dale (mdale), will be improving the Cortado applet for video playback (I’m a bit fuzzy on the details for this one)
The official list with links to (parts of) the proposals can be found at the Google website; lists for other organizations can be reached through the list of participating organizations.
The next event on the GSoC timeline is the community bonding period, during which the students are supposed to get to know their mentors and the community. This period lasts until May 23rd, when the students actually begin coding.
Starting now and continuing at least until the end of GSoC in August, you will probably see and hear from the students on IRC and the mailing lists and hear about the projects they’re working on. To repeat the crux of an earlier thread on this list: be nice to these special newcomers, make them feel welcome and comfortable, and try not to bite them :)
To the mentors and students: have fun!
Roan Kattouw (Catrope)
OpenStreetMap maps will be added to Wikimedia projects
Posted by Ævar Arnfjörð Bjarmason in open-source, wikimedia on April 4th, 2009
There has been rapid progress on the subject of adding OpenStreetMap maps to Wikimedia projects (e.g. Wikipedia) during the MediaWiki Developer Meet-Up taking place right now in Berlin.
We now have a clear plan of action for getting OpenStreetMap maps embedded in Wikimedia wiki (e.g. Wikipedia) pages:
- Wikimedia will set up a database to mirror the OSM data (Planet.osm)
- Wikimedia will set up its own rendering infrastructure for rendering tiles & other maps from the OSM data
- The existing MediaWiki extensions for displaying OSM data in a MediaWiki article will be improved to work acceptably in production on Wikimedia servers
To prototype all this we’ll be using new infrastructure provided by Wikimedia Deutschland. Once things have been tested there they’ll eventually be deployed on the main Wikimedia sites.
After discussion with the Wikimedia operations people (including Brion Vibber, Mark Bergsma et al) there seem to be no objections to the above plan as long as:
- The maps will work not only for JavaScript enabled browsers but also non-JavaScript enabled ones
- The tools involved are improved to be relatively stable & deployable on Wikimedia, e.g. being able to embed more than one slippy map, the internationalization of error messages etc.
- The end product (the generated tiles or map files) are cachable so that they can be thrown at the frontend squids, as they’re static images this should be easy.
The featureset that we’re aiming for to be able to deploy this on Wikimedia sites from the view of the user (more can be added later once we’ve got it working) is:
- The ability to embed OSM maps in articles with something like the Simple image extension, perhaps automagically turning into a Slippy Map if the browser supports it
- A static or slippy map that can be used by geotagged articles so we can have maps without explicit inclusion of a <map> tag.
We’ll also set up a map toolserver for experimenting with other uses of OpenStreetMap data on Wikimedia. People with relevant projects can get access to this toolserver to try out their ideas for tools that could eventually be integrated on the main Wikimedia sites.
This project is seeking help from anyone who’s interested who’d like to be a part of making this happen, if you want to be a part of adding free maps to the world’s largest encyclopedia please subscribe to this mailing list:
And/or read/edit/comment on the relevant wiki coordination pages:
Google Summer of Code student applications open for Wikimedia!
Posted by brion in open-source, summer of code on March 30th, 2009
Google Summer of Code is now open for student applications!
We’ve had 5 submissions come in so far… don’t be shy! :) Also don’t be shy about hanging out on our mailing lists and IRC channels and getting feedback from other MediaWiki developers on your project ideas.
The more feedback you get, the better you can make your submission… and the awesomer the result will be!
The application period ends April 3, 2009 at 19:00 UTC — don’t be late!
(We could still use a couple more project mentors too…)
Google Summer of Code 2009
Posted by brion in mediawiki, open-source, software, summer of code, wikimedia on March 18th, 2009
On 3/10/09 5:17 PM, Brion Vibber wrote:
I’ve just put in Wikimedia’s org application for Google Summer of Code 2009… Hopefully we’ll get in.
We’re officially in! Woo!
Student applications will open starting March 23, ending April 3. This intermediate week before applications open is a good time to chat us up with your ideas and try to pair up with potential mentors. :)
Techblog is online.
Posted by RobH in open-source, software, wikimedia on March 13th, 2009
Starting today the Wikimedia Technical Team now has a single, unified blog for updates. This is a much more technical blog than the Wikimedia Blog (which covers Foundation issues and news). Software updates, server changes, and other such issues will be covered here.