Archive for April, 2009
Tor configuration changes, and IP block exemption rollout
Hi all,
Just a quick note to let everybody know that in a few days I’ll be changing the TorBlock configuration to require explicit block exemption rather than merely being logged in.
While we would rather this weren’t necessary, it seems that the edits coming through tor are mostly unconstructive; and we’ve had all kinds of nasty harassment come through that way — the community feedback we asked for was overwhelmingly that the ideological benefits of allowing truly anonymous editing are outweighed by the pragmatic concerns of
harassment and vandalism.
To facilitate this, I will also be activating explicit IP block exemption on all wikis. Like on English Wikipedia and many other wikis, administrators will be able to add users to an “IP block exempt” group, which exempts its holder from IP blocks, range blocks and autoblocks, but not explicit user blocks. This is a helpful, albeit inaccessible way to defray some of the problems associated with
blocking Tor users carte blanche.
Please let me know if you have any questions, comments, concerns or suggestions about these changes!
–
Andrew Garrett
Contract Developer
Wikimedia Foundation
Google Summer of Code student projects accepted!
Posted by brion in open-source, summer of code on April 22nd, 2009
Reposting the announce from Roan’s wikitech-l mailing list post:
Yesterday, the selection of GSoC projects was officially announced. For MediaWiki, the following projects have been accepted:
- Niklas Laxström (Nikerabbit), mentored by Siebrand, will be working on improving localization and internationalization in MediaWiki, as well as improving the Translate extension used on translatewiki.net
- Zhe Wu, mentored by Aryeh Gregor (Simetrical), will be building a thumbnailing daemon, so image manipulation won’t have to happen on the Apache servers any more
- Jeroen de Dauw, mentored by Yaron Koren, will be improving the Semantic Layers extension and merging it into the Semantic Google Maps extension
- Gerardo Antonio Cabero, mentored by Michael Dale (mdale), will be improving the Cortado applet for video playback (I’m a bit fuzzy on the details for this one)
The official list with links to (parts of) the proposals can be found at the Google website; lists for other organizations can be reached through the list of participating organizations.
The next event on the GSoC timeline is the community bonding period, during which the students are supposed to get to know their mentors and the community. This period lasts until May 23rd, when the students actually begin coding.
Starting now and continuing at least until the end of GSoC in August, you will probably see and hear from the students on IRC and the mailing lists and hear about the projects they’re working on. To repeat the crux of an earlier thread on this list: be nice to these special newcomers, make them feel welcome and comfortable, and try not to bite them :)
To the mentors and students: have fun!
Roan Kattouw (Catrope)
Wikimedia Foundation opting out of Phorm
After some internal discussion on whether opting out of the Phorm user-profiling system in the UK would legitimize it, we’re going ahead and requesting an opt-out for all the domains under the Wikimedia Foundation’s control:
Subject: Phorm opt-out for Wikipedia.org and related domains Date: Thu, 16 Apr 2009 14:28:11 -0700 From: Brion Vibber <brion@wikimedia.org> To: website-exclusion@webwise.com CC: private-l@lists.wikimedia.org To whom it may concern -- The Wikimedia Foundation requests that our web sites including Wikipedia.org and all related domains be excluded from scanning by the Phorm / BT Webwise system, as we consider the scanning and profiling of our visitors' behavior by a third party to be an infringement on their privacy. Here is a list of our domains which should be excluded (please exclude any and all subdomains as well): wikipedia.org wikipedia.com wikipedia.co.uk wikipedia.cz wikipedia.fr wikipedia.info wikipedia.lt wikipedia.net wikipedia.nl wikipedia.org.br mediawiki.com mediawiki.org quickipedia.net quickipedia.org toolserver.org vikipedio.com vikipedio.org wikibook.com wikibooks.com wikibooks.cz wikibooks.org wikicitaty.cz wikidata.org wikidisclosure.com wikidisclosure.org wikidruhy.cz wikifamily.com wikifamily.org wikigis.com wikigis.org wikijunior.com wikijunior.net wikijunior.org wikiknihy.cz wikimania2006.org wikimania2007.org wikimaps.com wikimaps.net wikimediacommons.co.uk wikimediacommons.de wikimediacommons.eu wikimediacommons.info wikimediacommons.mobl wikimediacommons.net wikimediacommons.org wikimedia.cz wikimediafoundation.com wikimediafoundation.net wikimediafoundation.org wikimedia.hu wikimedia.li wikimedia.lt wikimedia.org wikimedia.pl wikimedia.se wikimedia.us wikimemory.org wikimorial.com wikimorial.org wikinews.org wikipaedia.net wikipedie.cz wikiquote.com wikiquote.cz wikiquote.net wikiquote.org wikislovnik.cz wikisource.com wikisource.cz wikisource.org wikispecies.com wikispecies.cz wikispecies.net wikispecies.org wikiversity.com wikiversity.cz wikiversity.org wikiverzita.cz wikizdroje.cz wikizpravy.cz wiktionary.com wiktionary.cz wiktionary.org Thank you for your time. -- brion vibber (brion @ wikimedia.org) CTO, Wikimedia Foundation San Francisco +1 (415) 839-6885
Received autoreply:
Subject: Publisher Exclusion Request Autoreply Date: Thu, 16 Apr 2009 14:28:30 -0700 From: website-exclusion To: Brion Vibber Thank you for your submission to the Phorm website exclusion list. If there are no obvious grounds to doubt the legitimacy of the request the URL will be blocked as soon as possible, usually within 48 hours. Requests must be made by the legitimate owner of the domain. If we have questions regarding your domain Phorm may take a number of steps, including attempting to contact the domain administrator by email for confirmation of this request. If the request remains questionable and is not confirmed within 10 days, the URL will be removed from the exclusion list and an email will be sent informing you of this decision. Where applicable, please ensure that the Administrative Contact details for this domain are up to date. If you need to update them, please resubmit your request when the amended details are visible in the WhoIs database - (use a public whois service such as _http://who.godaddy.com/whoischeck.aspx_ if you are unsure it has been updated)
European network outage
We’re encountering some networking problems between our Tampa and Amsterdam data centers, which is breaking access to the sites for people in Europe. Mark’s poking to see if it can be resolved; if necessary we’ll reroute European visitors directly to the Tampa center.
Update: Has been resolved.
Deadline for WMDE contract application
At the developer meetup, I announced that Wikimedia Deutschland is offering contracts for a couple of projects we feel are important. We again invite you to apply for any project that interests you.
The DEADLINE for applying is SUNDAY, APRIL 19!
We did not receive any offer for the most urgent project: Evaluate the impact of using flagged revisions on the German Wikipedia. We feel that it would be very helpful to run a full analysis on this before the English language Wikipedia decides on how to implement flagged revisions. It’s a powerful tool, and we should make sure we use it to it’s full potential.
Below, the other projects are listed again:
- Rewrite CatScan, a tool for finding pages in a set of categories recursively, based on various criteria.
- Store interwiki-links in the database, just like we already store interlanguage-links.
- Improve the Gadgets extension to allow for gadgets to be enabled per default, be restricted to specific user groups, etc.
- Implement full support for TIFF files, including multi-page TIFFs, similar to how DjVu is handled.
If you would like to help with any of the above, please contact me at <daniel.kinzler AT wikimedia.de> and provide the information specified at http://www.mediawiki.org/wiki/WMDE_contract_offers
Thank you all for your interest!
Skin & JS cleanup and jQuery
Michael Dale is working on some cleanup of how the various JavaScript bits are loaded by the skins to centralize some of the currently horridly spread-out code and make it easier to integrate in a centralized loader so we can serve more JS together in a single compressed request.
Unless there’s a strong objection I’d be very happy for this to also include loading up the jQuery core library as a standard component.
The minified jQuery core is 19k gzipped, and can simplify other JS code significantly so we can likely chop down wikibits.js, mwsuggest.js, and the site-customized Common.js files by a large margin for a net savings.
If you’ve done browser-side JavaScript development without jQuery and wanted to kill yourself, I highly recommend you try jQuery — it’s sooooo nice. :)
csw2-knams seems to have gone down
CSW2-knams is down and with it a few servers: pascal, ragweed, clematis, iris, fuchsia and a couple of sql-text*.knams.
It seems this issue mostly affects the toolserver environment.
I am still working on figuring out a way of fixing this and will update once the issue has been resolved.
Sorry for the inconvenience.
Update: Mark was able to resolve the issue. Apparently, the excess temperature due to the HVAC malfunction at the datacenter caused servers to automatically shutdown.
Developer Meet-Up Wrap-Up
During the Wikimedia Conference on April 3. to 5., MediaWiki developers and other technologically inclined people met at the c-base in Berlin. The idea was to meet people and get new (and old) projects moving. And I think it worked!
The biggest topic was probably the Integration of OpenStreetMap into Wikipedia (notes). This project should soon provide us with automatic maps for places, rivers, countries, etc. It will also provide interactive maps which can be panned and zoomed, similar to Google Maps. Highlighting places and objects and integrating sattelite images is being worked on. Wikimedia Germany is supporting the project by providing servers.
Another prominent topic was the Usability Initiative (notes) which has the goal of improving the user interface. Recently, the interaction of new users with the Wikipedia website has been studies in order to identify the things people have most problems with. The results should be available soon and will be used right away to improve the site. Meanwhile, Wikia is working on a WYSIWYG-Editor which greatly simplifies not only editing text but also makes the creation of tables and infoboxes much easier. Better methods for uploading and embedding images and other media are also being researched.
We talked about a lot of other things too, liked a system for systematically testing extensions, improving the handling of reports in Bugzilla, and the WikiTrust system, which can identify and highlight dubious changes in article text. The improved search was also discussed – some of the improvements can already be seen across Wikimedia sites, like the type-ahead suggestions when typing in the search box; others, like result from sister projects, are still limited to the English language Wikipedia, but should be available to all soon.
I
n the course of the event, I also presented WikiWord (Notes), which is the basis for the impending multi-lingual image search for Commons. The idea is to list the articles a word could refer to in a given language, find the corresponding pages or categories on commons, and present images from them. A prototype is available on the toolserver, but it has no images yet.
I found the meet-up very exciting and I hope that all participants has at least as much fun as I did. I would like to thank again all the people who made this event possible: Sebastian and Guillom, who organized the Wikimedia Conference as a whole, Henriette, Thomas and the rest of the Wikimedia Germany team, who managed all the things big and small that make such an event work, from accomodation to city maps. I want to thank all the volunteers who contributed their time, and especially the kitchen crew, who provided us with such great meals. My special thanks go to Lucas who helped me moderate the event and acted as my voice while I could hardly speak. Thanks also to the local folks of the c-base for their support and hospitality. And finally, I want to thank all the participants, who filled the event with life!
In conclusion I want to mention a few lessons learned: we need to have registrations much earlier, and have to make it clear that registrations are not definite until confirmed. We need to know how many people are coming before we decide on a venue, next time. The fact that we had to close registration early and even had to tell some people to stay home was really really sad. It really shouldn’t happen again.
Another point is that a schedule of prepared presentations might have helped giving an overview of topics and people. On the other hand, the flexible OpenSpace approach worked nicely to bundle topics and get people to sit down and talk. Perhaps next time we will do it like this: have a schedule of presentations ready for the first day, and leave the second day for more spontaneous discussions and demos. That way, we keep the creative chaos alive.
Indeed, next time… I’m sure that there will be a next time, hopefully next year already. Until then, we have Wikimania!
Pictures: Raymond
WMF needs additional datacenter space
The Wikimedia Foundation is looking into the option of expanding into a new datacenter. Currently the plans are tentative, but are expected to become much firmer once discussions with various Datacenter Providers takes place.
Currently, the servers for the projects reside in Tampa, Florida, USA, and in Amsterdam, Netherlands. We actually have moved the servers recently in Amsterdam. Now the time has come to move/expand in the US. We are looking at moving to an area OUTSIDE of Florida, where every single hurricane season is not the cause of distress.
We are currently looking in the Virginia and DC areas, but are not adverse to other areas given the space/power/transit issues. I have already been in contact with a number of vendors, but that doesn’t mean I do not want more options.
Things we require:
- We are looking for Datacenters that offer co-location services with 24/7 access.
- We also require racks have both primary and redundant power drops, from different feeds and circuits.
- The drops also need to be 3phase 208V power.
- Offers a low cost out of band access for our mangement network ONLY (no production traffic.)
- Some kind of NOC in residence in the event of ‘horrible end of the world’ happenings and we need remote hands. (We have LOM and remote reboot capabilities, but having a NOC is never horrible.)
Any interested sales folks at a datacenter can email me rob at wikimedia dot org. Put ‘Datacenter Relocation Project’ in the subject so I am sure I see it!
Also, any folks out there who have decent recommendations, let me have em!
Server named Singer has a sore throat?
In working on the servers, some apache config files were made inoperable. This is on a misc. services computer named Singer. This is the host for our blogs, as well as some other web-facing info. As such, the cached blogs are affected, but not the tech blog. (It was, but it was the easiest to get back online.)
Apologies for any annoyance this single server downtime may have caused anyone. Rest assured, it will be fixed and steps will be taken to prevent it from occurring in the future.
Wikimedia Deutschland offering short term contracts
Wikimedia Deutschland is offering contracts for a couple of projects we feel are important. If you are interested in earning some money by helping Wikimedia to improve our Wikis, have a look at these projects:
- Evaluate the impact of using flagged revisions on the German Wikipedia
- Rewrite CatScan, a tool for finding pages in a set of categories recursively, based on various criteria
- Store interwiki-links in the database, just like we already store interlanguage-links
- Improve the Gadgets extension to allow for gadgets to be enabled per default, be restricted to specific user groups, etc
- Implement full support for TIFF files, including multi-page TIFFs, similar to how DjVu is handled.
If you would like to help with any of the above, please contact User:Duesentrieb aka daniel.kinzler AT wikimedia.de and provide the following information:
- Your real name and country of residence
- How you plan to go about implementing the desired function
- Any experience working with MediaWiki
- How many working hours you would spend on it, and how much you ask for it
- In what time frame you would be able to do the job
Thanks for helping us make the web a better place!
OpenStreetMap maps will be added to Wikimedia projects
Posted by Ævar Arnfjörð Bjarmason in open-source, wikimedia on April 4th, 2009
There has been rapid progress on the subject of adding OpenStreetMap maps to Wikimedia projects (e.g. Wikipedia) during the MediaWiki Developer Meet-Up taking place right now in Berlin.
We now have a clear plan of action for getting OpenStreetMap maps embedded in Wikimedia wiki (e.g. Wikipedia) pages:
- Wikimedia will set up a database to mirror the OSM data (Planet.osm)
- Wikimedia will set up its own rendering infrastructure for rendering tiles & other maps from the OSM data
- The existing MediaWiki extensions for displaying OSM data in a MediaWiki article will be improved to work acceptably in production on Wikimedia servers
To prototype all this we’ll be using new infrastructure provided by Wikimedia Deutschland. Once things have been tested there they’ll eventually be deployed on the main Wikimedia sites.
After discussion with the Wikimedia operations people (including Brion Vibber, Mark Bergsma et al) there seem to be no objections to the above plan as long as:
- The maps will work not only for JavaScript enabled browsers but also non-JavaScript enabled ones
- The tools involved are improved to be relatively stable & deployable on Wikimedia, e.g. being able to embed more than one slippy map, the internationalization of error messages etc.
- The end product (the generated tiles or map files) are cachable so that they can be thrown at the frontend squids, as they’re static images this should be easy.
The featureset that we’re aiming for to be able to deploy this on Wikimedia sites from the view of the user (more can be added later once we’ve got it working) is:
- The ability to embed OSM maps in articles with something like the Simple image extension, perhaps automagically turning into a Slippy Map if the browser supports it
- A static or slippy map that can be used by geotagged articles so we can have maps without explicit inclusion of a <map> tag.
We’ll also set up a map toolserver for experimenting with other uses of OpenStreetMap data on Wikimedia. People with relevant projects can get access to this toolserver to try out their ideas for tools that could eventually be integrated on the main Wikimedia sites.
This project is seeking help from anyone who’s interested who’d like to be a part of making this happen, if you want to be a part of adding free maps to the world’s largest encyclopedia please subscribe to this mailing list:
And/or read/edit/comment on the relevant wiki coordination pages:
Planning to tighten TorBlock restrictions
en.wikipedia.org and others have seen a rash of abuse coming via Tor in the form of account creations with abusive names and such; this is taking up a large chunk of CheckUser and oversighter time and effort, which is apparently not too fun.
It looks like the current TorBlock settings don’t generally restrict various actions to a logged-in user when accessing through Tor; is there any objection to tightening this up to restrict edits, account creations, etc via Tor except when the account is explicitly excepted?