Apache Kafka – [[WM:TECHBLOG]]

Wikimedia’s Event Data Platform – Event Intake

Part 3 of 3 posts on Wikimedia’s event data platform.

https://commons.wikimedia.org/wiki/File:Vue_a%C3%A9rienne_raproch%C3%A9e_du_grand_r%C3%A9cif_de_Gatope_et_son_trou_bleu.jpg

Wikimedia’s Event Data Platform – JSON & Event Schemas

In the previous post, we talked about why Wikimedia chose JSONSchema instead of Avro for our Event Data Platform. This post will discuss the conventions we adopted and the tooling we built to support an Event Data Platform using JSON and JSONSchema.

https://commons.wikimedia.org/wiki/File:Dobson_Stream_by_Wharfedale_hut_with_the_moon,_Mt_Oxford_area,_Canterbury,_New_Zealand.jpg

Wikimedia’s Event Data Platform, or JSON is ok too

The Wikimeda Foundation has been working with event data since 2012. Over time, our event collection systems have transitioned from being used only to collect analytics data to being used to build important user facing features. This 3 part series will focus on how Wikimedia has adapted these ideas for our own unique technical environment.