If you’re interested in learning more about Wikinews, I have created two more instructional pages: What is News? and Documenting your original sport reporting. A full list of instructional materials I have created for English Wikinews are available on Bookshelf. At some point, I feel like I should put these together into a book of sorts that could be easily distributed to encourage people to write for Wikinews. If anyone wants to do that for me… let me know. 🙂
Last weekend, I had the pleasure of attending the Wikimedia CEE conference where I was scheduled to speak about the Paralympics. I had spent a fair amount of time before the start of the conference talking to people about metrics, so five minutes before the start of the presentation, I informed the conference organizer I was going to do a presentation about metrics. Original session notes from the presentation are available here. I’ll discuss the same themes a bit here.
What is the value of a single edit to Wikipedia? What is the value of a single file upload to Commons? This is the metric we are often asked to use in the movement to assess the strength of our programming. Give me a dollar amount. How much is too much to spend dollar wise for a single edit? I asked the audience this, and no one knew. We know that 100 edits is good. We know editor retention as defined by making multiple single edits over a period of time is desirable. But what does that mean?
The answer to the question of what is the value of a single edit is nothing. In and of itself, edit counts are not a valuable form of assessing the effectiveness of programming. In order to effectively your programming, you need a basket of metrics to understand exactly what is going on. The basket of metrics need to be understood against the backdrop of your clearly defined objectives as they relate to your programming. Inside the movement, there is a tendency to use edit counts because they are an easy measure to do. Really easy data to get. Proper data analysis takes work. It takes contextualization. It actually starts before you even do your programming by understanding your goals and objectives.
I didn’t go into this much, but let’s do a radical rethink here. We don’t sit there after the fact and measure. We start out by writing objectives. (Objectives and goals are different. Objectives are clearly definable. Goals are broad general things. A goal is to learn to edit Wikipedia. An objective is to have a contributor make 50 total edits, where they add 100 KBs of content, four references and 3 pictures.) What are your objectives? Are they reasonable? How do your objectives compare to similar projects? What were the outcomes for similar projects? What can you change to improve your outcomes related to shared objectives?
Back to metrics. The example I used at the conference was asking the WMF to give Wikimedia Serbia US$500,000 to sent printed copies of Serbian and Macedonian Wikipedia into space. For this blog post, let’s go simpler.
Lots of Wikimedia work is local, and there are a lot of volunteers doing things that are not always in a chapter context. Much of my own work currently falls into this category, and I really like self-assessment. The European Wheelchair Basketball Championships are coming up next year in the Czech Republic. I’m in Spain. It is feasible for me to go. 8 days of wheelchair basketball with media accreditation. €40 a night for 9 nights is €360. €180 for airfare. Figure another €200 for food and transport. That’s €740 which is kind of eek. As a volunteer, I might be willing to spend that because ZOMG! WHEELCHAIR BASKETBALL IS AWESOME! As some one who likes to understand what they are doing, I need to define my objectives before going. I can tell you the budget does align with some what closely with the IPC Alpine World Championships, would be much less than the IPC NorAm Cup and the London Paralympics. It is probably about twice the cost of the Rollers and Gliders World Championships. These costs are important to know because they provide a budget template, an outcomes template, and can lay the ground work for objectives. They can assist me with goals.
Goals are broadly defined goals. A lot of these are simple, and can easily align with the strategic priorities. I want to improve content about a group under represented content and participation wise. I want to increase visibility of Wikimedia in these groups and more broadly speaking. I want to strengthen partnerships with strategic partners. I want to increase collaboration between sister projects and languages on the same project. A lot of this can be done cheaply, on the internet and at little cost to myself, which is why defining objectives is important. (I can write articles, look for pictures on Flickr, send an e-mail and comment on a water cooler.)
Objectives here would include creating and improving content about wheelchair basketball players in advance of the Rio Paralympic Games, take pictures of wheelchair basketball players to illustrate articles in multiple language Wikipedias, and publish news stories about the championships on multiple languages. I want to increase the visibility of wheelchair basketball content on Wikimedia projects by reaching a non-Wikipedia audience. I want to strengthen my relationship with the Spanish Paralympic Committee, and develop a relationship with the International Wheelchair Basketball Federation. I want to encourage people with disabilities to edit Wikipedia. I want to increase participation of the Wikimedia community in editing content about wheelchair basketball.
These all align with previous successful work. They are all aligned with past work. (In this case, my own. It doesn’t need to be. If some one else is doing something similar, ask them for their own metrics, what their outcomes are, what their goals are, what their objectives were and if they thought they aligned. This sort of talk in the community needs to happen more. Get relevant information from the source. These relationships can and do come in handy. Related objective to asking people: Improving your own performance when it comes to program delivery. Setting up your contact network to facilitate a move to another country, or getting a job in another chapter by creating your successes using what you learned. If you’re doing WLM in a country that hasn’t done so before, it only makes sense to model that work. Why re-invent the wheel? Discuss. Ask. Forget about what others think of your work. Think about your own definitions of success, or as this example suggests, spending €740 out of pocket for a benefit that makes you happy.)
So we have our objectives. These need to be more measurable. About $350 spent resulted in 8 Wikinews articles in one language and about 250 pictures uploaded to Commons. This was for a three day event. London had about 60 articles written in about 10 days at a cost of AU$14000. Insane, two people. Not a reasonable outcome. This is me comparing outcomes. With one reporter, I’d probably shoot for around 20 articles. This aligns most closely with the IPC Alpine World Championships. Why did I look at this objective first? Because it is the one that is the most important for me. English Wikinews articles frequently outperform comparable English Wikipedia articles, especially for articles about non-English speaking subjects. This goes to audience reach. I want to connect with that audience. Output here for English Wikinews articles? It actually builds in a fair amount of reach because all Wikinews content is linked on Facebook and on Twitter. Twitter links are often retweeted by one to three different accounts. Facebook material is often shared by one or more people from the official Wikinews account page. Facebook has a reach of about 30,000 people. Twitter has a potential reach of about 100,000 people. Thus, I know I need to be tracking all links on Facebook and Twitter for all 20 of the stories I want to write. How many of these were retweeted/reposted? Who did the retweeting? Was it IWBF? If so, this assists in building a strategic partnership and building awareness, which I have established as objectives. Who did that content resharing is just as important as how many people. When I am spend €740, I am not looking at total edits. (Which for Wikinews can be two edits or forty edits. Talk about useless metric when the outcome that matters is published articles.) I am looking at creating Wikinews content, engaging with people I want to partner with, building awareness with the general public and influencers in the space. These are all easy enough to measure, but they take time. The data is public. It is a basket of data to give context for success. If my content isn’t tweeted or Facebooked, it becomes a problem. I will not be as successful. It isn’t just content itself. (If on the other hand I had data to suggest Facebook and Twitter were not useful, I would pass on this and on objectives related to them. It is well worth the time to look at the effectiveness of your Wikimedia project’s effectiveness of driving traffic to content. Some accounts are better drivers of traffic than others. Some accounts are not about generating traffic, but about saving staff time by sharing messages in the same place so as to avoid having to answer the same time consuming e-mails on a topic. There, the ROI for social media is volunteer and staff time saving. This is where metrics get messy. What exactly are you measuring and why?)
Pictures matter. They tell a story and convey ideas. I want to take pictures of wheelchair basketball players beyond what I have for the Wikinews articles. I know from having done the Australian Paralympic material that sometimes uploading a less than great picture of a person results in a better image being donated by some one connected to the person. I also know articles with pictures appear to get more page views than those without pictures. (I have the data around some place.) Often articles with pictures appear to be of higher quality. Thus, taking pictures builds into multiple measurable objectives. First, it creates an incentive for people with disabilities or an interest in disabilities to upload similar content on Commons. It also incentives them to edit Commons. It incentives them to edit Wikipedia. Sometimes, it incentives them to edit Wikipedia in different languages. I could probably build in an objective at this point to create 8 DYKs about wheelchair basketball players as a way of personally measuring the quality of content improvement being done, increasing awareness of the content type, and the flow on effect of getting more contributors from Wikimedia projects to edit this content. Here, I can measure the number of editors to the Wikipedia content, and it makes sense because the context is improving content. I think I can safely define changing an image as generally improving an article, typo fixing as improving an article. I’d like more but realistically, previous cases say no. Still, it can be measured. Another thing that can be measured is how many times those pictures are used. What’s the value of 100 pictures on Commons if none of them are getting used? Using them on articles adds value and that should be recorded. Also, I’d want to be looking again at page views because back to that objective about reach, which includes Wikimedia page views.
So if looking at the metrics to assess personal value based on my objectives, I am generally speaking going to look at the volume of content of content created, traffic volume, content linking and content inclusion, the social media audience, and editor participation. These begin to give a true measure of impact. They are based on realistic objectives. They do not assume one ultimate goal. They have the general underlying knowledge that a single metric is not feasible in terms of understanding what is going on. If I measured based on edit count alone, I probably would not spend the €740 or only spend it with the knowledge that I’m a true sport fan going for the love of the Game. A well rounded approach, with clearly defined objectives that you know how to measure and collect, balanced against a historical overview can tell you the value of your programming, be it personal or be it chapter run.
I have accreditation for the Sochi Paralympics and I am going with a number of Ukrainian Wikimedians to cover the Games. Currently, Ukrainian Wikipedia has about 158 articles about the Paralympic Games, which have been viewed about 58,000 times since the start of the year. With accreditation coming from the USOC, it seems very important to create articles about USA Paralympians on the project. It also seems important to create articles about Ukrainian Paralympians as they are lacking. This would provide an excellent long term resource for the project. It would also get a lot of views during Sochi, and encourage local disability sport people in the region to edit newly created articles about themselves and those they may know.
This past month, I’ve attended two conferences where I have had the pleasure of meeting with and talking to people from a number of language communities about my efforts with Ukrainian Wikipedia. Several expressed interest in potentially doing the same for their languages if the data was good, and a way to translate and implement it could be found. There are a lot of benefits to doing this, including better engagement with and content creation about a diverse group of people including people with disabilities, women and people from outside the English speaking world. Normalizing the disability sport also assists society by showing that people with disabilities can do things. If a blind woman can successfully ski down a mountain at 100 kilometers an hour and be a professional skier, maybe you should consider hiring the blind person with an excellent resume as a computer programmer or marketing.
By manually datamining a number of sources including IPC medalists and competitor lists, world, regional and national record holder lists, world championship rosters, sportpeople lists from national Paralympic committees, classification lists, newspaper lists of medalists, and other sources, I have created a file that is fully referenced that can be used to show intrinsic notability (medalist or world record holder) and has some supplemental information. Each medalists should, in theory, have at least two sources as some Wikipedia projects require that.
In any case, I have tried to normalize and add more information to my dataset so people could start working on it in their own languages. The disciplines column and the birth location, current location columns are not as normalized as they could be. It is not 100% complete and unless I spent several months combing through various sources, I do not think it would be possible to easily get it much more complete.
https://simple.wikipedia.org/wiki/User:LauraHale/Paralympic_medalist_stub is a sample of how the data can be formatted. On English and Simple English Wikipedia, Paralympians are generally considered notable if they set a world record or if they won a Paralympic medal. I can explain the logistics more of how as a non-programmer the data file needs to be viewed. (Everything is sourced, but not every line is complete and notability may be derived from one line, but other key information from another line.)
If you have any questions, please ask me. 🙂 I would be happy to try to expand the data for specific countries upon request if they think there are holes in terms of missing medalists or world record holders. I would love to see articles created about Paralympians in multiple languages.
I love the Paralympics, and I love writing about them in the context of Wikimedia projects for a variety of reasons. If all goes well, I should be going to the Sochi Paralympics to cover US Paralympians. In preparing, I am trying to write more about the Paralympic Games from a sociological perspective of sorts. What are the Games? Why are they not better received by the US public? Who are US Paralympians as sport people? How do they fit into a broader US sporting landscape? This feels like important background information so when we cover US Paralympians at the Sochi Games, readers have this understanding and a better appreciation for the challenges in this sporting landscape. (With the Paralympics, it is too easy to fall into this trap of compelling personal story after compelling personal story. It really takes away from the awesome high level of sport taking place.)
In any case, in doing this reporting, I am trying to push boundaries both personally in terms of my reporting goals and with what Wikinews has done. My most recent effort is trying to get US Congresspeople to comment on the Paralympics. I contacted about 20 Congresspeople through their media people seeking comment on a variety of issues related to the Paralympics, and I got a response from Jackie Spieir, a San Francisco Democrat with the response now being published on Wikinews. Getting a response from a US House person is awesome for me as a journalist and for Wikinews as a project. 🙂
Almost two years ago, I became a Wikinews reporter. It has offered me a number of opportunities I would not have had otherwise. Along the way, I have learned a great many things. I’ve started this blog as a place to discuss Wikinews, to vent and to share about the project.