Confront the brutal facts: Five ways to measure your wiki's success.
February 19th, 2010
When you are introducing a wiki, you want to know how you are doing. Also you may like to confront your team with the actual usage numbers. You can only improve, what you measure.
The research on what other wikis typically use didn't bring a lot of insight. Apart from some helpful statistics on good'ol wikimatrix that simply compares wiki solutions, we didn't find a good set of indicators that tell you something about the health of your wiki. So we sat down with the team, and below you can find what we came up with. We'd love to get comments on what you found useful and what worked for you in the past.
First thing to decide is about the frequency of your measurements. Daily nor weekly make sense according to our experience. Remember, when setting up a wiki, you are in for the long run. Slowly you need to get it in the back of everybody's mind that the wiki needs updating, amending, correcting,... Showing the monthly view to your team is the optimal frequency.
1. Who's who
The (variation of) the height of the graph below shows how your user base is growing. One requirement of course is that you capture who is accessing the wiki. Many wikis do not require registration, some not even for writers. In a business environment, which is anyhow WordonWiki's target, this is less common. Management typically wants to 'control' who has what access.
The various stack segments show how many of our users 'can' read or 'can' write. Note that this only tells something about the segmentation of the user base, not about the actual use. The sample graph below is a typical example for a relatively small wiki, small in terms of users. In line with popular believe, most people should be able to correct something if they want to. So it is not uncommon to have most if not all users with write access.
2. Wiki Activity
Unless you have a very outspoken team, you will most likely see that most people are simply viewers. Only a minority is actually editing pages. Of course, a wiki is primarily used to get information, so growth in absolute numbers of logins is certainly a good thing. If you are brave, you could even divide the total number of logins (filtering out users that log in for a second or third time) by the number of (work) days. This gives you how many times per day, somebody turns to the wiki to get information or add something.
During our brainstorming, some one suggested to break down this number even further: to the team/unit level or even to the individual user. In the end, we're not convinced that this would be helpful to move things forward. However, it would be very confrontational to see who is almost never using the wiki, or who has rarely added something to the wiki. Please send us your idea's on this.
The case of the graph above clearly shows a wiki that is relatively new and a lot of information needs to be added. Even after 4-5 months of operation, a lot of edits still go on. When this is wiki for a more process oriented business, you will see the number of edits drop over time.
3. How recent is your wiki's information?
Next we present how many users did NOT log in for two weeks or more. Also we plot the number of pages that were not updated or read. Both of these lines have of course a correlation with the number of users and pages. For all lines on the graph, less is better.
4. Time for spring cleaning?
The longer you are active, the more junk you accumulate. This is true for all the stuff in your house, and it also applies to your wiki. When the ratio of pages that are not 'in use' becomes to high compared to the total number of pages, you are probably in need of some serious cleaning. Now people hesitate to get rid of old pages. 'We may still need it', is an often used excuse for not cleaning up.
One way to solve this is to create an 'Old' folder and move all outdated pages to that folder. You move stuff out of the way but you can still reach it if you need to.
One anecdote that painfully illustrates this point is our own story. Due to some software programming problem, the page delete option could not be used in one of our early releases. It took more than a month before we noticed this. None of our users had complained. It seems like a feature that nobody really missed!
5. Who is doing all the work?
The last graph shows two dimensions. Firstly, the size of the pie charts show the number of users over time. Secondly, each pie shows the percentage of the users that do 80% of the page updates. With sufficiently number of edits&users, this should ideally reach 80%. In reality that figure is likely to drop to 40% or even lower.
Over to you...
Please comment on this post to post your wishes for meaningful indicators and graphs.