Tuesday December 22nd, 2009 by Björn Balazs
It is great to see how many people participated in the first icon test for KMail: 3327 started the test and 2399 finished. This is a drop-out of about 28% across the whole study and fairly good for this kind of study – especially seeing the technical difficulties we had when we started the study.
Here is a first brief summary of the results:
- Selected language: English: 2244, German: 659, Spanish: 313, Polish: 109
- Gender: Male: 98%, Female: 2%
- Average Age: 29 years (from 10 to 88)
It is great to see that we got sufficient participants for reliable results in all languages!
As you might know, our test combines multiple indicators and calculates a single value for each icon-term relationship. The maximum value an icon-term-relation can reach is 10,0. Following I split up the results into 3 groups:
1. Icons working really well
All of the following icon-term-relationships show average values above 8 with no individual value below 7 for any language. We can assume all these icons to work well enough in all tested languages.
|9,6||Open Full Search|
|9,2||Open a new tab|
|8,2||Change Sort Order|
|8,2||Select View Appearance (Theme)|
2. Terms with no working Icons
For the following terms no fitting icons were identified. They definitely need to be redone.
|4,4||Select aggregation mode|
|3,2||Filter by Status|
3. Icons that need some attention
For the following terms the icons show to be the best, but only in comparison to what was presented. They could not convince. The results here need to be further examined and icons should be discussed and – if needed – improved.
Note that for Inbox and Check mail the same icon won. There is the same problem with Sent-mail and Outbox. These concepts are obviously not correctly differentiated by the users.
Some cultural notes
With these studies we also want to learn about cultural differences and their impact on the quality of icons. So here are some interesting facts found in the results – perhaps some native speakers can help to understand why…
While this icon was well understood in most languages Spanish (’Responder’) and Polish (’Odpowiedź’) speakers had much bigger troubles with this icon.
Only German speakers (’Weiterleiten’)understood this icon really well – all other had much bigger difficulties.
Only Spanish speakers (’borrador’) understood this icon really well.
Spanish speakers (’Crear tarea pendiente’) had much more problems with this icon.
Should anyone need the results in greater detail, please let me know. Under certain circumstances, I can make it possible to directly access the results. I planned to start the next test before christmas, but I won’t manage to do so. But we will definitely have Italian available then, and perhaps also Norwegian… Wishing all of you a happy christmas!