Results of KMail Icon Test #1

More than 3000 people took part in the KMail Icon Test. Most icons are working well, but some need attention. Interesting results are touching cutural aspects of the design of icons.

It is great to see how many people participated in the first icon test for KMail: 3327 started the test and 2399 finished. This is a drop-out of about 28% across the whole study and fairly good for this kind of study – especially seeing the technical difficulties we had when we started the study.

Here is a first brief summary of the results:

  • Selected language: English: 2244, German: 659, Spanish: 313, Polish: 109
  • Gender: Male: 98%, Female: 2%
  • Average Age: 29 years (from 10 to 88)

It is great to see that we got sufficient participants for reliable results in all languages!

As you might know, our test combines multiple indicators and calculates a single value for each icon-term relationship. The maximum value an icon-term-relation can reach is 10,0. Following I split up the results into 3 groups:

1. Icons working really well

All of the following icon-term-relationships show average values above 8 with no individual value below 7 for any language. We can assume all these icons to work well enough in all tested languages.

IconRatingTerm
Print10,0Print
Previous10,0Previous
Trash10,0Trash
OpenFullSearch9,6Open Full Search
Next9,2Next
OpenNewTab9,2Open a new tab
Reply9Reply
CreateToDo8,8Create To-do
ChangeSortOrder8,2Change Sort Order
SelectView8,2Select View Appearance (Theme)
NewMessage8,2New message

2. Terms with no working Icons

For the following terms no fitting icons were identified. They definitely need to be redone.

Best IconRatingTerm
SelectAggregationModus4,4Select aggregation mode
Template4,2Templates
SentMail4,0Sent-mail
FilterByStatus3,2Filter by Status

3. Icons that need some attention

For the following terms the icons show to be the best, but only in comparison to what was presented. They could not convince. The results here need to be further examined and icons should be discussed and – if needed – improved.

Best IconRatingTerm
Outbox7,8Outbox
Forward6,8Forward
Inbox6,2Inbox
Drafts6,2Drafts
CheckMail5,6Check mail

Note that for Inbox and Check mail the same icon won. There is the same problem with Sent-mail and Outbox. These concepts are obviously not correctly differentiated by the users.

Some cultural notes

With these studies we also want to learn about  cultural differences and their impact on the quality of icons. So here are some interesting facts found in the results – perhaps some native speakers can help to understand why…

Reply Reply

While this icon was well understood in most languages Spanish (’Responder’) and Polish (’Odpowiedź’) speakers had much bigger troubles with this icon.

Forward Forward

Only German speakers (’Weiterleiten’)understood this icon really well – all other had much bigger difficulties.

 DraftsDrafts

Only Spanish speakers (’borrador’) understood this icon really well.

CreateToDo Create To-do

Spanish speakers (’Crear tarea pendiente’) had much more problems with this icon.

Should anyone need the results in greater detail, please let me know. Under certain circumstances, I can make it possible to directly access the results. I planned to start the next test before christmas, but I won’t manage to do so. But we will definitely have Italian available then, and perhaps also Norwegian… Wishing all of you a happy christmas!