Results of KMail Icon Test #1

More than 3000 people took part in the KMail Icon Test. Most icons are working well, but some need attention. Interesting results are touching cutural aspects of the design of icons.

It is great to see how many people participated in the first icon test for KMail: 3327 started the test and 2399 finished. This is a drop-out of about 28% across the whole study and fairly good for this kind of study – especially seeing the technical difficulties we had when we started the study.

Here is a first brief summary of the results:

  • Selected language: English: 2244, German: 659, Spanish: 313, Polish: 109
  • Gender: Male: 98%, Female: 2%
  • Average Age: 29 years (from 10 to 88)

It is great to see that we got sufficient participants for reliable results in all languages!

As you might know, our test combines multiple indicators and calculates a single value for each icon-term relationship. The maximum value an icon-term-relation can reach is 10,0. Following I split up the results into 3 groups:

1. Icons working really well

All of the following icon-term-relationships show average values above 8 with no individual value below 7 for any language. We can assume all these icons to work well enough in all tested languages.

Icon Rating Term
Print 10,0 Print
Previous 10,0 Previous
Trash 10,0 Trash
OpenFullSearch 9,6 Open Full Search
Next 9,2 Next
OpenNewTab 9,2 Open a new tab
Reply 9 Reply
CreateToDo 8,8 Create To-do
ChangeSortOrder 8,2 Change Sort Order
SelectView 8,2 Select View Appearance (Theme)
NewMessage 8,2 New message

2. Terms with no working Icons

For the following terms no fitting icons were identified. They definitely need to be redone.

Best Icon Rating Term
SelectAggregationModus 4,4 Select aggregation mode
Template 4,2 Templates
SentMail 4,0 Sent-mail
FilterByStatus 3,2 Filter by Status

3. Icons that need some attention

For the following terms the icons show to be the best, but only in comparison to what was presented. They could not convince. The results here need to be further examined and icons should be discussed and – if needed – improved.

Best Icon Rating Term
Outbox 7,8 Outbox
Forward 6,8 Forward
Inbox 6,2 Inbox
Drafts 6,2 Drafts
CheckMail 5,6 Check mail

Note that for Inbox and Check mail the same icon won. There is the same problem with Sent-mail and Outbox. These concepts are obviously not correctly differentiated by the users.

Some cultural notes

With these studies we also want to learn about  cultural differences and their impact on the quality of icons. So here are some interesting facts found in the results – perhaps some native speakers can help to understand why…

Reply Reply

While this icon was well understood in most languages Spanish (’Responder’) and Polish (’Odpowiedź’) speakers had much bigger troubles with this icon.

Forward Forward

Only German speakers (’Weiterleiten’)understood this icon really well – all other had much bigger difficulties.

 DraftsDrafts

Only Spanish speakers (’borrador’) understood this icon really well.

CreateToDo Create To-do

Spanish speakers (’Crear tarea pendiente’) had much more problems with this icon.

Should anyone need the results in greater detail, please let me know. Under certain circumstances, I can make it possible to directly access the results. I planned to start the next test before christmas, but I won’t manage to do so. But we will definitely have Italian available then, and perhaps also Norwegian… Wishing all of you a happy christmas!