Intermediate results of the icon tests: Crystal

With a series of icon tests we currently study effects on the usability of icon design. This article however does not focus on these general design effects but presents findings specific to the Crystal icon set.

Introduction

The introduction of the new Breeze icon set in KDE let us again wonder, what aspects of an icon set actually takes what impact on the usability of it. We investigated Oxygen and Tango Icons for the LibreOffice project before, but our focus then was on checking all icons of the standard tool bar. This time we focus on different icon sets and will use 13 common actions to compare them.

With this series we are going to test at least 10 different free icon sets: Breeze, Oxygen, Tango, Faenza, Nuvola, Nitrux, Elementary, Crystal Project, Humanity and Treepata. These icon sets differ on various aspects: use of color and details, flat or not and sometimes even on the metaphor used.

So, we generally want to analyze effects of icon design on the overall performance of an icon set. Statistics on this issue can obviously only be done after all icon sets have been tested. But with every test, we win some specific insights in strengths and weaknesses of each icon set tested.

In this post we share some findings about the Crystal icon set.

The study was finished by 513 participants (drop-out rate 5%) with an average handling time of 1:43 min.

Results of Crystal icons

Table 1 lists the aggregated quality indicators. They show how well all icons that we used for the test were suited to symbolize the different terms. It has a range from 1 (no fit) to 10 (perfect fit), whereas you would expect values of at least 9 for well represented terms.

Table 1: Quality of the icon set for different terms based on assignment ratio (percentage of missing assignments) and conspicuity (or speed of picking icons).
TermQuality Indicator
Link7.5
Open9
New9.5
Copy9.5
Paste9.5
Redo9.5
Undo9.5
Add10
Remove10
Save10
Print10
Search10
Cut10

Table 2 shows a cross-table with the percentage of false associations. These are terms where the intended icon was not chosen by the users, but some other icon was.

 Table 2: Cross-table of icons and terms with percentage of false associations. The direct match is inverted (1-value, e.g. 0.99 for Remove) to obtain comparable data.

Results_Icontest-Crystal

Discussion

The Crystal icon set was the default in KDE 3.x and is still available in LibreOffice for instance. The style aims to have a distinctive “crystal” look, mainly consisting in icons that appear to be made of highly reflective surfaces.

The icon set shows a couple of issues. First of all, New is just a plain sheet without any indicator of the underlying function. Actually, the Open icon is similarly reduced, at least in low resolution, but works however. Copy uses the same metaphor as the other sets, and receives a comparable high amount of mix-ups. Redo/Undo seems to be less structured because both icons uses the same color, but the results are very comparable to other sets. Obviously, the color information does not help here. And last but not least the Link icon shows some issues. The original set doesn’t contain any link icon, so it was taken from the Open Office repository that provides only a small size and not the conventional freedesktop organization. But to avoid the bias due to different set sizes we kept those non-standard icons in the study.

If you know how to design icons and would like to help us to identify metaphors that work better, please contact us. Also, all raw results are publicly available on our open usability platform UserWeave.

As mentioned before: These results only reflect the internal quality of the Crystal icon set. The final interpretation will be done soon. So stay tuned. And, of course, feel free to discuss these findings with us.