Results of the Tine 2.0 ‘Kristina’ Usability Benchmarking

We are investigating the users' satisfaction with the groupware Tine 2.0 for many years now. The results of the current benchmarking witness the success of our continuous usability work.

For many years we are investigating together with Metaways Infosystems the user experience (UX) of Tine 2.0. The benchmarking we developed on the one hand captures user satisfaction with the product in general. Also it assesses the impact and success of UX related developments. Typically we ask for the satisfaction in general and for specific assessments of the individual modules.

In this year’s survey, we have introduced an additional category to capture ‘soft’ factors. These should allow the users to also assess aspects that are functionally not relevant but often provide a large contribution to the experience of the software. These questions are geared to the ISO 9241-110. In principle, it is expected that software for example meets the expectations of the user, that the user is supported doing (complex) entries and that incorrect operations were considered during the development and therefore gross errors are avoided.

Results

The current questionnaire was completed by 24 users. The answers prove the consistently high level of acceptance of the software: all values ​​are in the positive range between 4.02 for the CRM module and 5.13 for the Address Book (see legend in Figure 1). The question about the satisfaction in general even achieved a slight increase over the last few years.

tine_general-satisfaction

Figure 1: Evolution of satisfaction with Tine 2.0. The whiskers indicate the 95% confidence interval. In the legend, the mean values ​​and variances of the individual questions for the current survey are given.

The user also rated the questions about the soft criteria with good to very good values ​​(Figure 2). The highest values ​​obtained the question of ‘Getting familiar with Tine 2.0’, which may result from the fact that only experienced users answered the questionnaire. The lowest value was given to ‘Speed of processing’.

tine_soft

Figure 2: Assessment of the current product in terms of ‘soft’, non-functional criteria.

Discussion

Tine 2.0 is much liked by the users. On average, high levels of satisfaction are given to all modules, both in terms of functional as well as generic aspects. However, this statement is ambiguous because of the small number of participants. So there are certainly also single negative responses (rather for less frequently used modules such as CRM). But a thorough assessment of individual responses is not possible. To this end, several similar statements need to be taken by the users, which can then be put into dependency with other variables. For example, the low rating for the CRM module, does not need to question the whole concept, but can also arise from a single missing functionality.

We will further investigate these questions in the next surveys.