Using the Kano method to prioritize requirements

On the basis of data from the user survey about KGet we show how to apply the Kano method for use in massively multi-user online surveys.

KGet is the download manager of KDE. As part of the current porting to the KF5 frameworks its functionality was taken under verification. In an online survey users were asked about their opinions, needs and requirements. The evaluation for the current program can be found in the posting on “What people think about KGet“. The analysis of requirements follows in this posting.

Introduction

The analysis of customer needs is an essential part of the development process. Requirements have to get registered thoroughly and prioritized concerning their relevance. There are a couple of self-evident functions usually not documented as particular requirements, like the function to save changed data. Other needs are quite specific and determined by the user’s workflow. Or the function might be even undesirable, like the implicit save of documents which makes the explicit action needless.

Kano method

The Kano method addresses this subject by inquiring the relevance for every feature:

  • I like it that way
  • I expect it to be that way
  • I am neutral
  • I can live with it that way
  • I dislike it that way

This procedure is done for both the positive, functional questions (‘How would you like if the document is saved automatically on changes?’) as well as the inverted, dysfunctional type (‘How would you like if the document is NOT saved automatically on changes?’).

This discrimination in a functional and dysfunctional dimension hearkens back to the ‘motivation-hygiene theory’ (Herzberg, 1966). The basic concept is that job satisfaction cannot be increased by ‘hygiene factors’ but decreases if those conditions are not met. ‘Motivators’ on the other hand always improve satisfaction. Regarding software the consistency of data at any time is a hygiene factor. A motivator could be that data changes are stored implicit and automatically. This point makes the evidence for quantitative analysis clear: some users might perceive the implicit save feature as convenient. Others are offended by the loss of control, however.

Categories

The Kano model characterizes requirements on five categories:

  • Must-be attributes (also known as basic needs) are taken for granted resulting in dissatisfaction when not fulfilled (“show stopper”)
  • Additional one-dimensional attributes (performance needs) diminish dissatisfaction or increase it when perfectly implemented
  • Attractive attributes (also known as exciters, delighters) are exiting features not expected by the user (“killer feature”)
  • Indifferent attributes refer to aspects that are neither good nor bad
  • Reverse attributes would lead to a rejection if implemented in this way

The idea of the model is often illustrated by a diagram with extend of implementation on the x-axis and satisfaction on the y-axis. The one-dimension, performance attributes contribute linear to satisfaction in this diagram, basic must-be features do not decrease dissatisfaction non-linearly, and exciters so non-linear increase satisfaction.

Analysis

The first step is to transpose the two-dimensional replies of participants to the five categories.

Table 1: Kano categorization.

Dysfunctional form of the question
Like itMust-beNeutralLive withDislike
Like itQAAAO
Must-beRIIIM
NeutralRIIIM
Live withRIIIM
Functional form of the question
DislikeRRRRQ
A=attractive, O=one-dimensional, M=must-be, I=indifferent, Q=questionable

The data are then usually normalized and the dysfunctional questions inverted (Berger et al., 1993). Since the used test platform only provides aggregated values we had to relinquish from this calculation.

The next step is to calculate satisfaction and dissatisfaction:

x=(O+M)/(A+O+M+I) #Dissatisfaction-1
y=(A+O)/(A+O+M+I) #Satisfaction

These values can be shown in a scatter plot. The distance of a point to the origin can be interpreted as importance index (KI), and the angle to the x-axis as satisfaction index (KI).

KI = sqrt(x²+y²)
KS = atan(y/x)

KGet

Requirements for KGet seem to be clear on the first glance: a download manager has to support the access of resources from the Internet. Whereas some years ago the major benefit was clearly for the management of bandwidth and monthly data cap this feature is today not relevant for most users. Because of the dissemination of Firefox and Chrome other solutions come into operation, furthermore. Therefor the whole concept of KGet needs verification.

Requirements

For new products the customer needs would be found by interviews with potential users. Since KGet and similar tools are well known it is possible to start with KGet’s functions plus solutions found in competitive products. To improve the readability of individual themes we introduce four epic tasks:

  • #1: KGet’s core feature is to download resources.
  • #2: KGet contains supportive functions to manage downloads.
  • #3: KGet is well integrated into KDE.
  • #4: KGet offers a plenty of settings for individual configuration.

This categorization is arbitrary and subjective. As well the association of KDE personas to themes is not verified. Learn more about the results concerning the current program in “What people think about KGet“.

The next table lists requirements (themes) and contains Kano indexes for the importance of an items (KI) between 0 and 2, and the influence to satisfaction (KS) between 0 and π/2. The higher the index the better.

Table 2: List of requirements.

#RequirementKIKS
1.1Susan wants to download from FTP and HTTP sources to store a local copy of files.1.090.57
1.2Matt wants to download via torrent protocol to make installation of KTorrent unnecessary. 0.530.96
1.3Berna wants to upload and synchronize data in order to have the all-in-one comfort.0.381.26
1.4Matt wants to use metalinks to verify the download and to retrieve data from multiple sources.0.400.96
1.5Susan wants to store complete websites to read it offline including formatting and images.0.311.38
1.6Susan wants to save video and audio streams to watch and see it offline.0.731.16
2.1Susan wants to download multiple files at once to have freedom in her workflow.1.120.65
2.2Susan wants to (re)start/stop a download or a group of downloads to shut down her computer over night.
1.080.65
2.3Susan wants to pause/resume a download or a group of downloads to get the full bandwidth for a video call.
1.100.66
2.4Susan wants to drag and drop URLs onto the KGet UI to start the download immediately.
0.600.85
2.5Susan wants to get her clipboard monitored in order to confirm if a downloadable ressource can be accessed by KGet.
0.361.23
2.6Berna wants to import/export tasks to keep the local copy of files up-to-date.0.231.09
2.7Berna wants to schedule actions to run bandwidth consuming tasks over night.
0.301.18
2.8Susan wants to organize downloads in groups to run default actions in respect to the predefined group.0.271.16
3.1Matt wants to run integrity and sanity checks on downloaded files for safety purpose.0.840.86
3.2Susan wants the organization of files to be done automatically to store all music at one folder and all pictures in another.
0.471.26
3.3Berna wants to prioritize downloads to adopt the process to her workflow.0.731.02
3.4Susan wants to get notified when a download is finished to make monitoring useless.
0.800.70
3.5Susan wants to have KGet well integrated into KDE’s activity concept to separate her private downloads from business.
0.391.00
3.6Berna wants to log all activities for documentation purpose.0.420.83
4.1Configuration of bandwidth limitation0.650.79
4.2Configuration of maximum download size per month/day
0.201.37
4.3Configurable action(s) on start (e.g. minimize)
0.450.94
4.4Configurable action(s) in case of interruption (e.g.run a script)
0.301.22
4.5Configurable action(s) when finished (e.g. remove from list, start program…)0.421.09
KI = Kano index of importance for users, KS = Kano index of user satisfaction

Kano results

With our evaluation, we follow the analytical approach by Xu et al. (2009) who do not split the evaluation matrix into four equally sized quadrants but use segments for interpretation. However, the thresholds of those segments are not defined by a particular value.

The equivalent Kano indexes importance (KI; distance of a point to the origin) and satisfaction (KS; angle to the x-axis) are listed in table 2 too. The next figure shows the results for hygiene factors (dissatisfaction) vs. motivators (satisfaction).

Figure 1: Kano representation of requirements.

Sources

  • Herzberg, F.I. (1966). Work and the Nature of Man. Cleveland: World Publishing.
  • Berger, C. et al. (1993): Kano’s method for understanding customer-defined quality, Center for Quality of Management Journal, 2(4), 3-35.
  • Xu, Q., Jiao, R. J., Yang, X., Helander, M., Khalid, H. M., & Opperud, A. (2009): An analytical Kano model for customer need analysis, Design Studies, 30(1), 87-110. [link]

If you want to reanalyze the study we offer raw data and R scripts: KGet-Kano.tar.gz

Discussion

The Kano method is used successfully for many years, however not as a ‘massively multi-user online survey” (MMOS). The present study is a first approach but leaves room for methodological improvements. The platform used provides access to aggregated data only, and the additional (explicit) question about relevance is not available.

In respect to KGet the study yields quite much indifference. That means a good proportion of participants does not have a clear opinion on features and understand options as nice but could live without. The results for classic downloads via ftp/http and the option to control the action (pausing, resuming etc.) are noticeable. The known features about prioritization, automatically organization, and the integration into Plasma receive medium acceptance as on-dimensional performance criteria. Users obviously expect extended functionality. We did not find clear excitements though. Most likely functions known from other tools like access to whole web sites or to store video/audio streams may be seen as an advantage. In this respect the integration into KDE’s activity concept and the ability to synchronize data should be taken into consideration as well.

These findings correspond with the results from the parallel study where only a few functions proofed to be important. On the other hand, this result could be attributed as well to the missing knowledge and familiarity of functions like ‘grouping’ or ‘metalinks’, that are not missed by the users therefore. But people who learned how to use those features love KGet for it, as it becomes apparent in the comments.

The future development has two options: concentration on only core functions and simplification of the user interface, or a better presentation of all the awesome features and enhancement of the program as a central management tool for resources.

What do you think about the Kano method? Which conclusion do you draw from the results for KGet? We are looking forward your comments.