Harbor UCLA Medical Center Factors that Impact Usability Discussion

User Generated

SSSSErq

Writing

Harbor UCLA Medical Center

Description

Which issue(s) do you think have been assigned salience by audiences due to the agenda-setting process that might not otherwise be considered salience? Why might the media drive the transfer of salience for that issue(s)?

Why do you think usability is important? Use examples (both personal and from the readings) to explain your point of view.

Each queation one page

Unformatted Attachment Preview

Contents lists available at ScienceDirect Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo User-centred web design, usability and user satisfaction: The case of online banking websites in Iran Iman Dianata,*, Pari Adelia, Mohammad Asgari Jafarabadib, Mohammad Ali Karimic a b c Department of Occupational Health and Ergonomics, Tabriz University of Medical Sciences, Tabriz, Iran Tabriz Health Service Management Research Center, Tabriz University of Medical Sciences, Tabriz, Iran Department of Computer Engineering, Bushehr Branch, Islamic Azad University, Bushehr, Iran A R T I C LE I N FO A B S T R A C T Keywords: E-banking E-satisfaction Internet banking Web site The relationship of Web design attributes (personalisation, structure, navigation, layout, search and performance) and users' personal characteristics to website usability and user satisfaction was investigated among 798 online banking users in Iran. The design and usability of the evaluated websites were not satisfactory from the users’ perspectives. Multivariate regression models indicated that Web layout and performance were the main predictors of website usability, while personal characteristics including gender, age and Web usage experience of users had no effect. User satisfaction was also influenced only by the Web design attributes (particularly Web structure) and not by the personal characteristics of the users. There was also a significant relationship between website usability and user satisfaction. The findings suggest that the website designers should focus more on the Web design attributes (particularly Web layout and structure), regardless of the personal characteristics of their users, to improve the usability and user satisfaction of websites. 1. Introduction Websites are the most important communication tool and primary interface for users who are searching for online information and products (Kim and Stoel, 2004). A website can be defined as a set of connected interfaces and functional attributes designed to deliver high levels of performance and usability to users (Lee and Koubek, 2010). Organisations and companies generally use websites to achieve marketing and business objectives. In designing websites, highest priority should be given to the users, as the main source of profit for the organisations/companies (Lee and Koubek, 2010). User preference is, therefore, an important factor that should be taken into account in relation to website design. The user preference, usually measured through an interview or a questionnaire, is generally expressed in terms of design, usability, performance, aesthetics, satisfaction, online shopping, information quality, brand, etc. (Abelse et al., 1998; Helander and Khalid, 2000; Cyr and Bonanni, 2005; Zviran et al., 2006; Lee and Koubek, 2010; Tandon et al., 2016). However, usability and user satisfaction in terms of system use and acceptance have been regarded as two aspects of user preference that are of utmost importance in Web success (Palmer, 2002; Muylle et al., 2004; Zviran et al., 2006; Lee and Koubek, 2010). This is particularly the case for online banking, which its users are growing rapidly worldwide due to its convenience and ease of transactions and customer services (Yoon, 2010). Currently, almost all Iranian banks are offering online banking. Iran, with one of the highest rates of internet-using population in the world (over 33 million users), has a constantly growing part of its population using online banking (Hojjati et al., 2015). It is, therefore, important to investigate and understand the usability and user satisfaction to ensure usercentred design of these websites. 1.1. User-centred web design guidelines In recent years, increasing attention has been paid to the design of websites by researchers from various disciplines. A number of publications have suggested guidelines and recommendations regarding website design (For example, see Abelse et al., 1998; Shneiderman, 1998; Head, 1999; Palmer, 2002; Garrett, 2003; McCracken and Wolfe, 2004). However, as acknowledged by several investigators (Zviran et al., 2006; Lee and Koubek, 2010), these design guidelines are either insufficient or not detailed enough to be applicable to specific cases. Abelse et al. (1998) proposed six criteria (including use, content, linkage, structure, special features, and appearance) for constructing Web sites with the user in the focus. Other investigators have * Corresponding author. E-mail addresses: dianati@tbzmed.ac.ir (I. Dianat), pari.adeli@yahoo.com (P. Adeli), m_asgharl862@yahoo.com (M. Asgari Jafarabadi), karimi_a@live.com (M.A. Karimi). https://doi.org/10.1016/j.apergo.2019.102892 Received 31 May 2017; Received in revised form 6 November 2018; Accepted 30 June 2019 I. Dianat, et al. considered three elements including information content, navigation design, and visual design for website design (Cyr and Bonanni, 2005; Cyr, 2008). Structure, layout, and aesthetic aspects were recommended by Lee and Koubek (2010) as Web design attributes. 1.2. Website usability Usability is a relatively complex concept because of the complexity of human beings as well as the product features itself. For this, the usability has been defined in several different ways and various methods have been proposed to measure it. Flavián et al. (2006) define the website usability as “the perceived ease of navigating the site or making purchases through the Internet”. The contexts of use and user's experience of interacting with the system are key elements of usability in human–computer interaction (HCI) (Baber, C., 2005). To the authors' knowledge, the relationship between usability and different Web design attributes has not been explored in a multivariate context. As a result, it is difficult to determine which Web design attributes are more important from a usability point of view. In addition, while several previous studies have examined the effect of website usability on user satisfaction when shopping online (Fu and Salvendy, 2002; Schaupp and Bélanger, 2005; Flavián et al., 2006; Casaló et al., 2008; Belanche et al., 2012; Tandon et al., 2016), much less attention has been paid to other online services such as online banking. And some of these studies have shown that usability is the most important predictor of user satisfaction (Belanche et al., 2012; Tandon et al., 2016), while others have not shown this to be the case (Schaupp and Bélanger, 2005). Therefore, additional research may be needed to shed further light on this issue. 1.3. Website user satisfaction The rapid growth of online services emphasises the need for increased attention to user satisfaction as a key factor in designing websites. Website user satisfaction can be defined as: “the attitude toward the website by a hands-on user of the site” (Muylle et al., 2004). Research on user satisfaction is very important because it not only influences the customer behavioural outcomes such as loyalty, trust and purchase intention (Shankar et al., 2003; Gustafsson et al., 2005; Flavián et al., 2006), but also is a key factor to profitability (Chiou and Shen, 2006). Researchers have established various models from different perspectives to describe the user satisfaction (Doll et al., 1994; Szymanski and Hise, 2000; McKinney et al., 2002; Muylle et al., 2004; Schaupp and Bélanger, 2005). Doll et al. (1994) proposed a model of website user satisfaction that consisted of five system attributes including content, format, accuracy, timeliness, and ease of use. A number of researchers have highlighted the impact of website's general design (Szymanski and Hise, 2000; Liu et al., 2008; Yoon, 2010; Guo et al., 2012; Tandon et al., 2017) or some specific aspects of website design such as interface consistency (Ozok and Salvendy, 2000) or website quality (McKinney et al., 2002; Kim and Stoel, 2004; Ha and Im, 2012) on user satisfaction. However, these studies have not specifically focused on the impact of different Web design attributes on user satisfaction. In one of the few attempts to address this issue, Zviran et al. (2006) found that content and search capabilities of commercial websites were predictors of user satisfaction. From a user-centred design point of view, this is particularly important because better understanding in this area may lead to the development of more specific guidelines and recommendations with regard to website design, and consequently to more user satisfaction. In addition, there are variations between different cultures in terms of design preferences and user satisfaction (Marcus and Gould, 2000; Cyr, 2008; Liu et al., 2008), which underscore the need for further research in this area. 1.4. Users’ personal characteristics A review of literature reveals that very few studies have examined the role of demographic differences in relation to website design preferences. In an e-business setting, Simon (2001) examined differences in perception of website design (in terms of information richness, communication effectiveness, and the communication interface) between genders and found that female users prefer websites with less clutter and fewer graphics compared to male users. Relatively similar findings were reported by Simon and Peppas (2005), who evaluated simple and complex product websites. Another study, conducted by Cyr and Bonanni (2005) on website design in e-business, demonstrated gender differences in perception of website design (in terms of information design, navigation design and visual design) and website satisfaction. Some other researchers have examined the influence of a specific design factor (e.g. aesthetic perception of websites) and reported significant gender differences in this regard (Cousaris et al., 2008; Tuch et al. (2010). The users’ attitudes and behaviours may vary according to their other personal characteristics and culture (Yoon, 2010). It is, therefore, apparent that further research is required to examine the individual differences such as gender, age, occupation, education and Web usage experience in relation to various aspects of Web design as well as to usability and user satisfaction. Better understanding and knowledge in this area will have important implications for the Web design process and help designers to consider the needs and expectations of different users and design inclusively for them. 1.5. Rationale Based on the above mentioned background, the aim of this study was to answer the following research questions: 1) How are the user-centred Web design, usability, and user satisfaction of online banking websites in Iran? 2) How is the relationship of user-centred Web design and users' personal characteristics (including gender, age, occupation, education and Web usage experience) to usability and user satisfaction of online banking websites? 3) How is the relationship between usability and user satisfaction of online banking websites? 2. Methodology 2.1. Study setting, participants and procedure A total of 798 online banking users of four main banks (Table 1) in Iran participated in this study. The study was conducted over a fivemonth period between August and December 2016 in Tabriz–Iran. Users who were older than 18 years, and had used the corresponding online banking system at least for one year, once a week, were considered as the target population for the study. Permission to access the research sites and conduct the study was obtained from the bank authorities involved. One investigator then visited the selected bank branches for data collection. A questionnaire was administered to collect data about demographic factors (including age, gender, occupation, education, and Web usage experience), Web design attributes (by using Table 1 Four online banking websites evaluated in the study. Websites URL Website Website Website Website https://www.bmi.ir http://www.banksepah.ir http://www.bsi.ir http://www.tejaratbank.ir 1 2 3 4 I. Dianat, et al. the user-centred Web design (UCWD) questionnaire), website usability (by using the system usability scale [SUS]), and user satisfaction of websites (by using the end-user satisfaction (EUS) assessment scale). The participating users were instructed to complete the questionnaire in the bank premises and return it to the investigator before leaving. Participation in this study was voluntarily and written consent was obtained from each user before participation. Ethical approval for the study was granted from the ethical review committee of the Tabriz University of Medical Sciences. 2.2. Instrument and outcome measures The instruments used in the study included the following: 1) UCWD questionnaire (Abelse et al., 1998), 2) SUS (Brooke, 1996), and 3) EUS assessment scale (Doll et al., 1994). Demographic details including age, gender, occupation (office worker, manual worker, university student, and other), education level (diploma, undergraduate and postgraduate), and Web usage experience (years) were also recorded. Items regarding the UCWD were based on the questionnaire developed originally by Abelse et al. (1998), and further developed by Zviran et al. (2006). This is a valid and reliable tool which contains 17 questions regarding the website personalisation (3 items, personalisation of the site and complete information on basic facts and product features), structure (2 items, logical categorisation of the content and multimedia/graphics that strictly support the site purpose), navigation (2 items, error handling and navigation aids), layout (4 items, nonoverlapped and consistent navigation bars, update clues indicating user's location on the site and page arrangement), search (3 items, dealing with misspellings and synonyms and presenting the results in relevant order), and performance (3 items, cross-browser compatibility and browser-specific requirements as well as errors such as JavaScript error messages of the site). Using a 5-point Likert scale (from 1 – very low to 5 – very high) respondents rated their reactions to the usercentred design of banking websites. Website usability was measured using the SUS proposed by Brook (1996). The SUS is a widely used and validated tool for measuring the perceived usability of a wide range of products and user interfaces, including websites (Brooke, 1996; Zviran et al., 2006). The SUS is a 10item scale that evaluates various aspects of usability of a system or product. All items have 5-point Likert scale responses (from 1 – strongly disagree to 5 – strongly agree), and the scale gives an overall usability score ranging from 0 to 100. Higher scores indicate higher levels of usability. Finally, the EUS assessment scale (Doll et al., 1994), which is a valid and reliable instrument and consists of 12 items, was used to measure the users’ satisfaction from banking websites. It consists of five criteria for assessing the user satisfaction including content (4 items, provision of required, precise and sufficient information), accuracy (2 items, accuracy of the system and satisfaction with it), format (2 items, useful format and clear information), ease of use (2 items, ease of use and user friendliness of the system) and timeliness (2 items, required and up to date information). Each item is rated on a 5-point Likert scale (from 1 – very low to 5 – very high), where a higher score indicates a higher satisfaction level. 2.3. Validity and reliability issues The original version of the SUS has been translated and revised into Farsi (Iranian language), and therefore has an established reliability and validity (Dianat et al., 2014, 2015). The English versions of UCWD and EUS scales were also translated into Farsi and verified for content validity by ten experts (ergonomists and psychologists). In addition, internal consistency reliability (using Cronbach's alpha) and stability reliability (using Intraclass correlation coefficient – ICC) of the measures on a sample of 81 participants were also found to be good. The Cronbach's alpha for the SUS was 0.82. The Cronbach's alpha for the Table 2 Users’ personal characteristics and their relationships with the study variables (n = 798). n (%) Gender Male Female Age (yr) < 30 30–45 > 45 Occupation Office worker Manual worker University student Other Education Diploma Undergraduate Postgraduate Web usage experience ≤2 3–4 ≥5 Web type Website 1 Website 2 Website 3 Website 4 Total UCWD score SUS score EUS score Mean (SD) Mean (SD) Mean (SD) 515 (64.5) 283 (35.5) 42.5 (6.0) 41.3 (5.8) 51.8 (7.5) 52.3 (8.2) 41.3 (6.3) 40.8 (6.3) 86 (10.8) 601 (75.3) 111 (13.9) 43.2 (4.8) 41.9 (6.2) 42.3 (5.5) 52.1 (6.9) 52.0 (7.9) 51.6 (7.5) 41.5 (6.3) 41.2 (6.4) 40.8 (5.5) 458 (57.4) 96 (12.0) 86 (10.8) 158 (19.8) 41.8 43.3 41.4 52.3 51.7 53.2 53.1 51.9 41.3 42.1 40.2 40.5 88 (11.0) 486 (60.9) 224 (28.1) (yr) 296 (37.1) 303 (38.0) 199 (24.9) 200 199 200 199 (25.1) (24.9) (25.1) (24.9) 798 (100) (6.1) (5.3) (5.1) (6.1) (7.6) (7.8) (8.5) (7.8) (6.3) (5.2) (6.9) (6.5) 42.3 (5.9) 42.2 (5.8) 41.8 (6.3) 53.6 (7.3)* 52.3 (7.9) 50.9 (7.2) 41.9 (7.4) 41.2 (6.2) 41.0 (6.4) 42.8 (5.7)* 41.6 (6.2) 41.7 (5.9) 52.4 (7.7) 51.5 (8.0) 52.0 (7.3) 41.3 (6.4) 40.6 (6.0) 41.7 (6.6) 42.6 (5.2)*** 42.9 (5.1) 39.0 (6.1) 43.9 (6.2) 53.3 (7.3)*** 52.7 (7.5) 49.5 (9.0) 52.5 (6.2) 41.4 40.7 40.6 41.9 42.1 (5.9) 52.0 (7.7) 41.1 (6.3) (5.6) (5.7) (7.2) (6.5) Statistically significant values are shown in bold. *p < 0.05; **p < 0.01; ***p < 0.001. UCWD and its subscales were as: total UCWD (0.78), personalisation (0.69), structure (0.73), navigation (0.71), layout (0.75), search (0.71), and performance (0.72). The Cronbach's alpha for the EUS and its subscales were as: total EUS (0.88), content (0.86), accuracy (0.76), format (0.72), ease of use (0.68) and timeliness (0.70). The ICC index for each of the three measures was 0.99 (95% CI = 0.98 to 1.00). 2.4. Data analysis Statistical analysis of the data was performed using STATA software, V.13 (StataCorp, College Station, Texas 77845 USA). Normality of the numeric variables was checked by Kolmogorov–Smirnov test. Data were presented as mean (standard deviation−SD) and median (minmax) for the numeric normal and non-normal variables, respectively, as appropriate, and frequency (%) for categorical variables. Between–group comparisons of baseline measures and demographic variables were performed using independent t-test, analysis of variance (ANOVA) or Chi-square test, as appropriate. Univariate and multivariate regression modelling was also used to examine the relationship between study variables. In the multivariate models, the effects of confounders were adjusted and the categorical variables were entered as indicators. To comprehensively examine the relationship between usability and user satisfaction with other study variables, and considering the effect of confounders, a three step hierarchical modelling was utilised in three models. In the first model, the gender, age, educational level, occupation, Web type and Web usage experience were selected as the independent variables and the usability (SUS score) and user satisfaction (total EUS score) were selected as dependent variables. The second model examined the relationship of independent variables in the Model 1 plus Web design attributes (dimensions of UCWD including rationalisation, structure, navigation, layout, search and performance) to usability and user satisfaction as dependent variables. The third model examined the relationship of independent variables in the I. Dianat, et al. Table 3 Univariate regression analyses for the relationship between study variables. Variables Usability B Gender Male Female Age (yr) < 30 30-45 > 45 Educational level Diploma Undergraduate Postgraduate Occupation Office worker Manual worker Student Others Web type Website 1 Website 2 Website 3 Website 4 Web usage experience (yr) 1–2 3–4 ≥5 UCWD Personalization Structure Navigation Layout Search Performance Usability (SUS score) End-user satisfaction SE 95% CI B Lower bound Upper bound SE 95% CI Lower bound Upper bound 1.00 0.444 – 0.575 – −0.68 – 1.57 1.00 −0.532 – 0.469 – −1.45 – 0.38 1.00 −0.062 −0.484 – 0.897 1.117 – −1.82 −2.67 – 1.69 1.70 1.00 −0.361 −0.731 – 0.732 0.913 – −1.80 −2.52 – 1.07 1.06 1.00 −1.249 −2.647 – 1.300 1.359 – −3.80 −5.31 – 1.30 0.02 1.00 −0.762 −0.946 – 1.067 1.115 – −2.85 −3.13 – 1.33 1.24 1.00 1.509 1.405 0.275 – 0.872 1.344 0.649 – −0.20 −1.23 −0.99 – 3.22 4.04 1.55 1.00 0.760 −1.142 −0.802 – 0.711 1.097 0.530 – −0.63 −3.29 −1.84 – 2.15 1.01 0.23 1.00 −0.636 −3.850*** −0.762 – 0.765 0.764 0.765 – −2.13 −5.35 −2.26 – 0.86 −2.34 0.73 1.00 −0.751 −0.795 0.429 – 0.635 0.634 0.635 – −1.99 −2.03 −0.81 – 0.49 0.44 1.67 1.00 −0.940 −0.393 – 0.635 0.712 – −2.18 −1.79 – 0.30 0.100 1.00 −0.721 0.365 – 0.518 0.581 – −1.73 −0.77 – 0.29 1.50 0.995*** 1.840*** 1.845*** 1.566*** 1.040*** 1.698*** – 0.162 0.206 0.020 0.097 0.175 0.293 – 0.67 1.43 1.44 1.37 0.69 1.12 – 1.31 2.24 2.24 1.75 1.38 2.27 – 1.505*** 3.158*** 2.060*** 1.381*** 0.930*** 1.204*** 0.295*** 0.125 0.136 0.157 0.077 0.142 0.241 0.027 1.25 2.89 1.75 1.22 0.64 0.73 0.24 1.75 3.42 3.69 1.53 1.21 1.67 0.34 Statistically significant values are shown in bold. *p < 0.05; **p < 0.01; ***p < 0.001. Model 2 plus usability (SUS score) to user satisfaction as dependent variable. Standardised regression coefficients (β) and their corresponding 95% confidence intervals (CIs) as well as explanatory power (adjusted R-square – R2) were presented to indicate the predictive strength of independent variables and the explanatory power of each model, respectively. P values less than 0.05 were considered as statistically significant for all tests. (p < 0.001) (Table 2). Web usage experience had also a significant effect on the mean UCWD score (p < 0.05), so that a longer experience (more than 2 years) of using the internet resulted in lower scores on the UCWD scale. However, no significant difference was found in the UCWD score in terms of the age and gender of the participants. 3. Results The mean (SD) SUS score was 52.0 (7.7). The ANOVA results indicated a statistically significant difference between the four evaluated websites in term of their usability (p < 0.001). A significant difference was also found in the SUS scores in terms of the education level of the respondents (p < 0.05). This indicated that those participants with a lower level of education (e.g. diploma) rated higher scores on the SUS than those with a higher level of education. Other demographic variables had no significant effect on the SUS scores. 3.1. Demographic details A total of 900 online banking customers were approached, of which 833 (response rate = 92.5%) accepted to participate in the study. Of those that completed the questionnaire, 35 were excluded for incomplete questionnaires. Therefore, the responses from 798 (515 males and 283 females) were included in the analyses. The age of participants ranged from 19 to 55 years (mean = 33.9 years; SD = 6.1 years). They had been using the internet banking between 1 and 6 years (mean = 3.3 years; SD = 1.4 years). More than half of the respondents (57.4%) were office workers, and had undergraduate (60.9%) education. 3.3. Usability 3.4. End-user satisfaction The mean (SD) score for the EUS scale was 41.1 (6.3). The results of the study showed no statistically significant difference within the subgroups of the demographic factors in terms of the EUS score. 3.2. User-centred web design 3.5. Regression models The mean (SD) score of the total UCWD scale was 42.1 (5.9). According to the ANOVA results, the mean UCWD score was significantly different between the four websites evaluated in the study The results of the univariate and multivariate regression analyses are summarised in Table 3 through 5. According to the results of multivariate regression modelling presented in Table 4, both Web I. Dianat, et al. Table 4 Multivariate regression analysis for the relationship between study variables and website usability. Model 1 β Gender Male Female Age (yr) < 30 30-45 > 45 Educational level Diploma Undergraduate Postgraduate Occupation Office worker Manual worker Student Others Web type Website 1 Website 2 Website 3 Website 4 Web usage experience (yr) 1–2 3–4 ≥5 UCWD Personalization Structure Navigation Layout Search Performance Model 2 β 95% CI Lower bound Upper bound 1.00 0.044 – −0.40 – 1.86 1.00 0.040 −0.006 – −2.09 −2.99 – 0.96 0.84 1.00 −0.127 −0.209** – −4.67 −6.33 – 0.46 −0.89 1.00 0.053 0.075 0.021 – −0.57 −0.01 −1.04 – 3.11 5.68 1.78 1.00 −0.042 −0.227*** −0.040 – −2.29 −5.59 −2.29 – 0.78 −2.55 0.83 1.00 −0.022 0.029 – −1.67 −1.09 – 0.95 2.15 95% CI Lower bound Upper bound 1.00 −0.087 −0.139* – −3.62 −4.67 – 0.73 −0.13 1.00 −0.060 −0.141*** −0.048 – −2.36 −3.92 −2.21 – 0.21 −1.14 0.48 −0.038 0.052 0.004 0.456*** −0.006 0.204*** −0.51 -.13 −0.43 1.19 −0.39 1.20 0.15 0.77 0.49 1.69 0.32 2.25 Model 1: Adjusted for gender, age, educational level, occupation, Web type and Web usage experience. Model 2: Model 1 plus Web design attributes (personalization, structure, navigation, layout, search and performance). Adjusted R2: Model 1 = 0.042; Model 2 = 0.293. Statistically significant values are shown in bold. *p < 0.05; **p < 0.01; ***p < 0.001. design attributes and individual factors were associated with website usability. However, the relationships of Web design attributes such as layout (β = 0.456, 95% CI: 1.19 to 1.69, p < 0.001) and performance (β = 0.204, 95% CI: 1.20 to 2.25, p < 0.001) were stronger than the association of individual factors such as education level (β = −0.139, 95% CI: -4.67 to −0.13, p < 0.05). About 29% of the variance in usability was accounted for by variables in the model. With regard to the relationship of Web design attributes and individual factors to user satisfaction, the results of the multivariate regression analysis (shown in Table 5) indicated that Web design attributes had strong association with user satisfaction, whereas individual factors (such as age, gender and Web use experience) had no effect. Among the Web design attributes, Web structure had the strongest association (β = 0.494, 95% CI: 2.16 to 2.76, p < 0.001), while other attributes such as layout (β = 0.171, 95% CI: 0.26 to 0.62, p < 0.001), personalisation (β = 0.131, 95% CI: 0.28 to 0.71, p < 0.001), search (β = 0.120, 95% CI: 0.25 to 0.74, p < 0.001) and performance (β = 0.085, 95% CI: 0.23 to 0.94, p < 0.001) had weaker associations. Altogether, these five variables explained 51% of the variance in user satisfaction model. The results also showed a significant association between website usability and user satisfaction (β = 0.060, 95% CI: 0.01 to 0.09, p < 0.05). 4. Discussion With dramatic increasing number of websites and users who have to use them as a routine in their living and work activities, the issue of usability and user satisfaction of these interfaces are getting more and more important for both companies/organisations and users. In this study, online banking websites were selected to model the relationship between Web design attributes and users' personal characteristics to usability and user satisfaction in this regard. The relationship of website usability and user satisfaction was also examined in this research. The main findings of the study were that the design and usability of the evaluated websites, despite their high importance, were not satisfactory from the users' perspectives. Users' personal characteristics influenced the Web design and usability scores, but not the user satisfaction. Usability and user satisfaction differed by the manner in which they were influenced by the Web design attributes and users' personal characteristics. The former was influenced by both Web design attributes and users' personal characteristics, while the latter was only affected by Web design attributes. Website usability itself had a significant impact on user satisfaction. These findings can help to better understand the users’ needs and expectations, and consequently to optimise the website design. With regard to website usability, the SUS scores obtained for different banking websites ranged between 49 and 53, which indicates a moderate level of usability. This finding highlights a call for better ergonomic design of banking websites evaluated in the study. From an ergonomic point of view, poor or unacceptable website usability may be attributed to the fact that most Web developers have been mistakenly guided by the challenges of technology rather than by users’ needs I. Dianat, et al. Table 5 Multivariate regression analysis for the relationship between study variables and end-user satisfaction of websites. Variables Model 1 β Gender Male 1.00 Female −0.036 Age (yr) < 30 1.00 30-45 −0.038 > 45 −0.058 Educational level Diploma 1.00 Undergraduate −0.082 Postgraduate −0.088 Occupation Office worker 1.00 Manual worker 0.032 Student −0.039 Others −0.057 Web type Website 1 1.00 Website 2 −0.060 Website 3 −0.049 Website 4 0.029 Web usage experience (yr) 1–2 1.00 3–4 −0.059 ≥5 0.014 UCWD Personalisation Structure Navigation Layout Search Performance Usability (SUS score) Model 2 β 95% CI Lower bound Upper bound – −1.42 – 0.46 – −2.09 −2.99 – 0.96 0.84 – −3.25 −3.50 – 1.02 1.01 – −0.89 −3.59 −2.00 – 2.17 1.16 0.34 – −2.16 −1.99 −0.86 – 0.39 0.54 1.73 – −1.86 −1.14 – 0.32 1.56 0.128*** 0.498*** −0.007 0.199*** 0.121*** 0.096*** Model 3 β 95% CI Lower bound Upper bound 0.27 2.18 −0.35 0.34 0.26 0.32 0.70 2.79 0.27 0.68 0.74 1.01 95% CI Lower bound Upper bound 0.131*** 0.494*** 0.28 2.16 0.71 2.76 0.171*** 0.120*** 0.085*** 0.060* 0.26 0.25 0.23 0.01 0.62 0.74 0.94 0.09 Model 1: Adjusted for gender, age, educational level, occupation, Web type and Web usage experience. Model 2: Model 1 plus Web design attributes (personalization, structure, navigation, layout, search and performance). Model 3: Model 2 plus suability (SUS score). Adjusted R2: Model 1 = 0.005; Model 2 = 0.510; Model 3 = 0.512. Statistically significant values are shown in bold. *p < 0.05; **p < 0.01; ***p < 0.001. (Helander and Khalid, 2000). It is, therefore, essential to overcome the challenges associated with website functionality, while not compromising its usability. Based on our findings, two aspects of Web design including Web layout and performance were the main predictors of usability, while personal characteristics of users such as gender, age and Web usage experience had no effect. This means that, in terms of usability, special attention should be given to the website layout issues such as navigation bar design and page arrangement as well as to the Web page performance metrics including browser-related and error management issues to improve the usability of banking websites under study. However, it should be pointed out that usability problems may be unique to each type of website depending on its content and application, and therefore further research is needed to draw firm conclusions on this issue. As shown in this research, all the Web design attributes were highly related to user satisfaction, except for the navigation, which is generally not surprising in view of previous research (Cyr, 2008; Liu et al., 2008; Yoon, 2010). But more importantly, the website structure had the strongest relationship in this regard. This result is, in part, consistent with the finding of Zviran et al. (2006), who found that user satisfaction was more related to content and search than other attributes of different types of commercial websites (e.g. online shopping, customer self-service, trading, and publish/subscribe). Although the amount, heterogeneity and applications of websites are so great that it is difficult to give general guidelines and recommendation for the Web design, it seems that the structure of a website is perhaps a more important design consideration than other attributes for this type of websites. More specifically, improved organisation of the information in the site (e.g. logical categorisation of the content and use of appropriate multimedia/graphics) is likely the most effective way to increase user satisfaction. It should, however, not be forgotten that the relationship between user-centred design and user satisfaction may be differentially affected depending on the website content and application. For example, some investigators have highlighted the importance of information content and system quality in terms of delivery of the information as most important indicators of user satisfaction in online shopping environments (McKinney et al., 2002). Thus, further studies evaluating this relationship in other types of websites are recommended. However, such studies should preferably focus on the impact of different Web design attributes on user satisfaction in a multivariate context, rather than on the website's general design or on some specific aspects of website design. It is also of interest to note that according to the multivariate regression models, demographic and individual variables such as gender, age and Web usage experience had no influence on the user satisfaction of websites. There are very few studies that have examined users' personal characteristics and preferences in relation to website satisfaction. In a study by Cyr and Bonanni (2005) about website design in e-business, the authors found significant gender differences in perceptions of website design and website satisfaction, which contradicts the findings I. Dianat, et al. of this study. This may be attributed to either differences in the types of websites being evaluated or to cultural differences in these studies. Similarly, the regression models showed no relationship between age or Web usage experience of users and their satisfaction with websites. These findings are relatively new to the literature and have important implications for the website design. These findings may rule out the possibility of individual differences in terms of user satisfaction of websites. Based on these findings, it can be concluded that website designers should focus more on Web design attributes as a more important design consideration than individual differences for increasing the satisfaction of their users. Again, these findings provide a basis for informing the relationship between users’ personal characteristics and their satisfaction from banking websites but would need further validation for other types of websites and cultural backgrounds. One of the main contributions of this study is that the relationship of Web design attributes and users’ personal characteristics to website usability and user satisfaction was evaluated in a multivariate context. However, it should be noted that the development of recommendations regarding website design attributes in this study was based on online banking websites with emphasis on usability and user satisfaction as two important measures of Web success. Obviously, UCWD can be evaluated from different point of view such as performance, error, etc. Moreover, Web design attributes considered important by users may differ depending on the website content and application. In addition, this study was conducted in Iran, where the culture and social structure differ greatly from those of the developed or other developing countries. Therefore, further studies on different types of websites, using different measures of UCWD, in other settings or communities are required to have a better understanding and knowledge in this regard. Moreover, the findings presented in this research identify the most important website design parameters in terms of usability and user satisfaction which then provide a basis for further research. To take account of this, the website layout and structure which have shown to be the most important parameters in this regard should be evaluated by both subjective (e.g. user preference, ease of use, etc.) and objective (e.g. use of eye-tracking in navigation bar design) assessments. 5. Conclusion In conclusion, the results presented in this study provide an insight into the design of banking websites, with emphasis on usability and user satisfaction as two important measures of Web success. Designing a good website seems to be one of the challenges that Iranian banking industry may encounter to achieve their goals, particularly in a limited and competitive market. Two Web design attributes including Web layout (e.g. navigation bar design and page arrangement) and performance (e.g. browser-related and error management issues) were shown to be the main predictors of perceived usability, while users' personal characteristics including gender, age and Web usage experience had no effect in this regard. User satisfaction was also influenced only by the Web design attributes and not by the personal characteristics of the users. Among the Web design attributes, Web structure (e.g. improved organisation of the information in the site) showed the strongest association with user satisfaction, followed by layout, personalisation, search and performance. Website usability also influenced the user satisfaction. The lack of association of users’ personal characteristics with usability and user satisfaction is important and suggests that the Web designers should focus more on the Web design attributes, regardless of the personal characteristics of their users, to improve the usability and user satisfaction of websites. Acknowledgements The authors wish to acknowledge the support and assistance provided by the bank authorities, and all the participants who collaborated in this research. References Abelse, E.G., White, M.D., Hahn, K., 1998. A user-based design process for Web sites. Internet Res. 8, 39–48. Baber, C., 2005. Evaluation in human–computer interaction. In: Wilson, J.R., Corlett, N. (Eds.), Evaluation of Human Work. Taylor & Francis, Boca Raton. Belanche, D., Casaló, L.V., Guinalíu, M., 2012. Website usability, consumer satisfaction and the intention to use a website: the moderating effect of perceived risk. J. Retail. Consum. Serv. 19, 124–132. Brooke, J., 1996. SUS: a “quick and dirty” usability scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, A.L. (Eds.), Usability Evaluation in Industry. Taylor & Francis, London. Casaló, L., Flavián, C., Guinalíu, M., 2008. The role of perceived usability, reputation, satisfaction and consumer familiarity on the website loyalty formation process. Comput. Hum. Behav. 24, 325–345. Chiou, J., Shen, C., 2006. The effects of satisfaction, opportunism, and asset specificity on consumers' loyalty intention toward internet portal sites. Int. J. Serv. Ind. Manag. 17, 7–23. Cousaris, C., Swierenga, S., Watrall, E., 2008. An empirical investigation of color temperature and gender effects on web aesthetics. J. Usability Stud. 3, 103–117. Cyr, D., 2008. Modeling website design across cultures: relationships to trust, satisfaction and e-loyalty. J. Manag. Inf. Syst. 24, 47–72. Cyr, D., Bonanni, C., 2005. Gender and website design in e-business. Int. J. Electron. Bus. 3, 565–582. Dianat, I., Ghanbari, Z., Asghari-Jafarabadi, M., 2014. Psychometric properties of the Persian language version of the system usability scale. Health Promot. Perspect. 4, 82–89. Dianat, I., Nedaei, M., Nezami, M.A.M., 2015. The effects of tool handle shape on hand performance, usability and discomfort using masons' trowels. Int. J. Ind. Ergon. 45, 13–20. Doll, W.J., Xia, W., Torkzadeh, G., 1994. A confirmatory factor analysis of the end-user computing satisfaction instrument. MIS Q. 18, 453–461. Flavián, C., Guinalíu, M., Gurrea, R., 2006. The role played by perceived usability, satisfaction and consumer trust on website loyalty. Inf. Manag. 43, 1–14. Fu, L., Salvendy, G., 2002. The contribution of apparent and inherent usability to a user's satisfaction in a searching and browsing task on the Web. Ergonomics 45, 415–424. Garrett, J.J. (Ed.), 2003. The Elements of User Experience: User-Centered Design for the Web. New Riders Publications, Indiana. Guo, X., Ling, K.C., Liu, M., 2012. Evaluating factors influencing customer satisfaction towards online shopping in China. Asian Soc. Sci. 8, 40–50. Gustafsson, A., Johnson, M.D., Ross, I., 2005. The effects of customer satisfaction, relationship commitment dimensions, and triggers on customer retention. J. Mark. 69, 210–218. Ha, Y., Im, H., 2012. Role of web site design quality in satisfaction and word of mouth generation. J. Serv. Manag. 23, 79–96. Head, A.J. (Ed.), 1999. Design Wise: A Guide for Evaluating the Interface Design of Information Resources. Information Today Inc., Medford, NJ. Helander, M.G., Khalid, H.M., 2000. Modeling the customer in electronic commerce. Appl. Ergon. 31, 609–619. Hojjati, S.N., Monavvarian, A., Roozbahani, F.S., 2015. Evaluation of factors influencing internet banking acceptance by internet users in Iran. Int. J. Adv. Netw. Appl. 6, 2503–2519. Kim, S., Stoel, L., 2004. Apparel retailers: website quality dimensions and satisfaction. J. Retail. Consum. Serv. 11, 109–117. Lee, S., Koubek, R.J., 2010. The effects of usability and web design attributes on user preference for e-commerce web sites. Comput. Ind. 61, 329–341. Liu, X., He, M., Gao, F., Xie, P., 2008. An empirical study of online shopping customer satisfaction in China: a holistic perspective. Int. J. Retail Distrib. Manag. 35, 919–940. Marcus, A., Gould, E.W., 2000. Crosscurrents: cultural dimensions and global Web userinterface design. Interactions 7, 32–46. McCracken, D.D., Wolfe, R.J. (Eds.), 2004. User-centered Website Development: A Human–Computer Interaction Approach. Pearson Prentice Hall Inc., Upper Saddle River, NJ. McKinney, V., Yoon, K., Zahedi, F., 2002. The measurement of Web customer satisfaction: an expectation and disconfirmation approach. Inf. Syst. Res. 13, 296–315. Muylle, S., Moenaert, R., Despontin, M., 2004. The conceptualization and empirical validation of web site user satisfaction. Inf. Manag. 41, 543–560. Ozok, A.A., Salvendy, G., 2000. Measuring consistency of web page design and its effects on performance and satisfaction. Ergonomics 43, 443–460. Palmer, J.W., 2002. Web site usability, design, and performance metrics. Inf. Syst. Res. 13, 151–167. Shankar, V., Smith, A.K., Rangaswamy, A., 2003. Customer satisfaction and loyalty in online and offline environments. Res. Market. 20, 153–175. Schaupp, L.C., Bélanger, F., 2005. A conjoint analysis of online consumer satisfaction. J. Electron. Commer. Res. 6, 95–111. Shneiderman, B., 1998. Designing the User Interface: Strategies for Effective Human–Computer Interaction. Addison-Wesley Longman Publishing Co. Inc., Boston, MA. Simon, S., 2001. The impact of culture and gender on web sites: an empirical study. ACM SIGMIS - Data Base 32, 18–37. Simon, J., Peppas, S., 2005. Attitudes towards product website design: a study of the effects of gender. J. Mark. Commun. 11, 18–37. Szymanski, D.M., Hise, R.T., 2000. E-satisfaction: an initial examination. J. Retail. 76, 309–322. I. Dianat, et al. Tandon, U., Kiran, R., Sah, A.N., 2016. Customer satisfaction using website functionality, perceived usability and perceived usefulness towards online shopping in India. Inf. Dev. 32, 1657–1673. Tandon, U., Kiran, R., Sah, A.N., 2017. The influence of website functionality, drivers and perceived risk on customer satisfaction in online shopping: an emerging economy case. Inf. Syst. E Bus. Manag. 16, 57–91. Tuch, A.N., Bargas-Avila, J.A., Opwis, K., 2010. Symmetry and aesthetics in website design: it's a man's business. Comput. Hum. Behav. 26, 1831–1837. Yoon, C., 2010. Antecedents of customer satisfaction with online banking in China: the effects of experience. Comput. Hum. Behav. 26, 1296–1304. Zviran, M., Glezer, C., Avni, I., 2006. User satisfaction from commercial web sites: the effect of design and use. Inf. Manag. 43, 157–178 . Received February 10, 2020, accepted March 16, 2020, date of publication March 19, 2020, date of current version March 30, 2020. Digital Object Identifier 10.1109/ACCESS.2020.2981892 Usability of Mobile Applications: A Systematic Literature Study PAWEä WEICHBROTH Department of Software Engineering, Faculty of Electronics, Telecommunications and Informatics, Gda´sk University of Technology, 80-233 Gda´sk, Poland e-mail: pawel.s.weichbroth@gmail.com ABSTRACT Since the release of the first mobile devices, the usability of on-board applications has been the concern not only of software vendors but hardware manufacturers as well. The academia community later willingly joined the discussion on usability in terms of theory and empirical measurement, having experience and knowledge in desktop settings. At first sight, such a background should guarantee a solid foundation to conduct research on software usability in a new setting. However, a preliminary study on the subject matter revealed methodological disorder in contemporary literature. As a matter of fact, a need emerged to review existing usability definitions, attributes and measures to recognize all associated aspects. In order to fill this void, we conducted a systematic literature review on usability studies indexed by the Scopus database and devoted to mobile applications. The input volume covers 790 documents from 2001 to 2018. The data analysis shows that the ISO 9241-11 usability definition has been adopted in an unchanged form and popularized as the standard by the HCI community. Secondly, in total, 75 attributes were identified and analysed. The most frequent are efficiency (70%), satisfaction (66%) and effectiveness (58%), which directly originate from the above definition. Subsequently, the less frequent are learnability (45%), memorability (23%), cognitive load (19%) and errors (17%). The last two concern simplicity (13%) and ease of use (9%). Thirdly, in the evaluation of usability, controlled observation and surveys are two major research methods applied, while eye-tracking, thinking aloud and interview are hardly used and serve as complementary to collect additional data. Moreover, usability evaluations are often confused with user experience dimensions, covering not only application quality characteristics, but also user beliefs, emotions and preferences. All these results indicate the need for further research on the usability of mobile applications, aiming to establish a consensus in the theory and practice among all interested parties. INDEX TERMS Mobile applications, usability, attributes, measures, usability evaluation methods, systematic literature review. I. INTRODUCTION Amobile application is defined as ‘‘a software application developed specifically for use on small, wireless computing devices, such as smartphones and tablets, rather than desktop or laptop computers’’ [1]. A recent Statista report shows that in 2017 smartphones had a share of 77% of the global mobile device market [2], and more than 32% of the global population used a smartphone [3]. Although technological progress has been made regarding mobile devices equipped with computing power, leading to a shift from desktop computers, many limitations and The associate editor coordinating the review of this manuscript and approving it for publication was Mario Luca Bernardi VOLUME 8, 2020 . challenges still remain [4]. From the many identified, usability has been the main concern, since the users of an application, and their judgment, ultimately decide on its success or failure [5]–[7]. Since the inception of the first smartphones, the subject of mobile application usability has gained attention both in academia communities and in the software vendors industry. While researchers are focused on formulating theories [8], modelling frameworks [9], and constructing methods and techniques [10], [11] for new settings, manufacturers simply desire to deliver high quality products [12]. Despite the abundance of research devoted to studies of mobile application usability on the one hand, and design patterns, prototyping tools and software frameworks on the This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 55563 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study other, the term tends to be vague and loose, weakening the ability to capture its real facets and impeding the construction of measures. As a consequence, such methodological disorder violates the core assumptions and principles laying beneath the foundations of the usability notion. Therefore, considering the need for the emergence of a usability definition, its attributes and measures, along with evaluation methods, valid for mobile applications, in this paper we made an attempt to find reliable answers by conducting a systematic literature review. We expect that the obtained results can be used not only by researchers to perform further studies in this area, but also for practitioners engaged in mobile application development and quality-in-use evaluation to better understand the characteristics and measures of the notion. The main contributions of this study include: (i) an evidence-based discussion of the usability definition, its attributes and measures, (ii) and an up-to-date map of the state of the art in usability evaluation methods (UEMs), adopted for and adapted to mobile applications, covering publications from 2001 to 2018. The rest of the paper unfolds as follows. Section 2 provides the background on the subject addressed, and related work. Section 3 describes the research methodology. The definition and execution of the literature review are respectively presented in Sections 4 and 5. Section 6 provides an analysis of the extracted data, while the results are further discussed in Section 7, along with the future research directions. The conclusions are raised in Section 8. II. BACKGROUND Most people tend to use products that are easy to understand, work as expected, and eventually deliver value. In the context of the software engineering, system usability plays the crucial role in shaping perceived quality in use by its users [13], [14]. Usability is the study of the intersection of between systems and users, tasks and expectations in the context of use. Since many software products have been determined to be insufficient to meet user needs, several comprehensive studies have been conducted so far under the term usability, which move towards a better understanding and relevant measurement, aiming to cover all valid phenomena in one framework or model [15]–[17]. The results of the study, introduced by Weichbroth [18], show that over time the definition of usability has evolved. In 1991 the Organization for Standardization (ISO), in response to the emergence of the need of the software community to standardize some facets of software products, publicized the 9126 standard, which defines usability as ‘‘a set of attributes of software which bear on the effort needed for use, and on an individual assessment of such use, by a stated or implied set of users’’ [19]. Then, in 1998, ISO refashioned the usability definition in the ISO 9241-11 norm, which actually states that usability is ‘‘the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use’’ [20], [21]. 55564 While some argue that it is the most recognizable definition [18], others maintain that ‘‘a generally accepted usability definition still does not exist, as its complex nature is hard to describe in one definition’’ [22], [23]. The other usability definition can be found in ISO/IEC 25010 [24], which replaced the ISO/IEC 9126 standard from 2001 [25], and specifies usability as the ‘‘degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use’’. Here, it is worth noting that these two latest paraphrased definitions, however differently particularized, still share exactly the same three virtues which affect the user’s ability to achieve specified goals. Since the inception of the first official usability definition, one might argue that a great plethora of usability attributes have been taken into consideration regarding the ability to use particular software products, ranging from monolithic systems to lightweight web pages. Having said that, based on the literature search and analysis, in view of usability attributes that contribute to the quality in use of the desktop software, the latest study [18] shows that the most frequent are efficiency, satisfaction, learnability and effectiveness. The least frequent are understandability and operability, memorability, errors, attractiveness and accessibility. To collect all necessary data in order to improve the quality of particular software facets, a variety of usability evaluation methods (UEMs) have been developed and empirically tested. One of the most recognized UEMs concern the family of user testing methods [26]–[28], in particular think-aloud protocol [29]–[31], question-asking protocol [32]–[34], performance measurement [35]–[37], log analysis [38]–[40], eye tracking [41]–[43], and remote testing [44]–[46]. Secondly, inspection methods, intended to be used by experts [47], refers to heuristic evaluation [48]–[50], cognitive walkthrough [51]–[53], perspective-based inspection [54]–[56], and guideline reviews [57]–[59]. Thirdly, inquiry methods, designed to gathering subjective data from users, utilize both quantitative (questionnaires [60]–[62]) and qualitative (interviews [63]–[65] and focus groups [66]–[68]) techniques. Furthermore, some authors also distinguish analytical modelling methods such as cognitive task analysis [69]–[71], task environment analysis [72]–[74] and GOMS analysis (Goals, Operators, Methods and Selection rules) [75]–[77]. Regarding the context of this study, Zhang and Adipat (2005) propose a generic framework for conducting usability tests for mobile applications through discussing existing methodologies and usability attributes [78]. As challenges, they point to the unique features of mobile devices and wireless networks which influence the usability of mobile applications, including mobile context, multimodality, connectivity, small screen size, different display resolutions, limited processing capability and power, and restrictive data entry methods. In the case of research methodologies for usability testing, they point to controlled laboratory experiments and field studies. While former limitations are ignorance of the mobile context and the preservation of reliable VOLUME 8, 2020 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study network conditions and other environmental factors, then later, the lack of sufficient control over participants in a study, and dealing with issues such as the selection of environmental conditions, evaluation performance, data collection and condition control. They also identify nine generic usability attributes: learnability (ease of use), efficiency, memorability, errors, user satisfaction, effectiveness, simplicity, comprehensibility (readability) and learning performance. Hussain and Kutar (2009) introduce a hierarchical GQM (Goal Question Metric) model to evaluate mobile usability [79]. On the top level, they place three quality characteristics: effectiveness, efficiency and satisfaction. On the middle level, six guidelines are conceptualized: simplicity, accuracy, time taken, features, safety and attractiveness. Eventually, on the bottom, there is a mapping between questions and metrics, which enables the collection of quantitative data in order to evaluate usability. Kronbauer et al. (2012) propose a hybrid model for the evaluation of smartphone application usability [80]. In this study, the hybrid approach blends two methods for data capture, namely, Logging and ESM (Experience Sampling Method). The first one is based on data collection related to user interaction with an application. Using sensors available in smartphones for contextual data collection, such as luminosity intensity and the device’s position, allows the performance of statistical analysis regarding usability. The second one is based on the collection of users’ feelings towards a specific product through questions. These two methods are respectively used to measure efficiency, effectiveness and satisfaction. Harrison et al. (2013) developed the PACMAD (People At the Centre of Mobile Application Development) usability model, which identifies three major dimensions affecting the overall usability of a mobile application: the user, the task and the context of use [81]. However, the last one plays a crucial role, as an application may be used in multiple and very different contexts (e.g. environment, physical location, user’s state or activity performed). The model encompasses seven attributes, which together reflect the usability of an application: effectiveness, efficiency, satisfaction, learnability, memorability, errors and cognitive load. In some studies the model has been adopted to set up testing and evaluation frameworks [82], [83]. The novelty of the model concerns cognitive load as a new usability attribute. The authors claim that it can be observed that users of mobile applications often perform additional tasks, such as walking, while using the mobile device. For this reason, these additional tasks impact the user’s performance, arguing by example of a walking user who in the same time is texting a message which reduces walking speed as s/he is concentrating on typing (sending) the message. More recently, cognitive load has been acknowledged [84], or disregarded [85], as one of the usability notions. Actual usability, located in the frames of the quality-in-use model by Lew and Olsina (2013), comprises effectiveness, efficiency, learnability in use, and communicability [86]. VOLUME 8, 2020 They also emphasize the difference between the context of mobile applications and traditional, desktop or web applications. The context does not only concern hardware limitations (e.g. size of the screen), but also other factors, such as: user activity, day/time of day, location, user profile, device and network performance. Obviously, there are many more usability models, individually applicable to particular domains, such as mobile banking [87], or healthcare [88]; however, they were excluded from the discussion due to their specific attributes, classified as superior with respect to the others. III. RESEARCH METHODOLOGY A systematic literature review (SLR) in its nature differs from traditional narrative reviews by adopting a replicable, scientific and transparent process methodology. By design, it aims to reduce cognitive bias by providing an audit trail of the associated assumptions and procedures, reviewers decisions and conclusions on the one hand, and by identifying and documenting key scientific contributions to a field or question on the other. In order to provide a body of knowledge on the usability of mobile applications, we performed a systematic literature review by adopting and adapting the approach provided by Kitchenham and Charters [89], [90], since a large majority of the reported SLRs in software engineering has been carried out in respect to their guidelines [91]. According to the research design employed, this study consists of three steps, performed in a fixed sequence. Interdependency is revealed in the one-way output/input relations. Step 1 in the research methodology involves defining the research questions and the review protocol, which encompasses the data source and search strategy, the inclusion and exclusion criteria and the definition of the search string. The outcome of this step is described in Section 4. Step 2 in the research methodology involves executing the search string carried out on the database engine. Next, the obtained results are extracted and further processed. The outcome of this step is given in Section 5. Step 3 in the research methodology involves reviewing, analysing and reporting each data record, in order to consequently find and document answers for a defined set of the research questions. The outcome of this step is described in Section 6. IV. SYSTEMATIC LITERATURE REVIEW DEFINITION A. RESEARCH QUESTIONS DEFINITION Investigating the gap in usability between desktop and mobile settings, the following three questions arose: RQ1. How has usability for mobile applications been defined? RQ2. What are the usability attributes for mobile applications? RQ3. How have usability attributes for mobile applications been defined, and which measures and evaluation methods have been used? 55565 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study TABLE 1. The general search query criteria. TABLE 3. The exclusion criteria (EXCLUDE) to the subject area (SUBJAREA). TABLE 2. The inclusion criteria (LIMIT-TO) to the subject area (SUBJAREA). These three interrogative statements provide the overall framework for conducting this study, by giving direction and setting up boundaries. B. DATA SOURCE AND SEARCH STRATEGY In line with the research methodology, step 1 involves a systematic search of the scientific literature on the topic of mobile application usability. Performed on Scopus, the largest abstract and citation database of peer-reviewed literature, counting over 71 million records [92], the search strategy aims at identifying indexed publications. A key issue when formulating a search strategy is to define the period of time to set up time boundaries. Being in our interest to obtain reliable and concise answers to the questions, we determined the closing date in December 2018. TABLE 4. The inclusion criteria (LIMIT-TO) for the document type (DOCTYPE). TABLE 5. The inclusion (LIMIT-TO) and exclusion (EXCLUDE) criteria for the language. C. SEARCH QUERY DEFINITION The search query was defined by the presence of ‘‘usability’’ and the string ‘‘mobile application’’ in titles, abstracts and keywords. These unique and specific terms, joined together in that order and in the extent of such meta-data, embody the authors’ common declaration of their research objectives and the adopted context of their performed studies. The summary, in terms of the search query construct, is given in Table 1. D. INCLUSION AND EXCLUSION CRITERIA In accordance with our research objective and questions, the first applied inclusion criterion relates to the subject area, which alternatively includes: computer science, engineering, mathematics, social sciences, or decision sciences. Table 2 presents the summary of the search query construct in this scope. In this study, usability is considered in the context of software, which is a concern of computer science and is also closely associated with the other abovementioned disciplines. In this line of thinking, we exclude irrelevant subject areas (e.g. Medicine, Health Professions, Chemistry and others). Table 3 depicts the summary of the search query construct in this scope. The second inclusion (exclusion) criterion was the document type which alternatively encompasses: conference proceedings, journal articles or book chapters. On the other 55566 hand, we did not take into account conference reviews and other reviews, which present non-scientific contributions. Table 4 outlines the summary of the search query in this scope. Not all scientists regard conference proceedings as a reliable and valuable source of knowledge. However, from our point of view, our judgement was not solely based on the document type, but on scrupulous reading and conscientious content analysis. The third inclusion (exclusion) criterion was the language, exclusively limited to English. Therefore, two other (Portuguese and French) were excluded. Table 5 depicts the summary of the search query construct in this regard. English has become the modern lingua franca in the modern world. The major international standardization bodies publish norms and standards in English, and communication channels between experts and communities follow the same rule as well. V. SEARCH EXECUTION A. SEARCH AND SELECTION In the first run, the search query (Table 1) produced 1,615 document results. To this volume, the inclusion VOLUME 8, 2020 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study FIGURE 1. The distribution of the number of publications per year. and exclusion criteria were applied, defined respectively in Tables 2–5. The search strings, given in all these tables, were eventually combined by the relevant Boolean operators. The final search query construct, which entirely fulfils all the requirements, is given below. TITLE-ABS-KEY (usability AND ‘‘mobile application’’) AND (LIMIT-TO (SUBJAREA, ‘‘comp’’) OR LIMIT-TO (SUBJAREA, ‘‘engi’’) OR LIMIT-TO (SUBJAREA, ‘‘math’’) OR LIMIT-TO (SUBJAREA, ‘‘soci’’) OR LIMIT-TO (SUBJAREA, ‘‘deci’’) OR EXCLUDE (SUBJAREA, ‘‘medi’’) OR EXCLUDE (SUBJAREA, ‘‘heal’’) OR EXCLUDE (SUBJAREA, ‘‘ceng’’) OR EXCLUDE (SUBJAREA, ‘‘envi’’) OR EXCLUDE (SUBJAREA, ‘‘phys’’) OR EXCLUDE (SUBJAREA, ‘‘mate’’) OR EXCLUDE (SUBJAREA, ‘‘bioc’’) OR EXCLUDE (SUBJAREA, ‘‘ener’’) OR EXCLUDE (SUBJAREA, ‘‘psyc’’) OR EXCLUDE (SUBJAREA, ‘‘arts’’) OR EXCLUDE (SUBJAREA, ‘‘eart’’) OR EXCLUDE (SUBJAREA, ‘‘nurs’’) OR EXCLUDE (SUBJAREA, ‘‘chem’’) OR EXCLUDE (SUBJAREA, ‘‘neur’’) OR EXCLUDE (SUBJAREA, ‘‘econ’’) OR EXCLUDE (SUBJAREA, ‘‘agri’’) OR EXCLUDE (SUBJAREA, ‘‘immu’’) OR EXCLUDE (SUBJAREA, ‘‘phar’’)) AND (LIMIT-TO (DOCTYPE, ‘‘cp’’) OR LIMIT-TO (DOCTYPE, ‘‘ar’’) OR LIMIT-TO (DOCTYPE, ‘‘ch’’)) AND (EXCLUDE (PUBYEAR, 2019)) AND (LIMIT-TO (LANGUAGE, ‘‘English’’)) AND (EXCLUDE (LANGUAGE, ‘‘Portuguese’’) OR EXCLUDE (LANGUAGE, ‘‘French’’)) The results summary was checked in order to verify whether all the criteria were successfully applied. In total, the final search query eventually produced 887 documents, published between 2001 and 2018. The details of the volume data are as follows, while the numbers in brackets indicate the total number of publications: (a) published in English (887), VOLUME 8, 2020 (b) the subject area is from: computer science (803), decision sciences (40), engineering (198), mathematics (197) and social sciences (103), and (c) the document type is: conference proceedings (666), journal articles (196) or book chapters (25). The peak year is 2017 (140), followed by the years 2015 (110), 2018 (104) and 2016 (101), with an average of 74 documents published annually between 20082018 (Figure 1). The distribution of the number of publications increases in linear. However, in 2018 a fall was observed in comparison to the previous year, but still above the year 2016. The majority of documents were published by Springer in Lecture Notes in Computer Science, including sub-series Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics (148), while the largest contributor among journals is the Journal of Telecommunication, Electronic and Computer Engineering (12). The top three countries, the USA (136), Germany (81) and Malaysia (66), accounted for over 31% of the countries the authors were affiliated to. B. DATA EXTRACTION Having imported the reference data (authors, document title, year, and digital object identifier) to an external spreadsheet, we systematically searched for each record in full-text databases hosted by particular publishers and indicated as the source of the document. From the list of 887 records, in total 790 (89%) documents were fully available, while using a HAN system licensed account. To extract the data, three independent reviewing procedures were prepared and executed, respectively for each research question. In the first run, each available document was screened with the aim to identify and recognize a usability definition 55567 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study TABLE 6. The list of adopted usability definitions for mobile settings. TABLE 7. The shares of adopted usability definitions for mobile settings. referenced by the author(s). The document was classified as relevant if: (a) usability, as a term, was explicitly defined and (b) correctly referenced. If the authors provided more than one definition and did not indicate a particular one as valid, then the first one given was assumed to be adopted. Eventually, 66 (8%) documents were classified as relevant as the input for analysis, with the aim of formulating an answer to the first research question. In the second run, each document was screened again to determine the overall quality and its relevance. A document was classified as relevant if: (a) the subject of the research was addressed to the usability of mobile applications, and (b) was not biased by a context of the research, such as: (i) application type or (ii) user-specific properties, such as: age, occupation, sex or (iii) system-specific support features, like visually impaired or disability. The review of the list produced 53 (7%) documents as relevant as the input for analysis with the aim of formulating an answer to the second research question. In the third run, the above list was reviewed and examined again with the aim of extracting attribute definitions, measures and UEMs. The document was classified as relevant if: (a) usability attributes being the subject of the study were explicitly defined, whereas a measure was valid if it captures the quantitative data which accurately describes one particular usability attribute. Ultimately, 39 (5%) documents were classified as relevant as the input for analysis with the aim of formulating an answer to the third research question. VI. DATA ANALYSIS This section addresses the analysis of the data extracted from the studies in accordance with the three defined 55568 research questions. We used a qualitative content analysis, which focuses on the characteristics of language as a communication channel, with attention to the specific subjects, narrowed and directed by particular research questions. RQ1. How has usability for mobile applications been defined? To this day, none of the authors have introduced any formal definition of usability associated with an application (system) running on a mobile device. Therefore, all identified and recognized definitions have been adopted from the existing general norms, standards and definitions. The great majority of authors (88%) have defined usability solely in terms of the ISO 9241-11 norm, while others have also made reference to ISO 25010 (4,5%) and ISO 9126 (3%) norms, as well as to the IEEE Glossary (1,5%), the Nielsen (1,5%) and Bevan (1,5%) definitions. Table 6 includes the full text of these six definitions, whereas Table 7 depicts findings of the shares of adopted usability definitions for mobile settings. RQ2. What are the usability attributes for mobile applications? In total, 75 usability attributes were identified and analysed. Among them, the most frequent are efficiency (70%), satisfaction (66%) and effectiveness (58%). Less frequent are learnability (45%), memorability (23%), cognitive load (19%) and errors (17%). The last two concern simplicity (13%) and ease of use (9%). The remaining attributes occurred four times or less. Table 8 outlines the details in this regard (the attributes which occurred only once are not included). VOLUME 8, 2020 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study TABLE 8. The list of adopted usability attributes for mobile settings. RQ3. How have usability attributes for mobile applications been defined, and which measures and evaluation methods have been used? The foremost attribute, efficiency is the ability of a user to complete a task with speed and accuracy. Efficiency is measured in a number of ways, such as the duration spent on each screen, the duration to complete a given task (a set of tasks), and the user’s error rate. Two evaluation methods are used: controlled observation and survey. Satisfaction is a user’s perceived level of comfort and pleasure, or a user’s perceived level of fulfilment of his expectations and needs. Satisfaction is measured only by using survey, with predefined statements with the Likert-scale rating system, which is typically used to capture a user’s intangible attitude towards an application. Effectiveness is the ability of a user to complete a task in a given context. It is measured by the number of successfully completed tasks, the number of steps required to complete a task, the number of double taps unrelated to the operation of an application, and the number of times that a back button is used by the mobile device (not the application). Learnability is defined twofold. First-time learnability refers to the degree of ease with which a user can interact with a newly-encountered system without getting guidance or referring to documentation. It is measured by the number of attempts to solve a task, the number of assists VOLUME 8, 2020 during performing a task, and the number of errors performed by a user. Learnability over time, on the contrary, is the capacity of a user to achieve proficiency with an application. Typically, a user’s performance during a series of tasks is observed to measure how long it takes these participants to reach a pre-specified level of proficiency. Similarly to effectiveness, two evaluation methods are used: controlled observation and survey. Memorability is the degree of ease with which a user can remember how to use an application effectively. It is measured by asking users to perform a series of tasks after having become proficient with the use of the application, and afterwards asking them to perform similar tasks after a period time of inactivity. To determine how memorable the application was, a comparison is made between the two sets of results. In this case, the eye-tracking technique is also used as the method to collect gaze data which is further used to evaluate usability. Cognitive load refers to the amount of mental activity imposed on a user’s working memory during application usage. Cognitive load theory differentiates cognitive load into three types: extraneous, intrinsic and germane. Firstly, extraneous cognitive load refers to instructional and presentation schemas, caused by the mental activities and elements that do not directly support application usage. Secondly, intrinsic cognitive load refers to the task complexity, caused by 55569 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study TABLE 9. The top most frequent usability attributes, their measures and associated usability evaluation methods (UEMs) for mobile settings. the number of elements in a task and the degree to which these elements are related to each other. Thirdly, germane cognitive load refers to the amount of mental effort used to form schemas and actively integrate new information with prior knowledge during application usage. In the practice of cognitive load measurement, instruments such as a subjective rating scale, a thinking aloud dual task protocol or eye tracking are in common use. Errors refers to the amount and type of errors which occur during task performance by a user. On the other hand, it is the ability of an application to recover from occurred errors. Both these definitions also respectively reflect the measures of attribute. Simplicity is the degree of being easy to understand or being uncomplicated in form or design, described by such characteristics as the number of menu levels, the number of performed gestures to reach a destination object, and the duration of searching a button to perform a specific function. On the other hand, simplicity is the level of comfort with which a user is able to complete a task, measured by predefined statements with the Likert-scale rating. Ease of use is the perceived level of the user’s effort related to usage of the application. The survey instrument is used to collect data from users on perceptions concerning their experienced interaction with the application. Table 9 presents a summary in which each attribute is associated with the valid measures, along with the usability 55570 evaluation methods used to collect the necessary data to improve particular software artefacts. From the variety of available methods, the most frequent is survey, based on the questionnaire instrument, which has been used to collect data from a sample of the participants, as a representation of the population of interest. Controlled observation of the user while interacting with an application is the second most frequent method applied to usability evaluation. The remaining three, namely eye-tracking, thinking aloud and interview, are hardly used and serve as complementary to collect additional data. Table 10 presents the details showing the number of occurrences of all identified UEMs applied for particular attributes. VII. DISCUSSION Based on the obtained results, we argue that the ISO 9241-11 definition has been widely accepted in a non-changed form, and since the inception of research on mobile application usability, has been, de facto, popularized as the standard by the HCI community. Having said that, it is worth noting that other definitions are not contrary to each other. Moreover, they have in common the software capability to interact with a user, yet emphasize different aspects of his/her proficiency. On the other hand, usability is always associated with the product, except for the IEEE Glossary and Bevan definitions, which focus first and foremost on the user. VOLUME 8, 2020 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study TABLE 10. The number of occurrences of usability evaluation methods (UEMs) applied to particular usability attributes. TABLE 11. The percentage of studies concerning usability attributes of mobile applications. The most frequent attributes originate from the usability definition adapted to mobile applications. In such a case, the main usability characteristics are device-agnostic. In other words, efficiency, satisfaction and effectiveness are valid for studying the usability of both desktop and mobile applications. In a similar manner, however with minor extensions, the remaining attributes have been assimilated as well. By design, cognitive load is related to the mental effort required by the user to perform tasks using a mobile device. While it is neither novel, nor high-ranking in usability research, it has now gained a larger audience due to the fact that a user’s attention is usually divided among other simultaneously performed tasks. If one breaks down usability into two parts, one gets two nouns: ‘‘use’’ and ‘‘ability’’. According to this line of thinking, the ability to use an application, in particular, means the ability to learn, memorize, navigate and operate. Besides this, ease of use seems related to the sense of presence of these abilities, facilitated by the errorless behaviour of the application. If one takes into account the research methods applied to the problem of usability evaluation, the attributes of the studied phenomena can be divided into two groups: quantitative and qualitative. However, based on such criteria, every attempt to formulate distinct groups will have its pros and cons, because each attribute has been measured depending on observation and survey. Nevertheless, the existing measures can be unambiguously classified if one still makes a clear distinction between facts and opinions. In other words, quantitative-oriented attributes have the advantage of being clearly definable and objectively measurable, using measures that are not influenced by the user’s personal judgement. VOLUME 8, 2020 On the other hand, one can point to user-oriented measures, and on the contrary, to application-oriented measures. Last but not least, it appears that existing measures intertwine user and application performance in one integrated artefact. It seems obvious that observational data are required to discover an application’s bottlenecks and general areas for improvement, thereby optimizing its operational capabilities by reducing the time and effort involved in its usage. To collect quantitative and qualitative data, questionnaires and controlled observation, respectively, have been typically applied, occasionally supported by eye-tracking, thinking aloud and interview techniques (see Table 10). In order to obtain numerical measures, a retrospective audio/video analysis is performed, while in some studies, third-party tools were installed which log all user interaction with an application with the date and time of the event, including the buttons that they chose, the gestures that were made and the functions that were recalled. After completing the task scenario, a user was asked to rank their agreement (disagreement) with predefined statements on a Likert scale or other rating scale. In comparison with the results obtained from studies with similar objectives, conducted by Coursaris and Kim [93] and Harrison et al. [81], our findings are consistent in the extent of the top three attributes, which concern efficiency, satisfaction and effectiveness (see Table 11). An increased interest in learnability and memorability can also be noticed, while errors and cognitive load are less appreciated. While simplicity and ease of use have not been indicated before, and being complementary, are neither novel nor visionary. The rest of the attributes seem to be extensions of existing ones, however, unbigoted by usability, they correspond to both explicit and implicit application properties. On the other 55571 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study hand, if one explores the user’s preferences instead of his/her ability to use an application, the results refer more to the user experience domain, yet less to usability. A. LIMITATIONS Although this study contributes to the field of humancomputer interaction, certain limitations exist within the research design. Firstly, one of the major limitations is that only one data source was involved. However, the indexation process covers varied sources of scientific content, ranging from conference proceedings to journal articles, which are reviewed each year to ensure quality standards are maintained. Secondly, inclusion and exclusion criteria, as they permit documents only published in English, may be a subject of critique. In this manner, our intention was not to disregard other foreign languages, but was determined by the global status of the English language in modern science. Thirdly, regarding the search query construct, including only the terms ‘‘usability’’ and ‘‘mobile applications’’ might have excluded potentially relevant documents concerning other related studies (i.e. user experience or design thinking) from the scope of the search results, and later, from the analysis, though one should bear in mind that evidence is defined as the ‘‘synthesis of the best quality scientific studies on a specific topic (. . . )’’ [90]. Nevertheless, by design, the goal of this study was to provide an evidence-based contribution on the usability of mobile applications, thus this limitation is simply the result of the application of SLR methodology principals. Ultimately, the applied reviewing procedures might be seen as too strict or hard to follow. However, we assumed to identify only such attributes and measures which can be replicated in any extent, and arbitrarily extended if necessary. B. FUTURE RESEARCH The obtained results uncover the trend in time of producing ‘‘new’’ attributes which unnecessarily contribute to the usability of mobile applications. And yet, one might try to assume that there are still some vulnerable properties laying beneath their quality of use. Nevertheless, one of the issues which unfolds definitely concerns how to consolidate the existing attributes into one compact model which reflects all identified and relevant usability facets. Moreover, in addressing the topic of usability evaluation, there is still little known about the simulation methods which might admittedly replace both experts and users in application evaluation in view of selecting its properties and behaviour. On the one hand, software vendors will benefit by reducing engaged time and effort, while on the other, the users will take advantage of the better application in daily usage. Therefore, a second suggestion is practical in nature, and relates to the matter of developing a tool in which implemented methods to automate and simulate the users’ interaction with the application may reduce the participation of both experts and users. 55572 Currently, we are developing a usability inspection method which aims to fully automate application testing due to evaluating its compliance with efficiency and effectiveness requirements and to detecting bugs and errors. The latest application version enables usability engineers to perform video analysis annotation, which aims at measuring the duration of actions on a time scale, embedded on a ribbon within a visual diagram editor. Moreover, it also allows the tasks on the layers to be graphically decomposed into smaller units (subtasks). The first results are promising, showing that if we isolate user activities from application responses, then it allows us to analyse and evaluate both the user and the application separately, which adequately produces a reliable outcome for interface designers and developers. VIII. CONCLUSION The results of the systematic literature review show that the ISO 9241-11 definition has been adapted by the majority as the baseline in the studies of mobile application usability. In total, 75 attributes were distinguished in the body of 790 documents, indexed by the Scopus database. The most frequent are efficiency (70%), satisfaction (66%) and effectiveness (58%), which originate from the above definition. Afterwards, the less frequent are learnability (45%), memorability (23%), cognitive load (19%) and errors (17%). The last two concern simplicity (13%) and easy of use (9%). The remaining attributes occurred four times or less. We observed that 91% of documents lack a usability definition. While not providing a formal and legitimate definition even seems to be acceptable in some circumstances, measuring and explaining the facets of the phenomena exclusively on the grounds of common sense might be questionable. As a matter of fact, over 90% of documents did not meet the inclusion criteria for analysis, although some report valuable results. On the other hand, a small number of the remaining documents zealously ‘‘produced’’ new attributes with associated ‘‘unique’’ measures, which usually concerned unobservable properties, measured by a set of explicit statements. Many of these constructs lack theoretical foundations and empirical evidence to expose their worthiness. To complicate the matter even further, most of the introduced attributes have focused on user beliefs, emotions, preferences, perceptions, physical and psychological responses, behaviours and accomplishments that occur before, during and after application use, which concern, in particular or as a whole, dimensions of user experience [21]. Such a combination of objective and subjective assessments eventually produces an outcome which refers to neither application usability nor user experience. This reflects an ignorance to methodological rigour, which negatively affects the validity of the results. REFERENCES [1] M. Rouse and I. Wigmore. (2013). Definition. Mobile App. [Online]. Available: https://whatis.techtarget.com/definition/mobile-app VOLUME 8, 2020 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study [2] Statista. (2019). Global Mobile Device Market Share in 2017 and 2018, by Device Type. [Online]. Available: https://www.statista.com/ statistics/183530/worldwide-market-share-mobile-device/ [3] Statista. (2019). Smartphone User Penetration as Percentage of Total Global Population From 2014 to 2021. [Online]. Available: https:// www.statista.com/statistics/203734/global-smartphone-penetration-percapita-since-2005/ [4] H. M. Salman, W. F. Wan Ahmad, and S. Sulaiman, ‘‘Usability evaluation of the smartphone user interface in supporting elderly users from experts’ perspective,’’ IEEE Access, vol. 6, pp. 22578–22591, 2018. [5] M. Sikorski, ‘‘Beyond product usability: User satisfaction and quality management,’’ in Proc. CHI Extended Abstr. Hum. Factors Comput. Syst. (CHI). New York, NY, USA: ACM, 2000, pp. 61–62. [6] H. Feng, T. Hoegler, and W. Stucky, ‘‘Exploring the critical success factors for mobile commerce,’’ in Proc. Int. Conf. Mobile Bus., Jun. 2006, p. 40. [7] E. Garcia-Lopez, A. Garcia-Cabot, C. Manresa-Yee, L. de-Marcos, and C. Pages-Arevalo, ‘‘Validation of navigation guidelines for improving usability in the mobile Web,’’ Comput. Standards Interfaces, vol. 52, pp. 51–62, May 2017. [8] N. Tractinsky, ‘‘The usability construct: A dead end?’’ Hum.-Comput. Interact., vol. 33, no. 2, pp. 131–177, Mar. 2018. [9] B. Shackel, ‘‘Usability—Context, framework, definition, design and evaluation,’’ Interacting Comput., vol. 21, nos. 5–6, pp. 339–346, Dec. 2009. [10] J. Kjeldskov and J. Stage, ‘‘New techniques for usability evaluation of mobile systems,’’ Int. J. Hum.-Comput. Stud., vol. 60, nos. 5–6, pp. 599–620, May 2004. [11] K. Gulzar, J. Sang, M. Ramzan, and M. Kashif, ‘‘Fuzzy approach to prioritize usability requirements conflicts: An experimental evaluation,’’ IEEE Access, vol. 5, pp. 13570–13577, 2017. [12] P. Weichbroth, ‘‘Delivering usability in IT products: Empirical lessons from the field,’’ Int. J. Softw. Eng. Knowl. Eng., vol. 28, no. 7, pp. 1027–1045, Jul. 2018. [13] M. S. Bashir and A. Farooq, ‘‘EUHSA: Extending usability heuristics for smartphone application,’’ IEEE Access, vol. 7, pp. 100838–100859, 2019. [14] R. Parente Da Costa, E. D. Canedo, R. T. De Sousa, R. De Oliveira Albuquerque, and L. J. Garcia Villalba, ‘‘Set of usability heuristics for quality assessment of mobile applications on smartphones,’’ IEEE Access, vol. 7, pp. 116145–116161, 2019. [15] A. Seffah, M. Donyaee, R. B. Kline, and H. K. Padda, ‘‘Usability measurement and metrics: A consolidated model,’’ Softw. Qual. J., vol. 14, no. 2, pp. 159–178, Jun. 2006. [16] S. Winter, S. Wagner, and F. Deissenboeck, ‘‘A comprehensive model of usability,’’ in Proc. IFIP Int. Conf. Eng. Hum.-Comput. Interact. Berlin, Germany: Springer, 2007, pp. 106–122. [17] K. Majrashi and M. Hamilton, ‘‘A cross-platform usability measurement model,’’ Lect. Notes Softw. Eng., vol. 3, no. 2, p. 132, 2015. [18] P. Weichbroth, ‘‘Usability attributes revisited: A time-framed knowledge map,’’ in Proc. Federated Conf. Comput. Sci. Inf. Syst., Sep. 2018, pp. 1005–1008. [19] Software Enginnering—Product Quality, Standard ISO/IEC 9126:1991, ISO/IEC, Geneva, Switzerland, 1991. [20] Ergonomic Requirements for Office Work With Visual Display Terminals (VDTs)—Part 11: Guidance on Usability, Standard ISO 924111:1998(en), ISO, Geneva, Switzerland, 1998. [Online]. Available: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-1:v1:en [21] Ergonomics of Human-System Interaction—Part 11: Usability: Definitions and Concepts, ISO 9241-11:2018(en), ISO, Geneva, Switzerland, 2018. [Online]. Available: https://www.iso.org/obp/ui/#iso:std:iso:9241:11:ed-2:v1:en [22] J. R. Lewis, ‘‘Usability: Lessons learned. . . and yet to be learned,’’ Int. J. Hum.-Comput. Interact., vol. 30, no. 9, pp. 663–684, Sep. 2014. [23] D. Quiñones and C. Rusu, ‘‘How to develop usability heuristics: A systematic literature review,’’ Comput. Standards Interfaces, vol. 53, pp. 89–122, Aug. 2017. [24] Systems and Software Engineering—Systems and Software Quality Requirements and Evaluation (SQuaRE)—System and Software Quality Models, Standard ISO/IEC 25010:2011, ISO, Geneva, Switzerland, 2011. [25] Software Engineering—Product Quality—Part 1: Quality Model, Standard ISO/IEC 9126-1:2001, ISO, Geneva, Switzerland, 2001. [Online]. Available: https://www.iso.org/standard/22749.html [26] J. Nielsen, ‘‘Iterative user-interface design,’’ Computer, vol. 26, no. 11, pp. 32–41, Nov. 1993. VOLUME 8, 2020 [27] N. Bevan, ‘‘Usability is quality of use,’’ in Advances in Human Factors/Ergonomics, vol. 20. Amsterdam, The Netherlands: Elsevier, 1995, pp. 349–354. [28] J. M. C. Bastien, ‘‘Usability testing: A review of some methodological and technical aspects of the method,’’ Int. J. Med. Inform., vol. 79, no. 4, pp. e18–e23, Apr. 2010. [29] T. Boren and J. Ramey, ‘‘Thinking aloud: Reconciling theory and practice,’’ IEEE Trans. Prof. Commun., vol. 43, no. 3, pp. 261–278, 3rd Quart., 2000. [30] S. Richardson, R. Mishuris, A. O’Connell, D. Feldstein, R. Hess, P. Smith, L. McCullagh, T. McGinn, and D. Mann, ‘‘‘Think aloud’ and ‘Near live’ usability testing of two complex clinical decision support tools,’’ Int. J. Med. Inform., vol. 106, pp. 1–8, Oct. 2017. [31] G. Deniz and P. O. Durdu, ‘‘A comparison of mobile form controls for different tasks,’’ Comput. Standards Interfaces, vol. 61, pp. 97–106, Jan. 2019. [32] T. Kato, ‘‘What ‘question-asking protocols’ can say about the user interface,’’ Int. J. Man-Mach. Stud., vol. 25, no. 6, pp. 659–673, Dec. 1986. [33] T. Obata, T. Daimon, and H. Kawashima, ‘‘A cognitive study of invehicle navigation systems: Applying verbal protocol analysis to usability evaluation,’’ in Proc. Vehicle Navigat. Inf. Syst. Conf. (VNIS), Oct. 1993, pp. 232–237. [34] R. Nagpal, D. Mehrotra, and P. K. Bhatia, ‘‘The state of art in Website usability evaluation methods,’’ in Design Solutions for UserCentric Information Systems. Harrisburg, PA, USA: IGI Global, 2017, pp. 275–296. [35] et. al. R. Nagpal, ‘‘FAHP approach to rank educational Websites on usability,’’ Int. J. Comput. Digit. Syst., vol. 4, no. 4, pp. 251–260, Oct. 2015. [36] A. Hinderks, M. Schrepp, F. J. Domínguez Mayo, M. J. Escalona, and J. Thomaschewski, ‘‘Developing a UX KPI based on the user experience questionnaire,’’ Comput. Standards Interfaces, vol. 65, pp. 38–44, Jul. 2019. [37] J.-P. Jain and W.-C. Shen, ‘‘A study of the optimization of app design with regards to usability and user experience—a case study of SunlineApp,’’ in Proc. Eng. Innov. Design, 7th Int. Conf. Innov., Commun. Eng. (ICICE), Hangzhou, China. CRC Press, May 2019, p. 175. [38] A. Holzinger, ‘‘Usability engineering methods for software developers,’’ Commun. ACM, vol. 48, no. 1, pp. 71–74, Jan. 2005. [39] L. F. Gonçalves, L. G. Vasconcelos, E. V. Munson, and L. A. Baldochi, ‘‘Supporting adaptation of Web applications to the mobile environment with automated usability evaluation,’’ in Proc. 31st Annu. ACM Symp. Appl. Comput. (SAC). New York, NY, USA: ACM, 2016, pp. 787–794. [40] J. Grigera, A. Garrido, J. M. Rivero, and G. Rossi, ‘‘Automatic detection of usability smells in Web applications,’’ Int. J. Hum.-Comput. Stud., vol. 97, pp. 129–148, Jan. 2017. [41] J. Falkowska, J. Sobecki, and M. Pietrzak, ‘‘Eye tracking usability testing enhanced with EEG analysis,’’ in Proc. Int. Conf. Design, User Exper., Usability. Cham, Switzerland: Springer, 2016, pp. 399–411. [42] P. Weichbroth, K. Redlarski, and I. Garnik, ‘‘Eye-tracking Web usability research,’’ in Proc. Federated Conf. Comput. Sci. Inf. Syst., Oct. 2016, pp. 1681–1684. [43] P. Realpe-Muñoz, C. A. Collazos, J. Hurtado, T. Granollers, J. Muñoz-Arteaga, and J. Velasco-Medina, ‘‘Eye tracking-based behavioral study of users using e-voting systems,’’ Comput. Standards Interfaces, vol. 55, pp. 182–195, Jan. 2018. [44] T. Tullis, S. Fleischman, M. McNulty, C. Cianchette, and M. Bergel, ‘‘An empirical comparison of lab and remote usability testing of Web sites,’’ in Proc. Usability Professionals Assoc. Conf., 2002, pp. 1–8. [45] J. Dong, S. L. Martin, J. M. Mullaly, and A. R. Tannenbaum, ‘‘Method, system and program for performing remote usability testing,’’ U.S. Patent 6 526 526, Feb. 25, 2003. [46] J. Sauer, A. Sonderegger, K. Heyden, J. Biller, J. Klotz, and A. Uebelbacher, ‘‘Extra-laboratorial usability tests: An empirical comparison of remote and classical field testing with lab testing,’’ Appl. Ergonom., vol. 74, pp. 85–96, Jan. 2019. [47] M. J. Scott, F. Spyridonis, and G. Ghinea, ‘‘Designing for designers: Towards the development of accessible ICT products and services using the VERITAS framework,’’ Comput. Standards Interfaces, vol. 42, pp. 113–124, Nov. 2015. [48] J. Díaz, C. Rusu, and C. A. Collazos, ‘‘Experimental validation of a set of cultural-oriented usability heuristics: E-commerce Websites evaluation,’’ Comput. Standards Interfaces, vol. 50, pp. 160–178, Feb. 2017. 55573 P. Weichbroth: Usability of Mobile Applications: Systematic Literature Study [49] T. C. Lacerda and C. G. von Wangenheim, ‘‘Systematic literature review of usability capability/maturity models,’’ Comput. Standards Interfaces, vol. 55, pp. 95–105, Jan. 2018. [50] D. Quiñones, C. Rusu, and V. Rusu, ‘‘A methodology to develop usability/user experience heuristics,’’ Comput. Standards Interfaces, vol. 59, pp. 109–129, Aug. 2018. [51] I. Connell, A. Blandford, and T. Green, ‘‘CASSM and cognitive walkthrough: Usability issues with ticket vending machines,’’ Behav. Inf. Technol., vol. 23, no. 5, pp. 307–320, Sep. 2004. [52] T. Hollingsed and D. G. Novick, ‘‘Usability inspection methods after 15 years of research and practice,’’ in Proc. 25th Annu. ACM Int. Conf. Design Commun. (SIGDOC). New York, NY, USA: ACM, 2007, pp. 249–255. [53] L. Alonso-Virgós, J. P. Espada, and R. G. Crespo, ‘‘Analyzing compliance and application of usability guidelines and recommendations by Web developers,’’ Comput. Standards Interfaces, vol. 64, pp. 117–132, May 2019. [54] Z. Zhang, V. Basili, and B. Shneiderman, ‘‘Perspective-based usability inspection: An empirical validation of efficacy,’’ ...
Purchase answer to see full attachment
User generated content is uploaded by users for the purposes of learning and should be used following Studypool's honor code & terms of service.

Explanation & Answer

Attached. Please let me know if you have any questions or need revisions.

1

Running Head: DISCUSSION

Discussion
Institutional Affiliation
Name
Date

DISCUSSION

2

Question One
The issues that I think have been assigned salience by audiences due to the agenda-setting
process that might not be otherwise considered salient include memorability and cognitive load.
Memorability refers to the degree to which the device user can remember how to use and apply
the application effectively. Memorability is usually measured by asking the users to do a series
of tasks aimed at proving their proficiency with the application. To determine the memorability
level of the application, a direct comparison is made between the results obtained. It helps
determine whether the users find the application salient or not. In this case, memorability is not a
factor that should be given when it comes to assigning the importance of applications to
audiences. This is because most applications come with instructions and have unique features
that make it memorable to the users.
On the other hand, the cognitive load should not be a salient feature when it comes to
checking the effectiveness of an application. Cognitive load defines the amount of mental
activity that a user is allowed when using an application. There are three...


Anonymous
Really helped me to better understand my coursework. Super recommended.

Studypool
4.7
Trustpilot
4.5
Sitejabber
4.4

Similar Content

Related Tags