Advanced Topics in End User Computing - Mahmood M. Adam

Author: Mahmood M. Adam

Tags: computer engineering

ISBN: 1-59140-474-6

Year: 2005

Similar

Distributed Computing Fundamentals, Simulations, and Advanced Topics

Advances on Intelligent Informatics and Computing

CPU. Computer Power User. Vol.17 Issue 09

User Experience (UX). User Interface (UI)

Text

TEAM LinG

Advanced Topics in
End User Computing
Volume 4
M. Adam Mahmood
University of Texas, El Paso, USA

IDEA GROUP PUBLISHING
Hershey • London • Melbourne • Singapore

TEAM LinG

Acquisitions Editor:
Development Editor:
Senior Managing Editor:
Managing Editor:
Copy Editor:
Typesetter:
Cover Design:
Printed at:

Renée Davies
Kristin Roth
Amanda Appicello
Jennifer Neidig
Sue VanderHook
Cindy Consonery
Integrated Book Technology
Integrated Book Technology

Published in the United States of America by
Idea Group Publishing (an imprint of Idea Group Inc.)
701 E. Chocolate Avenue, Suite 200
Hershey PA 17033
Tel: 717-533-8845
Fax: 717-533-8661
E-mail: cust@idea-group.com
Web site: http://www.idea-group.com
and in the United Kingdom by
Idea Group Publishing (an imprint of Idea Group Inc.)
3 Henrietta Street
Covent Garden
London WC2E 8LU
Tel: 44 20 7240 0856
Fax: 44 20 7379 3313
Web site: http://www.eurospan.co.uk
Copyright © 2005 by Idea Group Inc. All rights reserved. No part of this book may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including
photocopying, without written permission from the publisher.
Product or company names used in this book are for identification purposes only. Inclusion of the
names of the products or companies does not indicate a claim of ownership by IGI of the
trademark or registered trademark.
Advanced Topics in End User Computing, Volume 4 is part of the Idea Group Publishing series
named Advanced Topics in End User Computing Series (ISSN: 1537-9310).
ISBN: 1-59140-474-6
Paperback ISBN: 1-59140-475-4
eISBN: 1-59140-476-2
British Cataloguing in Publication Data
A Cataloguing in Publication record for this book is available from the British Library.
All work contributed to this book is new, previously-unpublished material. The views expressed in
this book are those of the authors, but not necessarily of the publisher.

TEAM LinG

Advanced Topics in
End User Computing
Volume 4

Table of Contents
Preface ............................................................................................................ vi
M. Adam Mahmood, University of Texas at El Paso, USA
SECTION I: ORGANIZATIONAL AND END U SER COMPUTING ISSUES,
PERFORMANCE, PRODUCTIVITY
Chapter I
End User Computing Research Issues and Trends (1990-2000) ...... 1
James P. Downey, University of Central Arkansas, USA
Summer E. Bartczak, Air Force Institute of Technology, USA
Chapter II
The Effect of End User Development on End User Success ........... 21
Tanya McGill, Murdoch University, Australia
Chapter III
Testing the Technology-to-Performance Chain Model ...................... 42
D. Sandy Staples, Queen’s University, Canada
Peter B. Seddon, The University of Melbourne, Australia
Chapter IV
The Role of Personal Goal and Self-Efficacy in Predicting
Computer Task Performance ................................................................... 65
Mun Y. Yi, University of South Carolina, USA
Kun S. Im, Yonsei University, South Korea

TEAM LinG

Chapter V
Measurement of Perceived Control in Information Systems ........... 90
Steven A. Morris, Middle Tennessee State University, USA
Thomas E. Marshall, Auburn University, USA
Chapter VI
The Technology Acceptance Model: A Meta-Analysis of
Empirical Findings ................................................................................... 112
Qingxiong Ma, Central Missouri State University, USA
Liping Liu, University of Akron, USA
SECTION II: C OLLABORATIVE TECHNOLOGIES AND IMPLEMENTATION ISSUES
Chapter VII
Success Factors in the Implementation of a Collaborative
Technology and Resulting Productivity Improvements in a
Small Business: An Exploratory Study .................................................. 1 2 9
Nory B. Jones, University of Maine, USA
Thomas R. Kochtanek, University of Missouri in Columbia,
USA
Chapter VIII
Supporting the JAD Facilitator with the Nominal Group
Technique ................................................................................................... 1 5 1
Evan W. Duggan, University of Alabama, USA
Cherian S. Thachenkary, Georgia State University, USA
Chapter IX
Applying Strategies to Overcome User Resistance in a Group of
Clinical Managers to a Business Software Application:
A Case Study .............................................................................................. 1 7 4
Barbara Adams, Cyrus Medical Systems, USA
Eta S. Berner, University of Alabama at Birmingham, USA
Joni Rousse Wyatt, Norwood Clinic, USA
Chapter X
Responsibility for Information Assurance and Privacy:
A Problem of Individual Ethics? ........................................................... 1 8 6
Bernd Carsten Stahl, De Montfort University, UK
Chapter XI
Organizational Knowledge Sharing in ERP Implementation:
Lessons from Industry ............................................................................ 2 0 8
Mary C. Jones, University of North Texas, USA
R. Leon Price, University of Oklahoma, USA

TEAM LinG

SECTION III: E-COMMERCE PROCESSES AND PRACTICES
Chapter XII
Electronic Banking and Information Assurance Issues:
Survey and Synthesis ............................................................................... 2 3 3
Manish Gupta, State University of New York, USA
Raghav Rao, State University of New York, USA
Shambhu Upadhyaya, State University of New York, USA
Chapter XIII
Computer Security and Risky Computing Practices:
A Rational Choice Perspective ............................................................. 2 5 7
Kregg Aytes, Idaho State University, USA
Terry Connolly, University of Arizona, USA
Chapter XIV
A TAM Analysis of an Alternative High-Security User
Authentication Procedure ...................................................................... 2 8 0
Merrill Warkentin, Mississippi State University, USA
Kimberly Davis, Mississippi State University, USA
Ernst Bekkering, Northeastern State University, USA
Chapter XV
A Blended Approach Learning Strategy for Teacher
Development ............................................................................................. 3 0 1
Kalyani Chatterjea, Nanyang Technological University,
Singapore
About the Editor ....................................................................................... 3 2 2
About the Authors ................................................................................... 3 2 3
Index ........................................................................................................... 3 3 1

TEAM LinG

Preface

This scholarly book is a collection of some of the best manuscripts published in the Journal of Organizational and End User Computing. This introduction is mainly a collection of abstracts provided by the authors for their
manuscripts. The book is divided into three segments: Section I, which covers
organizational and end user computing issues, trends, and success; Section II,
which addresses collaborative technologies and implementation issues; and
Section III, which discusses e-commerce processes and practices.
Section I consists of six chapters. Chapter 1, by Downey and Bartczak,
starts the section by providing a comprehensive framework for research that
allows one to examine the trends and issues in end user computing. It is based
on a comprehensive review of research articles from some of the leading journals in the information systems area. The review is precipitated, according to
the author, by the fact that during the 1980s and early 1990s, end user computing was reported to be among the key concerns facing managers and organizations. The authors claim that the framework is parsimonious and allows a comprehensive classification of three major dimensions of end user computing: end
user, technology, and organization. The authors conclude by discussing emerging trends, important themes, and journal differences in the area.
Chapter II of this scholarly volume is penned by McGill. She discusses the
contribution of systems developed by users on systems success. Her contention is that since end user systems development is a significant part of organizational systems development, it deserves attention. She investigated the role an
application developed by the user developer plays on the eventual success of
the application itself. The results of her study are intuitive but very important.
She suggest that end users are likely to be more satisfied with systems they
develop than with ones developed by others. More interestingly, the author found
that end users also perform better with these systems.

TEAM LinG

vii

To help end users and organizations understand and make more effective
use of information technology, Staples and Seddon proposed the Technologyto-Performance Chain (TPC) model in 1995. According to the authors, the TPC
model combines insights from research on user attitudes as predictors of utilization and insights from research on task-technology fit as a predictor of performance. In Chapter III of this scholarly book, the same authors tested the
TPC model in two settings—voluntary use and mandatory use. In both settings,
they found strong support for the impact of task-technology fit on performance,
as well as on attitudes and beliefs about use. Social norms also had a significant
impact on utilization in the mandatory use setting. They also found that beliefs
about use only had a significant impact on utilization in the voluntary use setting. Overall, the authors found support for the predictive power of the TPC
model.
In Chapter IV, Yi and Im suggest that computer task performance is an
essential driver of end user productivity. Recent research, according to the
authors, indicates that computer self-efficacy (CSE) is an important determinant of computer task performance. They argue that understanding the role of
personal goal (PG) is also important in predicting and determining computer
task performance. Employing CSE, PG, age, and experience, the authors developed a theoretical model that predicts individual computer task performance.
They validate this model using PLS on data derived from a Microsoft Excel
training class of 41 MBA students. They found PG, along with past experience
and age, play a significant role in predicting computer task performance. Interestingly, the authors found no significant relationship between post-training CSE
and task performance.
In Chapter V, Morris and Marshall claimed that several disciplines have
already identified and validated the importance of control in explaining human
behavior and motivation. They report an exploratory investigation that assesses
perceived control within the information systems (IS) area. The authors developed a survey instrument, based on the research literature in the IS area, to
assess perceived control as a multi-dimensional construct. They validated this
instrument using 241 subjects. They analyzed their results to produce a set of
five factors that represent a user’s perceptions of control when working with
an interactive information system: timeframe, feedback signal, feedback duration, strategy, and metaphor knowledge.
In Chapter VI, the final chapter in this section, Ma and Liu conducted a
meta analysis to synthesize and summarize the findings of 26 prior research
studies on perceived ease of use and usefulness that used the technology acceptance model (TAM) as a framework to predict the acceptance of information technology. A number of past studies have empirically investigated these
relationships, but, as the authors indicated, the findings of these research studies are mixed. The authors found that both the correlations between usefulness
and acceptance, and between usefulness and ease of use are somewhat strong.

TEAM LinG

viii

They found the relationship between ease of use and acceptance as weak.
As stated earlier, Section II addresses collaborative technologies and implementation issues. It consists of five chapters: Chapters VII, VIII, IX, X, and
XI. In Chapter VII, Jones and Kochtanek recognize that literature provides
many examples of performance improvements resulting from adoption of different technologies. The authors, at the same time, claim that they found very
little evidence demonstrating specific and generalizable factors that contribute
to these improvements. The authors’ qualitative study examined the relationship between four classes of potential success factors on the adoption of a
collaborative technology and whether they are related to performance improvements in a small service company. They interviewed the users of a newly adopted
collaborative technology to explore which factors contributed to the users’ initial adoption and subsequent effective use of this technology. Their results showed
that several factors were strongly related to adoption and effective implementation. They further explored the impact on performance improvements. Their
results showed a qualitative link to several performance improvements, including time savings and improved decision-making.
In Chapter VIII, Duggan and Thachenkary start by suggesting that the
Joint Application Development (JAD) was introduced to solve many of the
problems system users experienced with the conventional methods used in determining systems requirements. They recognize that JAD helped produce noteworthy improvements over these methods. They suggest that a JAD session
conducted with freely interacting groups is susceptible, however, to some problems that may curtail the effectiveness of groups. They further suggest that
JAD outcomes are also critically dependent on excellent facilitation for minimizing dysfunctional group behaviors, and many JAD efforts fail because some
group members are often unavailable. According to the authors, the nominal
group technique (NGT) was designed to reduce the impact of negative group
dynamics. The authors integrate JAD and NGT to reduce the burden of the
JAD facilitator in controlling group sessions for determining systems requirements. They empirically tested their approach, which was found to outperform
JAD in the areas tested and seemed to contribute to group outcomes even
without excellent facilitation.
Adams, Berner, and Wyatt in Chapter IX suggest that user resistance is a
common occurrence when new information systems are introduced to health
care organizations. They further suggest that individuals responsible for overseeing the implementation process of these systems in the health care environment may encounter more resistance than facilitators in other environments.
The authors claim that proper training of end users is an important strategy for
minimizing this resistance. Their research reviews the literature on the reasons
for user resistance to health care information systems and the implications of
this literature on the design of training programs. They illustrate principles for
reducing user resistance (e.g., communication, user involvement, strategic use

TEAM LinG

of consultants) using a case study that involved training clinical managers on
business applications. The authors recommend that individuals responsible for
health care information system implementations should recognize that end user
resistance can lead to system failure and should employ these best practices
when embarking on new implementations.
In Chapter X, Stahl suggests that decisions regarding information assurance, IT security, and privacy can affect individuals’ rights and obligations. The
author explores the question of whether individual responsibility is a useful construct to address ethical issues of this complexity. After introducing a theory of
responsibility, he discusses the conditions that an individual typically is assumed
to fulfill in such an environment. The author argues that individuals lack some
of the essential preconditions necessary for handling responsibility. According
to the author, individuals have neither the power, the knowledge, nor the intellectual capacities to successfully deal with the ethical challenges in the tension
of privacy and information assurance. The author ends by suggesting that the
concept of responsibility may be useful nevertheless in this setting, but it will
have to be expanded to allow collective entities as subjects.
In Chapter XI, Jones and Price put forth that knowledge sharing in ERP
implementation is somewhat unique, because ERP requires end users to have
more divergent knowledge than is required in the use of traditional information
systems. They claim that, because of the length of the time and commitment
that ERP implementation requires, end users often are more involved in ERP
implementations than they are in more traditional information systems implementations. Their study presents findings about organizational knowledge sharing during ERP implementation in three firms. They collected data through interviews using a multi-site case study methodology. The authors analyzed the
findings in an effort to provide a basis on which practitioners can more effectively facilitate knowledge sharing during ERP implementation.
The last and final section in this compiled volume deals mainly with ecommerce processes and practices. It includes four chapters: Chapters XII,
XIII, XIV, and XV. In Chapter XII, Gupta, Rao, and Upadhyaya assert that
information assurance is a key component in e-banking services. They investigate the information assurance issues and tenets of e-banking security that
would be needed for the design, development, and assessment of an adequate
electronic security infrastructure. They present the technology terminology and
frameworks with an understanding to equip the reader with a glimpse of the
state-of-art technologies that may help toward making better decisions regarding electronic security.
In Chapter XIII, Aytes and Connolly present the Check-Off Password
System (COPS) for entering passwords that combines a high level of security
with easy recall features for end users. They claim that COPS is more secure
than self-selected passwords as well as high-protection assigned-password procedures (FIPS). The authors provide a preliminary assessment of the efficacy

TEAM LinG

of COPS by comparing COPS with three traditional password-assigning procedures. They showed that end users perceive all password-assigning procedures
tested to have equal usefulness, but the perceived ease of use of COPS equals
that of an established high-security password. They claim that the COPS interface does not negatively affect user performance compared with that of highsecurity passwords generating systems.
In Chapter XIV, Warkentin, Davis, and Bekkering state that the main objective of information system security management is information assurance.
The authors claim that user authentication is an important means toward achieving this objective, and password procedures have historically been the primary
method for user authentication. As expected, the authors found an inverse relationship between the level of security provided by a password procedure and
ease of recall for users. Also, as expected, the authors found the longer the
password and the more variability in its characters, the higher the level of security provided by such a password. They state that such a password, however,
tends to be more difficult for end users to remember, particularly when the
password does not spell a recognizable word. Conversely, when end users select their own passwords that are easier to memorize and recall, the passwords
may also be easier to crack.
In Chapter XV, the last chapter in this scholarly volume, Chatterjea states
that in-service upgrading has been provided for retraining teachers in Singapore
to help them keep abreast of changing curriculum requirements as well as a
way of infusing information technology in teaching and learning. She further
states that upgrading courses are offered to the teachers primarily asynchronously, using the Internet platform, with some integrated synchronous sessions.
The author provides rationales for the development of such Web-based teacherupgrading systems and discusses the developmental issues related to such systems. She also addresses issues of adult learning in a learner-controlled adaptive learning environment that provides the much-needed freedom to the participants for managing their own time. The author concludes by discussing the
participants’ responses to such an upgrading system.

TEAM LinG

Acknowledgments

I wish to recognize contributions made by the reviewers and associate
editors in bringing this scholarly book to fruition. I thank them for diligently and professionally reviewing the manuscripts included in this volume. My thanks go to the authors for being highly responsive to reviewers’ and associate editors’ comments and promptly meeting the
deadline imposed on them. They have made outstanding contributions
to this volume.
I express my special thanks to Hettie Houghton in the Department of
Information and Decision Sciences at the University of Texas at El
Paso. She was extremely diligent in keeping the project on track. Her
effort toward the project is truly appreciated. I also want to thank Jan
Travers for her help with the project.
M. Adam Mahmood
University of Texas at El Paso, USA

TEAM LinG

Section I:
Organizational
and
End User Computing
Issues,
Performance,
Productivity

TEAM LinG

End User Computing Research Issues and Trends (1990-2000)

Chapter I

End User Computing
Research Issues and Trends
(1990-2000)
James P. Downey, University of Central Arkansas, USA
Summer E. Bartczak, Air Force Institute of Technology, USA

ABSTRACT

During the 1980s and into the early 1990s, end user computing (EUC) was
reported to be among the key concerns facing managers and organizations.
Is EUC still an important topic? This study examines academic research
during this period. A research-focused framework is offered to provide a
conceptual structure for examining the trends and issues in EUC. This
framework is parsimonious and also allows a comprehensive classification
of end user computing’s three major dimensions: end user, technology, and
organization. The study examines every article from five leading information
systems (IS) journals (ISR, MISQ, JMIS, I&M, and JEUC) for the 11 years
1990-2000. The results indicate that there has been no diminishing of EUC
interest and studies during this time, either overall or in any journal or
dimension. A discussion of emerging trends, important themes, and journal
differences concludes this examination.

INTRODUCTION

EUC has been evolving since the appearance of mainframe end users in the
late 1960s; it was mainstreamed with the introduction of the personal computer
more than 20 years ago. As organizations and individuals discovered the
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

2 Downey and Bartczak

advantages and capabilities of personal computing technology, new competencies and efficiencies were developed that transformed the workplace. The
academic study of EUC grew out of an attempt to provide direction and control
to managers, executives, and knowledge workers who persisted in using this new
technology.
The importance of EUC was evident early on as academics and practitioners consistently rated it as one of the key areas of concern. In a list of the most
important managerial issues, Dickson, Leitheiser, Wetherbe, and Nechis (1984)
reported EUC as the second-most important. Brancheau and Wetherbe (1987)
placed it at number six. More recently, EUC has been ranked high in a number
of different settings and nations. Managers of small businesses ranked training
and education of end users as no. 2 and end user support as no. 4 (Riemenschneider
& Mykytyn, 2000). U.S. multinational corporations ranked EUC as no. 4 of 32
top issues (Deans, Karwan, Goslar, Ricks, & Toyne, 1990-91), while U.S. public
sector organizations ranked it no. 4, with office automation no. 5 (Caudle, Gorr,
& Newcomer, 1991). Taiwanese managers ranked communications with end
users no. 2 (Yang, 1996), while a similar study in China listed the same issue no.
1 (Wang, 1994).
The importance of EUC, however, is not reflected in other studies. In the
last few years, for example, the relative importance of EUC in the workplace has
reportedly been diminishing, particularly in the U.S. Niederman, Brancheau, and
Wetherbe (1991) reported that facilitating and managing EUC was the no. 18
most important managerial issue. Four years later Brancheau, Janz, and Wetherbe
(1996) placed it as no. 16, as did Lee, Trauth, and Farwell (1995) in their study
of critical IS activities. Clearly, there are some inconsistencies present regarding
EUC’s importance.
Part of the reason for these conflicting studies is the lack of concurrence
as to what comprises EUC today. It is important to note that there is a distinction
between managerial EUC and EUC as used in academic literature. To the
manager in the organizational setting, end user computing comprises the functions of planning, managing, and supporting the computer needs of end users. As
organizations gain computing experience and expertise, EUC becomes less
important as a management issue, as is evident in some of the larger or more
technologically advanced organizations (Essex, Magal, & Masteller, 1998;
Guimaraes & Igbaria, 1994).
To the IS academic community, however, EUC covers a wide range of
themes and research, from investigations into the nature of individual attitudes
and behaviors toward IT to organizational strategies for project development. In
fact, there is disagreement as to what should be included in such research. In
more than 20 years of research in EUC, there is no consensus as to what EUC
success means or how organizations should assess their EUC needs (Harris,
2000). Despite this lack of agreement as to what constitutes EUC, a comprehensive examination of relevant EUC research reveals some consistent patterns and
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

End User Computing Research Issues and Trends (1990-2000)

themes. This study specifically examines all EUC articles from five leading IS
journals for the years 1990-2000. The research indicates that EUC is still a wellresearched and relevant topic for practitioners and academics alike.
The objectives of this book are (1) to examine the nature and characteristics
of EUC and end users; (2) to present a comprehensive framework for the study
of EUC based on the dimensions of end user, technology, and organization; (3)
to assess how this framework characterizes the various themes of EUC as
present in the literature between 1990 and 2000; and (4) to explore the position
of EUC within the IS academic community by detecting and establishing EUC
research trends and issues.

END USER COMPUTING AND END USERS

EUC as a subset of IS has been examined since before 1980. In an early
study, Benson (1983) noted the shift from mainframe computing to microcomputers and reported on relevant management issues concerning this change. As
computing became available and useful to users and managers outside the data
processing centers, it evolved along three paths: growth in number of users,
growth in the hardware and software technologies, and growth in computer skills
of users (Harris, 2000).
Disagreement persists over what EUC actually is and even the identity of
end users (Rainer & Harrison, 1993). There exist two widespread views or
definitions of EUC, one broad and one that focuses on applications development.
The more restricted definition states that EUC is the adoption and use of
information systems by users outside the IS department to develop software
applications to support organizational tasks and decision making (Aggarwal,
1994; Brancheau & Brown, 1993; Shah & Lawrence, 1996; Shayo, Guthrie, &
Igbaria, 1999).
Others define EUC more generally. Ein-Dor and Segev (1992) characterize
it as any hands-on use of PCs. Essex, Magal, and Masteller (1998) describe it
as the direct use of information technology by end users. Barker (1995) defines
EUC as the application of computing resources for the purpose of producing
information. Rainer and Harrison (1993) define EUC as the direct, individual use
of computers encompassing all the computer-related activities required or
necessary to accomplish one’s job. These definitions clearly recognize a more
ubiquitous end user. Recent evidence suggests a rapidly closing gap between the
typical end user and the data processing professional of 10 years ago (Aggarwal,
1996; McLean & Kappelman, 1992-93).
As depicted in IS literature, end users are individuals who develop and/or
use IS. In one of the earliest and most influential taxonomies of end users,
Rockart and Flannery (1983) categorize users according to their skills and use
of IS. Subsequently, users have been described as managers, professionals, and
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

4 Downey and Bartczak

supervisors (Aggarwal, 1994), software developers (Brancheau & Brown,
1993), and those who develop, interact, and otherwise utilize application systems
(Glorfeld & Cronan, 1993).
For the purpose of this study, EUC is defined as the use and/or development
of computing technology and software applications by end users to solve
organizational problems and assist in decision making. End users are non-IS
department individuals who directly use and/or develop computing technology
and application systems in an organizational setting. Thus, end users are direct
(not indirect) IS users, using a variety of technologies that include group support
systems, decision support systems, executive information systems, and a host of
common software application systems such as word processing, spreadsheets,
and databases.

EUC RESEARCH-FOCUSED FRAMEWORK

A framework should partition and organize a topic into manageable parts to
enable the user to easily traverse the subject (Kochen, 1985-86). In order to
examine applicable EUC studies, a research-focused framework was developed
to provide a conceptual structure for EUC literature that is parsimonious yet
allows a comprehensive classification of its themes. This framework was
adopted after careful examination of those available in the literature. Three in
particular were used as a basis: the IS success framework of DeLone and
McLean (1992), the EUC management research framework of Brancheau and
Brown (1993), and the general framework of Harris (2000), which divided IS
success into three factors—behavioral, technological, and organizational.
As shown in Figure 1, EUC research can be divided logically into three
dimensions, depending on the focus of the study.
These three dimensions are the end user, the technology, and the organization. Although derived from the three frameworks mentioned, these three
dimensions have, in fact, been used in the past in IS/MIS literature. For example,
in one of the early frameworks for MIS, Mason and Mitroff (1973) recognized
that the information system (application) consisted of a person attempting to
solve a problem within an organizational context. Nolan and Wetherbe (1980)
introduced a process model through which personnel transform inputs using MIS
technology within an organizational context. Galliers and Land (1987) submitted
a taxonomy of IS approaches that divides research into society, organization (or
groups within the organization), individual, technology, and methodology. In a
survey of academic and business practitioners, Aggarwal (1994) categorized
their responses concerning IT into three categories: technical, organizational,
and people. The division of IS/EUC research into the three areas of end user,
technology, and organization has been a useful technique to describe and
categorize IS and end user computing.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

End User Computing Research Issues and Trends (1990-2000)

Figure 1. EUC research-focused framework

End User
Dimension

Technology
Dimension

Satisfaction
Usage Anxiety
Self-efficacy
Attitudes
Skills Norms
Tech. Acceptance
Training
Task

TECHNOLOGY
IMPACT:
Systems
Information
Individual
Group
Organization

Organization
Dimension

Strategy
Project Develop.
EU Support
Tech. Acceptance

There are some key differences between the proposed research-focused
framework and others. DeLone and McLean (1992) studied the broad spectrum
of IS, thereby including a variety of studies that did not directly involve EUC or
end users. Many organizational or systems quality studies do not have an EUC
component; for example, Benarock and Kauffman (1999) studied a nominal
EUC issue—project development—but from a capital budget standpoint with no
end user constructs. DeLone and McLean relied on empirical studies, which
excluded some exceptional conceptual studies that focused on EUC.
Brancheau and Brown (1993) defined end users as software developers
only, which limited their framework and ability to categorize all EUC studies. The
success framework of Harris (2000) is most similar, in that it included the same
three dimensions of EUC. But it differs in usage and generalizability. For
example, he examined only 16 articles, most employing some form of user
satisfaction as the dependent variable. Classifying research based only on the
dependent variable limits the rich aggregation of all EUC variables, both
dependent and independent.
The proposed EUC research-focused framework provides a taxonomy of
EUC research. It divides EUC research based on the focus of the study.
Fundamentally, an article can be classified into one of the three dimensions based
on whether it concentrates on a particular technology, the individual end user, or
some organization aspect. A description of these dimensions follows.
End User Dimension. In one respect, all IS/EUC research involves an end
user. An end user is an individual who directly uses and/or develops application
systems in an organizational setting. In order for a study to be included in this
dimension, it must focus on the end user, either empirically through constructs,
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

6 Downey and Bartczak

through variables at the individual level, or through a conceptually relevant
theme. This includes identifying end users and their behaviors, attitudes, skills,
and applicable antecedents.
The end user themes identified in the literature include the two most
common dependent variables—satisfaction and usage (DeLone & McLean,
1992). Other themes in the end user dimension include attitudes (toward a
technology), skills, self-efficacy, anxiety, and technology acceptance and diffusion. They commonly measure cognitive or affective attributes of the end user.
Technology Dimension. This component is reserved for those studies that
focus on the technology itself. These articles typically include research on newer
technologies of interest, such as group support systems (GSS), decision support
systems (DSS), group decision support systems (GDSS), expert systems,
executive information systems, and databases. The focus of such research is
always on methods to assess and/or improve the effectiveness or efficiency of
the technology.
Merely focusing on such a technology is not enough for an article to be
labeled as an EUC study. In order to be included in this study, the reference to
the end user must be clearly established through a measured construct or
variable, or other data that is unmistakable (such as may be present in a case or
field study). An article that studies only the technology and not its relationship
with an end user cannot be included in EUC. This end user relationship with
technology is generally incorporated in these studies through an assessment of
the impact on the end user. This impact assessment, or end user component, is
what distinguishes the study as belonging to EUC literature.

Levels of Technology Dimension Impact

Information systems (technologies) make an impact at four different levels
(Brancheau & Brown, 1993; DeLone & McLean, 1992; Harris, 2000; Powell &
Moore, 2002; Seddon, 1997): system or information level, individual level, group
level, and organizational level. These are summarized below:
•
•
•

System or Information Level: Studies of the relationship between end user
and the impact on system or information quality (i.e., impact of distortion
effects by end users in Sussman and Sproull, 1999).
Individual Level: Impact of technology on individual performance (such as
decision-making time or accuracy).
Group Level: Because end users may be members of groups, the effectiveness and/or efficiencies of group performance impacts the end user.
DeLone and McLean (1992) fuse group impact into departmental performance, one of the descriptions of individual impact.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

End User Computing Research Issues and Trends (1990-2000)

•

Organizational Level: Although there are many studies that examine the
organizational and IS relationship, those in this category are constrained in
that the article’s focus must be on the technology while simultaneously
including an explicit end user and organizational measure. An example of
this type of article is a field study by Vandenbosch and Huff (1997) that
examined the factors affecting executive retrieval behavior using EIS
technology and the impact of that behavior on organizational performance.
It is useful to note the distinction between this subcategory and the
organizational component covered next. Here, the focus is on the technology, with organizational measures to support this effect.

Organizational Dimension. As DeLone and McLean (1992) point out,
there is inherent difficulty in assessing the “business value of information
systems” (p. 80). Even so, there are several research streams that examine EUC
in its organizational context. These studies do not necessarily measure IS
success, but other facets of the relationship, including management, support, and
planning. To be included in the organizational dimension of EUC, the article must
have an explicit end user association. Measuring this end user relationship is
frequently done through satisfaction and usage.
The following subcategories are identified from the literature:
•

•

Project and Applications Development: These articles focus on the
management of systems or project development. Measures include satisfaction with the development process and usage of the developed system,
as well as user participation in the process and the degree participation
affects the developmental outcome. Because development typically remains an organization guided and managed function, and to promote
parsimony, project and applications development studies (with explicit end
user variables) will be categorized in this subgroup.
End User Support: The information center (and other support mechanisms) and its effects have been the topic of much research in terms of
EUC (Bowman, Grupe, Lund, & Moore, 1993; Guimaraes & Igbaria, 1992).
These studies typically measure effectiveness at the individual level.
EUC Strategy and Management: There is a plethora of strategy and
management studies; in fact, in one respect almost all IS and EUC articles
are management-directed. What distinguishes an EUC study, however, is
its explicit end user measure(s).

In categorizing EUC studies along these three dimensions of end user,
technology, and organization, an attempt was made to place them in only one
dimension. So, for example, if the focus of the article was on applications
development, it was placed in organizational project development, even though

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

8 Downey and Bartczak

it may have used satisfaction (with the process) as a dependent variable and
end user skills as an independent variable. Likewise, a study of effective GSS
systems, focusing on the technology, is placed in the technology dimension,
even though group satisfaction with the system may be one of the dependent
variables.

METHODOLOGY

In order to examine the frequency, types, and themes of EUC research, five
leading IS journals were scrutinized in their entirety for the years 1990-2000.
Every article in each journal during these 11 years was examined and either was
included as EUC-related or eliminated. For those articles labeled EUC, a
subsequent assessment was conducted to categorize each.
The journals selected included the following: Information Systems Research (ISR), MIS Quarterly (MISQ), Journal of Management Information
Systems (JMIS), Information and Management (I&M), and Journal of End
User Computing (JEUC)1. In selecting the journals, two criteria were used.
The first was its reputation as reported in five recent reviews (Gillenson & Stutz,
1991; Hardgrave & Walstrom, 1997; Holsapple, Johnson, Manakyan, & Tanner,
1994; Walstrom & Hardgrave, 2001; Walstrom, Hardgrave, & Wilson, 1995). In
each of these reviews, MISQ and ISR were in the top three in overall ratings
(excepting ISR in the 1991 and 1994 studies, when it was a relatively new
publication). In the five ratings, JMIS was rated between 3 and 7, while I&M was
rated between 8 and 20. JEUC, rated 44 in 1997 and 34 in 2001, was included
not only because of its reputation, but also because it is one of the only pure-EUC
journals.
The second criteria was based on IS emphasis; (i.e., whether a journal
published primarily IS research or not). Because EUC research is a subset of IS
research, it was considered important that the journals be recognized for top
quality IS research. As ranked in Walstrom and Hardgrave (2001), the top four
“pure” IS journals were ISR, JMIS, MISQ (all tied at number one) and I&M,
rated no. 4. JEUC was rated no. 8 in this list. Some leading journals, such as
Management Science and Communications of the ACM, were listed as
“hybrid” or “partial” IS journals (p. 122) and were, therefore, not considered in
this report.
Examining each article in each journal for the given years was a meticulous
process. The first step involved reading the abstract and checking the included
variables (for empirical articles). If, at this point, the article did not involve EUC,
it was eliminated from consideration. If the article was not eliminated, it was
further examined and then categorized in a number of ways. Categorization
involved examining each variable, dependent and independent, as well as themes
for non-empirical articles.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

End User Computing Research Issues and Trends (1990-2000)

Any classification of previous literature involves a certain amount of
arbitrariness (DeLone & McLean, 1992). Steps were taken to reduce this by
following some consistent procedures. If an article clearly pertained to just one
dimension, it was categorized as such. This was determined by noting the focus
of the article, which was usually ascertained by the dependent variable(s). For
conceptual articles, the major theme determined the focus. Through this examination, the articles were categorized as end user, technology, or organizational.
The themes or variables then were recorded. All variables were included (except
demographics) in order to provide an accurate account of measures.
When EUC articles appeared to belong in multiple dimensions, a different
procedure was used. Generally, this resulted when an article had variables that
measured multiple dimensions (such as organizational and end user variables).
Placing it in more than one dimension required either a second dependent
variable (in another dimension) or two or more independent variables in another
dimension. There turned out to be only 15 articles (out of 463) classified in
multiple dimensions.

RESULTS

The examination of the five journals for the years 1990-2000 yielded a total
of 463 EUC articles. Of these, 414 (89.4%) were empirical in nature. I&M had
the highest number of EUC articles, a total of 179, due primarily to an increased
number of articles per issue and shorter article length. The other four journals
averaged 71 articles for the 11 years.
To assess the findings, the results include an analysis of the three dimensions and a comparison of journals. Based on these findings, the discussion
section addresses EUC trends found in the literature.

Dimensions of EUC

The three dimensions of end user computing are displayed in Table 1. The
end user dimension included 203 total articles (42.4%), while both the technology

Table 1. Articles per EUC dimension
Dimension

Total

Empirical%

End User

203

183/90%

Technology

138

127/92%

Organizational

138

116/84%

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

10 Downey and Bartczak

and organizational dimensions had 138 total articles (28.8%). Although there
were 463 total articles, the 479 presented below includes fourteen articles
classified in two dimensions, and one article placed in all three dimensions. The
percentage and number of empirical articles within each dimension are included.
End User Dimension. Articles that focused on the end user dimension
reflected more than 11 different themes (see Table 2). In these 203 articles, there
were a total of 417 themes or variables, because most articles measured multiple
variables. Excluding demographics, each study measured an average of 2.1
variables. These were aggregated into 11 themes.
Usage and satisfaction comprised 36% of all variables. Acceptance/
diffusion comprised 15%. Ten percent of the total included variables or themes
in an “other” category, each with less than 1%, such as computer playfulness,
end user privacy/security, innovativeness, and end user personality.
Table 2. End user dimension themes
Theme/Variable

Number

Usage

22%

Acceptance/Diffusion

15%

Satisfaction

14%

Training

11%

Skills

10%

Attitudes

Self-Efficacy

Anxiety

Task
Norms

Other

10%

Total

417

100%

Figure 2. End user themes trend

Satis faction

Us age

99
20
00

20
15
10
5
0

Te ch. Acc.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

End User Computing Research Issues and Trends (1990-2000)

Figure 2 indicates the trends of the top three themes within the end user
dimension. Usage, the most common variable in this dimension, averaged 8.3 per
year and shows a crest in 1995. Satisfaction, with an average of 5.2 articles per
year, was notable only in that there were no articles in 2000 (significant at p <
.10). Technology acceptance, with an average of 5.6 per year, shows a slow
steady rise from 1996-2000. However, none of the differences in these three
themes for the 11 years was significant at p < .05.
Most of the remaining themes also were consistent during this time period.
Only two points are noteworthy. In 1995, there were eight articles on computer
skills, significantly (p < .05) more than any other year. Likewise, the number of
self-efficacy articles in 2000 was significantly higher than in other years (p < .05).
Technology Dimension. Table 3 lists the themes/variables in the technology dimension. These articles focused on the impact (to the end user) of the
object technology. There were a total of 138 articles in this dimension. Most of
the variables pertaining to this dimension considered the impact of the technology
on either the decision process or group processes. Decision time and quality
were the most common for DSS and GDSS technologies. For GSS technologies,
participation and number of unique ideas or solutions were most common.
In terms of the themes under which the variables were categorized, group
impact represented 54% of the total. This underscores the relative importance
of studies of GSS and GDSS technologies. Individual impact, at 30%, primarily
studied technologies such as DSS, ES, and databases.
Figure 3 presents the trends for the two most common technology dimension
themes—individual and group impact. While group impact studies were most
numerous (average of 6.9 articles per year), there was not a significant
difference in numbers per year (p < .05). In 1996 there was a high of 11 articles,
(significant at p <.10). For individual impact articles, there was a significantly

Table 3. Technology dimension themes
Theme

Number

Percent

Group Impact

54%

Individual Impact

30%

System/Info Impact

12%

Organiz. Impact

Other

Total

140

100%

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

12 Downey and Bartczak

Figure 3. Technology themes trend
15
10
5

Individual

99
20
00

Gr oup

higher number in 1995 (p < .05), a total of nine. Individual impact articles
averaged 3.82 articles per year.
Organizational Dimension. Table 4 provides the themes for EUC articles
that focused on the organization (with an end user variable). Project development studies were most common, with 37%. In the empirical studies involving
development, the most common dependent variable was satisfaction with the
project or application, followed by user participation and system usage.
EUC support consisted of 40 articles (27%), and most were information
center based (26 of 40, or 65%). All but one were empirical with the most
common dependent variable being satisfaction with the information center.
Studies concerning EUC strategy were 20% of the total in the organizational
dimension. Of the 30 articles, 24 were empirical (80%). All had strategic
independent variables, such as management climate, organizational size or
maturity, IS alignment, and planning activities.
Figure 4 presents the three most common organizational dimension themes
by year. EUC support articles fluctuated by year, with a high of 6 in both 1992
and 1995, and a low of 0 in 1999 (significant at p <.10). Strategy studies,
averaging 2.7 articles per year, did not differ significantly by year and showed
Table 4. Organizational dimension themes
Theme

Number

Percent

Project/Applic. Dev.

37%

EUC Support

27%

Strategy

20%

Acceptance/Diffusion

12%

Other

Total

148

100%

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

End User Computing Research Issues and Trends (1990-2000)

Figure 4. Organizational themes trend
8
6
4
2

Proj. Dev

Support

99
20
00

Strategy

a high of five in 2000. The only significant finding in the organizational dimension
was the low of one in project development articles in 2000 (p < .05). The most
common organizational dimension theme—project development—had an average per year of five.

JOURNAL COMPARISON

Table 5 gives the number of actual EUC articles per journal for the years
1990-2000. The mean for all five journals was 92. There were more articles (p
< .10) from Information & Management, a result of shorter article length and
more issues per year. I&M usually published 12 issues per year (10 per year in
1990-1991, 1997-1999; only 8 in 2000), whereas the others publish quarterly.
In comparing the three dimensions within each journal, interesting differences were noted. As portrayed in Table 5, four of the journals had more end user
dimension articles than either of the other two dimensions (figured by percentage). In fact, MISQ and JEUC had more end user dimension articles than the
other two combined. JMIS, however, had more technology focused articles
(51%) than end user and organizational articles combined.
Table 5. Number of articles by journal
Jrnl

Tech

Org

179+

41%

26%

33%

JMIS

18%

51%

31%

MISQ

55%

23%

22%

JEUC

41%

26%

33%

ISR

43%

32%

25%

I&M

p < .10

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

14 Downey and Bartczak

DISCUSSION

The evidence gathered from the 11 years between 1990-2000 clearly
indicates that EUC is still a prevalent research topic. With an average of 42
articles per year in just these five journals, there is still wide academic and
practitioner interest in the themes of EUC. This section examines the top themes
and trends, and discusses limitations to this study.
Table 6 lists the top 10 themes or variables in EUC. Six of the themes were
in the end user dimension, with two each in technology and organization. This list
provides a keen insight into the state of EUC research for these 11 years. Usage
and satisfaction, the most common dependent variables (DeLone & McLean,
1992) in 1992, remain almost a decade later in the forefront of research (at no.
1 and no. 4). Group impact articles were at no. 2, with 16% of the total.
Surprisingly, technology acceptance was no. 3, documenting the importance
throughout the 1990s of promoting and studying users and their technology.
Table 6. Top ten themes in EUC: 1990-2000
Theme

Dimension

Usage

End User

19%

Group Impact

Technology

16%

Tech. Aceptnce

End User

13%

Satisfaction

End User

12%

Project Dev.

Organiz.

11%

Training

End User

Skills

End User

Indiv. Impact

Technology

EUC Support

Organiz.

Attitudes

End User

Figure 5. Trend by year of EUC: articles/dimension
80
60
40
20
0

Article s

Dim e ns ions

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

End User Computing Research Issues and Trends (1990-2000)

Trends in EUC. Perhaps the most interesting and compelling findings are
the trends that were uncovered. Figure 5 presents the trends in EUC articles
from 1990 through 2000. Both actual articles (n = 463) and dimensions within the
actual articles (n = 479) are plotted. The difference of 16 is a result of a few
articles being categorized in multiple dimensions. There was an average of 42
articles per year in the five journals. The number of articles ranged from a low
of 33 in 1991 to a high of 62 in 1995. Except for a crest in 1995-1996, the average
number of EUC articles remained fairly constant. In 1995 the crest of 62 for
dimensions was significantly higher than the other years, as was the 59 actual
articles (p < .05).

Crest of 1995

Figure 6 presents the trends by year for the three dimensions of end user,
technology, and organization. As can be seen, the end user dimension increased
from 10 articles in 1994 to 27 in 1995. In 1996 it dropped slightly to 25 articles.
Technology-focused articles also showed an increase, with a high of 16 in 1995
that was significant at p < .10. Thus, the two dimensions of end user and
technology contributed to the peak of 1995, with the end user dimension making
a larger impact.
In analyzing individual themes within the dimensions in 1995, there were
three that reached maximums that year. In the end user dimension, both usage
(15 articles) and computer skills (8 articles) had highs. In the technology dimension,
individual impact articles reached a high of nine (significant at p < .05).

Persistent Trends

Although the crest of 1995 may be interesting, the most important finding in
this study is the persistency of EUC research across all 11 years. EUC remains
an important topic for researchers, one that has not diminished during this time.
There was no significant difference (p < .05) in number of articles in any of the
three dimensions for any of the 11 years. This provides some compelling
evidence of the importance of EUC for both researchers and practitioners.
Figure 6. Trends over time by dimension
30
25
20
15
10
5
0
90 91

93
EU

Tech

Org

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

16 Downey and Bartczak

Across all three dimensions, for each individual theme, none showed a
significant decline (p < .05). Usage, satisfaction, training, attitudes, group and
individual impact studies, and organizational studies were persistent through
these 11 years. However, there were some emerging trends that should be noted.

Developing Trends

Three developing trends should be noted. The first is the upward trend in
articles concerning the end user dimension. Figure 6 illustrates that from 1990
through 1994, all three dimensions were equally represented in research. But
beginning in 1995, there were more end user dimension articles than either of the
other two. That disparity remained until the last year of the study (2000), when
all three dimensions were again close to each other in number of articles: end user
dimension – 16 articles; technology – 14 articles; organization – 9 articles. It
remains to be seen whether the 2000 data are an anomaly for the end user
dimension or whether the gap is actually closing between the three.
The second developing trend involves individual themes. Technology acceptance articles show a slow steady rise from 1996-2000. Group impact articles
(technology dimension) and strategy articles (organization dimension) also show
increases. Perhaps the most significant increase is in the number of self-efficacy
articles. There were six articles in 2000, significantly higher than in other years
(p < .05). In observing the trend in self-efficacy, 15 of 17 articles on self-efficacy
were published in the last six years.
The final developing trend involves articles dealing with the Internet or the
World Wide Web- (WWW-) based studies. Of the 13 articles identified in this
study, all were submitted in the last five years. This is clearly an important and
developing theme in the EUC research and practitioner community.
Limitations. There are limitations in this study. Only five journals were
selected for review, making generalization across the EUC field more problematic. The journals were predominately North American based, with less coverage extended to Europe or Asia. That the five journals are among the top in IS
research is advantageous, but also a limitation in that hybrid IS journals may
provide a different but equally important perspective of EUC. Some of the pvalues calculated are derived from small sample sizes (such as the trends
displayed in Figures 2-4), which limit its statistical power. Finally, as noted, the
classification of articles into dimensions involves some subjectivity.

CONCLUSION

End user computing research is now well over 20 years old. In that time,
great strides have been made in developing and refining the many relationships

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

End User Computing Research Issues and Trends (1990-2000)

of end user technology in some organizational setting. This study confirms that
EUC research is prevalent and important.
Missing was a simple yet practical framework to help organize and
compartmentalize the broad dimensions and themes of EUC. The EUC research-focused framework provides that structure and allows an examination of
the trends and important themes still being studied.
The importance of this study is not that it adds yet another framework for
consideration; rather, it is to assess and categorize the themes of EUC and the
important issues still remaining. This study provides some compelling conclusions. The themes of EUC, including the end user, technology, and the organization, remain important and pervasive. EUC research remains predominantly
empirical in nature; almost 90% of the articles during this timeframe were
empirical. The upward trend in the end user dimension suggests that many
unanswered questions remain on how and why individuals respond and use
technology. The persistence of usage and satisfaction, along with the rise in
technology acceptance and computer self-efficacy, clearly indicates that the
research focus on the individual remains. The Internet brings a new dimension
to individuals and organizations, and is moving to the vanguard in EUC research.
New technology in the future will continue to provide both researchers and
practitioners with opportunities. But the critical link between the technology and
the user remains one of the most viable interests in EUC.

REFERENCES

Aggarwal, A.K. (1994). Trends in end user computing: A professional’s
perspective. Journal of End User Computing, 6(3), 32-33.
Aggarwal, A.K. (1996). End user computing: Revisited. Journal of End User
Computing, 8(1), 31-32.
Barker, R.M. (1995). The interaction between end user computing levels and job
motivation and job satisfaction: An exploratory study. Journal of End User
Computing, 7(3), 12-19.
Benarock, M., & Kauffman, R.J. (1999). A case of using real options pricing
analysis to evaluate information technology project investments. Information Systems Research, 10(1), 70-86.
Benson, D.H. (1983). A field study of end user computing: Findings and issues.
MIS Quarterly, 7(4), 35-45.
Bowman, B., Grupe, F.H., Lund, D., & Moore, W.D. (1993). An examination of
sources of support preferred by end user computing personnel. Journal of
End User Computing, 5(4), 4-11.
Brancheau, J.C., & Brown, C.V. (1993). The management of end-user computing: Status and directions. ACM Computing Surveys, 25(4), 437-482.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

18 Downey and Bartczak

Brancheau, J.C., Janz, B.D., & Wetherbe, J.C. (1996). Key issues in information
systems management: 1994-1995 SIM Delphi results. MIS Quarterly,
20(2), 225-242.
Brancheau, J.C., & Wetherbe, J.C. (1987). Key issues in information systems
management. MIS Quarterly, 11(1), 23-45.
Caudle, S.L., Gorr, W.L. & Newcomer, K.E. (1991). Key information systems
management issues for the public sector. MIS Quarterly, 15(2), 23-45.
Deans, P.C., Karwan, K.R., Goslar, M.D., Ricks, D.A., & Toyne, B. (19901991). Identification of key international information systems issues in
U.S.-based multinational corporations. Journal of Management Information Systems, 7(3), 27-50.
DeLone, W.H., & McLean, E.R. (1992). Information systems success: The
quest for the dependent variable. Information Systems Research, 3(1),
60-95.
Dickson, G.W., Leitheiser, R.L., Wetherbe, J.C., & Nechis, M. (1984). Key
information systems issues for the 1980’s. MIS Quarterly, 8(3), 135-159.
Ein-Dor, P., & Segev, E. (1992). Information resources management for end
user computing: An exploratory study. Journal of End User Computing,
4(3), 14-20.
Essex, P.A., Magal, S.R., & Masteller, D.E. (1998). Determinants of information center success. Journal of Management Information Systems,
15(2), 95-117.
Galliers, R.D., & Land, F.F. (1987). Choosing appropriate information systems
research methodologies. Communications of the ACM, 30(11). 900-902.
Gillenson, M.L., & Stutz, J.D. (1991). Academic issues in MIS: Journals and
books. MIS Quarterly, 15(4), 447-452.
Glorfeld, K.D., & Cronan, T.P. (1993). Computer information satisfaction: A
longitudinal study of computing systems and EUC in a public organization.
Journal of End User Computing, 5(1), 27-36.
Guimaraes, T., & Igbaria, M. (1992). Determinants of turnover intentions:
Comparing IC and IS personnel. Information Systems Research, 3(3),
273-303.
Guimaraes, T., & Igbaria, M. (1994). Exploring the relationship between IC
success and company performance. Information & Management, 26(3),
133-141.
Hardgrave, B.C., & Walstrom, K.A. (1997). Forums for MIS scholars. Communications of the ACM, 40(11), 119-124.
Harris, R.W. (2000). Schools of thought in research into end-user computing
success. Journal of End User Computing, 12(1), 24-34.
Holsapple, C.W., Johnson, L.E., Manakyan, H., & Tanner, J. (1994). Business
computing research journals: A normalized citation analysis. Journal of
Management Information Systems, 11(1), 131-140.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

End User Computing Research Issues and Trends (1990-2000)

Kochen, M. (1985-1986). Are MIS frameworks premature? Journal of Management Information Systems, 2(3), 92-100.
Lee, D.M.S., Trauth, E.M., & Farwell, D. (1995). Critical skills and knowledge
requirements of IS professionals: A joint academic/industry investigation.
MIS Quarterly, 19(3), 313-340.
Mason, R.O., & Mitroff, I.I. (1973). A program for research on management
information systems. Management Science, 19(5), 475-487.
McLean, E.R., & Kappelman, L.A. (1992-1993). The convergence of organizational and end-user computing. Journal of Management Information
Systems, 9(3), 145-155.
Niederman, F., Brancheau, J.C., & Wetherbe, J.C. (1991). Information systems
management issues for the 1990s. MIS Quarterly, 15(4), 475-500.
Nolan, R.L., & Wetherbe, J.C. (1980, June). Toward a comprehensive framework for MIS research. MIS Quarterly, 1-19.
Powell, A., & Moore, J.E. (2002). The focus of research in end user computing:
Where have we come since the 1980s? Journal of End User Computing,
14(1), 3-22.
Rainer, R.K., & Harrison, A.W. (1993). Toward development of the end user
computing construct in a university setting. Decision Sciences, 24(6),
1187-1202.
Riemenschneider, C.K., & Mykytyn, Jr., P.P. (2000). What small business
executives have learned about managing information technology. Information & Management, 37(5), 257-269.
Rockart, J.F., & Flannery, L.S. (1983). The management of end user computing.
Communications of the ACM, 26(10), 776-784.
Seddon, P.B. (1997). A respecification and extension of the DeLone and
McLean model of IS success. Information Systems Research, 8(3), 240253.
Shah, H.U., & Lawrence, D.R. (1996). A study of end user computing and the
provision of tool support to advance end user empowerment. Journal of
End User Computing, 8(1), 13-21.
Shayo, C., Guthrie, R., & Igbaria, M. (1999). Exploring the measurement of end
user computing success. Journal of End User Computing, 11(1), 5-14.
Sussman, S.W., & Sproull, L. (1999). Straight talk: Delivering bad news through
electronic communication. Information Systems Research, 10(2), 150166.
Vandenbosch, B., & Huff, S.L. (1997). Searching and scanning: How executives obtain information from executive information systems. MIS Quarterly, 21(1), 81-107.
Walstrom, K.A., & Hardgrave, B.C. (2001). Forums for information systems
scholars: III. Information & Management, 39(2001), 117-124.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

20 Downey and Bartczak

Walstrom, K.A., Hardgrave, B.C., & Wilson, R.L. (1995). Forums for management information systems scholars. Communications of the ACM, 38(3),
93-107.
Wang, P. (1994). Information systems management issues in the Republic of
China for the 1990s. Information & Management, 26(6), 341-352.
Yang, H.L. (1996). Key information management issues in Taiwan and the US.
Information & Management, 30(5), 251-267.

ENDNOTE

From its inception in 1989 until 1992, this journal was named Journal of
Microcomputer Systems Management.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Effect of End User Development on End User Success 21

Chapter II

The Effect of End User
Development on
End User Success
Tanya McGill, Murdoch University, Australia

ABSTRACT

End user development of applications forms a significant part of
organizational systems development. This study investigates the role that
developing an application plays in the eventual success of the application
for the user developer. The results of this study suggest that the process of
developing an application not only predisposes an end user developer to
be more satisfied with the application than they would be if it were
developed by another end user, but also leads them to perform better with
it. Thus, the results of the study highlight the contribution of the process of
application development to user developed application success.

INTRODUCTION

An end user developer is someone who develops applications systems to
support his or her work and possibly the work of other end users. The applications
developed are known as user developed applications (UDAs). So, while the
technical abilities of user developers may vary considerably, they are required
basically to analyze, design, and implement applications. End user development
of applications forms a significant part of organizational systems development
with the ability to develop small applications forming part of the job requirements
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

22 McGill

for many positions (Jawahar & Elango, 2001). In a survey to determine the types
of applications developed by end users, Rittenberg and Senn (1990) identified
more than 130 different types of applications. More than half of these were
accounting related, but marketing, operations, and human resources applications
also were represented heavily. The range of tasks for which users develop
applications has expanded as the sophistication of both software development
tools and user developers has increased, which has led to a degree of convergence with corporate computing so that the tasks for which UDAs are developed
are less distinguishable from tasks for corporate computing applications (McLean,
Kappelman, & Thompson, 1993). In addition to the traditional tasks that UDAs
have been developed to support, Web applications are becoming increasingly
common (Nelson & Todd, 1999; Ouellette, 1999).
Much has been written in the end user computing literature about the
potential benefits and risks of end user development. It has been suggested that
end user development offers organizations better and more timely access to
information, improved quality of information, improved decision making, reduced
application development backlogs, and improved information systems department/user relationships (Brancheau & Brown, 1993; Shayo, Guthrie, & Igbaria,
1999). In the early UDA literature, the proposed benefits of UDA were seen to
flow mainly from a belief that the user has a superior understanding of the
problem to be solved by the application (Amoroso, 1988). This superior understanding then should enable end users to identify information requirements more
easily and to thus create applications that provide information of better quality.
This, in turn, should lead to better decision making. Other proposed benefits also
should flow from this; that is, user development of applications should allow the
information systems staff to focus more on the remaining and presumably larger
requests and hence reduce the application development backlog. This, in turn,
should improve relationships between information systems staff and end users.
Despite the potential benefits to an organization of user development of
applications, there are many risks associated with it that may lead potentially to
dysfunctional consequences for the organization’s activities. These risks result
from a potential decrease in application quality and control as individuals with
little information systems training take responsibility for developing and implementing systems of their own making (Cale, 1994) and include ineffective use of
monetary resources, threats to data security and integrity, solving the wrong
problem (Alavi & Weiss, 1985-1986), unreliable systems, incompatible systems,
and use of private systems when organizational systems would be more
appropriate (Brancheau & Brown, 1993).
As end user development forms a large proportion of organizational systems
development, its success is of great importance to organizations. The decisions
made by end users using UDAs influence organizational performance every day.
Organizations carry out very little formal assessment of fitness for use of UDAs
(Panko & Halverson, 1996); therefore, they have to rely very heavily on the
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Effect of End User Development on End User Success 23

judgment of end users, both those who develop the applications and others that
may use them, as end user developers are not the only users of UDAs. Bergeron
and Berube (1988) found that 44% of the end user developers in their study had
developed applications that were used by more than two people, and Hall (1996)
found that only 17% of the spreadsheets contributed by participants in her study
were solely for the developer’s own use. Therefore, it is essential that more be
known about UDA success, including whether end users are disadvantaged
when they use applications developed by other end users. This chapter explores
the contribution of the development process to UDA success and, hence,
highlights differences between the success of UDAs when used by the developer and when used by other end users.
The literature on user participation and involvement proposes benefits that
are thought to accrue from greater inclusion of users in the system development
process. The benefits that have been proposed include higher levels of information system usage, greater user acceptance of systems, and increased user
satisfaction (Lin & Shao, 2000). The end user’s superior knowledge of the
problem to be solved is certainly one factor influencing these benefits, but the
process of participating, per se, also is thought to have benefits. Those who have
participated in systems development have a greater understanding of the
functionality of the resulting application (Lin & Shao, 2000); a greater sense of
involvement with it (Barki & Hartwick, 1994); and, hence, a greater commitment
to making it successful. User development of applications has been described as
the ultimate user involvement (Cheney, Mann, & Amoroso, 1986). Thus, it could
be expected to lead to systems that gain the benefit of a better understanding of
the problem and to end users with a better understanding of the application and
greater commitment to making it work.
This study was designed to isolate the effect of actually developing a UDA
on the application’s eventual success for the user developer, and to measure that
success in terms of a range of possible success measures. There has been little
empirical research on user development of applications (Shayo et al., 1999), and
most of what has been undertaken has used user satisfaction as the measure of
success because of the lack of direct measures available (Etezadi-Amoli &
Farhoomand, 1996). User satisfaction refers to the attitude or response of an end
user toward an information system. While user satisfaction has been the most
widely reported measure of success (Gelderman, 1998), there have been
concerns about its use as the major measure of information systems success
(Etezadi-Amoli & Farhoomand, 1996; Galletta & Lederer, 1989; Melone, 1990;
Thong & Chee-Sing, 1996).
The appropriateness of user satisfaction as a measure of system effectiveness may be even more questionable in the UDA domain. Users who assess their
own computer applications may be less able to be objective than users who
assess applications developed by others (McGill, Hobbs, Chan, & Khoo, 1998).
The actual development of an application, which may involve a significant
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

24 McGill

investment of time and creative energy, may be satisfying other needs beyond the
immediate task. User satisfaction with a UDA, therefore, could reflect satisfaction with the (highly personal) development process as much as with the
application itself.
Other proposed measures of information systems success that might be
appropriate for UDAs include system quality, information quality, involvement,
use, individual impact, and organizational impact (DeLone & McLean, 1992;
Seddon, 1997). System quality refers to the quality of an information system, as
opposed to the quality of the information it produces. It is concerned with issues
such as reliability, maintainability, ease of use, and so forth. As this study relates
to the success of a UDA for the eventual user, the user’s perception of system
quality is considered important. Information quality relates to the characteristics
of the information that an information system produces. It includes issues such
as timeliness, accuracy, relevance, and format. As discussed above, improved
information quality has been proposed as one of the major benefits of user
development of applications.
Involvement is defined as “a subjective psychological state, reflecting the
importance and personal relevance of a system to the user” (Barki & Hartwick,
1989, p.53). Seddon and colleagues (Seddon, 1997; Seddon & Kiew, 1996)
included involvement in their extensions to DeLone and McLean’s (1992) model
of information systems success. Use refers to how much an information system
is used. It has been widely used as a measure of organizational information
systems success (Gelderman, 1998; Kim, Suh, & Lee, 1998) but is only
considered appropriate if use of a system is not mandatory (DeLone & McLean,
1992).
Individual impact refers to the effect of an information system on the
behavior or performance of the user. DeLone and McLean (1992) claimed that
individual impact is the most difficult information systems success category to
define in unambiguous terms. For example, the individual impact of an UDA
could be related to a number of measures such as impact on performance,
understanding, decision making, or motivation. Organizational impact refers to
the effect of an information system on organizational performance. According
to DeLone and McLean’s model, the impact of an information system on
individual performance should have some eventual organizational impact. However, the relationship between individual impact and organizational impact is
acknowledged to be complex. Organizational impact is a broad concept, and
there has been a lack of consensus about what organizational effectiveness is
and how it should be measured (Thong & Chee-Sing, 1996). DeLone and
McLean (1992) recognized that difficulties are involved in “isolating the effect
of the I/S effort from the other effects which influence organizational performance” (p. 74). Again, this issue likely is to be magnified in the UDA domain,
where system use may be very local in scope.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Effect of End User Development on End User Success 25

The fact that vital organizational decision making relies on the individual end
user’s perception of fitness for use suggests that more insight is needed into the
role of application development in the success of applications and that, as well
as user satisfaction, additional measures of success should be considered. This
chapter reports on a study designed to address this need by considering a range
of both perceptual and direct measures of UDA success in the same study and
by isolating the role that actually developing an application plays in the eventual
success of the application.

RESEARCH QUESTIONS

The primary research question investigated in this study was:
Does the process of developing an application enhance the success of that
application for the user developer?
In order to isolate the effect of actually developing an application on its
success for the user, this study compares end user developers using applications
they have developed themselves with end users using applications developed by
another end user, on a number of key variables that have been considered in the
information systems success literature. Spreadsheets are the most commonly
used tool for end user development of applications (Taylor, Moynihan, & WoodHarper, 1998). Therefore, in this study, a decision was made to focus on end
users who develop and use spreadsheet applications.
In a study that investigated the ability of end users to assess the quality of
applications they develop, McGill (2002) found significant differences between
the system quality assessments of end user developers and independent expert
assessors. In particular, the results suggested that end users with little experience might erroneously consider the applications they develop to be of high
quality. If this is the case, then end user developers also may consider their
applications to be of higher quality than do other users. Therefore, it was
hypothesized that:
H1: End user developers will perceive applications they have developed themselves to be of higher system quality than applications developed by another
end user with a similar level of spreadsheet knowledge.
Doll and Torkzadeh (1989) found that end user developers had much higher
levels of involvement with applications than did users who were involved in the
development process, but where the application was primarily developed by a
systems analyst or by another end user. Therefore, it was hypothesized that:

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

26 McGill

H2: End user developers will have higher levels of involvement with applications they have developed themselves than with applications developed by
another end user with a similar level of spreadsheet knowledge.
End user developers have been found to be more satisfied with applications
they have developed themselves than with applications developed by another end
user (McGill et al., 1998) or with applications developed by a systems analyst
(despite involvement in the systems development process) (Doll & Torkzadeh,
1989). Therefore, it was hypothesized that:
H3: End user developers will have higher levels of user satisfaction when using
applications they have developed themselves than when using applications
developed by another end user with a similar level of spreadsheet knowledge.
Increased user satisfaction has been shown to be associated with increased
individual impact (Etezadi-Amoli & Farhoomand, 1996; Gatian, 1994; Gelderman,
1998; Igbaria & Tan, 1997). As end user developers are believed to be more
satisfied with applications they have developed than are other users of these
applications, it is to be expected that they will also perceive that these applications
have a greater impact on their work. Therefore, it was hypothesized that:
H4: End user developers will have higher levels of perceived individual impact
when using applications they have developed themselves than when using
applications developed by another end user with a similar level of spreadsheet knowledge.
As previously discussed, the end user computing literature has claimed that
end user development leads to more timely access to information, improved
quality of information, and improved decision making (Brancheau & Brown,
1993; Shayo et al., 1999). While this may be partially due to end users having a
better understanding of the problems to be solved by information systems
(Amoroso, 1988), the actual process of developing an application also may lead
to benefits resulting from a superior knowledge of the application. Hence, it was
hypothesized that:
H5: End user developers will make more accurate decisions when using
applications they have developed themselves than when using applications
developed by another end user with a similar level of spreadsheet knowledge.
H6: End user developers will make faster decisions when using applications
they have developed themselves than when using applications developed by
another end user with a similar level of spreadsheet knowledge.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Effect of End User Development on End User Success 27

METHOD
Participants

The target population for this study was end users who develop their own
applications using spreadsheets. In order to obtain a sample of end user
developers with a wide range of backgrounds, participants were recruited for the
study in a variety of ways. It was recognized that the time required for
participation would make recruitment difficult, so participants were offered a
one-hour training course entitled “Developing Spreadsheet Applications” as an
incentive. This session focused on spreadsheet planning, design, and testing.
They also were given $20 to compensate them for parking costs, gas, and
inconvenience. Recruitment occurred first through a number of advertisements
which were placed in local newspapers calling for volunteers, these werefollowed by e-mails to three large organizations that had expressed interest in the
study, and finally, word of mouth brought forth some additional participants. The
criteria for inclusion in the study was previous experience using Microsoft Excel.
While essentially a convenience sample, the participants covered a broad
spectrum of ages, spreadsheet experience, and training.

Procedure

Fourteen separate experimental sessions of approximately four hours were
held over a period of five months. Each session involved between seven and 17
participants, depending on availability, a total of 159 end users participated. Each
experimental session consisted of three parts (see Table 1). The study used a
within-subjects research design, as this has been shown to provide superior
control for individual subject differences (Maxwell & Delaney, 1990).
In Part 1, participants were asked to complete a questionnaire to provide
demographic information about themselves and information about their back-

Table 1. Experimental session outline
Part

Activities

Approximate
Duration
.5 hour

Collect background information and assess spreadsheet knowledge

Develop spreadsheets (see Appendix 1 for the problem statement)

Use spreadsheets to answer decision questions and complete
perceived system quality, involvement, user satisfaction, and
perceived individual impact questions (see Appendix 2 for the
questionnaire items)

1 hour

Training session (see Appendix D)

1 hour

1.5 hours

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

28 McGill

ground with computers and spreadsheets. The questionnaire also tested their
knowledge of spreadsheets. They were not told the objective of the study.
In Part 2, the participants were given a problem statement and asked to
develop a spreadsheet to solve it using Microsoft Excel. The problem related to
making choices between car rental companies (see Appendix 1 for the problem
statement). Participants were provided with blank paper to use for planning, if
they wished, but otherwise were left to develop the application as they wished.
They were encouraged to treat the development exercise as they would a task
at work rather than as a test. Participants could use online help or ask for
technical help from the two researchers present in the laboratory during each
session.
Once all participants in the session had completed their spreadsheet, they
undertook Part 3 of the session. Each participant was given a floppy disk
containing both the spreadsheet they had developed and a spreadsheet from
another participant in the session. Matching of participants was done on the basis
of the spreadsheet knowledge scores from Part 1 in the expectation that
participants with a similar level of spreadsheet knowledge would develop
spreadsheets of similar sophistication.
To control for presentation order effects, each participant was randomly
assigned to use either the participant’s own spreadsheet or the other participant’s
spreadsheet first. Then, they used the spreadsheet to answer 10 questions
relating to making choices about car rental hire. The time taken to answer these
questions was recorded. Then, they completed a questionnaire containing items
to measure (i.e., perceived system quality, involvement, user satisfaction, and
perceived individual impact). Once the questionnaire and their answers to the car
rental decision questions were collected, the participants then repeated the
process with the other spreadsheet on their floppy disk. A different but
equivalent set of car rental decision questions was used. Eighty participants
ended up using the application they had developed first, and 79 participants used
the other application first.

Instruments

The development of the research instruments for this study involved a
review of many existing survey instruments. To ensure the reliability and validity
of the measures used, previously validated measurement scales were adopted
wherever possible. Factor analysis of the items used to measure the constructs
that were not directly measured was undertaken to examine discriminant validity
of the constructs. Discriminant validity appeared to be satisfactory for all
operationalizations except for user satisfaction and perceived individual impact,
which were highly correlated (r = 0.95, p < 0.000). However, as these
instruments were used in a closely related study on end user success (McGill,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Effect of End User Development on End User Success 29

Hobbs, & Klobas, 2003) and discriminant validity demonstrated for that study, a
decision was made to accept these operationalizations.
Spreadsheet Application Development Knowledge
Spreadsheet application development knowledge relates to the knowledge
that end user developers make use of when developing UDAs. The instrument
used to measure spreadsheet development knowledge was based upon an
instrument used by McGill and Dixon (2001). That instrument was developed
using material from several sources, including: Kreie’s (1998) instrument to
measure spreadsheet features knowledge; spreadsheet development methodologies from Ronen, Palley, and Lucas (1989), and Salchenberger (1993); and
Rivard et al.’s (1997) instrument to measure the quality of UDAs. The final
instrument contained 25 items. Each item was presented as a multiple-choice
question with five options. In each case, the 5th option was “I don’t know” or “I
am not familiar with this feature.” Nine of the items related to knowledge about
the features and functionality of spreadsheet packages; eight items related to
development process; and eight items related to spreadsheet quality assurance.
The instrument was shown to be reliable with a Cronbach’s alpha of 0.78
(Nunnally, 1978).
Involvement
The involvement construct was operationalized using Barki and Hartwick’s
(1991) instrument. They developed the scale for information systems based on
the general involvement scale proposed by Zaichkowsky (1985). The resulting
scale is a seven-point bi-polar semantic differential scale with 11 items. See
Appendix 2 for a list of the questionnaire items used to measure involvement.
The instrument, as used in this study, was shown to be reliable with a
Cronbach’s alpha of 0.95, and involvement was created as a composite variable
using the factor weights obtained from a measurement model development using
AMOS 3.6.
Perceived System Quality
The items used to measure perceived system quality were obtained from the
instrument developed by Rivard et al. (1997) to assess the quality of UDAs.
Rivard et al.’s instrument was designed to be suitable for end user developers
to complete yet sufficiently deep to capture their perceptions of components of
quality. For this study, items that were not appropriate for the applications under
consideration (e.g., specific to database applications) were excluded. Minor
adaptations to wording were also made to reflect the environment in which
application development and use occurred. The resulting perceived system
quality scale consisted of 20 items, each scored on a Likert scale of 1 to 7 where

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

30 McGill

1 was “strongly agree” and 7 was “strongly disagree” (see Appendix 2 for a list
of the questionnaire items used to measure perceived system quality).
The instrument was shown to be reliable with a Cronbach’s alpha of 0.94,
and perceived system quality was created as a composite variable using the
factor weights obtained from measurement model development using AMOS
3.6.
User Satisfaction
Given the confounding of user satisfaction with information quality and
system quality in some previous studies (Seddon & Kiew, 1996), items measuring
only user satisfaction were sought. Seddon and Yip’s (1992) four-item sevenpoint semantic differential scale that attempts to measure user satisfaction
directly was used in this study. A typical item on this scale is “How effective is
the system?” measured from 1 as “effective” to 7 as “ineffective” (see
Appendix 2 for a list of the questionnaire items used to measure user satisfaction).
The instrument was shown to be reliable with a Cronbach’s alpha of 0.96,
and user satisfaction was created as a composite variable using the factor
weights obtained from a one-factor congeneric measurement model developed
using AMOS 3.6.
Individual Impact
In this study, it was explicitly recognized that an individual’s perception of
the impact of an information system on the individual’s performance might not
be consistent with other direct measures of individual impact, and, hence, three
measures of individual impact were included in the study. These were individual
impact as perceived by the end user, accuracy of decision making, and time taken
to answer a set of questions.
Perceived individual impact was measured using items derived from Goodhue
and Thompson (1995) in their study on user evaluations of systems as surrogates
for objective performance. The instrument was shown to be reliable with a
Cronbach’s alpha of 0.96 (see Appendix 2 for a list of the questionnaire items
used to measure perceived individual impact).
In addition to the end user’s perception of individual impact, two direct,
easily quantifiable aspects of individual impact were also measured. These
aspects—decision accuracy and time taken to answer a set of question—also
were used by Goodhue, Klein, and March (2000) in their study on user
evaluations of systems.
Two sets of 10 different but equivalent questions involving the comparison
of costs of car rental companies under a variety of scenarios were created. The
questions ranged from comparison of the three firms when no excess mileage
charges are imposed to questions where excesses are applied and basic

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Effect of End User Development on End User Success 31

parameters are assumed to have changed from those given in the original
problem description. A typical question is, “Which rental company is the
cheapest if you wish to hire a car for six days and drive approximately 100 miles
with it?” Participants were asked to provide both the name of the cheapest firm
and its cost. The questions were piloted by four end users, and slight changes
were made to clarify them. The equivalent of the two sets of questions in terms
of difficulty and time to complete also were confirmed by measuring the time
taken to answer each set using the four applications created during piloting of the
task.

RESULTS

Of the 159 participants, 32.7% were male and 67.3% were female (52
males, 107 females). Their ages ranged from 14 to 77 with an average age of
42.7. Participants reported an average of 4.5 years experience using spreadsheets (with a range from 0 to 21 years). One hundred and twelve (70.4%)
reported using spreadsheets at work, and 92 (57.9%) reported using spreadsheets for personal use.
Table 2 provides descriptive information about each of the variables of
interest. Data analysis was undertaken using MANOVA. Pillai’s Trace (F =
5.45; df = 6, 306; p < 0.001), indicated that there was a significant multivariate
effect for being the developer. Each of the hypotheses was then addressed using
univariate F-tests (see Table 2). As a number of comparisons were being made,
the level of significance was conservatively set at 0.01.
End users perceived applications they had developed themselves to be of
higher quality than applications developed by other end users. On average, there
was a 16.6% difference in perceived quality when the developer was assessing
his or her own application. This increase was significant (F = 17.96; df = 1, 311;
p < 0.001). End user developers also were significantly more involved with their
own applications (F= 12.42; df = 1, 311; p < 0.001) and significantly more
Table 2. End user developer perceptions and performance when using their
own or another application
Perceived system
quality
Involvement
User satisfaction
Perceived individual
impact
Number of decisions
correct (/10)
Time to make
decisions (minutes)

Developer + User
Mean Std. dev
N
4.64
1.27 157

User Only
Mean Std. dev
N
3.98
1.48 156

Comparison
% incr.
Sign.
16.6
<0.001

9.36
4.44
9.38

2.73
1.86
3.94

157
157
157

8.17
3.63
7.26

3.20
2.07
4.30

156
156
156

14.6
22.3
29.2

<0.001
<0.001
<0.001

4.43

3.33

157

3.47

3.22

156

27.7

0.010

17.75

10.00

157

15.31

7.22

156

15.9

0.014

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

32 McGill

satisfied with them (F = 13.22; df = 1, 311; p < 0.001). The average difference
in involvement if the user was also the developer was 14.6%, and the average
difference in user satisfaction was 22.3%. Thus, Hypotheses 1 through 3 were
supported.
End users perceived applications they had developed themselves as having
a significantly greater impact on their decision performance (F = 20.65; df = 1,
311; p < 0.001), which was confirmed as they made a significantly larger number
of correct decisions (F = 6.70; df = 1, 311; p = 0.010). The average difference
in perceived individual impact of the application was 29.2%, and the average
difference in the number of decisions correct was 27.7%. Thus, Hypotheses 4
and 5 were supported. It was also hypothesized that end user developers would
make faster decisions when using the application they had developed themselves. However, this hypothesis was not supported. End users took longer on
average to answer the questions using their application (F = 6.10; df = 1, 311; p
= 0.014). On average, the difference in decision time was 15.8%.

DISCUSSION

The results of this study suggest that the process of developing an
application predisposes an end user developer to be more satisfied with the
application than if it were developed by another end user, and also leads them to
perform better with the application than if it were developed by another end user.
While previous research has established the positive impact of the process of end
user development on subjective measures such as involvement (Doll & Torkzadeh,
1989) and user satisfaction (Doll & Torkzadeh, 1989; McGill et al., 1998), its
impact on directly measured performance previously has not been established.
The results of this study highlight the contribution of the process of application
development to application success. This contribution appears to be beyond the
advantages achieved by an increased knowledge of the problem situation, as, in
this study, the effects of domain knowledge were controlled for by the withinsubjects design. Thus, end user developers benefit not only from better understanding of the problem to be solved (Amoroso, 1988), but also from the process
of application development.
The end user developers in this study had significantly higher levels of
involvement, user satisfaction, and perceived individual impact when using
applications they had developed themselves than they did when using applications developed by another end user with approximately the same levels of
spreadsheet development knowledge. They also perceived their applications to
be of higher system quality. These results are consistent with the results in the
literature on user involvement in the development of organizational systems. For
example, Doll and Torkzadeh (1988) found user participation in design to be
positively correlated with end user computing satisfaction, and Lawrence and

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Effect of End User Development on End User Success 33

Low (1993) found that the more users felt involved with the development
process, the more satisfied they were with the system. The results are also
consistent with McGill et al.’s (1998) study in the end user developer domain,
where end user developers were found to be more satisfied with their own
applications.
The results also strongly support Cheney, Mann, and Amoroso’s (1986)
claim that end user development can be considered as the ultimate user
involvement. The higher levels of perceived system quality for end users’ own
applications highlight the subjectivity of system quality for end users. This issue
has been raised by Huitfeldt and Middleton (2001) who argued that the standard
system quality criteria are oriented toward information technology maintenance
staff rather than toward end users and that “it is still difficult for an end user, or
software development client, to evaluate the quality of the delivered product” (p.
3). Although the instrument used to measure perceived system quality in this
study was designed specifically for end users (Rivard et al., 1997), informal
feedback from participants suggests that they found quality assessment a
difficult task. In contrast to software engineering definitions of system quality
(Boehm et al., 1978; Cavano & McCall, 1978), Amoroso and Cheney (1992)
implicitly acknowledge this difficulty by defining UDA quality as a combination
of end user information satisfaction and application utilization. This, however,
ignores the underlying necessity for the more technical dimensions of system
quality to be taken account of in order to have reliable and maintainable
applications.
End user developers made significantly more correct decisions when using
their own applications than when using an application developed by another end
user. In this study, all participants had been provided with the same problem
statement, and all had spent time considering the problem in order to develop an
application. All participants also had used both the application they had developed and another application, so domain knowledge was not a factor. The
improved performance could be due to a greater familiarity with the application
itself, achieved through the development process. Successful use of user
developed spreadsheet applications appears to require substantial end user
knowledge because of the lack of separation of data and processing that is
commonly found (Hall, 1996; Ronen et al., 1989). Users of UDAs usually do not
receive formal training in the particular application; yet training is associated with
successful use (Nelson, 1991). Developing an application allows the user to
develop a robust understanding of it that makes it easier to use and makes it
possible for them to successfully adjust aspects of it when necessary. The
development process can be seen as a form of training for future use of the
application and can circumvent problems that might otherwise occur because of
lack of training and/or documentation.
The improved performance also could be due to a greater determination to
achieve the correct answers because of the higher levels of involvement. This
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

34 McGill

explanation receives support from the additional time user developers spent
making the decisions. On average, the user developers spent an extra two and
a half minutes trying to answer the 10 questions. This was unexpected, as it
would be logical to expect end users to spend less time using the applications they
understand best, but it may be due to the end user developer’s greater
commitment to succeeding with his or her own application. Comments from
participants during the sessions support this possible explanation. In addition,
many participants continued working on their applications, once the formal part
of the experiment was completed; some even continued to adapt their applications over a number of days.
McGill et al. (1998) questioned the usefulness of user satisfaction as a
measure of UDA success after finding that developers of UDAs were significantly more satisfied with applications that they had developed than other end
users were with the same applications. They speculated that increased satisfaction might be a reflection of the role of attitude in maintaining self- esteem, and
expressed concerns that this increased satisfaction might blind end user developers to problems that exist in the applications that they have developed.
However, no measures of performance were included in that study. This study
suggests that the raised levels of user satisfaction and other perceptual variables
were appropriate, as they were consistent with better levels of performance.
Both subjective and direct measures of UDA success have an important
role to play in research on user development of applications. Shayo et al. (1999)
noted that subjective measures are less threatening and easier to obtain, thus
making end user computing research easier to conduct. Subjective measures
also can reflect a wider range of success factors than can be captured using
direct measures such as decision accuracy. However, exclusive use of subjective measures can be problematic, because users are asked to place a value on
something about which they may not be objective. By including both types of
measures, this study has demonstrated a range of benefits attributable to end
user development and has provided a measure of confidence that increases in
subjective measures that also are associated with increases in some direct
measures.
The results of this comparison between end user developers using their own
applications and end users using applications developed by other end users has
implications for staff movement in organizations. If an end user develops an
application for his or her own use, and its use has a positive impact on
performance, this does not guarantee that the same will be true if another end
user starts to use it. Organizations should recognize that the use of UDAs by end
users other than the developer may carry with it greater risks. If an end user
developer has developed an application for his or her own use and then leaves
the position or organization, it cannot be assumed that another end user
necessarily will be able to use it successfully. In addition, if users are developing

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Effect of End User Development on End User Success 35

applications for others to use, particular attention must be paid to ensure that
these applications are of sufficient quality for successful use not to rely on
additional insight gained during the development process. As previously discussed, the development process provides a form of preparation for future use
of an application and may reduce dependence on training and documentation.
However, users of a UDA who were not involved in its development still rely
heavily on documentation and training, and the importance of them must be
emphasized.
Several limitations of the research are apparent and should be considered
in future investigations of end user development success. First, the only
application development tool considered was spreadsheets. While spreadsheets
have been the most commonly used end user application development tool
(Taylor et al., 1998), the generalizability of the results to users of other
development tools (i.e., database management systems and Web development
tools) needs to be investigated in future research. A second limitation of the
research was the constraints resulting from the use of a laboratory experiment
research approach. The spreadsheets that participants developed were probably
smaller than the majority of spreadsheets developed by users in support of
organizational decision making (Hall, 1996). In addition, because of the finite
nature of the experiment, end users did not have the same incentive to succeed
as would be expected in a work situation. The artificial nature of the environment
and task may have influenced the results. While the research situation chosen
provided the benefit of control of external variability and, hence, internal validity,
it was not ideal in terms of providing external validity. It would be valuable to
undertake a field study in a range of organizations to extend the external validity
of the research.

CONCLUSION

In conclusion, this study suggests that the process of developing an
application leads to significant advantages for the end user developer. In the past,
the proposed benefits of user development of applications have been attributed
mainly to a belief that the user has a superior understanding of the problem to be
solved by the application system (Amoroso, 1988). In this study, all end users
should have had equal knowledge and understanding of the problem when using
both the application they had developed and the other application, so any
differences in domain knowledge were not a factor.
The relative success of the end user developers when using their own
applications in this study may flow from their superior knowledge of their own
applications, thus confirming one of the proposed advantages of user involvement in organizational information systems development. The advantage of
superior knowledge of the application is likely to be particularly important with
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

36 McGill

spreadsheet applications where data and processing are usually integrated (Hall,
1996; Ronen et al., 1989). Future research should investigate whether these
findings also hold when other application development tools are used and with
other groups of end user developers.
There have been concerns expressed in the literature about user development of applications as an inefficient use of personnel time, distracting end users
from what they are supposed to be doing (Alavi & Weiss, 1985-1986; Davis &
Srinivasan, 1988; O’Donnell & March, 1987). However, this study suggests that
the potential risk of inefficient use of personnel time may be compensated for by
superior decision making later, based upon insights gained from system development. While development of applications by more experienced user developers
or by information systems professionals may ensure a more reliable and
maintainable application (Edberg & Bowman, 1996), end user development
currently is a pervasive form of organizational system development, and it is
encouraging to identify this benefit of it. However, the findings relating to
differences in end user success between those who have developed the
application they are using and those who haven’t emphasize that organizations
should recognise that the use of UDAs by end users other than the developer may
carry with it greater risks, and that these must be addressed by particular
attention to documentation of applications and training for other users. It is not
appropriate that successful use relies on insight gained during the development
process. UDAs must be sufficiently robust and reliable to be used by a wide
range of users.

REFERENCES

Alavi, M., & Weiss, I.R. (1985-1986). Managing the risks associated with enduser computing. Journal of Management Information Systems, 2(3), 5-20.
Amoroso, D.L. (1988). Organizational issues of end-user computing. Data
Base, 19(Winter/Fall), 49-57.
Amoroso, D.L., & Cheney, P.H. (1992). Quality end user-developed applications: Some essential ingredients. Data Base, 23(1), 1-11.
Barki, H., & Hartwick, J. (1989). Rethinking the concept of user involvement.
MIS Quarterly, 13(March), 52-63.
Barki, H., & Hartwick, J. (1991). User participation and user involvement in
information system development. Proceedings of the Twenty-Fourth
Annual Hawaii International Conference on System Sciences, 4, 487492.
Barki, H., & Hartwick, J. (1994). Measuring user participation, user involvement, and user attitude. MIS Quarterly, 18(1), 59-79.
Bergeron, F., & Berube, C. (1988). The management of the end-user environment:
An empirical investigation. Information & Management, 14, 107-113.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Effect of End User Development on End User Success 37

Boehm, B.W., Brown, J.R., Caspar, H., Lipow, M., MacLeod, E.J., & Merritt,
M.J. (1978). Characteristics of software quality. Amsterdam: NorthHolland.
Brancheau, J.C., & Brown, C.V. (1993). The management of end-user computing: Status and directions. ACM Computing Surveys, 25(4), 450-482.
Cale, E.G. (1994). Quality issues for end-user developed software. Journal of
Systems Management (January), 36-39.
Cavano, J.P., & McCall, J.A. (1978). A framework for the measurement of
software quality. Proceedings of the Software Quality and Assurance
Workshop, 133-140.
Cheney, P.H., Mann, R.I., & Amoroso, D.L. (1986). Organizational factors
affecting the success of end-user computing. Journal of Management
Information Systems, 3(1), 65-80.
Davis, J., & Srinivasan, A. (1988). Incorporating user diversity into information
system assessment. In N. Bjorn-Andersen & G. Davis (Eds.), Information
systems assessment (pp. 83-98). New York: Knowledge Industry.
DeLone, W.H., & McLean, E.R. (1992). Information systems success: The quest
for the dependent variable. Information Systems Research, 3(1), 60-95.
Doll, W.J., & Torkzadeh, G. (1988). The measurement of end-user computing
satisfaction. MIS Quarterly, 12(2), 259-274.
Doll, W.J., & Torkzadeh, G. (1989). A discrepancy model of end-user computing
involvement. Management Science, 35(10), 1151-1171.
Edberg, D.T., & Bowman, B.J. (1996). User-developed applications: An empirical study of application quality and developer productivity. Journal of
Management Information Systems, 13(1), 167-185.
Etezadi-Amoli, J., & Farhoomand, A.F. (1996). A structural model of end user
computing satisfaction and user performance. Information & Management, 30, 65-73.
Galletta, D.F., & Lederer, A.L. (1989). Some cautions on the measurement of
user information satisfaction. Decision Sciences, 20, 419-438.
Gatian, A.W. (1994). Is user satisfaction a valid measure of system effectiveness? Information & Management, 26, 119-131.
Gelderman, M. (1998). The relation between user satisfaction, usage of information systems and performance. Information & Management, 34, 1118.
Goodhue, D.L., Klein, B.D., & March, S.T. (2000). User evaluations of IS as
surrogates for objective performance. Information & Management, 38,
87-101.
Goodhue, D.L., & Thompson, R.L. (1995). Task-technology fit and individual
performance. MIS Quarterly, 19(2), 213-236.
Hall, M.J.J. (1996). A risk and control oriented study of the practices of
spreadsheet application developers. Proceedings of the Twenty-Ninth
Hawaii International Conference on System Sciences, 364-373.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

38 McGill

Huitfeldt, B., & Middleton, M. (2001). The assessment of software quality from
the user perspective: Evaluation of a GIS implementation. Journal of End
User Computing, 13(1), 3-11.
Igbaria, M., & Tan, M. (1997). The consequences of information technology
acceptance on subsequent individual performance. Information & Management, 32(3), 113-121.
Jawahar, I.M., & Elango, B. (2001). The effect of attitudes, goal setting and selfefficacy on end user performance. Journal of End User Computing,
13(3), 40-45.
Kim, C., Suh, K., & Lee, J. (1998). Utilization and user satisfaction in end-user
computing: A task contingent model. Information Resources Management Journal, 11(4), 11-24.
Kreie, J. (1998). On the improvement of end-user developed systems using
systems analysis and design. Unpublished Ph.D, University of Arkansas.
Lawrence, M., & Low, G. (1993). Exploring individual user satisfaction within
user-led development. MIS Quarterly, 17(2), 195-208.
Lin, W.T., & Shao, B.B.M. (2000). The relationship between user participation
and system success: A simultaneous contingency approach. Information
& Management, 37, 283-295.
Maxwell, S.E., & Delaney, H.D. (1990). Designing experiments and analyzing data. Belmont, CA: Wadsworth Publishing Company.
McGill, T., Hobbs, V., & Klobas, J. (2003). User developed applications and
information systems success: A test of DeLone and McLean’s model.
Information Resources Management Journal, 16(1), 24-45.
McGill, T.J. (2002). User developed applications: Can end users assess quality?
Journal of End User Computing, 14(3), 1-15.
McGill, T.J., & Dixon, M.W. (2001). Spreadsheet knowledge: An exploratory
study. Proceedings of the 2001 IRMA International Conference:
Managing Information in a Global Economy (pp. 621-625).
McGill, T.J., Hobbs, V.J., Chan, R., & Khoo, D. (1998). User satisfaction as a
measure of success in end user application development: An empirical
investigation. In M. Khosrowpour (Ed.), Proceedings of the 1998 IRMA
Conference (pp. 352-357). Boston, MA: Idea Group Publishing.
McLean, E.R., Kappelman, L.A., & Thompson, J.P. (1993). Converging enduser and corporate computing. Communications of the ACM, 36(12), 7992.
Melone, N.P. (1990). A theoretical assessment of the user-satisfaction construct in information systems research. Management Science, 36(1), 7691.
Nelson, R.R. (1991). Educational needs as perceived by IS and end-user
personnel: A survey of knowledge and skill requirements. MIS Quarterly,
15(4), 503-525.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Effect of End User Development on End User Success 39

Nelson, R.R., & Todd, P. (1999). Strategies for managing EUC on the Web.
Journal of End User Computing, 11(1), 24-31.
Nunnally, J.C. (1978). Psychometric theory. New York: McGraw-Hill.
O’Donnell, D., & March, S. (1987). End user computing environments—Finding
a balance between productivity and control. Information & Management,
13(1), 77-84.
Ouellette, T. (1999, July 26). Giving users the keys to their Web content.
Computerworld, 66-67.
Panko, R.R., & Halverson, R.P. (1996). Spreadsheets on trial: A survey of
research on spreadsheet risks. Proceedings of the Twenty-Ninth Hawaii
International Conference on System Sciences, 2, 326-335.
Rittenberg, L.E., Senn, A., & Bariff, M. (1990). Audit and control of end-user
computing. Altamonte Springs, FL: The Institute of Internal Auditors
Research Foundation.
Rivard, S., Poirier, G., Raymond, L., & Bergeron, F. (1997). Development of a
measure to assess the quality of user-developed applications. The DATA
BASE for Advances in Information Systems, 28(3), 44-58.
Ronen, B., Palley, M.A., & Lucas, H.C. (1989). Spreadsheet analysis and
design. Communications of the ACM, 32(1), 84-93.
Salchenberger, L. (1993). Structured development techniques for user-developed systems. Information & Management, 24, 41-50.
Seddon, P.B. (1997). A re-specification and extension of the Delone and
McLean model of IS success. Information Systems Research, 8(3), 240253.
Seddon, P.B., & Kiew, M.Y. (1996). A partial test and development of Delone
and McLeans model of IS success. Australian Journal of Information
Systems, 4(1), 90-109.
Seddon, P.B., & Yip, S.K. (1992). An empirical evaluation of user information
satisfaction (UIS) measures for use with general ledger accounting software. Journal of Information Systems, 6(1), 75-92.
Shayo, C., Guthrie, R., & Igbaria, M. (1999). Exploring the measurement of end
user computing success. Journal of End User Computing, 11(1), 5-14.
Taylor, M.J., Moynihan, E.P., & Wood-Harper, A.T. (1998). End-user computing and information systems methodologies. Information Systems Journal, 8, 85-96.
Thong, J.Y.L., & Chee-Sing, Y. (1996). Information systems effectiveness: A
user satisfaction approach. Information Processing and Management,
12(5), 601-610.
Zaichkowsky, J.L. (1985). Measuring the involvement construct. Journal of
Consumer Research, 12, 341-352.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

40 McGill

APPENDIX 1
The Problem Statement Given to Participants in Part 2
of the Experimental Session
CAR RENTAL PROBLEM
Deciding which car rental company to choose when planning a holiday can be quite difficult. A local
consumer group has asked you to set up a spreadsheet to help people make decisions about car rental
options. The spreadsheet will enable users to determine which company provides the cheapest option for
them, given how long they need to hire a car and how much driving they intend to do.
After investigating the charges of the major companies, you have the following information about the
options for hiring a compact size car in Australia.
•
•
•

Advantage Car Rentals charges $35 per day for up to 50 miles per day. Extra driving beyond 50 miles
per day is charged at $0.25/mi.
OnRoad Rentals charges $41 per day. This rate includes 100 free miles per day. Extra miles beyond
that are charged at the rate of $0.30/mi.
Prestige Rent-A-Car charges $64 per day for unlimited miles.

Your task is to create a spreadsheet that will allow you or someone else using it to type in the number of
days they will need the car and the number of miles they expect to drive over the time of the rental. The
spreadsheet should then display the rental cost for each of the above three companies.

APPENDIX 2
Items Included in Questionnaire to Measure End User
Perceptions
Perceived system quality

strongly
disagree
1 2

Using the spreadsheet would be easy, even after a long period of not
using it
Errors in the spreadsheet are easy to identify
The spreadsheet increased my data processing capacity
The spreadsheet is easy to learn by new users
Should an error occur, the spreadsheet makes it straightforward to perform
some checking in order to locate the source of the error
The data entry sections provide the capability to easily make corrections to
data
The same terminology is used throughout the spreadsheet
This spreadsheet does not contain any errors
The terms used in the spreadsheet are familiar to users
Data entry sections of the spreadsheet are organized so that the different bits
of data are grouped together in a logical way
The data entry areas clearly show the spaces reserved to record the data
The format of a given piece of information is always the same, whereever it
is used in the spreadsheet
Data is labeled so that they can be easily matched with other parts of the

strongly
agree
6 7

1
1
1
1

2
2
2
2

3
3
3
3

4
4
4
4

5
5
5
5

6
6
6
6

7
7
7
7

1
1
1
1

2
2
2
2

3
3
3
3

4
4
4
4

5
5
5
5

6
6
6
6

7
7
7
7

1
1

2
2

3
3

4
4

5
5

6
6

7
7

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Effect of End User Development on End User Success 41
Data is labeled so that they can be easily matched with other parts of the
spreadsheet
The spreadsheet is broken up into separate and independent sections
Use of this spreadsheet would reduce the number of errors you make when
choosing a rental car
Each section has a unique function or purpose
Each section includes enough information to help you understand what it is
doing
Queries are easy to make
The spreadsheet provides all the information required to use the spreadsheet
(this is called documentation)
Corrections to errors in the spreadsheet are easy to make

1
1

2
2

3
3

4
4

5
5

6
6

7
7

1
1

2
2

3
3

4
4

5
5

6
6

7
7

1
1

2
2

3
3

4
4

5
5

6
6

7
7

Involvement

This car rental spreadsheet is:
unimportant
not needed
nonessential
trivial
insignificant
meaningless to me
unexciting
of no concern to me
not of interest to me
irrelevant to me
not making a
difference to me

1
1
1
1
1
1
1
1
1
1
1

2
2
2
2
2
2
2
2
2
2
2

3
3
3
3
3
3
3
3
3
3
3

User Satisfaction
How adequately do you feel the
spreadsheet meets your information
processing needs when answering car
rental queries?
How efficient is the spreadsheet?
How effective is the spreadsheet?
Overall, are you satisfied with the
spreadsheet?

Perceived Individual Impact
The spreadsheet has a largely positive
impact on my effectiveness and
productivity in answering car rental queries.
The spreadsheet is an important and
valuable aid to me in answering car rental
queries.

4
4
4
4
4
4
4
4
4
4
4

5
5
5
5
5
5
5
5
5
5
5

6
6
6
6
6
6
6
6
6
6
6

7
7
7
7
7
7
7
7
7
7
7

inadequately 1

inefficient 1
ineffective 1
dissatisfied 1

important
needed
essential
fundamental
significant
meaningful to me
exciting
of concern to me
of interest to me
relevant to me
making a
difference to me

2 3 4 5 6 7 adequately

2 3 4 5 6 7 efficient
2 3 4 5 6 7 effective
2 3 4 5 6 7 satisfied

disagree 1 2

3 4 5 6 7 agree

disagree 1 2

3 4 5 6 7 agree

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

42 Staples and Seddon

Chapter III

Testing the
Technology-to-Performance
Chain Model
D. Sandy Staples, Queen's University, Canada
Peter B. Seddon, The University of Melbourne, Australia

ABSTRACT

Goodhue and Thompson (1995) proposed the Technology-to-Performance
Chain (TPC) model in 1995 to help end-users and organizations understand
and make more effective use of information technology. The TPC model
combines insights from research on user attitudes as predictors of utilization
and insights from research on task-technology fit as a predictor of
performance. In this chapter, the TPC model was tested in two settings—
voluntary use and mandatory use. In both settings, strong support was
found for the impact of task-technology fit on performance as well as on
attitudes and beliefs about use. Social norms also had a significant impact
on utilization in the mandatory use setting. Beliefs about use only had a
significant impact on utilization in the voluntary use setting. Overall, the
results found support for the predictive power of the TPC model; however,
the results show that the relationships among the constructs in the model
will vary, depending on whether or not the users have a choice to use the
system.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Testing the Technology-to-Performance Chain Model 43

INTRODUCTION

The purpose of the study reported in this chapter was to test the TPC
proposed by Goodhue and Thompson (1995) in their 1995 paper published in MIS
Quarterly. The TPC model seeks to predict the impact of an information system
on an individual user’s performance. One of the main objectives of information
systems research is to help end users and organizations make more effective use
of information technology. Understanding and measuring the success of investments in information systems is important to both researchers and practitioners.
Practitioners naturally want to learn about ways to make their investments more
effective and ways to improve their decision making about which investments to
make. Researchers desire to build and test theories that explain and predict
performance. The TPC model potentially helps us do that; however, it has not
been tested fully. Contributing to the understanding of the predictive validity of
the TPC is our goal in this chapter.
Goodhue and Thompson’s (1995) TPC model (see Figure 1) combined
insights from research on user attitudes as predictors of utilization and insights
from research on task-technology fit as a predictor of performance. Past
research on user attitudes as predictors of utilization is largely based on theories
of attitudes and behavior (Fishbein & Ajzen, 1975; Triandis, 1980). Aspects of
information technology lead to user attitudes about the system, and these
attitudes, along with social norms and other situational factors, lead to utilization.
Task-technology fit theory suggests that information systems affect performance, depending on the fit or correspondence between the task requirements
of the users and the functionality of the system. Task-technology fit theory also
suggests that the impact on performance depends on the fit between individual
characteristics of the users and functionality of the system. The basic argument
of the model is that for an information technology to have a positive impact on
individual performance, the technology must fit with the tasks it is supposed to
support, and it has to be used (Goodhue & Thompson, 1995).
Goodhue (1995) and Goodhue and Thompson (1995) tested part of the TPC
model (see the dotted paths in Figure 1). They found support for the link between
task-technology fit and performance impacts, as well as some support for the
proposed antecedents of task-technology fit. They did not test the linkages proposed
in their model between task-technology fit and the precursors of utilization, which
is one of the main objectives of our chapter. Our chapter builds on Goodhue’s
work and adds to it by investigating additional links in the TPC that have not been
explored previously. Although the impact of task-technology fit has been
investigated by several researchers in various settings (Dishaw & Strong, 1998a,
1998b; Ferratt & Vlahos, 1998; Goodhue, Klein, & March, 2000; Goodhue,
Littlefield, & Straub, 1997; Kanellis, Lycett, & Paul, 1999; Lim & Benbasat,
2000; Pendharkar, Rodger, & Khosrwopour, 2001; Shirani, Tafti, & Affisco,
1999), no one of whom we are aware has tested directly the proposed TPC model.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

44 Staples and Seddon

Figure 1. Goodhue and Thompson’s technology-to-performance chain
model (adapted from Goodhue & Thompson, 1995, p. 216)
Task
Characteristics
Technology
Characteristics

Task
Technology
Fit
Performance
Impacts

Individual
Characteristics

Precursors of Utilization:
-Expected Consequences of
Use (beliefs)
-Affect Toward Use
-Social Norms
-Habit
-Facilitating Conditions

Utilization

Dotted lines show the
paths tested by Goodhue
& Thompson, 1995

Referring to the link between users’ evaluations of task-technology fit and
performance, Goodhue (1998) suggested:
Conceptual and empirical research is needed critically to address the issue
of whether there is a link, and, if so, under what circumstances it is strong
or weak (p. 128).
Our study meets this call for research by examining this link (along with
other aspects of the TPC model) and does so in two important contexts—
voluntary use and mandatory use. In mandatory use settings, Goodhue and
Thompson (1995) suggested that the fit of the system and its tasks and individual
characteristics, as well as the social norms, may be the key drivers on
performance. Where use is voluntary, Goodhue and Thompson (1995) suggested
that the individual’s beliefs and feelings should play a stronger role than fit of the
system. Their suggestions imply that the strength of the relationships in the TPC
model should vary for the two different use conditions. We examine the validity
of these suggestions in this study.
The next section will describe our research model and present the associated hypotheses that were tested. The research design and methodology is
described in the third section, followed by a description of the results. The final
section discusses the results and offers suggestions for future research.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Testing the Technology-to-Performance Chain Model 45

THE RESEARCH MODEL AND HYPOTHESES

The research model tested in this chapter is illustrated in Figure 2.
Specifically, the relationships between user evaluations of task-technology fit
and perceived performance expected consequences of use and users’ affect
toward using the system are examined. The impact of expected consequences
of use, affect toward use, social norms, and one specific potential facilitating
condition (i.e., support staff effectiveness) on utilization (i.e., use of the system)
also are examined in the analysis. Finally, the linkage between utilization and
perceived performance will be examined. First, we describe the reasoning
behind the relationships specified in the model and then describe how these
relationships are expected to vary for voluntary versus mandatory users. For
brevity, we summarize Goodhue & Thompson’s (1995) arguments in our
presentation of the hypotheses. Readers who want more background on the
model are pointed to their original article. The antecedents of task-technology fit
(i.e., interactions between task, technology, and individual characteristics) are
not tested in this study. Support has been found in previous studies (Goodhue,
1995; Goodhue & Thompson, 1995) for that part of the TPC model.
Consistent with Orlikowski and Iacono’s (2001) classifications of technology as both tool and perception, Goodhue and Thompson (1995) define tasktechnology fit (TTF) as follows:
… the degree to which a technology assists an individual in performing his
or her portfolio of tasks. More specifically, TTF is the correspondence

Figure 2. The research model
Expected
Consequences
of Use

H1
H4

H3
Performance
Impacts

Affect
Toward
Use

Social
Norms

Task
Technology
Fit

H5
Utilization

H7
Facilitating
Conditions

Precursors of Utilization

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

46 Staples and Seddon

between task requirements, individual abilities, and the functionality of the
technology. (Goodhue & Thompson, 1995, pp. 216-218)
Fitness for task can be assessed in two ways. One is to identify important
facets of the task requirements and assess whether the proposed tool—in the
hands of the intended user—meets each of these facet-of-task requirements.
We call this the facets-of-fit approach to assessing task-technology fit. The other
is to predict the outcomes of tool use—again, in the hands of the intended user—
and see if they are as desired. We call this the predicted-outcomes approach to
assessing fit. The following example illustrates these two approaches to measuring fit. Suppose the task is to cook spaghetti, and three sets of tools are being
considered. Potential Toolset One is a large metal pot and a gas cooker; Toolset
Two is a small metal pot and flame thrower; Toolset Three is a large plastic bowl
and an open fire. The facets-of-fit approach to assessing fit would involve asking
whether certain key facets of the task requirements are met. For cooking
spaghetti, these might be (a) the container should hold sufficient water; (b) there
should be a strong, reliable, controllable heat source available; and (c) the
proposed container should withstand the heat as the water boils for 10-15
minutes. Toolset One meets all three requirements, so it is fit for the task. Toolset
Two fails the first test (it would not hold sufficient water) and the second test (the
heat source is hard to control). Toolset Three fails the third requirement (the
plastic bowl would melt or burn), so it, too, is unfit for the task. By contrast, the
predicted-outcomes approach to assessing task-technology fit asks: Would this
toolset, in the hands of this user, lead to the desired outcome? To answer this
question, the respondent imagines using the toolset and attempts to predict the
outcome. Using this approach, since Toolset One seems likely to produce cooked
spaghetti, there is a good fit between Toolset One and this particular task. By
contrast, because the outcomes from Toolset Two and Toolset Three seem
unlikely to result in cooked spaghetti, fit would be low in each case.
This example makes it clear that there are two ways to assess tasktechnology fit: facets-of-fit and predicted outcomes. If the facets of TTF are
correctly identified, both measures should be highly correlated. From this
perspective, Davis’ (1989) famous questionnaire on perceived usefulness and
perceived ease of use contains two measures of TTF—a facets-of-fit measure
and a predicted-outcomes measure. Specifically, the six ease-of-use questions
(e.g., “I would find CHART-MASTER easy to use”) ask about one aspect of
facets-of-fit; namely, ease of use. By contrast, the six perceived-usefulness
questions (e.g., “Using CHART-MASTER in my job would increase my
productivity”) ask respondents to assess TTF based on predicted outcomes.
Goodhue and Thompson (1995) used eight facets of fit (relationship, quality,
timeliness, compatibility, locatability, ease/training, reliability, and authority) to
explain the variance in performance 1, an outcomes measure of task-technology
fit. Curiously, they did not attempt to combine scores for the various facets of fit
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Testing the Technology-to-Performance Chain Model 47

into some overall measure of facet-based task-technology fit. They simply used
their eight facets-of-fit measures in three regression equations that explained
variance in performance. Like Goodhue and Thompson (1995), in this study we
use four facets of fit to explain variance in two potential outcomes indicators of
fit—Expected Consequences of Use and Performance Impacts. But, unlike
Goodhue and Thompson (1995), we use the formative-indicators technique in
PLS (Chin, 1998, 2001) to create a second-order factor called Task-Technology
Fit (TTF). That second-order factor is depicted at the top of Figure 2.
Precursors of utilization (see the box on the left of Figure 2) include beliefs
about the consequences of using a system. The logic behind Goodhue and
Thompson’s (1995) TPC model’s links between TTF and Expected Consequences of Use and Affect Toward Use (i.e., beliefs and feelings about the
consequences of using a system) is based on the assumption that TTF should be
one important determinant of beliefs about the usefulness and importance of a
system and the advantage obtained from using a system (see Goodhue &
Thompson [1995] for fuller justification and logic behind their model). The better
the fit between the capabilities of the system, the task, and the individual, the
more positive the expected consequences and the higher users’ affect toward
using the system. Thus, we hypothesize:
H1: Task-Technology Fit will be associated positively with Expected Consequences of Use.
H2: Task-Technology Fit will be associated positively with Affect Toward Use.
If the system matches the needs of the users (i.e., their tasks) and their
abilities, then it should have a positive impact on performance, since it is more
useful to achieve the required task. The positive relationship between tasktechnology fit and performance has been examined and supported by previous
research. Goodhue and Thompson (1995) found support for this hypothesis in
their study of 25 different technologies in two organizations. Benbasat, Dexter
and Todd (1986) found support for the impact of system design fit with the task
on task performance, as did Dickson, DeSanctis and McBride (1986). Vessey
(1991) and Jarvenpaa (1989) also found strong support for the relationship
between cognitive fit and performance in their laboratory experiments. Therefore, we hypothesize:
H3: Task-Technology Fit will be positively associated with Performance Impacts.
Hypotheses 4 through 7 refer to the relationships between antecedents of
utilization and use (i.e., utilization). These hypotheses are based on theories of
attitudes and behaviors such as the Theory of Reasoned Action (TRA) (Fishbein
& Ajzen, 1975). TRA has been found to have strong predictive power in many
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

48 Staples and Seddon

studies and in a meta-analysis done by Sheppard, Hartwick, and Warshaw
(1988). In an information systems context, Davis (1989) developed the Technology Acceptance Model (TAM), which was adapted from TRA (Szajna, 1996).
The validity of TAM also has been demonstrated in many information systems
contexts (see Mathieson, Peacock, & Chin, 2001; Venketash, 1999; Venketash
& Davis, 2000 for recent reviews of TAM research), such that it is now a wellaccepted belief that attitudes of the users can influence use of systems. Social
norms and other situational factors also can influence utilization (Hartwick &
Barki, 1994; Moore & Benbasat, 1992). Thus, consistent with TRA, TAM, and
Goodhue and Thompson’s (1995) TPC model, we hypothesize:
H4: Expected Consequences of Use will be positively associated with Utilization.
H5: Affect Toward Use will be positively associated with Utilization.
H6: Social Norms will be positively associated with Utilization.
H7: Facilitating Conditions will be positively associated with Utilization.
Finally, use of a system or non-use of a system can impact performance
(i.e., accomplishment of some task or group of tasks). If the system is designed
well, then more use should have a positive impact on performance. Non-use of
a well-designed system should have a negative impact on potential performance,
since efficiency and effectiveness gains are lost by non-use. Hence, it is
hypothesized:
H8: Utilization will be positively associated with Performance Impacts.
Although Goodhue and Thompson (1995) intended the TPC model to be a
general model, they note, “to the extent that utilization is not voluntary, performance impacts will depend increasingly upon task-technology fit rather than
utilization” (p. 216). This implies that the strengths of the relationships in the
model should vary, depending on whether use is voluntary or not (i.e., mandatory). By testing the TPC model in both mandatory- and volitional-usage
situations, we are able to examine if the expected patterns described next
emerge. Under mandatory use, we expect strong paths from task-technology fit
to performance impacts, and from social norms to utilization. In mandatory use,
the users’ expectations and feelings toward use should not play as strong a role
as when they have a choice (i.e., in a voluntary use situation). In mandatory
usage, social norms also should play a strong role, since these would reflect the
expectations of others that the system is to be used (Goodhue & Thompson,
1995). The findings of Venkatesh and Davis (2000) support this compliance
effect. They found subjective norm had a significant direct effect on usage
intentions for mandatory users but not for voluntary users. Under voluntary use,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Testing the Technology-to-Performance Chain Model 49

we expect to see weaker paths from task-technology fit to performance impacts,
and stronger paths from utilization to performance impacts and from expected
consequences of use and affect toward use to utilization. In both situations, we
expect task-technology fit to impact the expectations and feelings toward use,
since the impact of fit on these constructs should be independent of why the
system is used.
A quantitative research design was chosen to examine the proposed
relationships among the various constructs in the research model. The next
section describes how the research model was tested.

RESEARCH DESIGN AND METHODOLOGY

This section describes the two samples, construct measures, and analysis
methods employed. Samples were obtained from users of two different systems.
The first set of users represents mandatory use, since they were required to use
the system to do their job. The second set of users represents voluntary use, since
it was up to the users to decide if they wished to use the system or not (i.e., they
could accomplish their job without using the system.) As highlighted in the
discussion section, internal validity could have been stronger, if it was possible
to find one system that had both mandatory and volitional users; however, we
were unable to find such a field setting, and to enhance external validity, we
wanted to sample real users of existing systems, not to conduct an experiment.

Sample A: Mandatory Use

To test the model in Figure 2 for mandatory use, we needed to collect
opinions from individual users of information systems who were required to use
the system as part of their job. We obtained agreement from the University
Librarian (i.e., the CEO) at a large university library to survey her staff
concerning the effectiveness of the library’s central cataloguing system. Although patterns of usage varied (i.e., acquisitions staff used the system to record
details of planned purchases; cataloguing staff used the system to add cataloguing information once books had arrived; loans-desk staff used the system when
patrons borrowed books and to collect fines for overdue fines; accounting staff
used the system to manage and budget for expenditure), all these people had to
interact with the system to do their job. Questionnaires were sent to 250
librarians, and after one follow-up letter, 140 usable responses (56% of 250)
resulted. To test for non-response bias, we split the data by work groups and
tested to see whether the percentages that responded in each group were
significantly different from the known percentages in the population. There were
no significant differences, so we believe that the dataset is representative of the
population.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

50 Staples and Seddon

Figure 3: Indications of mandatory use
100

Frequency

20
Std. Dev = 1.17
Mean = 6.5
N = 112.00

0
1.0

2.0

3.0

4.0

5.0

6.0

7.0

My employer requires me to use the system

To establish that respondents were, indeed, mandatory users, we asked
respondents about the degree of mandatory usage via a Likert question (1 =
strongly disagree; 7 = strongly agree). The response is shown in Figure 3. The
great majority felt that they were required to use the system by their employer,
supporting the position that this dataset represents mandatory use.

Sample B: Voluntary Use

For what we expected would be volitional users, we decided to survey
university students concerning their use of two types of productivity tools—word
processors and spreadsheets—for course-related work and personal activities.
These users were not required to use these productivity tools for their courses
(i.e., it was their choice, so they were voluntary users). Student opinions about
the benefits from their own personal use of computers are considered valid,
because the students were not asked to imagine themselves acting in some
unfamiliar management role. They were asked to think about one particular
application that they use and report on their experiences with it. The subjects
were senior-year undergraduate commerce students who all had completed an
introductory computing subject at least 12 months before the date of the survey.
Questionnaires were mailed out to a random sample of 600 students. After one
follow-up letter, 308 responses were received (a response rate of 50%). The 266
evaluations of word processing and spreadsheet packages were used as the
potential sample for testing our model. However, missing responses to some of
the questions (e.g., the information quality questions) reduced our usable dataset
to 114. Tests for non-response bias found no significant differences that would
affect the relationships in the study.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Testing the Technology-to-Performance Chain Model 51

Figure 4. Indications of voluntary use
30

Frequency

Std. Dev = 2.16
Mean = 3.7
N = 107.00

0
1.0

2.0

3.0

4.0

5.0

6.0

7.0

My instructor requires me to use the system

To establish that respondents were voluntary users, we again asked
respondents about the degree of mandatory usage via a Likert question (1 =
strongly disagree; 7 = strongly agree). The response is shown in Figure 4. While
the majority felt that they were not required to use the system, there were a
substantial number who perceived that they had little choice. Therefore, to get
a more valid set of voluntary users, we dropped any respondents who scored
above 4 on this question. This resulted in a sample of 66 voluntary use
respondents that was used to test the research model.

Construct Measurement

The questionnaire completed by the respondents contained multiple measurement items relating to each of the constructs in the research model (see
Appendix A for details of each question). Wherever possible, scales that had
demonstrated good psychometric properties in previous studies were used. As
indicated earlier, task-technology fit (TTF) was measured with a multi-faceted
measure. Goodhue (1995, 1998) and Goodhue & Thompson (1995) measured
facets of TTF dealing with the quality and accessibility of the information
provided by the system, ease of use, and training of the system. Twelve questions
were used in the current study to measure four similar facets of the TTF
construct: Work Compatibility, Ease of Use, Ease of Learning, and Information
Quality.
Expected Consequences of Use, a potential outcomes measure of TTF, was
measured with 10 items that dealt with issues of the usefulness of using the
system and the personal benefits of using the system. Affect Toward Use was
measured with eight items dealing with attitudes and feelings toward system use.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

52 Staples and Seddon

Table 1. Internal consistency of the constructs
Construct/Scale
First order indicators of tasktechnology fit
Work Compatibility
Ease of Use
Ease of Learning
Information Quality
Expected Consequences of Use
Affect toward Use
Social Norms
Facilitating Conditions
Utilization
Performance Impacts

Number of
Items

Internal
Consistency

Cronbach’s
Alpha

Average
Variance
Extracted

3
3
3
3
10
8
4
3
6
7

0.875
0.907
0.952
0.896
0.952
0.934
0.878
0.793
0.917
0.918

0.762
0.840
0.921
0.822
0.941
0.917
0.822
0.720
0.717
0.886

0.702
0.766
0.868
0.743
0.668
0.638
0.645
0.571
0.649
0.621

The Social Norms construct was measured with four questions dealing with
pressures and expectations from others (i.e., boss, coworkers, family, and
friends) to use the system. One specific type of facilitating condition was
investigated—relationship with support staff (three questions) for the mandatory
use setting only (information on support staff help was not relevant for the
voluntary usage setting). Utilization was measured with both self-reported
estimates of the number of hours respondents used the system in the past and
expected to use it in the future, along with Likert scale measures of whether they
felt they were light/heavy users and infrequent/frequent users. The Performance Impacts construct was measured with seven self-reported questions
(consistent with Goodhue and Thompson’s [1995] way of measuring Performance Impacts) of the overall net benefit of the system to the respondent,
including efficiency and effectiveness issues, overall advantages versus disadvantages, cost-effectiveness, and overall satisfaction.
The number of items used to measure each construct, along with indicators
of internal consistency, are provided in Table 1. As discussed in the results
section, internal consistency of all constructs was acceptable.

Analysis Method

Partial Least Squares (PLS), a structural equation modeling (SEM) technique, was chosen for analyzing relationships between variables in the research
model (for more information on PLS, see Barclay, Higgins, & Thompson, 1995;
Gefen, Straub, & Boudreau, 2000; Hulland, 1999). PLS was chosen as the
analytical tool over a covariance SEM approach (e.g., LISREL or AMOS) for
several reasons. PLS makes no assumptions about the distribution of the
variables (i.e., it is non-parametric), so it is more robust than a parametric
modeling approach, which assumes multi-variate normality. The purpose of PLS
is to explain variance in the model (similar to regression), and since that is the

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Testing the Technology-to-Performance Chain Model 53

purpose of the study, it fits well. Last, PLS generally can be used validly with
significantly smaller sample sizes than covariance SEM. The minimum sample
required is calculated by identifying the endogenous construct with the most
paths leading into it (Utilization, in this case, with four paths leading into it). The
minimum sample size is 10 times the number of paths leading into this construct,
so our sample sizes are adequate for PLS analysis (Chin, 1998). (The sample size
of the voluntary group would not be adequate for covariance SEM).
All constructs were modeled with reflective indicators. The multidimensionality of task-technology fit was handled by modeling task-technology fit as
a second-order factor with each facet of task-technology fit being a first-order
factor that formed the second-order factor. The hierarchical component approach described by Lohmoller (1989) was used to model the second-order
factor. In this approach, the second-order factor is measured using the indicators
that also are used as indicators for each of the first-order factors (i.e., dimensions
of task-technology fit). The hierarchical component approach is the easiest way
to model second-order factors in PLS and works best with equal number of
indicators for each first-order construct, as is the case here (Chin, Marcolin, &
Newsted, 1996; Chin, 2001).

RESULTS

Structural equation modeling involves two phases. First, the measurement
model is assessed. Second, once the measurement model has been shown to be
adequate, the explanatory and predictive power of the model (i.e., the structural
model) can be assessed. When second-order factors are part of the model, as
is the case here, examination of the measurement properties of the first-order
factors that form the second-order factor is done to ensure adequate psychometric properties (Chin, 2001). Paths between the second-order constructs and
other components of the model are then examined as part of the assessments of
the structural model. Details of the assessment of the measurement model are
shown in Table 2, followed by the evaluation of the structural model.
Table 1 reports internal consistency values for each of the constructs in the
research model using both a measure proposed by Fornell and Larcker (1981)
and Cronbach’s alpha. The internal consistency scores should exceed 0.7, which
they do for all scales in Table 1 for both measures of consistency. Table 1 also
reports average variance extracted. The square root of this measure is used in
Table 2 to assess discriminant validity. Table 2 presents the intercorrelations
among constructs for each dataset. The diagonal elements of Table 2 are the
square roots of the average variance extracted for each latent variable. For
discriminant validity, these diagonal elements should be larger than any of the
intercorrelations between the latent variables (Barclay et al. 1995), which they
are. Another test of discriminant validity is to assess the loadings of each
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

54 Staples and Seddon

Table 2. Discriminant validity analysis
1. Expected Consequences
of Use
2. Affect toward Use
3. Social Norms
4. Facilitating Conditions
5. Utilization
6. Performance Impacts
7. Task-Technology Fit
(TTF) Factor 1: Work
Compatibility
8. TTF Factor 2: Ease of
Use
9. TTF Factor 3: Ease of
Learning
10. TTF Factor 4:
Information Quality

1
0.818

0.663
0.372
0.398
0.353
0.751
0.687

0.799
0.384
0.459
0.288
0.691
0.622

0.803
0.218
0.433
0.244
0.225

0.755
0.084
0.390
0.396

0.806
0.183
0.193

0.788
0.675

0.838

0.753

0.569

0.219

0.329

0.225

0.717

0.602

0.875

0.624

0.457

0.184

0.236

0.187

0.551

0.531

0.778

0.932

0.593

0.498

0.286

0.321

0.326

0.603

0.674

0.572

0.430

0.862

The bold diagonal elements are the square roots of the variance shared between the constructs
and their measures (i.e., the square root of the average variance extracted) from Table 1. Offdiagonal elements are the correlations between latent constructs. For strong discriminant validity,
the diagonal elements should be larger than any other corresponding row or column entry
(Barclay et al., 1995; Hulland, 1999).

individual item. The items should load highest on their targeted construct and
have relatively low loadings on all the other constructs2. All the items loaded
highest on their target construct. Overall, the statistics in Tables 1 and 2, along
with the examination of the individual items, suggest that the measurement model
is adequate.
Details of evaluation of the structural model are reported in Figure 5.
Evaluation of the structural model also involves two parts. First, predictive power
of the model is assessed. Second, the strength of hypothesized relationships
among the constructs is analyzed.
The predictive power of the model for both datasets is summarized by the
R2s on the endogenous variables in Figure 5. For mandatory use, the model
predicts 58% of Performance Impacts, 64% of Expected Consequences of Use,
41% of Affect Toward Use, and 24% of the Utilization construct. For voluntary
use, the model predicts 48% of Performance Impacts, 43% of Expected
Consequences of Use, 7% of Affect Toward Use, and 17% of the Utilization
construct. High path coefficients are to be expected between TTF and both
Expected Consequences of Use and Performance Impacts, if the facets-of-task
requirements have been correctly identified in measuring TTF. In Figure 5, these
path coefficients are high for both mandatory and voluntary use. Overall, the
amount of variance explained in Performance Impacts, our last endogenous
variable, appears reasonable, given that there certainly would be other contextual variables affecting the respondents’ net benefit beliefs.
Table 3 contains a summary of the hypotheses tested, the path coefficients
obtained from the PLS analysis, and t-values for each path obtained through
bootstrapping, for both datasets. At the 5% confidence level, the statistically
significant paths (i.e., significantly different from zero) in the mandatory use
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Testing the Technology-to-Performance Chain Model 55

Figure 5. Research model analysis results

Expected
Consequences
of Use
a R2 = 0.64
b R2 = 0.43
Affect
Toward Use
a R2 = 0.41
b R2 = 0.07

a = 0.798***
Task
b = 0.657*** Technology
Fit

Performance
Impacts
a R2 = 0.58
b R2 = 0.48

a = 0.233
b = 0.412***

a = 0.048
b = -0.110

H8
Utilization

Social
Norms

H7
Facilitating
Conditions

a = 0.770***
b = 0.638***

a = 0.641***
b = 0.266

H4
H5

a = 0.351**
b = -0.216
a = -0.107
b = n/a

a = -0.029
b = 0.181

a R2 = 0.24
b R2 = 0.17

a = mandatory use dataset
b = voluntary use dataset

dataset were found to support Hypotheses 1, 2, 3, and 6. The paths from tasktechnology fit to the three constructs—Consequences of Use (H1), Affect
Toward Use (H2), and Performance Impacts (H3)—were all highly significant.
The only antecedent of Utilization that had a significant path was Social Norms
(H6). In the voluntary use dataset, support was found again for hypotheses 1 and
3 (task-technology fit to Expected Consequences of Use and Performance
Impacts). A significant relationship was also found from Expected Consequences of Use to Utilization, supporting H4. These results are discussed in the
next section.

DISCUSSION

We see three main contributions of our study. First, strong support was
found for the impact of facets-based task-technology fit on perceived performance (H3), consistent with previous research results. The amount of variance
explained by the model in both datasets, along with an examination of the
statistical significance and strength of the individual paths, shows that most of the
explanatory power comes from the task-technology fit (TTF) construct.
Our results were somewhat similar to those obtained by Goodhue and
Thompson (1995) in that both our study and their study found that more
explanatory power came from task-technology fit than from utilization (utilization added .02 in explained variance in Goodhue and Thompson’s study).
However, Goodhue and Thompson did not discuss the issue of voluntariness with
respect to their respondents, so we can’t comment on whether or not that could
have been an influence on their results (the influence of voluntariness on our
results are discussed later).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

56 Staples and Seddon

Second, strong support also was found for the part of the TPC model linking
facets-based TTF with Expected Consequences for Use and, for mandatory
users, Affect Toward Use (Hypotheses 1 and 2). Goodhue and Thompson (1995)
did not test these links. Instead, they assumed that the relationships existed and,
therefore, tested an indirect path from TTF to Utilization. They found little
empirical support for this path, raising questions about the validity of the H1 and
H2 part of the Technology Performance Chain model. By contrast, our results
support H1 and for mandatory users, H2 and demonstrate that individuals’
perceptions of the fit of the system with their task and individual characteristics
are important influences on their beliefs about the usefulness, importance, and
potential advantages to be obtained from using a system.
Third, the results do demonstrate that the relationships in the TPC model
vary under the two usage situations (i.e., voluntary and mandatory). As
expected, the relationships between beliefs and affects toward use in a mandatory setting were non-significant. The data confirm that when users do not have
a choice about system use, their beliefs and feelings about such use may be
largely irrelevant in predicting utilization. By contrast, as suggested by Goodhue
and Thompson (1995), the fit of the system and its tasks and individual
characteristics, as well as the social norms, may be the key drivers of performance. If social norms are strong, in a situation where mandatory use exists, the
norms may overpower beliefs about expected consequences of use and affect
toward use. That appears to be the pattern observed in this study. For voluntary
use, it was expected that beliefs and feelings toward use (H4 and H5) would be
Table 3. Summary of path coefficients and significance levels

Hypotheses and Corresponding
Path(s)
H1:

Mandatory
Use
Path
t-value
Support
Coefficient
for H ?
(degrees of
freedom = 498)

Path
Coef.

Task-technology fit to
Expected consequences
YES
.657
.798
20.87***
of use
H2:
Task-technology fit to
Affect toward use
.641
10.41***
YES
.266
H3:
Task-technology fit to
Performance impacts
.770
19.19***
YES
.638
H4:
Expected consequences
of use to Utilization
.233
1.78
NO
.412
H5:
Affect toward use to
Utilization
.048
0.38
NO
-.110
H6:
Social norms to
Utilization
.351
3.10**
YES
-.216
H7:
Facilitating conditions
not
to Utilization
-.107
-1.08
NO
tested 1
H8:
Utilization to
Performance impacts
-.029
-0.44
NO
.181
* p < .05; ** p < .01; ***p < .001 (2 tailed test)
- t-statistics were calculated using bootstrapping
1
information on support staff help was not relevant for the voluntary usage setting

Voluntary
Use
t-value
Support
(d.f. = 493) for H ?
7.04***

YES

1.54

9.74***

YES

3.78***

YES

-0.47

-1.10

1.72

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Testing the Technology-to-Performance Chain Model 57

associated significantly with use; social norms would not play a strong role (H6);
and the impact of use on performance would be stronger than in a mandatory
setting (H8). We did see this pattern, in that H4 was supported, and H6 was not
for the voluntary dataset. However, as will be discussed, H8 was not supported.
The hypothesized (H8) association of Utilization with Performance Impacts
in our study was found to be non-significant for both datasets. However, the path
for the voluntary users was very close to being significant (i.e. t = 1.72 for a path
of 0.181), and would be, if the p-value was relaxed to 0.10 or if a one-tail test was
used. This lends some support to Goodhue and Thompson’s (1995) suggestion
that in a voluntary use setting, the impact of utilization on performance impacts
should be stronger than in a mandatory use situation. Future research should
investigate this link more closely with a larger dataset, since the statistical power
of the voluntary-use dataset was somewhat limited. Also, future researchers
should be clear about the usage context being studied, so that the results can be
interpreted and can add to our understanding of the predictive power of the TPC
model.
The purpose of the current chapter was a simple one—to test the Technology Performance Chain model in both mandatory use and voluntary use
situations. The TPC model was proposed approximately eight years ago. Since
that time, the general level of computer literacy of end-users has increased
considerably. Future research could help identify the most critical predictors of
performance, and perhaps examine if the TPC model needs to be revised for
today’s users. Although many aspects of the TPC model have been examined
and potentially validated in other studies of technology acceptance and impact
(e.g., TAM, TRA), there could be additions to the TPC model that would
enhance its explanatory power. A valuable effort for future research would be
to investigate the TPC model and compare and contrast it with current other
research in the area of technology acceptance, use, and performance impacts.
Since 1995, there has been considerable research in some of these areas, so it
would be useful to compare the explanatory power of competing models in both
mandatory and voluntary use settings.
As with all studies, our results should be interpreted in light of the limitations
of the study. The empirical results apply for the specific types of systems
investigated. We deliberately chose two different systems, because we wanted
to test the TPC model in the field with actual users, and it was not possible to find
a system that had a suitable number of both mandatory and volitional users. Using
practitioners adds external validity to our findings, but we realize that using two
different systems potentially decreased internal validity by adding another factor
into the situation. Therefore, TPC needs to be tested across a broader collection
of systems before we can confidently generalize to information systems in
general. Testing the effect of mandatory versus volitional use on the TPC model
in an experimental setting would be valuable future research, since the type of
system could be controlled.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

58 Staples and Seddon

Another limitation was that only one type of facilitating condition was
examined (support staff) in this study, and this was only tested in the mandatory
use setting. No support was found for a relationship between this type of
facilitating condition and utilization; however, there are many other types of
facilitating conditions that could have a stronger impact on utilization. Other
types of facilitating conditions that could be examined in future studies include
things such as ease of access to the system itself (hardware and software).
Training is another important facilitating condition that could be examined in
future studies.
Our measurement of performance impacts was based on the respondent’s
perception of how the system affected his or her work and the worth of the
system to them. Future research to test the model and voluntary/mandatory use
using more objective measures of individual performance would enhance the
internal validity of the findings.
In conclusion, this study found support for some parts of the Technology
Performance Chain model and for the argument that task-technology fit increases in predictive power of performance as utilization become mandatory.
We did find support for the impact of task-technology fit on beliefs and attitudes
regarding use of a system, relationships that Goodhue and Thompson (1995)
proposed but did not test. This, along with direct effect of task-technology fit on
performance, suggests that even in voluntary use settings (where beliefs and
attitudes regarding use of a system have a stronger impact on utilization), a good
fit between the task, technology, and user characteristics is very important, if
users are to achieve desired performance outcomes from system use. Overall,
our results suggest that the Technology Performance Chain model is a useful tool
for understanding the potential impact of a system on task performance.
However, the relationships within the model appear to vary, depending upon
whether usage is mandatory or voluntary.

REFERENCES

Barclay, D., Higgins, C., & Thompson, R. (1995). The partial least squares
(PLS) approach to causal modeling: Personal computer adoption and use as
an illustration. Technology Studies, 2(2), 285-309.
Baroudi, J.J., & Orlikowski, W.J. (1988). A short-form measure of user
information satisfaction: A psychometric evaluation and notes on use.
Journal of MIS, 4(Spring), 44-59.
Benbasat, I., Dexter, A.S., & Todd, P. (1986). An experimental program
investigating color-enhanced and graphical information presentation: An
integration of the findings. Communications of the ACM, 29(11), 10941105.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Testing the Technology-to-Performance Chain Model 59

Chin, W.W. (1998). The partial least squares approach to structural equation
modeling. In G.A. Marcoulides (Ed.), Modern methods for business
research (pp. 295-336). New Jersey: Erlbaum Associates.
Chin, W.W. (2001, March). Partial least squares for researchers: An overview
and presentation of recent advances using the PLS approach [presentation
notes]. Proceedings of the Workshop on PLS, Kingston, Canada.
Chin, W.W., Marcolin, B.L. & Newsted, P.R. (1996). A partial least squares
latent variable modeling approach for measuring interaction effects: Results from a Monte Carlo simulation study and voice mail emotion/adoption
study. In J.I. DeGross, S. Jarvenpaa, & A. Srinivasan (Eds.), Proceedings
of the Seventeenth International Conference on Information Systems,
21-41. Cleveland, Ohio.
Davis, F.D. (1989). Perceived usefulness, perceived ease of use, and user
acceptance of information technology. MIS Quarterly, 13(3), 319-342.
Dickson, G.W., DeSanctis, G. & McBride, D.J. (1986). Understanding the
effectiveness of computer graphics for decision support: A cumulative
experimental approach. Communications of the ACM, 29(1), 40-47.
Dishaw, M.T., & Strong, D.M. (1998a). Assessing software maintenance tool
utilization using task-technology fit and fitness-for-use models. Journal of
Software Maintenance Research and Practice, 10(3), 151-179.
Dishaw, M.T., & Strong, D.M. (1998b). Supporting software maintenance with
software engineering tools: A computed task-technology fit analysis. The
Journal of Systems and Software, 44, 107-120.
Doll, W.J., & Torkzadeh, G. (1988, June). The measurement of end-user
computer satisfaction. MIS Quarterly, 12, 259-274.
Ferratt, T.W., & Vlahos, G.E. (1998). An investigation of task-technology fit for
managers in Greece and the US. European Journal of Information
Systems, 7(2), 123-136.
Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intentions and behavior: An
introduction to theory and research. Boston: Addison-Wesley.
Fornell, C., & Larcker, D. (1981). Evaluating structural equation models with
unobservable variables and measurement error. Journal of Marketing
Research, 18, 39-50.
Gefen, D., Straub, D.W., & Boudreau, M.C. (2000). Structural equation
modeling and regression: Guidelines for research practice. Communications of the Association for Information Systems, 4(7). Retrieved from
http://cais.isworld.org/articles/default.asp?vol=4&art=7
Goodhue, D.L. (1995). Understanding user evaluations of information systems.
Management Science, 41(12), 1827-1844.
Goodhue, D.L. (1998). Development and measurement validity of a tasktechnology fit instrument for user evaluations of information systems.
Decision Sciences, 29(1), 105-138.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

60 Staples and Seddon

Goodhue, D.L., Klein, B.D., & March, S.T. (2000). User evaluations of IS as
surrogates for objective performance. Information & Management,
38(2), 87-101.
Goodhue, D.L., Littlefield, R., & Straub, D. W. (1997). The measurement of the
impacts of IIC on the end-users: The survey. Journal of the American
Society for Information Science, 48(5), 454-465.
Goodhue, D.L., & Thompson, R.L. (1995). Task-technology fit and individual
performance. MIS Quarterly, 19(2), 213-236.
Hartwick, J., & Barki, H. (1994). Explaining the role of user participation in
information system use. Management Science, 40(4), 440-465.
Hulland, J.S. (199)9. Use of partial least squares (PLS) in strategic management
research: A review of four recent studies. Strategic Management Journal, 20(2), 195-204.
Jarvenpaa, S.L. (1989). The effect of task demands and graphical format on
information processing strategies. Management Science, 35(3), 285-303.
Kanellis, P., Lycett, M. & Paul, R. J. (1999). Evaluating business information
systems fit: From concept to practical application. European Journal of
Information Systems, 8(1), 65-76.
Lim, K.H., & Benbasat, I. (2000). The effect of multimedia on perceived
equivocality and perceived usefulness of information system. MIS Quarterly, 24(3), 449-471.
Lohmoller, J. (1989). Latent variable path modeling with partial least
squares. New York: Springer-Verlag.
Mathieson, K., Peacock, E. & Chin, W.W. (2001). Extending the technology
acceptance model: The influence of perceived user resources. Database
for Advances in Information Systems, 32(3), 86-112.
Moore, G.C., & Benbasat, I. (1991). Development of an instrument to measure
the perceptions of adopting an information technology innovation. Information Systems Research, 2(3), 192-222.
Moore, G.C., & Benbasat, I. (1992). An empirical examination of a model of
the factors affecting utilization of information technology by end
users [working paper]. University of British Columbia, Vancouver, B.C.
Orlikowski, W.J., & Iacono, C.S. (2001). Research commentary: Desperately
seeking the IT in IT research—A call to theorizing the IT artifact.
Information Systems Research, 12(2), 121-134.
Pendharkar, P.C., Rodger, J.A., & Khosrwopour, M. (2001). Development and
testing of an instrument for measuring the user evaluations of information
technology in health care. Journal of Computer Information Systems,
41(4), 84-89.
Seddon, P.B., & Kiew M-Y. (1996). A partial test and development of DeLone
and McLean’s model of IS success. Australian Journal of Information
Systems, 4(1), 90-109.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Testing the Technology-to-Performance Chain Model 61

Sheppard, B.H., Hartwick, J. & Warshaw, P.R. (1988). The theory of reasoned
action: A meta-analysis of past research with recommendations for modifications and future research. Journal of Consumer Research, 15, 325343.
Shirani, A.I., Tafti, M. H. A. & Affisco, J.F. (1999). Task and technology fit: A
comparison of two technologies for synchronous and asynchronous group
communication. Information & Management, 36(3), 139-150.
Szajna, B. (1996). Empirical evaluation of the revised technology acceptance
model. Management Science, 42(1), 85-92.
Thompson, R.L., Higgins, C. & Howell, J.M. (1991). Personal computing:
towards a conceptual model of utilization. MIS Quarterly, 15(1), 125-143.
Thompson, R.L., Higgins, C. & Howell, J.M. (1994). Influence of experience on
personal computer utilization: Testing a conceptual model. Journal of
Management Information Systems, 11(1), 167-187.
Triandis, H.C. (1980). Values, attitudes and interpersonal behavior. In H. E.
Howe (Ed.), Nebraska symposium on motivation, 1979: Beliefs, attitudes and values (pp. 195-259). Lincoln, NE: University of Nebrasksa
Press.
Venkatesh, V. (1999). Creation of favorable user perceptions: Exploring the role
of intrinsic motivation. MIS Quarterly, 23(2), 239-260.
Venkatesh, V., & Davis, F.D. (2000). A theoretical extension of the technology
acceptance model: Four longitudinal field studies. Management Science,
46(2), 186-204.
Vessey, I. (1991). Cognitive fit: A theory-based analysis of the graphs vs. tables
literature. Decision Sciences, 22(2), 219-240.

ENDNOTES

Their two questions to assess performance were (p.236): (A) The company
computer environment has a large positive impact on my effectiveness and
productivity in my job. (B) IS computer systems and services are an
important and valuable aid to me in the performance of my job.
A table of loadings and cross-loadings is available from the authors.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

62 Staples and Seddon

APPENDIX A: CONSTRUCT MEASUREMENT
1.

Task-technology fit Dimension 1 – Work Compatibility (Source: Moore and
Benbasat, 1991)
Item
1
2
3

Task-technology fit Dimension 2 – Ease of Use (Source: Items 1 and 2 from
the construct came from Doll and Torkzadeh, 1988. Item 3 is from Moore
and Benbasat, 1991)
Item
1
2
3

Wording (Questionnaire 1/Questionnaire 2)
The system is easy to use
The system is user friendly
It is easy to get the system to do what I want it to do

Task-technology fit Dimension 3 – Ease of Learning (Source: Davis, 1989)
Item
1
2
3

Wording
Using the new system fits well with the way I like to work
The system is compatible with all aspects of my work
I have ready access to the system when I need it

Wording
The system is easy to learn
It is easy for me to become more skillful at using the system
New features are easy to learn

Task-technology fit Dimension 4 – Information Quality (Source: All items
came from Doll and Torkzadeh, 1988)
Item
1
2
3

Wording (1 – 7 scale, ranging from Never to Always)
Do you think the output is presented in a useful format?
Is the system accurate?
Does the system provide up-to-date information?

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Testing the Technology-to-Performance Chain Model 63

Consequences of Use (Source: Items 1, 3, 4, 6, 7 and 8 came from Davis,
1989; Items 2 and 5 came from Moore and Benbasat, 1991; Items 9 and 10
came from Thompson, Higgins, and Howell, 1991, 1994)
Item
1
2
3
4
5
6
7
8
9
10

Affect Toward System Use (Source: All items from Hartwick and Barki,
1994)
Item
1
2
3
4
5
6
7
8

Wording
The system enables me to accomplish my tasks more quickly
Using the new system improves my job performance
Using the new system increases my productivity
Using the new system enhances my effectiveness in the job
Using the new system makes it easier to complete my tasks
Using the new system gives me greater control over my tasks
Overall, I find the new system useful in the work I do
Using the system improves the quality of the work I do
The new system is fun to use
The new system is interesting to use

Wording
My frequent use of the system is good vs. bad
My frequent use of the system is terrible vs. terrific
My frequent use of the system is useful vs. useless
My frequent use of the system is worthless vs. valuable
My being a heavy user of the system is good vs. bad
My being a heavy user of the system is terrible vs. terrific
My being a heavy user of the system is useful vs. useless
My being a heavy user of the system is worthless vs. valuable

Social Norms (Source: Hartwick & Barki, 1994)
Item
1
2
3
4

Wording
My boss/instructor believes it is important for me to use the system
My friends at work believe it is important for me to use the system
My family and friends at home believe it is important for me to use
the system
People respect those who can use systems like this

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

64 Staples and Seddon

Facilitating Conditions – Support Staff (Source: Baroudi & Orlikowski, 1988)
Item
1
2
3

Wording
The support staff makes it easy to use the system
I frequently have disagreements with the support staff (reverse coded)
The support staff is never available when I want them (reverse coded)

Utilization (Source: Items 3 to 6 from Hartwick & Barki, 1994; Items 1 and
2 developed for this study).
Item
1
2
3
4
5
6

Wording
On average, how many hours per week do you use the system (last few
months)? (answered in hours/week)
How many hours per week do you expect to use the system (next few months)?
Your present usage of the system (last few months) is: infrequent to frequent
(1-7 scale)
Your present usage of the system (last few months) is: light to heavy (1-7
scale)
Your expected future usage of the system (next few months) is: infrequent to
frequent (1-7 scale)
Your expected future usage of the system (next few months) is: light to heavy
(1-7 scale)

10. Performance Impact (Source: Item 1 was constructed for the study. Item
2 came from Moore and Benbasat, 1991, and the remaining items came
from Seddon and Kiew, 1996)
Item
1
2
3
4
5
6
7

Wording (Questionnaire 1/Questionnaire 2)
The system is a cost-effective solution to my needs
The advantages of using the system outweigh the disadvantages
The system is efficient
The system is effective
Overall, I am satisfied with the system
The system is worthwhile
I would have no difficulty telling others about the results of my use of
this system

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

Chapter IV

The Role of Personal Goal
and Self-Efficacy in
Predicting Computer
Task Performance
Mun Y. Yi, University of South Carolina, USA
Kun S. Im, Yonsei University, South Korea

ABSTRACT

Computer task performance is an essential driver of end user productivity.
Recent research indicates that computer self-efficacy (CSE) is an important
determinant of computer task performance. Contrary to the significant
interest in understanding the role of CSE in predicting computer task
performance, little attention has been given to understanding the role of
personal goal (PG), which can be as powerful as or more powerful than
CSE in predicting and determining computer task performance. Employing
CSE and PG, the present research develops and validates a theoretical
model that predicts individual computer task performance. The model was
tested using PLS on data from an intensive software (Microsoft Excel)
training program, in which 41 MBA students participated. Results largely
support the theorized relationships of the proposed model and provide
important insights into how individual motivational beliefs influence computer
skill acquisition and task performance. Implications are drawn for future
research and practice.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

66 Yi and Im

INTRODUCTION

Computer task performance is a major contributor to end-user productivity.
Most organizational activities are becoming increasingly dependent on computers and computer-based information systems (IS). The expected productivity
gains from the use of IS cannot be realized unless users are equipped with the
requisite computer skills. Many people experience substantial difficulty in
learning to use computers (Carroll & Rosson, 1987; Landauer, 1995; Wildstrom,
1998) and often abandon or underuse multi-million-dollar computer-based systems due to their lack of ability to use the systems effectively (Ganzel, 1998;
McCarroll, 1991). IS researchers have long recognized computer training as one
of the critical factors responsible for ensuring the success of end-user computing
(Bohlen & Ferratt, 1997; Cheney, Mann, & Amoroso, 1986; McLean, Kappelman,
& Thompson, 1993; Nelson & Cheney, 1987). A recent industry survey shows
that 99% of U.S. organizations teach their employees how to use computer
applications (Industry Report, 2001). Understanding the key mechanisms that
govern computer skill acquisition and task performance is a critical issue that has
a significant impact on daily employee functions, return on IS investment, and
ultimate organizational success.
Prior research examined a number of individual variables by which computer learning and task performance could be predicted. (Bostrom, Olfman, &
Sein, 1990; Evans & Simkin, 1989; Marcolin, Munro, & Campbell, 1997;
Martocchio & Judge, 1997; Webster & Martocchio, 1992). Lately, an increased
focus on the variables related to computer learning and task performance has
included a construct called computer self-efficacy (CSE), perception of one’s
capability to use a computer. In addition to being an important variable that
influences an individual’s decision to accept or use information technology
(Compeau & Higgins, 1995b; Hill, Smith, & Mann, 1987; Taylor & Todd, 1995;
Venkatesh, 2000), CSE has been found to significantly influence task performance in various training settings (Compeau & Higgins, 1995a; Gist, Schwoerer,
& Rosen, 1989; Johnson & Marakas, 2000; Martocchio & Dulebohn, 1994).
Contrary to the significant interest in understanding the role of CSE in
predicting computer learning and task performance, little attention has been
given to understanding the role of personal goal (PG), which is defined as the
performance standard an individual is trying to accomplish on a given task
(Locke & Latham, 1990). Goal setting theory (Locke & Latham, 1984, 1990)
views the constructs of both PG and self-efficacy as key determinants of task
performance that have powerful direct and independent effects. In various
studies conducted outside of the computer training domain, PG has been found
as powerful as and, in many cases, more powerful than self-efficacy in predicting
task performance (Bandura & Cervone, 1986; Earley & Lituchy, 1991; Locke
& Latham, 1990; Mitchell, et al., 1994; Wood & Locke, 1987). The joint effects
of self-efficacy and PG on performance indicates that performance is deter-

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

mined not only by how confident one is of being able to do the task at hand, but
also by how much one is trying to achieve. Goal setting theory also theorizes that
self-efficacy indirectly can influence task performance through its effect on PG.
Within the computer training domain, it is unknown how powerful PG is in
predicting trainee performance or how significantly CSE is linked to PG. Very
few studies, if any, have examined either the relative predictive power of CSE
and PG with regard to computer task performance or the relationship between
CSE and PG.
In an overview of past research on computer training, Gattiker (1992)
pointed out that many reports were based on studies of very short duration (less
than four hours), while literature suggested more extended hours of training and
skill practice for relatively complex tasks (Ackerman, 1992). In fact, most IS
training studies have focused on understanding the underlying mechanisms
behind only an initial skill set of a computer application. In sum, employing CSE
and PG, the present research develops a theoretical model that predicts
individual computer task performance and empirically validates the model in an
intensive computer software training program that lasted more than a month.
The rest of the chapter is organized as follows: Section 2 develops the
proposed theoretical model; Section 3 describes the study method employed for
this research; Section 4 presents the test of the proposed model using PLS; and
Section 5 discusses findings and concludes the paper by suggesting future
research directions and practical implications.

CONCEPTUAL BACKGROUND
AND RESEARCH MODEL

Figure 1 presents the research model. On the basis of social cognitive theory
(Bandura, 1977, 1986) and goal setting theory (Locke & Latham, 1984, 1990),
the model theorizes CSE and PG as the key determinants of computer task
performance. CSE is also hypothesized to influence computer task performance
through its effects on PG. The model includes two potentially relevant pretraining variables, prior experience and age, to isolate and control for pre-training
individual differences, thereby more precisely evaluating the theorized effects of
CSE and PG on computer task performance. Each element of the proposed
model and the specific hypotheses relating them are further described below.

Computer Self-Efficacy

Social cognitive theory (Bandura, 1977, 1986) posits that people are driven
neither by inner forces nor external stimuli only. Instead, human behavior is
explained by a model of triadic reciprocality in which behavior, cognitive and
personal factors, and environmental events all operate interactively as determi-

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

68 Yi and Im

Figure 1. Proposed research model
Computer
Self-Efficacy

H1
Computer
Task
Performance

H3
H2
Personal
Goal

Pre-training
Individual
Differences

Prior Experience

Age

nants of each other. A key regulatory mechanism in this dynamic relationship
that affects human behavior is self-efficacy, people’s judgments of their
capabilities to perform certain activities. The theory postulates that psychological procedures, whatever their form, serve as a means of creating and strengthening expectations of personal efficacy (Bandura, 1997), which in turn determines what actions to take, how much effort to invest, how long to persevere, and
what strategies to use in the face of challenging situations.
According to Bandura (1986, 1997), self-efficacy is a situation-specific
belief regarding a specific task accomplishment. Bandura opposes the idea of
measuring global efficacy belief without specifying the activities or conditions
under which they must be performed, but he also acknowledges that it is a multilevel construct. It is important to draw a distinction between general CSE, which
operates at the general computing level across multiple application domains, and
software-specific CSE, which operates at the application-specific software level
(Marakas, Yi, & Johnson, 1998). The present model focuses on softwarespecific CSE, because it more closely corresponds in specificity to the task
performance criterion of the current context (Bandura, 1997). Self-efficacy
formulated at the general computing level is more appropriate in estimating one’s
ability to use a computer across diverse application domains (Marakas et al.,
1998).
Computer software training provides an opportunity for an end user to
obtain the component skills and the confidence required for effective use of the
target software application. Social cognitive theory (Bandura, 1986, 1997) posits
individual self-perception of efficacy as a key determinant of skill acquisition and
task performance. A substantial body of research has reported significant
empirical relationships between self-efficacy and performance (Colquitt, LePine,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

& Noe, 2000; Kraiger, Ford, & Salas, 1993; Salas & Cannon-Bowers, 2001).
Previous research, specifically on computer training, has found post-training,
software-specific CSE to be a significant predictor of task performance (Compeau
& Higgins, 1995a; Gist et al., 1989; Johnson & Marakas, 2000; Martocchio &
Judge, 1997). However, our understanding of CSE in relation to task performance is limited, because most studies examined the predictive validity of CSE
with regard to a fairly simple task performance, focusing on the initial use of a
software program or one specific feature within the program. Extending prior
research, the present study examines the effect of post-training CSE on a
complex task performance, which requires the use of a comprehensive set of
software features, and hypothesizes that:
H1: Computer self-efficacy will positively influence computer task performance.

Personal Goal

The basic premise of goal setting theory (Locke & Latham, 1984, 1990) is
that conscious human behavior is purposeful and regulated by the individual’s
goal. Focusing on the question of why some individuals perform better on work
tasks than others, even when they are similar in ability and knowledge, the theory
seeks the answer from their differing levels of goals. Given that the person has
requisite ability and knowledge, the theory asserts that there is a positive linear
relationship between the level of goal and performance. That is, individuals with
more challenging goals exert more effort in line with the demands of the higher
performance standards (Bandura & Cervone, 1986; Terborg, 1976) and maintain
effort over more extended time (Sales, 1970; Singer, Korienek, Jarvis, McCloskey,
& Candeletti, 1981) than individuals with less challenging goals, thereby producing higher performance.
Although it is unknown specifically how PG is related to task performance
in the context of computer training, there is empirical evidence that PG affects
task performance over and above self-efficacy in a training or education context.
For example, Wood and Locke (1987) examined the relationship between PG
and performance in college courses. They found that grade goals were significantly related to academic course performance over and above the effects of
self-efficacy in three studies. In a meta-analysis based on the results of 13
studies, which measured each of the relationships between PG, self-efficacy,
and performance, Locke and Latham (1990) found the mean of the relationship
between PG and performance (r = .42) to be slightly higher than that of the
relationship between self-efficacy and performance (r = .39). These findings
suggest that PG can be a significant determinant of task performance in a
computer-training program. Thus, we hypothesize that:

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

70 Yi and Im

H2: Personal goal will positively influence computer task performance.
In addition to the direct effects of PG and self-efficacy on individual
performance, goal setting theory (Locke & Latham, 1984, 1990) posits that selfefficacy affects performance through PG. That is, other things being equal,
individuals with higher self-efficacy perceptions tend to set higher goals and
subsequently achieve superior performance. In a meta-analysis, Locke and
Latham (1990) found the link between self-efficacy and PG (r = .39) to be as
strong as the link between self-efficacy and performance (r = .39). Earley and
Lituchy (1991) compared three motivational models that described the relationships among self-efficacy, PG, and performance in alternative ways and found
the study results to support consistently the mediating role of PG in the relation
of self-efficacy and performance as proposed by Locke and Latham (1990).
Based on these findings, we hypothesize the following:
H3: Computer self-efficacy will positively influence personal goal.

Individual Differences: Prior Experience and Age

Even when trainees have the same levels of goals and self-efficacy, they
may not perform at the same level due to their pre-training individual differences.
Studies on goal setting theory and self-efficacy have found that prior experience
(sometimes called past performance) with the task was a significant predictor of
performance over and above PG and self-efficacy (Mitchell et al., 1994; Wood
& Bandura, 1989; Wood & Locke, 1987). Colquitt et al. (2000) conducted a
meta-analytic review of training literature for the past 20 years and showed that
the effect of age on training outcomes was mediated only partially by selfefficacy and other motivational variables. In the context of end-user training,
several researchers have confirmed the significant role of prior experience (Bolt,
Killough, & Koh, 2001; Compeau & Higgins, 1995a; Johnson & Marakas, 2000;
Martocchio & Dulebohn, 1994; Olfman & Bostrom, 1991; Webster & Martocchio,
1993) and age (Martocchio, 1994; Martocchio & Webster, 1992; Webster &
Martocchio, 1995) in determining training outcomes. In those studies, training
outcomes were related positively to prior experience, but negatively to age. By
controlling for potentially relevant pre-training individual difference variables
and accounting for variance in task performance that is unrelated to CSE and PG,
which would otherwise increase error variance, the present research model
seeks to provide a more precise evaluation of the CSE and PG effects on task
performance. Thus, we hypothesize that:
H4: Prior experience will positively influence computer task performance.
H5: Age will negatively influence computer task performance.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

METHOD
Procedure

A training program over a period of four weeks on an electronic spreadsheet
program (Microsoft Excel for Windows) was set up at a large university in the
eastern United States. Participants included 41 MBA students (41.5% female
and 58.5% male). The participant ages ranged from 24 to 48, with the average
of 29.4. Most participants (90.2%) reported using a spreadsheet program not
more than 10 hours a week. All the participants had work experience, with an
average of three to five years.
The training program started with the basic features of Excel and progressively covered more advanced features such as business modeling, charting and
graphing, financial and statistical analysis, database structuring and querying,
and development of complete business applications with macro programming
and interface design. The trainees met on four consecutive Saturdays and two
half-days—one half-day on the Friday just before the first Saturday, and the
other half-day on the Monday after the last Saturday. During the first half-day
session, trainees filled out a questionnaire that included demographic information, took a hands-on test designed to assess prior experience with Excel (25
minutes), and received a brief introductory lecture about basic spreadsheet
features. On the last Saturday, trainees again filled out a questionnaire that
included post-training software-specific CSE and PG measures. Two days later,
which was the last half-day, trainees took a comprehensive hands-on test for
computer task performance (150 minutes).
On each of the four Saturdays, trainees met from 9:00 A.M. to 5:00 P.M.
attending two lectures and two workshop sessions. Each lecture (one in the
morning and one in the afternoon) lasted for 90 minutes and introduced key
concepts, examples, and applications at a conceptual level to provide a frame of
reference within which the more detailed hands-on material could be assimilated.
The class was co-taught by two instructors, including one of the authors. The
instructors took turns covering different topics. The hands-on workshop session
lasted 90 minutes in the morning and 120 minutes in the afternoon. Trainees were
asked to solve assigned problems, from highly guided and detailed step-by-step
instruction to increasingly integrative case examples that required the trainees to
apply the newly acquired expertise in novel ways. Correct answers were
provided to allow trainees to self-check their own progress. To further reinforce
the training material, trainees were asked to solve a number of problems outside
of the training workshop. Table 1 summarizes the training procedures and
elements.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

72 Yi and Im

Table 1. Training procedures and elements
Week
1

Day
Fri.

1
2
3
4

Sat.
Sat.
Sat.
Sat.

Mon.

Training Elements
Pre-training Questionnaire
Prior Experience Assessment (25 min.)
Lecture & Workshop (1): Introduction to Excel
Lecture & Workshop (2 & 3): Building & Using Business Models
Lecture & Workshop (4 & 5): Analyzing & Managing Business Data
Lecture & Workshop (6 & 7): Developing Business Applications
Lecture & Workshop (8 & 9): Integrating with Other Applications
Post-training Questionnaire
Computer Task Performance Assessment (150 min.)

MEASURES
Computer Task Performance

The dependent variable of the study—computer task performance—was
measured by a comprehensive set of problems designed to evaluate the trainees’
overall competencies gained during the training (see Appendix A). Each problem
started typically with a description of a business problem, which was followed
by a list of computer tasks to be completed. The tasks required the use of
software functions such as present value analysis, two-input data table construction, charting, database filtering, pivot table analysis, interface design, and macro
programming. Upon completion of the test, the trainees submitted their results
on a provided diskette. Two graders independently graded the answers using a
scoring key on a scale from 0 to 100. The correlation between the grader scores
was high at .89 (p < .001). Each grader’s scores were used as indicators of the
task performance construct.

Computer Self-Efficacy

CSE was measured at the spreadsheet application level by five items
adopted from Johnson and Marakas (2000). Trainees were asked to indicate the
extent to which they agreed or disagreed with the following statements: “I
believe I have the ability to manipulate the way a number appears in a
spreadsheet;” “I believe I have the ability to use a spreadsheet to communicate
numeric information to others;” “I believe I have the ability to summarize
numeric information using a spreadsheet;” “I believe I have the ability to use a
spreadsheet to share numeric information with others;” and “I believe I have the
ability to use a spreadsheet to assist me in making decisions.” The self-efficacy
measure captured the magnitude (yes or no) and strength (on a scale from 1 to
10, where 1 = quite uncertain and 10 = quite certain) of each individual’s selfefficacy. For further analysis, the magnitude scale was converted to 0 (no) or 1
(yes), and then multiplied by the strength items per Lee and Bobko (1994).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

Personal Goal

The measure consisted of two items adopted from prior research (Locke &
Bryan, 1968; Wood & Locke, 1987). The items were (1) the grade the trainee
hoped to make in the course, and (2) the personal goal the trainee had for the
course grade.

Prior Experience

Each subject’s prior experience with the target computer program was
measured using a hands-on skill test designed to assess basic spreadsheet skills
with 12 computer tasks (Johnson & Marakas, 2000; Yi & Davis, 2001). The test
included entering a formula in multiple cells, using functions to calculate total and
average amounts, computing year-to-date sales and % change of sales, copying
the format of a cell, and changing the formats of numbers (see Appendix B).
Each trainee saved the test result in a diskette and submitted the diskette at the
end of the test. The grading of the answers was handled by the spreadsheet
program module developed through several stages of programming and accuracy
verification. Each task was scored with 1 point for totally correct answers, .5
points for partially correct answers, and 0 for incorrect or missing answers. The
percentage of correct answers was calculated from the total scores and used as
the prior experience measure.

Demographics

Age, sex, length and frequency of computer use and spreadsheet program
use, and work experience were measured by the pre-training questionnaire. Only
age was a significant predictor of the task performance among these demographic variables.

RESULTS

Cronbach alpha measures of internal consistency reliability were all high
and acceptable at .92 for CSE, and .80 for PG. Further measure validation and
model testing were conducted using PLS (Partial Least Squares) Graph Version
2.91.03.04 (Chin & Frye, 1998), a structural equation modeling tool that utilizes
a component-based approach to estimation. The PLS approach (Agarwal &
Karahanna, 2000; Barclay, Higgins, & Thomson, 1995; Chin, 1998; Compeau,
Higgins, & Huff, 1999; Falk & Miller, 1992; Wold, 1982), like other structural
equation modeling (SEM) techniques such as LISREL (Jöreskog & Sörbom,
1993) and EQS (Bentler, 1985), allows researchers to simultaneously assess
measurement model parameters and structural path coefficients. Whereas
covariance-based SEM techniques such as LISREL and EQS use a maximum
likelihood function to obtain estimators in models, the component-based PLS
uses a least squares estimation procedure.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

74 Yi and Im

PLS avoids many of the restrictive assumptions underlying covariancebased structural equation modeling (SEM) techniques such as multivariate
normality and large sample size (Barclay et al., 1995; Chin, 1998; Fornell &
Bookstein, 1982; Wold, 1982). Chin (1998, p. 311) advises that if one were to use
a regression heuristic of 10 cases per indicator, the sample size requirement
would be 10 times (1) the largest number of formative indicators or (2) the largest
number of independent variables impacting a dependent variable, whichever is
the greater. In our model, all items are modeled as reflective indicators, because
they are viewed as effects (not causes) of latent variables (Bollen & Lennox,
1991), and the largest number of independent variables estimated for a dependent variable is four. Thus, our sample size of 41 meets the requirement for the
PLS estimation procedures.

PLS Measurement Model

The measurement model in PLS is assessed by examining internal consistency and convergent and discriminant validity (Barclay et al., 1995; Chin, 1998;
Compeau et al., 1999). Internal consistency reliability (similar to Cronbach’s
alpha) of .7 or higher is considered adequate (Agarwal & Karahanna, 2000;
Barclay et al., 1995; Compeau et al., 1999). Convergent and discriminant validity
is assessed in two ways: (1) the square root of the average variance extracted
(AVE) by a construct from its indicators should be at least .707 (i.e., AVE > .50)
and should exceed that construct’s correlation with other constructs (Barclay et
al., 1995; Chin, 1998; Fornell & Larcker, 1981); and (2) item loadings (similar to
item loadings in principal components) should be at least .707, and items should
load more highly on constructs they are intended to measure than on other
constructs (Agarwal & Karahanna, 2000; Compeau et al., 1999).
Table 2 shows internal consistency reliabilities, convergent and discriminant
validities, and correlations among constructs. The internal consistency reliabilities
all were higher than .90, exceeding the reliability criteria of .70. As strong
evidence of convergent and discriminant validity, the square root of the AVE for
Table 2. Reliabilities, convergent and discriminant validities, and correlations
among constructs
Construct
ICR
1
2
3
4
5
.88
1. Computer Self-efficacy
.94
2. Personal Goal
.92
.39 .92
1.00
3. Prior Experience
1.00
-.04 .07
4. Age
1.00
-.02 .08
.25 1.00
.97
5. Computer Task Performance
.97
.28 .39
.38 -.16
Note. ICR = Internal Consistency Reliability, which should be greater than .70. Diagonal
elements (bold) are the square root of average variance extracted (AVE) between the
constructs and their indicator(s). Off-diagonal elements are correlations between constructs.
For convergent and discriminant validity, diagonal elements should be at least .707 (i.e., AVE
> .50) and larger than off-diagonal elements in the same row and column.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

Table 3. Factor matrix
Scale Items
1. Computer Self-efficacy
a. manipulate the way a number appears
b. use a spreadsheet to communicate
c. summarize numeric information
d. share numeric information
e. use a spreadsheet to assist me in making decisions

.76
.95
.95
.93
.76

.25
.35
.41
.37
.28

-.02
-.09
-.03
.01
-.02

-.01
-.01
-.01
-.01
-.04

.26
.27
.24
.23
.21

2. Personal Goal
a. grade I hope to make
b. personal goal for the course

.36
.35

.92
.92

.10
.03

.11
.04

.34
.38

3. Prior Experience
a. pre-training test score

-.03

.07

1.00

.24

.38

-.02

.08

.24

1.00

-.16

.21
.32

.30
.45

.40
.33

-.18
-.14

.97
.97

4. Age
a. trainee self-reported age
5. Computer Task Performance
a. grader 1 score
b. grader 2 score

each construct was greater than .707 (i.e., AVE > .50) and greater than the
correlation between that construct and other constructs, without exception.
Table 3 provides the factor structure matrix of loadings and cross-loadings.
The factor matrix shows that all items without exception exhibited high loadings
(> .707) on their respective constructs and no items without exception loaded
higher on the other constructs. Overall, the measured scales show excellent
psychometric properties with high reliability and appropriate convergent and
discriminant validity.

PLS Structural Model

The PLS structural model and hypotheses were assessed by examining path
coefficients (similar to standardized beta weights in a regression analysis) and
their significance levels. As recommended (Chin, 1998), bootstrapping (with 120
subsamples) was performed to test the statistical significance of each path
coefficient using t-tests. Inconsistent with H1, CSE had no significant effect on
task performance (b = .16, ns). Supporting H2, PG had a significant effect on
task performance (b = .32, p < .05). Supporting H3, CSE had a significant effect
on PG (b = .39, p < .05). Supporting H4, prior experience had a significant effect
on task performance (b = .43, p < .05). Supporting H5, age had a significant
effect on task performance in the expected direction (b = -.29, p < .05). The
model explained substantial variance in computer task performance (R2 = .38).
Figure 2 summarizes the results of model testing.
Although the earlier discussion of PLS with sample size requirements
justifies the use of PLS in our study, we also tested the research model using
ordinary least-squares regression method (Cohen & Cohen, 1983) to crossexamine the PLS testing results. Results from this analysis were almost identical
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

76 Yi and Im

Figure 2. Model testing results
Computer
Self-Efficacy

.16
Computer
Task
Performance

.39*
.32*
Personal
Goal

R2 = .38

.43*

-.29*

Prior Experience

Age

to the results from the PLS analysis. All the significant paths in the PLS model
remained significant, and the path coefficients were very similar—the difference
between the two corresponding paths between the PLS model and the regression
model was always less than .01.

DISCUSSION
Summary of Findings

Overall, there was significant empirical support for the proposed model. As
expected, PG was a significant predictor of computer task performance. Past
experience and age were also significant predictors of computer task performance. CSE was significantly related to PG. Four of five hypotheses were
supported. Contrary to expectation, the hypothesized effect of post-training CSE
on task performance (H1) was not supported, indicating a weaker contribution
of CSE for the given set of task skills than was expected. As trainees build their
confidence in using the software application in an extended training period, the
predictive strength of software-specific CSE seems to diminish rapidly. Mitchell
et al. (1994) found that PG and past performance were better predictors of
performance than self-efficacy as experience with an air traffic control task
increased. Our findings are consistent with their results.
The specific contributions of the present research can be articulated by
comparing it with other similar studies. In the computer-training context, a
number of studies showed a significant effect of software-specific CSE on
learning or task performance (Compeau & Higgins, 1995; Johnson & Marakas,
2000; Martocchio & Dulebohn, 1994), but did not examine the effect of PG as

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

a determinant of task performance. The current study shows that PG is a more
powerful determinant of computer task performance than CSE. Evans & Simkin
(1989) examined 34 independent variables to find that those individual difference
variables, while the number was considerable, could explain only 23% of the
variance in computer proficiency. The results, which were similar to the results
of other studies, show that the task of finding effective predictors of computer
proficiency and task performance is elusive. Using only four variables, the
current research model explained considerable variance in post-training handson task performance (R2 = .38) designed to measure trainee competencies
gained over a one-month period of computer software training in a field study
setting. Outside of the computer-training domain, Wood and Locke (1987)
examined self-efficacy, PG, and ability, explaining 25% to 28% variation in
academic performance. The current study model outperforms those models in
accounting for individual task performance.
The present study introduces a new variable—PG—which has been shown
to be an important determinant of task performance in other domains into the
computer-training domain. The findings show that PG affects computer task
performance over and above self-efficacy, prior experience, and age. The study
results support the applicability of goal-setting theory (Locke & Latham, 1990)
to an end-user training program and identify an important underlying mechanism
that governs an individual’s computer task performance. Despite the significant
amount of interest on self-efficacy, most computer training studies have demonstrated the predictive validity of CSE in relation to performance for fairly simple
tasks. Our findings suggest that PG plays a more important role in acquiring
complex computer skills. Also, it should be noted that the present study was
conducted in a longitudinal field study setting that lasted more than a month,
covering a full range of skills required for the effective use of a sophisticated
software program. Most previous studies on computer training were conducted
in a relatively short period of time (typically less than a day or two), focusing on
an initial skill set of a computer application. Consequently, our understanding of
the mechanisms that govern the process of computer skill acquisition (in
particular, beyond the initial phase) has been limited. The current study demonstrated that for complex computer skill acquisition, PG is a more powerful
predictor of task performance, and that self-efficacy has no significant effect on
task performance over and above PG. Instead, a self-efficacy belief with regard
to a specific software program is a significant determinant of PG, influencing
task performance indirectly via PG. Although future research should further
compare the relative effects of these two constructs under varying training
conditions, the current study extends prior work by empirically demonstrating the
predictive validity of PG in a computer-training context and testing the theorized
causal chains among CSE, PG, and task performance.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

78 Yi and Im

Limitations and Future Research Implications

Several limitations of the present study should be noted. One of the trainers
was the principal investigator who was aware of the study hypotheses. However, this study did not involve any treatments or manipulations. In addition,
trainees were fully informed that the content of their questionnaire responses
would not affect their grade in any way. The post-training variables were not
available during training, and the performance assessments were handled either
by a computer program or human graders who were not aware of the hypotheses. Thus, the possible threats of hypothesis guessing, evaluation apprehension,
and experimenter expectations to internal validity (Cook & Campbell, 1979)
were avoided or minimized for this study.
Recent motivation research has found that self-efficacy and PG are
influenced by certain personal factors such as goal orientation (Ford, Smith,
Weissbein, & Gully, 1998; Phillips & Gully, 1997; Steele-Johnson, Beauregard,
Hoover, & Schmidt, 2000), locus of control (Phillips & Gully, 1997), self-esteem
(Pilegge & Holtz, 1997; Tang & Sarsfield-Baldwin, 1991), cognitive abilities
(Kanfer & Ackerman, 1989; Kanfer, Ackerman, & Heggestad, 1996), and
achievement motivation (Mathieu, Martineau, & Tannenbaum, 1993; Phillips &
Gully, 1997). Also, many studies have demonstrated that self-efficacy and PG
affect the development and use of effective task strategies to solve problems
(Chesney & Locke, 1991; Gilliland & Landis, 1992; Wood & Bandura, 1989).
These antecedent and consequent variables have not been incorporated into our
current research model. Given that the model received empirical support in the
context of computer training, further relations between these variables and the
study variables of PG and CSE should be examined by future research in order
to properly extend the current model and develop more in-depth understanding
of the processes governing computer skill acquisition.
With regard to external validity, support for the study model should be tested
in different contexts. The present study was conducted with MBA students, all
of whom had work experience. The chosen software was a popular spreadsheet
program, highly useful in the workplace. The length of the training program was
more than a month, which is longer than most prior computer training studies. The
assessed performance outcomes included skills that can be used directly in real
work settings. Thus, the current study maintains many important characteristics
similar to organizational training settings. However, the findings should be
validated in other settings by future research beyond the specific conditions of
this study to ensure generalizability of the study findings.

Implications for Practice

The present study demonstrates the important roles PG and self-efficacy
play in the process of computer skill acquisition. Organizational or training

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

interventions that positively influence these variables are likely to produce
significant improvements in computer task performance, which is the main driver
of end-user productivity in the workplace. Over the past decades, prior studies
identified several interventions to enhance CSE such as behavior modeling
(Compeau & Higgins, 1995a; Gist et al., 1989), positive performance feedback
(Martocchio & Webster, 1992), induced conception of ability (Martocchio,
1994), and management support (Henry & Stone, 1994). Outside of the
computer-training context, the goal-setting research has identified a number of
interventions that can affect PG, including assigned goals (Meyer & Gellatly,
1988), group goals (Matsui, Kakuyama, & Onglatco, 1987), role modeling
(Rakestraw & Weiss, 1981), and normative information (Earley & Erez, 1991).
Although the specific effects of such an intervention on self-efficacy and PG and
subsequent task performance in the context of computer skill training still need
to be examined by future research, these interventions have the potential to
substantially improve an end user’s computer task performance.
We have found that prior experience and age have positive and negative
significant effects respectively on the development of computer skill acquisition.
People who enter a training program with relatively little or no prior exposure to
the target software should be given extra attention in order to be successful in
acquiring computer skills, and older trainees should be supported with more care
than younger trainees. Given that the effects of prior experience and age on task
performance were in the opposite directions, providing more hands-on experience with the software before training should help older trainees become
successful in acquiring computer skills.

CONCLUSION

In conclusion, this research developed a theoretical model that predicts
individual task performance in an end-user computer training context using the
central constructs of goal-setting theory (Locke & Latham, 1984, 1990) and
social cognitive theory (Bandura, 1977, 1986), and empirically validated the
proposed model in a longitudinal field setting. The model has received significant
empirical support. The present study extends previous research on end-user
training by introducing a new variable—PG—and empirically demonstrating its
importance in mediating the effect of CSE and predicting computer task
performance. Organizational or training interventions that make positive impacts
on these motivational variables should contribute to an end user’s improved
computer task performance, leading to increased work productivity.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

80 Yi and Im

REFERENCES

Ackerman, P.L. (1992). Predicting individual differences in complex skill
acquisition: Dynamics of ability determinants. Journal of Applied Psychology, 77(5), 598-614.
Agarwal, R., & Karahanna, E. (2000). Time flies when you’re having fun:
Cognitive absorption and beliefs about information technology usage. MIS
Quarterly, 24(4), 665-694.
Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: PrenticeHall.
Bandura, A. (1986). Social foundations of thought and action: A social
cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H.
Freeman and Company.
Bandura, A., & Cervone, D. (1986). Differential engagement of self-reactive
influences in cognitive motivation. Organizational Behavior and Human
Decision Processes, 38, 92-113.
Barclay, D., Higgins, C., & Thompson, R. (1995). The partial least squares
approach to causal modeling: Personal computer adoption and use as an
illustration. Technology Studies, 2(2), 285-309.
Bentler, P.M. (1985). Theory and implementation of EQS: A structural
equations program. Los Angeles: BMDP Statistical Software.
Bohlen, G.A., & Ferratt, T.W. (1997). End user training: An experimental
comparison of lecture versus computer-based training. Journal of End
User Computing, 9(3), 14-27.
Bollen, K.A., & Lennox, R. (1991). Conventional wisdom on measurement: A
structural equation perspective. Psychological Bulletin, 110, 305-314.
Bolt, M.A., Killough, L.N., & Koh, H.C. (2001). Testing the interaction effects
of task complexity in computer training using the social cognitive model.
Decision Sciences, 32(1), 1-20.
Bostrom, R.P., Olfman, L., & Sein, M.K. (1990). The importance of learning
style in end-user training. MIS Quarterly, 14, 101-117.
Carroll, J.M., & Rosson, M.B. (1987). Paradox of the active user. In J. M.
Carroll (Ed.), Interfacing though: Cognitive aspects of human-computer interaction (pp. 80-111). Cambridge, MA: MIT Press.
Cheney, P.H., Mann, R.I., & Amoroso, D.L. (1986). Organizational factors
affecting the success of end-user computing. Journal of Management
Information Systems, 3(1), 65-80.
Chesney, A.A., & Locke, E.A. (1991). Relationships among goal difficulty,
business strategies, and performance on a complex management simulation
task. Academy of Management Journal, 34(2), 400-424.
Chin, W.W. (1998). The partial least squares approach to structural equation
modeling. In G. A. Marcoulides (Ed.), Modern methods for business
research (pp. 295-336). Mahwah, NJ: Lawrence Erlbaum Associates.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

Chin, W.W., & Frye, T.A. (1998). PLS-Graph (Version 2.91.03.04).
Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation
analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence
Erlbaum Associates.
Colquitt, J.A., LePine, J.A., & Noe, R.A. (2000). Toward an integrative theory
of training motivation: A meta-analytic path analysis of 20 years of
research. Journal of Applied Psychology, 85, 678-707.
Compeau, D., Higgins, C.A., & Huff, S. (1999). Social cognitive theory and
individual reactions to computing technology: A longitudinal study. MIS
Quarterly, 23(2), 145-158.
Compeau, D.R., & Higgins, C.A. (1995a). Application of social cognitive theory
to training for computer skills. Information Systems Research, 6(2), 118143.
Compeau, D.R., & Higgins, C.A. (1995b). Computer self-efficacy: Development of a measure and initial test. MIS Quarterly, 19(2), 189-211.
Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation design and
analysis issues for field settings. Boston, MA: Houghton Mifflin.
Earley, P.C., & Erez, M. (1991). Time-dependency effects of goals and norms:
the role of cognitive processing on motivational models. Journal of
Applied Psychology, 76(5), 717-724.
Earley, P.C., & Lituchy, T.R. (1991). Delineating goal and efficacy effects: A
test of three models. Journal of Applied Psychology, 76, 81-98.
Evans, G.E., & Simkin, M.G. (1989). What best predicts computer proficiency?
Communications of the ACM, 32(11), 1322-1327.
Falk, R.F., & Miller, N.B. (1992). A primer for soft modeling. Akron, OH: The
University of Akron.
Ford, J.K., Smith, E.M., Weissbein, D.A., & Gully, S.M. (1998). Relationships
of goal orientation, metacognitive activity, and practice strategies with
learning outcomes and transfer. Journal of Applied Psychology, 83(2),
218-233.
Fornell, C., & Bookstein, L. (1982). Two structural equation models: LISREL
and PLS applied to consumer exit-voice theory. Journal of Marketing
Research, 19, 440-452.
Fornell, C., & Larcker, D.F. (1981). Evaluating structural equations models with
unobservable variables and measurement error. Journal of Marketing
Research, 18(1), 39-50.
Ganzel, R. (1998, April). Feeling squeezed by technology? Training, 62-70.
Gattiker, U. (1992). Computer skills acquisition: A review and future directions
for research. Journal of Management, 18(3), 547-574.
Gilliland, S.W., & Landis, R.S. (1992). Quality and quantity goals in a complex
decision task: Strategies and outcomes. Journal of Applied Psychology,
77(5), 672-681.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

82 Yi and Im

Gist, M.E., Schwoerer, C., & Rosen, B. (1989). Effects of alternative training
methods on self-efficacy and performance in computer software training.
Journal of Applied Psychology, 74(6), 884-891.
Henry, J.W., & Stone, R.W. (1994). A structural equation model of end-user
satisfaction with a computer-based medical information system. Information Resources Management Journal, 7(3), 21-33.
Hill, T., Smith, N.D., & Mann, M.F. (1987). Role of efficacy expectations in
predicting the decision to use advanced technologies: The case of computers. Journal of Applied Psychology, 72(2), 307-313.
Industry Report. (2001). Training, 38(10), 40-75.
Johnson, R.D., & Marakas, G.M. (2000). The role of behavior modeling in
computer skill acquisition—Toward refinement of the model. Information
Systems Research, 11, 402-417.
Jöreskog, K.G., & Sörbom, D. (1993). LISREL 8: User’s reference guide.
Chicago: Scientific Software, Inc.
Kanfer, R., & Ackerman, P.L. (1989). Motivation and cognitive abilities: An
integrative/aptitude-treatment interaction approach to skill acquisition.
Journal of Applied Psychology—Monograph, 74, 657-690.
Kanfer, R., Ackerman, P.L., & Heggestad, E.D. (1996). Motivational skills &
self-regulation for learning: A trait perspective. Learning and Individual
Differences, 8(3), 185-209.
Kraiger, K., Ford, J.K., & Salas, E. (1993). Application of cognitive, skill-based,
and affective theories of learning outcomes to new methods of training
evaluation. Journal of Applied Psychology, 78(2), 311-328.
Landauer, T.K. (1995). The trouble with computers: Usefulness, usability,
and productivity. Cambridge, MA: MIT Press.
Locke, E.A., & Bryan, J.F. (1968). Grade goals as determinants of academic
achievement. Journal of General Psychology, 79, 217-228.
Locke, E.A., & Latham, G.P. (1984). Goal-setting: A motivational technique
that works. Englewood Cliffs, NJ: Prentice-Hall.
Locke, E.A., & Latham, G.P. (1990). A theory of goal setting and task
performance. Englewood Cliffs, NJ: Prentice-Hall.
Marakas, G.M., Yi, M.Y., & Johnson, R.D. (1998). The multilevel and multifaceted character of computer self-efficacy: Toward clarification of the
construct and an integrative framework for research. Information Systems Research, 9(2), 126-163.
Marcolin, B.L., Munro, M.C., & Campbell, K.G. (1997). End user ability: Impact
of job and individual differences. Journal of End User Computing, 9(3),
3-12.
Martocchio, J.J. (1994). Effects of conceptions of ability on anxiety, selfefficacy, and learning in training. Journal of Applied Psychology, 79(6),
819-825.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

Martocchio, J.J., & Dulebohn, J. (1994). Performance feedback effects in
training: The role of perceived controllability. Personnel Psychology, 47,
357-373.
Martocchio, J.J., & Judge, T.A. (1997). Relationship between conscientiousness and learning in employee training: Mediating influences of selfdeception and self-efficacy. Journal of Applied Psychology, 82(5), 764773.
Martocchio, J.J., & Webster, J. (1992). Effects of feedback and cognitive
playfulness on performance in microcomputer software training. Personnel Psychology, 45, 553-578.
Mathieu, J. E., Martineau, J.W., & Tannenbaum, S.I. (1993). Individual and
situational influences on the development of self-efficacy: Implications for
training effectiveness. Personnel Psychology, 46, 125-147.
Matsui, T., Kakuyama, T., & Onglatco, M.L. (1987). Effects of goals and
feedback on performance in groups. Journal of Applied Psychology, 72,
407-415.
McCarroll, T. (1991). What new age? Time, 138, 44-46.
McLean, E.R., Kappelman, L.A., & Thompson, J.P. (1993). Converging enduser and corporate computing. Communications of the ACM, 36(12), 7992.
Meyer, J.P., & Gellatly, I.R. (1988). Perceived performance norm as a mediator
in the effect of assigned goal on personal goal and task performance.
Journal of Applied Psychology, 73, 410-420.
Mitchell, T.R., Hopper, H., Daniels, D., George-Falvy, J., & James, L.R. (1994).
Predicting self-efficacy and performance during skill acquisition. Journal
of Applied Psychology, 79(4), 506-517.
Nelson, R.R., & Cheney, P.H. (1987). Training end users: An exploratory study.
MIS Quarterly, 11(4), 547-559.
Olfman, L., & Bostrom, R. P. (1991). End-user software training: An experimental comparison of methods to enhance motivation. Journal of Information Systems, 1, 249-266.
Phillips, J.M., & Gully, S.M. (1997). Role of goal orientation, ability, need for
achievement, and locus of control in the self-efficacy and goal-setting
process. Journal of Applied Psychology, 82(5), 792-802.
Pilegge, A.J., & Holtz, R. (1997). The effects of social identity on the self-set
goals and task performance of high and low self-esteem individuals.
Organizational Behavior and Human Decision Processes, 70, 17-26.
Rakestraw, T.L., & Weiss, H.M. (1981). The interaction of social influences and
task experience on goals, performance, and performance satisfaction.
Organizational Behavior and Human Decision Processes, 27, 326-344.
Salas, E., & Cannon-Bowers, J.A. (2001). The science of training: A decade of
progress. Annual Review of Psychology, 52, 471-499.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

84 Yi and Im

Sales, S.M. (1970). Some effects on role overload and role underload. Organizational Behavior and Human Decision Processes, 5, 592-608.
Singer, R.N., Korienek, G., Jarvis, D., McColskey, D., & Candeletti, G. (1981).
Goal-setting and task persistence. Perceptual and Motor Skills, 53, 881882.
Steele-Johnson, D., Beauregard, R.S., Hoover, P.B., & Schmidt, A.M. (2000).
Goal orientation and task demand effects on motivation, affect, and
performance. Journal of Applied Psychology, 85(5), 724-738.
Tang, T.L., & Sarsfield-Baldwin, L. (1991). The effects of self-esteem, task
label, and performance feedback on goal setting, certainty, and attribution.
The Journal of Psychology, 125, 413-418.
Taylor, S., & Todd, P.A. (1995). Understanding information technology usage:
A test of competing models. Information Systems Research, 6(2), 144176.
Terborg, J.R. (1976). The motivational components of goal setting. Journal of
Applied Psychology, 61, 613-621.
Venkatesh, V. (2000). Determinants of perceived ease of use: Integrating
control, intrinsic motivation, and emotion into the technology acceptance
model. Information Systems Research, 11(4), 342-365.
Webster, J., & Martocchio, J.J. (1992). Microcomputer playfulness: Development of a measure with workplace implications. MIS Quarterly, 16(2),
201-226.
Webster, J., & Martocchio, J.J. (1993). Turning work into play: Implications for
microcomputer software training. Journal of Management, 19(1), 127146.
Webster, J., & Martocchio, J.J. (1995). The differential effects of software
training previews on training outcomes. Journal of Management, 21(4),
757-787.
Wildstrom, S.H. (1998). They’re mad as hell out there. Business Week, 3600,
32-33.
Wold, H. (1982). Systems under indirect observation using PLS. In C. Fornell
(Ed.), A second generation of multivariate analysis, volume I: Methods
(pp. 325-347). New York: Praeger.
Wood, R.E., & Bandura, A. (1989). Social cognitive theory of organizational
management. Academy of Management Review, 14, 361-384.
Wood, R.E., & Locke, E.A. (1987). The relation of self-efficacy and grade goals
to academic performance. Educational and Psychological Measurement, 47, 1013-1024.
Yi, M.Y., & Davis, F.D. (2001). Improving computer training effectiveness for
decision technologies: Behavior modeling and retention enhancement.
Decision Sciences, 32(3), 521-544.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

ENDNOTE

This chapter is based on the authors’ prior work that appeared in Journal
of Organizational and End User Computing, 16(2), pp. 20-37.

APPENDIX A:
COMPUTER TASK PERFORMANCE
Problem 1: Present Values, Data Tables, and Charts

A company is considering whether or not to undertake a project. If the
project is undertaken, there will be development costs during the first three years,
and then there will be projected positive cash flows for the next six years. (It is
estimated that after six years, the project will end.)
Development costs can be estimated with relative certainty. The estimated
development costs in Years 1, 2, and 3 are $150, $250, and $100, respectively.
(All dollar figures are in units of millions of dollars; that is, the currency unit is
$1,000,000. All costs are assumed to occur at the end of the year.)
Projected cash flows are relatively uncertain. In particular, the size of the
market for the product is uncertain, so the cash flow at the end of Year 4 (the
year the product will be launched) is uncertain.
Whatever the cash flow turns out to be in Year 4, it is estimated that the
Year-5 cash flow will exceed that in Year 4 by 20%; the cash flow in Year 6 will
exceed that in Year 5 by 15%; the cash flow in Year 7 will equal that in Year
6; the cash flow in Year 8 will be 10% less than that in Year 7; and the cash flow
in Year 9 will be 20% less than that in Year 8. (After Year 8, it is assumed the
project will be at an end.) These estimates are summarized in the following table:
Year
4
5
6
7
8
9

Cash Flow Relative to Preceding Year
Not Applicable
Up by 20%
Up by 15%
No Change
Down by 10%
Down by 20%

All cash flows are realized at the end of the corresponding year. Your goal
is to estimate the present value of the project under alternative assumptions about
the Year-4 cash flow (note that all subsequent cash flows are linked to the Year4 cash flow) and the Discount Rate (i.e., the interest rate). Details are described.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

86 Yi and Im

Part A
Assume that the Year-4 cash flow is $150 currency units, and that the
Discount Rate is 15%. In the Prob 1 worksheet in the exam workbook, where
a starting structure has already been provided, enter the numbers and formulas
needed to compute the projected cash flows and to compute the present value
of the project (Hint: Enter 150 in cell G12, and enter the formula =G12 in cell D9.
Use the NPV function in cell G14).
The beginning of Year 1 is the time point for which the present value is to
be computed.
Part B
Your boss wants to know how the present value of the proposed project
varies with alternative assumptions about the discount rate and the estimated
Year-4 cash flow.
In this regard, complete the two-input data table that has been started on the
Prob 1 worksheet. A series of alternative assumed discount rates and estimated
Year-4 cash flows has already been entered for you on the Prob 1 worksheet.
Part C
Your boss believes that a picture is worth a thousand words. You kind of
believe that yourself.
In any event, you’re not in a position to disagree. In order to keep your boss
happy, you decide to produce a chart based on the information in your two-input
data table from Part B. (Produce the chart directly from the two-input data table;
do not take the time needed to reproduce the numbers in the data table elsewhere
on the worksheet first.)
The chart you produce should match that shown below.
Present Value As a Function of Discount Rate
(The Plotting Param eter is the Assum ed Year 4 Cash Flow )
$100

Present Value

$400

$125

$200
$0
10%
-$200

$150
$175
15%

20%

25%

30%

$200

Discount Rate

Use your two-input data table to produce the chart shown above. Position
the chart beginning in cell C26 on the Prob 1 worksheet.
Pay careful attention to all details regarding such things as the scales on the
horizontal and vertical axes, the chart title (whose font size is 8), and the plot
background, which is “none.”

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

Problem 2: Excel Lists (Internal Databases)

Consider the set of records contained in the Prob 2 worksheet in the exam
workbook. The field names and the first several records in the database are
shown next.

Work Category
9
8
2
8
9
4
6
2
9

Skill Code
1
1
1
4
4
1
3
1
1

Years Experience
8
7
13
4
14
5
7
9
8

Salary
$37,665
$35,965
$35,682
$36,491
$46,717
$29,640
$36,191
$32,063
$38,343

Employee ID
1005
1011
1012
1014
1018
1020
1023
1025
1031

These records are from a temporary employment agency’s workforce
database and consist of fields named Work Category, Skill Code, Years
Experience, Salary, and Employee ID, as shown. These records are to be left
exactly as is on the Prob 2 worksheet. Do not sort them, do not filter them in
place, do not copy them to the clipboard and paste them elsewhere, and so forth.
Part A: Pivot Table Showing Average Employee Salaries
Build a Pivot Table on the Prob 2 worksheet that processes the database to
show average employee salaries by Work Category (row field) and Skill Code
(column field), indexed by Years Experience (page field).
Have the Pivot Table start in cell G2 of the Prob 2 worksheet. Do not include
grand totals for columns and rows in the table. After building the table,
autoformat the cell range G4:L15 in the Accounting 2 style. Be sure the Pivot
Table setting displays average salaries aggregated over All Years Experience.
Part B: List of Selected Employees
On the Prob 2 worksheet, use advanced filtering (Data / Filter / Advanced
Filter) to extract from the database a list of employees whose work-category/
skill-code combinations are as follows:
Work Category
2
3
6
6
8

Skill Code
2
2
3
4
5

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

88 Yi and Im

Caution: As indicated, the advanced filtering approach is to be taken in
building the extracted list. No credit will be given for using approaches other than
that of advanced filtering.
1.
2.
3.
4.

Details:
Locate the criteria range beginning in cell G18 of the Prob 2 worksheet.
(Field names for the criteria range will be in row 18, the first criteria record
will be in row 19, and so forth.)
Position the extracted records beginning in cell G29 of the Prob 2 worksheet.
Row 29 should contain the field names for the extracted records; row 30
should contain the first extracted record, and so forth.
The extracted records should be composed of the following fields in this
left-to-right order:
Skill Code; Work Category; Employee ID; Salary
After extracting the records, sort them in descending order on Skill Code.
Break Skill Code ties by sorting on Work Category in descending order.
Break Work Category ties by sorting on Salary in ascending order.

Problem 3: Macro Programming

Your goal is to create a form to sear a list of names. Include two buttons on
your form—Search and Cancel. The user provides the input of a last name. After
specifying the last name, when the user clicks on Search, your VBA-based
macro should search the list provided to you in Sheet Prob3 and print the person’s
first name, phone number, and office location in three separate labels on the same
form. Your search should work, irrespective of case. If no match is found, the
dialog box should say, “No match found.” Assume there are no duplicate entries
of last name. Cancel should close the form. You are not required to create any
custom menu items to make this form appear.
Hint: Use the function UCase() in Visual Basic to make your search case
insensitive.

APPENDIX B: PRIOR EXPERIENCE TEST

Using Excel, complete the following tasks. You have 30 minutes for the test.

1.
2.
3.

Enter a formula to compute profits (=sales - expenses) for each season in
cells B8:E8.
Using an appropriate function, compute the total amounts of sales, expenses, and profits. The computed amounts should be located in cells F6:F8.
Compute YTD (year to date) profits. The computed amounts should be
located in cells B9:E9.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Predicting Computer Task Performance

Calculate percent change of sales from the previous season. The computed
amounts should be located in cells C11:E11. The percent change of sales
is computed as current sales – previous sales / previous sales.
5. Change the spring sales amount in cell B6 from 320 to 390. Verify that the
numbers related to this cell have been updated correctly.
6. Using the MAX function, find the largest amounts of sales, expenses, and
profits. The computed amounts should be located in cells H6:H8.
7. Copy the format of the cell E5 to F5, G5, and H5.
8. Format all the numbers in the 6th row so that the numbers are displayed with
dollar signs, commas (when the numbers are greater than or equal to one
thousand), and two decimal places. Make sure all the numbers are
readable.
9. Format all the numbers in the 7 th, 8th, and 9th rows so that the numbers are
displayed with commas (when the numbers are greater than or equal to
1,000) and two decimal places. Make sure all the numbers are readable.
10. Format all the percent sales change numbers in the 11th row so that the
numbers are displayed with % symbols and one decimal place.
11. If there are any negative numbers in the worksheet, change their colors to
red.
12. Align the title “ABC Corporation” to the center of the screen. Change the
font of the title to Times New Roman.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

90 Morris and Marshall

Chapter V

Measurement of
Perceived Control in
Information Systems
Steven A. Morris, Middle Tennessee State University, USA
Thomas E. Marshall, Auburn University, USA

ABSTRACT

The importance of perceptions of control in explaining human behavior
and motivation has been identified, investigated, and found to be significant
in several disciplines. This study reports on an exploratory investigation
assessing perceived control within the information systems domain. A
survey instrument was developed based on the research literature to assess
perceived control as a multi-dimensional construct. The survey was
administered to 241 subjects. The results were analyzed to produce the
following five factors that represent a user’s perceptions of control when
working with an interactive information system: (1) timeframe, (2) feedback
signal, (3) feedback duration, (4) strategy, and (5) metaphor knowledge.

INTRODUCTION

While one of the ongoing efforts in information systems (IS) research is an
attempt to define the dependent variable in concrete terms, various attempts
have produced some widely accepted surrogates (DeLone & McLean, 1992;
Keen, 1980). Among these are user satisfaction and system usage, and much

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Measurement of Perceived Control in Information Systems

effort has been spent attempting to better understand these concepts (Bailey &
Pearson, 1983; Baroudi, Olson, & Ives, 1986; Hu, Chau, Sheng, & Tam, 1999;
Igbaria, & Nachman, 1990; Straub, Limayem, & Karahanna-Evaristo, 1995).
Though not well established in the IS literature, researchers in other disciplines
have linked perceived control to both emotional and behavioral characteristics,
such as user satisfaction, that are of interest to IS researchers (Stanton &
Barnes-Farrell, 1996). Most research in information systems that has included
user perceptions of control has not addressed the perceived control construct
from firm theoretical grounds; has not addressed perceived control as a complex,
multi-dimensional construct; and/or has not demonstrated consistency in definition or measurement to facilitate comparisons across studies (Kahai, Solieri, &
Felo, 1998; Sengupta & Te’eni, 1993). While this research has found perceived
control to be related to important constructs such as user satisfaction and task
performance, the cumulative impact of these studies is weakened by the
difficulties in making comparisons (Kahai, Solieri, & Felo, 1998; Stanton &
Barnes-Farrell, 1996). The purpose of the current research is to develop an
instrument for assessing perceived control in the information systems domain as
a multi-dimensional construct from a theoretical basis that can be used as a
common assessment tool, thereby facilitating future cross-study comparisons.
A review of the relevant literature that informed the creation of the
instrument appears in the next section. Following the review of the literature, an
explanation of the methodology used to create the instrument is provided, along
with a description of the methods used to collect data to refine the instrument.
The results of principal components factor analysis are then presented to explore
the dimensionality of the perceived control construct. These results and the
emergent factors are then discussed. Finally, remarks concerning the implications and limitations of the study are presented.

LITERATURE REVIEW

The importance that other disciplines have given perceived control is
highlighted by the proposal of Friedman and Lackey (1991) that control is the
universal motivator for all human activity. An individual’s perception of the
control that he or she can exert has been found to be a very strong predictor of
both behaviors and emotional outcomes and, therefore, has stimulated a great
deal of research in disciplines such as psychology, marketing, and organizational
behavior (Fox, Dwyer, & Ganster, 1993; Friedman & Lackey, 1991; Lacey,
1979; Robertson & Powers, 1990; Sargent & Terry, 1998; Skinner, 1995).
Perceived control can be viewed as the degree to which a person feels that he
or she can impact outcomes in his or her environment through voluntary actions
(Lacey, 1979). According to Skinner (1995), “Five decades of research have
established [perceived control] as a robust predictor of people’s behavior,
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

92 Morris and Marshall

emotion, motivation, performance, and success and failure in many domains of
life” (p. 3). Within these reference disciplines, control has been addressed as a
multi-dimensional construct.
Averill (1973) noted three dimensions of the control construct—cognitive
control, behavioral control, and decisional control. Decisional control addresses
the ability to choose among different courses of action. Cognitive control
addresses the interpretation of an event into a cognitive model or plan. Behavioral control deals with the existence of some means to exert influence over an
event. Behavioral control has been addressed in numerous studies involving
Azjen’s (1991) theory of planned behavior (Cordano & Frieze, 2000; Flannery
& May, 2000; Morris & Venkatesh, 2000; Venkatesh, Morris & Ackerman,
2000). Karasek (1979) investigated decisional control as decision authority and
decision latitude (Karasek & Theorell, 1991; Schaubroeck, Xie, & Lam, 2000).
Various scales to assess this perspective of control as a unidimensional construct
and as a multi-dimensional construct have been developed (Smith, Tisak, Hahn,
& Schmieder, 1997). Cognitive control has been investigated with less frequency, perhaps due to the difficulty in assessing this aspect of control (Faranda,
2001). Averill (1973) identified two facets of cognitive control—information
gathering and appraisal. Faranda (2001) used a multiple study approach to
develop a scale that would assess both of these facets as a unidimensional
measure of cognitive control and found support for the perspective of cognitive
control as a unidimensional construct. Figure 1 illustrates the three facets of
perceived control and indicates examples of the research that have addressed
each facet.
Within the field of experimental social psychology, control has been
suggested as the foundation of a new paradigm for understanding human
behavior. Historically, the Skinnerian paradigm for understanding behavior in
terms of stimulus–response has been the underpinning for much of the
behavioral research (Skinner, 1971). The new feedback–control paradigm
advocated by researchers such as Robertson and Powers (1990) shifts the
perspective of human behavior from being externally motivated to being internally motivated. Figure 2 depicts this perspective of human behavior. This
Figure 1. Facets of perceived control and sample research

Perceived
Control

Cognitive
Control

Sengupta & Te’eni (1993)
Faranda (2001)

Decisional
Control

Karasek (1979)
Schaubroeck, et al (2000)

Behavioral
Control

Azjen (1991)
Cordano & Frieze (2000)

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Measurement of Perceived Control in Information Systems

Figure 2. Illustration of feedback-contol paradigm
Environment

Reference Point:
Ideal Environment

Feedback

Comparison of
Environment to Ideal
Change
Ideal

Change
Environment
Cognitive
Control

Perceived Control

Decisional
Control

Behavioral
Control

Action

paradigm for explaining behavior posits that an individual sets an internal
reference point that represents the ideal state of his or her environment. Through
interaction with the environment, he or she receives feedback on the actual state
of the environment. A comparison is made of the feedback with the reference
point. If the environment varies significantly from the reference point, the
individual will assess his or her ability to change the environment in order to bring
it into alignment with the reference point. Based on the individual’s perceptions
of his or her ability to exercise control, the individual will take action either to
change the environment, change the reference point, or both. This perception of
control is a function of cognitive control, decisional control, and behavioral
control, as previously defined. This sequence of events iterates until either the
environment is aligned with the reference point (through changes in the environment, the reference point, or both), or the individual deems action to be futile,
potentially resulting in negative cognitive, emotional, or behavioral consequences
for the individual (Seligman, 1975).
Little has been done, however, to integrate the control aspect of human
behavior into the information systems arena. Nord and Nord (1994), in an
investigation of end-user computing (EUC), cite direct control of the system as
one of the benefits of EUC, yet no attempt to measure that control is made.
Williams and Nelson (1990) report frustration in users of decision support
systems (DSS) within petroleum companies when the users lack control of
certain system variables; however, no measure of the user’s perceived degree
of control is taken. Sengupta and Te’eni (1993) investigated the effects of
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

94 Morris and Marshall

cognitive feedback in a group decision support system (GDSS) on cognitive
control and strategy convergence. They found that cognitive feedback increases
cognitive control, and, over time, cognitive feedback produces a uniformly high
level of cognitive control. Control in the investigation was limited to “the extent
to which the decision maker controls the execution of his or her decision
strategy” (Sengupta & Te’eni, 1993, p. 90). While the execution of decision
strategy is certainly an important dimension of control, other dimensions such as
the internal, cognitive aspects of control are absent in their study. Luconi,
Malone, and Scott-Morton (1986) implicitly included control in their framework,
delineating types of information systems. Their framework uses individual
versus computer control of system activity to distinguish between types of
information systems, but they fail to provide an in-depth analysis of the construct
of user’s perceived control.
The only attempt at integrating the various aspects of control into the
information systems domain is provided by Frese (1987), who provides conceptual discourse on the aspects of control from an information systems perspective.
The model produced by Frese (1987) is grounded theoretically in the cumulative
development of the control construct and represents an extension of the
psychological view of control as developed by Glass and Singer (1972), Seligman
(1975), and Frese (1982) that has been applied to the information systems
context. The model posits factors, both internal and external, that influence the
ability to exert control. These factors, described below, correspond with the
aspects of behavioral, decisional, and cognitive control. Behavioral control
represents the individual’s belief that he or she is free to take actions that will
influence the system. Behavioral control is addressed through the factors that
deal with sequence and timeframe for completing tasks. These factors deal
directly with the individual’s ability to execute actions to change his or her
environment. Decisional control is the ability to make plans and decisions that will
lead to effective action to change the environment. In the Frese model, decisional
control is addressed by factors that deal with the individual’s ability to set goal
content and make decisions about the sequence of attempting goal fulfillment.
Cognitive control is the individual’s ability to interpret the functioning of the
system within his or her current understanding of the way the system works.
Within the Frese model, cognitive control is addressed by internal factors such
as knowledge, which considers the individual’s ability to conceptualize the
functioning of the system within his or her mental model of the system. The Frese
(1987) model combines these aspects into a single framework and addresses a
person’s degree of control that he or she can impose on an information system.
Frese (1987) postulates that there are both internal and external prerequisites for control. External prerequisites describe attributes of the system and the
environment that are external to the user but may impact the user’s ability to
influence the system in the desired manner. Ten external factors were identified
(Frese, 1987) as follows:
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Measurement of Perceived Control in Information Systems

Goal Content
Goal Sequence

(GC)
(GS)

Goal Timeframe
Task Content
Task Sequence

(GT)
(TC)
(TS)

Task
Timeframe
Feedback Content
Feedback
Sequence
Feedback
Timeframe
Feedback
Condition

(TT)
(FC)
(FS)
(FT)
(Fcn)

The ability to determine one’s own goals while using the system
The ability to determine the order in which multiple goals are
satisfied
The ability to determine the length of time a goal could be pursued
The ability to formulate one’s own plan for achieving a goal.
The ability to determine the order in which multiple plans to
achieve a goal are attempted
The ability to determine the length of time a plan to achieve a goal
could be pursued
The ability to determine the type of feedback the system provides
The ability to determine the order in which multiple feedbacks
may be received
The ability to determine the duration of a feedback signal
The ability to determine the conditions for which feedback is
received

Internal prerequisites describe the attributes of the user that are necessary
for the user to be able to influence the system in the desired manner. Internal
prerequisites were deemed to be of two types: skills and knowledge. Skills refer
to the ability of the user to perform effectively and efficiently. Five different skills
are identified, as follows:

Goal Realistic
Goal Stable
Task Realistic

(GR)
(GST)
(TR)

Task Flexible

(TF)

Task Organized

(TO)

The goal for which the user strives must be attainable.
The goal must endure even in the face of negative feedback.
The plan for achieving the goal must be capable of achieving the
goal.
The plan for achieving the goal must be flexible enough to
accommodate a changing environment.
Commonly performed tasks should be routinized in the user’s
mind.

Knowledge “refers to metaphors (Carroll & Thomas, 1982),
conceptualizations or mental models of the system (Rouse & Morris, 1986;
Norman, 1983)” (Frese, 1987, p. 318). Knowledge is the user’s understanding
of the system’s functioning. It is deemed an internal prerequisite to control since
the user must have sufficient knowledge of the system to make the system
respond in a predictable manner. Knowledge is defined for this study as follows:
Knowledge (K): The user’s understanding of the functioning of the system,
either directly or through analogy.
Extending the Frese (1987) research stream, this study operationalizes and
tests these factors through factor analysis determining a factor model. Items for
these 16 factors were identified, tested, and integrated to produce a research
instrument.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

96 Morris and Marshall

METHODOLOGY

Following the guidelines of Sethi and King (1991), an instrument to assess
perceived control was created. First, the identification of dimensions of the
construct was conducted through review of the previous literature, as previously
described. This review resulted in the use of the 16 factors from the Frese model,
since it contained the only integrated perspective of control that included all of
the dimensions suggested by the other literature. A panel of IS researchers
critiqued the dimensions, assessing the content validity of the instrument. Several
rounds of questionnaire feedback and revision were performed until consensus
agreement among the panelists was achieved. Second, based on the dimensions
identified, multiple questionnaire items were generated to operationalize each
dimension (Sethi & King, 1991). The same panel also pre-tested the questionnaire items to address issues of readability and clarity. Although the lack of a
formal pilot test may be a limitation, Boudreau, Gefen, and Straub (2001)
indicated that a pre-test can perform much the same function as the pilot test and
is the more important of the two.
Operationalizing the 16 factors influencing control of an information system
suggested by the literature and the expert panel produced a 55-item questionnaire
(see Appendix A). The correlation of questionnaire items to research model
factors is detailed in Appendix B. The questionnaire asked a series of 55
questions concerning the subject’s experiences using a particular information
system. Item responses were reported on a five-point Likert scale, with response
cues for questions ranging from “strongly disagree” to “strongly agree.” The
subjects were 241 undergraduate students taking an introductory management
course. Bonus credit was offered to the subjects as an incentive for participation
in the study. The information system targeted in the questionnaire was the
university’s automated registration system. For this system, the subjects of the
survey are the intended end users of the system, effectively eliminating problems
associated with subject surrogates.
Ideally, many researchers recommend a ratio of 5:1 subjects per questionnaire item (Hair, Anderson, Tatham, & Black, 1998). However, other researchers note that for exploratory factor analysis, much lower ratios are acceptable
(Essex, Magal, & Masteller, 1998; Templeton, Lewis, & Snyder, 2002; Yoon,
Guimaraes & O’Neal, 1995). Cattell (1998) suggested that a ratio of 2:1 is
permissible for exploratory work, and Baggaley (1982) goes so far as to suggest
that even a 1:1 ratio can be informative. Therefore, the current ratio of 4.38:1,
while less than ideal, is within the realm of acceptability for exploratory factor
analysis.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Measurement of Perceived Control in Information Systems

RESULTS

Instrument validity was addressed along multiple dimensions as suggested
by Boudreau, Gefen, and Straub (2001). Content validity, ensuring that the
instrument is representative of the pool of potential content, was addressed by
a combination of a thorough review of the literature and using an expert panel
(previously described), as recommended by Straub and Carlson (1989). Construct validity was addressed through the use of a principal components factor
analysis (Cattell, 1988). The results of the factor analysis, described later,
indicated a number of factors. Reliability was assessed using Cronbach’s alpha
for each of the factors and will be presented with the discussion of the respective
factors.
The responses were statistically analyzed to obtain a reduced set of factors
using exploratory principal components factor analysis. The data were examined
using both a varimax and a promax (oblique) rotation. Since the oblique rotation
produced the same factors as the varimax rotation, and the component correlation matrix indicated only modest correlation, the data were assumed to be
orthogonal (Bernstein, 1988). The remaining discussion will address only the
varimax rotation, since it tends to produce a simplified, interpretable factor
structure (Katchigan, 1982; Kim & Mueller, 1982; Sethi & King, 1991). Criteria
for selecting factors were utilized to eliminate weak or implausible factors. In
order to be retained, a factor must have an eigenvalue of 1.00 or greater.
Additionally, for an item to be considered as part of a factor, it must have a
minimum loading of 0.50 in its primary factor and not load higher than 0.30 in any
other factor. This rigorous standard was suggested by the literature and
produced conceptually sound factors with a clean separation of items (Cattell,
1988; Templeton, et al., 2002; Yoon, et al., 1995). Appendix C provides a list of
the survey question numbers (Number), proposed factor from the literature
(Item), and principal factor loading.
While the literature used to develop this instrument was based on 16
hypothesized factors (Frese, 1987), the factor analysis yielded 17 factors with
convergence in 31 iterations. Further, these 17 factors in all cases did not parallel
the hypothesized factors. A multiple criteria approach, including eigenvalue,
interpretability, and Scree-test, was used to evaluate and identify relevant
research factors within the data set (Ford et al., 1986; Cattell, 1988). In general,
the approach identifies the first factors that account for most of the observed
variance while eliminating non-relevant factors by using a multiple-criteria
evaluation. According to Ford, et al. (1986), the multiple criteria approach is
advised, because considering eigenvalues alone tends to overestimate the
relevant number of factors. Cattell (1988) recommends evaluation of the scree
plot of eigenvalues in addition to consideration of the eigenvalues themselves.
Evaluation of the scree plot is used to identify error factors by eliminating factors

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

98 Morris and Marshall

that fall off in a regular, linear pattern (Cattell, 1988). In addition to consideration
of the scree plot of the eigenvalues, a factor’s interpretability and contribution
to variance suggested that the optimal number of factors to be considered in this
study included the first five factors (Table 1).
The factors and the item loadings are detailed in Appendix C. The first
factor in the derived model is composed of a combination of items dealing with
goal timeframe and task timeframe (Table 2). This factor implies that respondents did not distinguish between the time spent pursuing a goal and the time
spent on various plans to achieve that goal. The combination of these items into
a single factor indicates that limitations on total working time on the system are
more important than the distinction of how that time is spent. This may be
explained by the view that developing a plan to achieve a goal is often a process
of decomposing a goal into a series of subgoals. These subgoals, in turn, then can
be decomposed further into smaller subgoals (Lesgold, 1988). From this perspective, executing a plan is achieving a sequence of goals; therefore, the plan
becomes a goal itself. This factor—timeframe—accounted for more than 15%
of the total variance observed.
Table 1. Factors considered for retention
Factor

Item Numbers

Eigenvalue

1
2
3
4
5

4, 19, 10, 23, 9
15, 16, 14, 24
21, 17
7, 6
28, 30, 54

8.33
3.40
2.92
2.44
1.97

Percent of
Variance
15.2
6.2
5.3
4.4
3.6

α
.8561
.7513
.5632
.6332
.7080

Table 2. Factor one: Goal and task timeframe
Item
Loading
.821
.803
.801
.727

Measurement Item

The system allows me to work on completing a task for as long as I want.
I may pursue an objective on the system for as long as I want.
There are no time restraints on my usage of the system.
The system does not limit the amount of time I can spend working on a method of
accomplishing my objectives.
.720
The system will let me pursue a plan to achieve a goal for as long as I want.
Cronbach’s Alpha for Factor 1 = .8561

Table 3. Factor two: Content and sequence feedback
Item Loading
.786
.773
.621

Measurement Item
I can choose the order in which I receive multiple feedback signals from the system.
The system allows me to choose the feedback signal that it uses.
I can select the method of feedback that the system uses to notify me of the
success or failure of the operations I perform on the system.
.531
When multiple potential feedback signals exist, the system allows me to decide the
order in which I will receive those signals.
Cronbach’s Alpha for Factor 2 = .7513

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Measurement of Perceived Control in Information Systems

Table 4. Factor three: Timeframe or duration of feedback
Item Loading
.660

Measurement Item
The feedback signal from the system, which notifies me of the
success or failure of the operations I perform on the system, lasts
only as long as I want it to.
.617
The system lets me choose how long the feedback notification
lasts when I receive feedback from the system.
Cronbach’s Alpha for Factor 3 = .5632

Table 5. Factor four: Goal and task content
Item Loading
.735
.660

Measurement Item
The system offers many choices of tasks to accomplish.
The system lets me make many decisions about the way I will achieve my
goals.
Cronbach’s Alpha for Factor 4 = .6632

Table 6. Factor five: Metaphor knowledge
Item
Loading
.799
.780

Measurement Item

I have used other systems that are fundamentally the same as this one.
The system seems to work similarly to other systems with which I am
familiar.
.680
There are parallels between this system and other systems I have operated.
Cronbach’s Alpha for Factor 5 = .7080

The second factor, detailed in Table 3, is a combination of feedback content
and feedback sequence. This combination of items implies that subjects did not
distinguish between the content of a feedback signal and the order in which
signals are received. The focus supports the identification of the feedback signal
itself as an important dimension of perceived control. This factor accounted for
more than 6% of the total variance observed.
The third factor is composed of items related to feedback timeframe or
feedback duration (Table 4). Feedback, as a factor of control, deals with the
timeframe or duration of the feedback signal that the subject receives to indicate
success or failure while using the system. This factor parallels one of the factors
that was hypothesized in the literature. This factor accounted for more than 5%
of the total variance observed.
The fourth factor, as presented in Table 5, represents a combination of goal
content and task content. The combination of goal and task content items is a
further indication that subjects did not distinguish between a goal and a plan to
achieve that goal. This supports the assertion made earlier that the decomposition
of goals into subgoals blurs the distinction between goals and plans. The thrust
of this factor appears to be work content, or the subject’s ability to manipulate
the system as desired, regardless of the hierarchical level of the goal that is to
be accomplished by that manipulation.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

100 Morris and Marshall

The fifth factor, as detailed in Table 6, is the initial factor dealing with the
internal prerequisites to control. The four preceding factors, with the greatest
explanative power, all deal with factors that were identified as being external
prerequisites. The items included in the fifth factor belong to the hypothesized
factor, knowledge. As noted earlier, however, knowledge is defined as being
composed of metaphors and mental models. Metaphor, in this case, refers to the
comparability of the current system to other systems with which the user has had
experience (Staggers & Norcio, 1993). Mental models are the cognitive constructs the subject has developed to understand the system functions. The items
that deal with the metaphor aspect of knowledge were clustered as the fifth
factor, suggesting that in terms of their degree of control of an information
system, individuals in this study use metaphor knowledge to achieve cognitive
control.

DISCUSSION

The current study has identified five factors that represent a subject’s
perceived degree of control when working with an interactive information
system. Although the factors hypothesized in the literature did not all hold in this
case, many hypothesized factors were supported. There was a trend in several
of the identified factors to aggregate goal- and task-oriented items that deal with
the external prerequisites for control, or the factors in the environment that
influence the user’s ability to impact the system. Viewing these factors in terms
of the feedback-control paradigm, which was previously discussed and presented graphically in Figure 2, supports an explanation of user’s perceived
degree of control and allows the following conclusions to be drawn.
Accepting control as the general motive, as proposed by Friedman and
Lackey (1991), a user motivated by a desire to control his or her environment will
set a reference point (Figure 2). This reference point represents the ideal
environment in which this user would prefer to exist. When the user receives
feedback that something in the environment is not in accord with the reference
point, the user will attempt to manipulate, or control, the environment to bring the
environment back in line with the reference point. Since the general motive is
control, and our paradigm dictates that all behavior is an effort to gain or maintain
control, then the goal of all activity, including information system usage, is control
(Robertson & Powers, 1990). This would explain the noted trend of users to view
external goals and tasks in a similar manner. The reason for this is that if the goal
of all activity is control, then any goals the user would set in regard to the external
environment (the system) are, in fact, tasks, or plans, for achieving the true,
internal goal—control. The findings of the current study help to indicate the
factors that the user believes are important to the success of this control activity.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Measurement of Perceived Control in Information Systems

101

Factor 1—timeframe—indicates that the user believes that it is important to
have sufficient time to develop a plan and to enact the required change on the
environment. This is consistent with the findings of other researchers who have
noted a relationship between time pressure and control (Doef & Maes, 1999;
Olson & Sarter, 2000; Yagil, 2001). For users to perceive that they have control
of the system, it must allow them enough time to create a plan for reducing the
deviation (decisional control). After a plan for returning the environment to the
reference point has been devised, the user requires sufficient time to perform the
activity (behavioral control). Additionally, if multiple plans are required, the user
needs time to create and to implement various plans until success is achieved.
From a temporal perspective, users do not seem to distinguish between decisional
and behavioral control, which means that once a variation from the reference
point is detected (factor two), the user needs adequate time to manipulate the
system so that the variation can be reduced to an acceptable level.
Factor 2—feedback signal—would be necessary to detect that the environment has deviated from the reference point, indicating that activity is necessary.
Additionally, the signal would be needed to determine when the environment has
been successfully manipulated, indicating that activity is no longer necessary.
Therefore, the signal prompts the user into action, and a feedback signal is
needed so that the user can judge the success of his or her activities. User
concerns with the feedback signal indicate that the signal must have sufficient
meaning to the user to detect and gauge any deviation. Related to this is factor
3—feedback duration. While factor 2 implies that the signal must be meaningful,
factor 3 suggests that the duration of the feedback must be sufficient for it to be
properly received and interpreted.
Factor 4 deals with the actual action of the user aimed at reducing the
deviation of the environment from the reference point. Once the deviation has
been detected, the user needs the ability to manipulate the environment in order
to reduce the deviation. The user must devise a means (strategy) of bringing the
environment back to the reference point (decisional control). Decision latitude
is constrained by the limitations of the system functions. Within the constraints
of the system’s capabilities, the user makes decisions concerning the means to
be employed in achieving the goal.
While factor 4 indicates that the manipulation options offered by the system
provide limitations to developing a means of achieving goals, other factors may
impact this, as well. Internally, factor 5—metaphor knowledge—helps the user
to determine the content of the activity. Cognitive control is the ability of the user
to interpret the functions of the system within his or her mental model of the
system. It is the user’s understanding of the causal relationships among the
various system components and the user’s actions. Factor 5 indicates that the
user’s ability to apply an understanding of other systems to the functioning of the
current system provides a greater sense of control over the current system. If

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

102 Morris and Marshall

the deviation has been experienced previously, then routinization may handle the
determination of activity content without much conscious effort. If the deviation
is a routine occurrence, then the user will draw on his or her experiences with
similar systems (metaphors) to help determine the content of the activity.

CONCLUSION

The current study contributes to the understanding of perceived control in
information systems. It developed and empirically tested a questionnaire based
on the literature to assess users’ perceptions of control of an information system.
The results of an exploratory factor analysis of responses to this survey were
then expressed in terms of the feedback-control paradigm. Based on these
results, the questionnaire was reduced to 16 items.
Many of the findings are consistent with previous research on perceived
control in other disciplines. In comparison to the Averill (1973) operationalizations
of control, the current findings clearly support the factors of cognitive and
decisional control. Factor 5 of the current study deals with metaphor knowledge.
This directly assesses the ability of the user to conceptualize the functioning of
the system and to integrate those functions within the user’s mental model of the
system. Factor 4 of the current study deals with the decision latitude of users in
regard to their ability to decide on a strategy for attempting to achieve their goals.
This clearly supports the work done by Karasek (1979) in decisional control.
The issue of behavioral control, however, is not as clearly supported. Factor
1 of the current study appears to relate to behavioral control as well as decisional
control. It appears that the ability to make decisions about the actions to perform
(decisional control) and the ability to take those actions (behavioral control) are
intertwined. This is not very surprising, since most users would not make
decisions to carry out an action that they knew they could not complete. Factor
1 deals with the issue of having sufficient time to make decisions and then
attempting to enact those decisions.
Factors 2 and 3 both deal with the feedback signal that the user receives
from the system. While these factors do not fall within the Averill operationizations
of control, other researchers have recognized that the feedback signal is an
integral component of any form of control (Frese, 1987; Robertson & Powers,
1990). Factor 2 deals with the content of the feedback signal and indicates that
the signal must contain sufficient information in order for the user to recognize
the deviation of the environment from the reference point. Factor 3 indicates that
the duration of the feedback signal must be sufficient in order for the user to
properly interpret it. With respect to perceived control of an information system,
the user must receive sufficient feedback to make a valid comparison of the
environment to the ideal (factor 2). The feedback must last long enough for the
user to conceptualize its meaning so that a proper strategy can be formulated

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Measurement of Perceived Control in Information Systems

103

(factor 3). The user must have sufficient latitude to devise a strategy that can
manipulate the system to achieve the user’s goal (factor 4) within the constraints
of the user’s understanding of the system (factor 5). Finally, the user must have
sufficient time to go through iterations of devising and implementing multiple
strategies to achieve success (factor 1).
Given the equivocal results of previous attempts to determine the nature of
control as unidimensional or multi-dimensional (Smith et al., 1997), the current
study contributes not only to the specific investigation of control in the information systems area, but also to the greater investigation of perceived control in
other disciplines. While this study is a theoretically based, empirical investigation
into the facets of perceived control, further empirical research within a variety
of information system contexts is recommended. Limitations of the current
study, such as the ratio of subjects to variables and the low reliability scores for
factors 3 and 4, are acceptable in exploratory research. However, confirmatory
research that holds these factors to more rigorous standards is necessary to
refine the instrument and improve the understanding of perceived control.
Further, research into the interactions between cognitive, behavioral, and
decisional control is critical to the understanding of control. Given a greater
understanding of perceived control, information system design, development and
implementation can be more effectively focused on the intrinsic needs of the
intended user population and its perception of perceived control.

REFERENCES

Averill, J.R. (1973). Personal control over aversive stimuli and its relationship to
stress. Psychological Bulletin, 80, 286-303.
Azjen, I. (1991). Theory of planned behavior. Organizational Behavior and
Human Decision Process, 50, 179-211.
Baggaley, A.R. (1982). Deciding on the ratio of the number of subjects to the
number of variables in factor analysis. Multivariate Experimental Clinical Research, 6(2), 81-85.
Bailey, J.E., & Pearson, S.W. (1983). Development of a tool for measuring and
analyzing computer user satisfaction. Management Science, 29(5), 530-545.
Baroudi, J.J., Olson, M.H., & Ives, B. (1986). Empirical study of the impact of
user involvement on system usage and information satisfaction. Communications of the ACM, 29(3), 232-238.
Bernstein, I.H. (1988). Applied multivariate analysis. New York: Springer-Verlag.
Boudreau, M.C., Gefen, D., & Straub, D.W. (2001). Validation in information
systems research: A state-of-the-art assessment. MIS Quarterly, 25(1), 1-16.
Carroll, J.M., & Thomas, J.C. (1982). Metaphor and the cognitive representation
of computing systems. IEEE Transactions on Systems, Man, and Cybernetics, 12, 1007-1116.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

104 Morris and Marshall

Cattell, R.B. (1988). Meaning and strategic use of factor analysis. In J.R.
Nesselroade & R.B. Cattell (Eds.), Handbook of multivariate experimental psychology (2nd ed.). New York: Plenum Press.
Cordano, M., & Frieze, I.H. (2000). Pollution reduction preferences of U.S.
environmental managers: Applying Azjen’s theory of planned behavior.
Academy of Management Journal, 43(4), 627-641.
DeLone, W. & McLean, E. (1992). Information systems success: Quest for the
dependent variable. Information Systems Research, 3(1), 60-95.
Essex, P., Magel, S.R., & Mosteller, D.E. (1998). Determinants of information
center success. Journal of Management Information Systems, 15(2),
95-117.
Faranda, W.T. (2001). A scale to measure the cognitive control form of
perceived control: Construction and preliminary assessment. Psychology
& Marketing, 18(12), 1259-1281.
Flannery, B.L., & May, D.R. (2000). Environmental ethical decision making in
the U.S. metal-finishing industry. Academy of Management Journal,
43(4), 642-662.
Ford, J.K., MacCallum, R.C., & Tait, M. (1986). The application of exploratory
factor analysis in applied psychology: A critical review and analysis.
Personnel Psychology, 39, 291-314.
Fox, M.L., Dwyer, D.J., & Ganster, D.C. (1993). Effects of stressful job
demands and control on physiological and attitudinal outcomes in a hospital
setting. Academy of Management Journal, 36(2), 289-318.
Frese, M. (1982). Occupational socialization and psychological development: An
underemphasized research perspective in industrial psychology. Journal
of Occupational Psychology, 55, 209-224.
Frese, M. (1987). Theory of control and complexity: Implications for software
design and integration of computer systems into the work place. In M. Frese,
E. Ulich, & W. Dzida (Eds.), Psychological issues of human computer
interaction in the work place. Holland: Elsevier Science Publishers.
Friedman, M.I., & Lackey, G.H., Jr. (1991). Psychology of human control: A
general theory of purposeful behavior. New York: Praeger Publishers.
Glass, D.C., & Singer, J.E. (1972). Experiments on noise and social stressors.
New York: Academic.
Hair, J.F., Anderson, R.E., Tatham, R.L., & Black, W. (1998). Multivariate
data analysis (5 th ed.). Upper Saddle River, New Jersey: Prentice Hall.
Hu, P.J., Chau, P.Y.K., Sheng, O.R.L., & Tam, K.Y. (1999). Examining the
technology acceptance model using physician acceptance of telemedicine
technology. Journal of Management Information Systems, 16(2), 91112.
Igbaria, M., & Nachman, S.A. (1990). Correlates of user satisfaction with end
user computing. Information & Management, 19(2), 73-82.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Measurement of Perceived Control in Information Systems

105

Kachigan, S.K. (1982). Multivariate statistical analysis. New York: Radius Press.
Kahai, S.S., Solieri, S.A., & Felo, A.J. (1998). Active involvement, familiarity,
framing, and the illusion of control during decision support system use.
Decision Support Systems, 23(2), 133-148.
Karasek, R.A. (1979). Job demands, job decision latitude, and mental strain:
Implications for job redesign. Administrative Science Quarterly, 24, 285308.
Karasek, R. A., & Theorell, T. (1991). Healthy work: Stress, productivity and
the reconstruction of working life. New York: Basic Books.
Keen, P.G.W. (1980). MIS research: reference disciplines and a cumulative
tradition. Proceedings of the First International Conference on Information Systems. Philadelphia, Pennsylvania.
Kim, J.O., & Mueller, C.W. (1982). Introduction to factor analysis. Beverly
Hills: Sage Press.
Lacey, H.M. (1979). Control, perceived control, and the methodological role of
cognitive constructs. In L.C. Perlmuter, & R.A. Monty (Eds.), Choice and
perceived control. Hillsdale, NJ: Lawrence Erlbaum Associates.
Lesgold, A. (1988). Problem solving. In R.J. Sternberg, & E.E. Smith (Eds.),
Psychology of human thought. New York: Cambridge Press.
Luconi, F.K., Malone, T.W., & Scott Morton, M. (1986). Expert systems: The
next challenge for managers. Sloan Management Review, 27(4), 3-15.
Morris, M.G., & Venkatesh, V. (2000). Age differences in technology adoption
decisions: Implications for a changing work force. Personnel Psychology,
53(2), 375-403.
Nord, G.D., & Nord, J.H. (1994). Perceptions & attitudes of end-users on
technology issues. Journal of Systems Management, 45(11), 12-15.
Norman, D. A. (1983). Some observations on mental models. In D. Gentner, &
A.L. Stevens (Eds.), Mental models. Hillsdale, NJ: Lawrence Erlbaum
Associates.
Olson, W.A., & Sarter, N.B. (2000). Automation management strategies: Pilot
preferences and operational experiences. International Journal of Aviation Psychology, 10(4), 327-341.
Robertson, R.J., & Powers, W.T. (1990). Introduction to modern psychology:
The control-theory view. Lexington, KY: Diamond Graphics.
Rouse, W.B., & Morris, N.M. (1986). On looking into the black box: Prospects and
limits in the search for mental models. Psychological Bulletin, 100, 349-363.
Sargent, L. D., & Terry, D.J. (1998). The effects of work control and job
demands on employee adjustment and work performance. Journal of
Occupational and Organizational Psychology, 71, 219-236.
Schaubroeck, J., Xie, J.L., & Lam, S.K. (2000). Collective efficacy versus selfefficacy in coping responses to stressors and control: A cross-cultural
study. Journal of Applied Psychology, 85(4), 512-525.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

106 Morris and Marshall

Seligman, M.E.P. (1975). Helplessness: On depression, development and
death. San Francisco: Freeman.
Sengupta, K., & Te’eni, D. (1993). Cognitive feedback in GDSS: Improving
control and convergence. MIS Quarterly, 17(1), 87-109.
Sethi, V., & King, W.R. (1991). Construct measurement in information systems
research: An illustration in strategic systems. Decision Sciences, 22(3),
455-472.
Skinner, B.F. (1971). Beyond freedom and dignity. New York: Knopf.
Skinner, E.A. (1995). Perceived control, motivation, & coping. Thousand
Oaks, CA: Sage Publications.
Smith, C.S., Tisak, J., Hahn, S.E., & Schmieder, R.A. (1997). The measurement
of job control. Journal of Organizational Behavior, 18(3), 225-237.
Staggers, N., & Norcio, A.F. (1993). Mental models: concepts for humancomputer interaction research. International Journal of Man-Machine
Studies, 38, 587-605.
Stanton, J.M., & Barnes-Farrell, J.L. (1996). Effects of electronic performance
monitoring on personal control, task satisfaction, and task performance.
Journal of Applied Psychology, 81(6), 738-745.
Straub, D., Limayem, M., & Karahanna-Evaristo, E. (1995). Measuring system
usage: implications for IS theory testing. Management Science, 41(8),
1328-1342.
Straub, D.W., & Carlson, C.L. (1989). Validating instruments in MIS Research.
MIS Quarterly, 13, 147-169.
Templeton, G. F., Lewis, B.R., & Snyder, C.A. (2002). Development of a
measure for the organizational learning construct. Journal of Management Information Systems, 19(2), 175-218.
van der Doef, M., & Maes, S. (1999). The Leiden quality of work questionnaire:
Its construction, factor structure, and psychometric qualities. Psychological Reports, 85(3, Part 1), 954-962.
Venkatesh, V., Morris, M. G., & Ackerman, P.L. (2000). A longitudinal field
investigation of gender differences in individual technology adoption decision-making processes. Organizational Behavior and Human Decision
Processes, 83(1), 33-60.
Williams, J., & Nelson, J.A. (1990). Striking oil in decision support. Datamation,
36(6), 83-86.
Yagil, D. (2001). Reasoned action and irrational motives: A prediction of drivers’
intentions to violate traffic laws. Journal of Applied Social Psychology,
31(4), 720-740.
Yoon, Y., Guimaraes, T., & O’Neal, Q. (1995). Exploring the factors associated
with expert systems success. MIS Quarterly, 19(1), 83-106.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Measurement of Perceived Control in Information Systems

107

APPENDIX A
Research Instrument
Strongly Disagree
1
2

Strongly Agree
5

It is easy for me to visualize the way in which the system works.
1
2
3
4

The system lets me set my own goals each time I use it.
1
2
3
4

The system does not dictate the sequence in which objectives can be accomplished.
1
2
3
4
5
The system allows me to work on completing a task for as long as I want.
1
2
3
4
5
The system supports the achievement of goals in any order I want.
1
2
3
4

The system lets me make many decisions about the way I will achieve my goals.
1
2
3
4
5
The system offers many choices of tasks to accomplish.
1
2
3
4

The system lets me devise my own plans for how to complete my tasks.
1
2
3
4
5
The system will let me pursue a plan to achieve a goal for as long as I want.
1
2
3
4
5
There are not time restraints on my usage of the system.
1
2
3
4

When more than one method of achieving a task is available, the system does not dictate the order in
which I apply the methods.
1
2
3
4
5
They system allows me to choose the way in which it provides feedback on the success or failure of
the operations that I attempt.
1
2
3
4
5
I am free to complete tasks in any order I want.
1
2
3

I can select the method of feedback that the system uses to notify me of the success or failure of the
operations I perform on the system.
1
2
3
4
5
I can choose the order in which I receive multiple feedback signals from the system.
1
2
3
4
5
The system allows me to choose the feedback signal that it uses.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

108 Morris and Marshall

Strongly Disagree
1
2

Strongly Agree
5

The system allows me to choose the feedback signal that it uses.
1
2
3
4

The system lets me choose how long the feedback notification lasts when I receive feedback from the
system.
1
2
3
4
5
I am free to attempt alternative plans for completing tasks in any sequence that I choose.
1
2
3
4
5
I may pursue an objective on the system for as long as I want.
1
2
3
4

The system provides many alternate methods of notifying me of the success or failure of the
operations I perform on the system.
1
2
3
4
5
The feedback signal from the system, which notifies me of the success or failure of the operations I
perform on the system, lasts only as long as I want it to.
1
2
3
4
5
There is no presser order in which I must try alternative methods of accomplishing my goals on the
system.
1
2
3
4
5
The system does not limit the amount of time I can spend working on a method of accomplishing my
objectives.
1
2
3
4
5
When multiple potential feedback signals exist, the system allows me to decide the order in which I
will receive those signals.
1
2
3
4
5
The tasks I do on the system are tasks that I have been able to complete in the past.
1
2
3
4
5
The system allows me to prioritize the feedback signals that it sends to me.
1
2
3
4
5
I can receive feedback from the system whenever I want feedback.
1
2
3
4

I have used other systems that are fundamentally the same as this one.
1
2
3
4
5
I receive feedback from the system only when I want it.
1
2
3
4

The system seems to work similarly to other systems with which I am familiar.
1
2
3
4
5
I understand, at least abstractly, the components of the system and how they work.
1
2
3
4
5

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Measurement of Perceived Control in Information Systems
Strongly Disagree
1
2

109

Strongly Agree
5

The system allows me to select when I will receive feedback from the system.
1
2
3
4
5
The objectives I attempt on the system are very similar to ones that I have successfully completed in
the past.
1
2
3
4
5
The system allows me to attempt to achieve my goals in any manner I want
1
2
3
4
5
Once I start a session on the system, I do not change the goals I am trying to accomplish.
1
2
3
4
5
I am certain that my goals in using the system are achievable.
1
2
3
4

I never attempt to do things on the system unless I am certain that I can succeed.
1
2
3
4
5
I immediately discontinue trying to achieve an objective on the system when the system reports
negative results.
1
2
3
4
5
I use a very flexible approach to completing tasks on the system.
1
2
3
4

I can change the conditions for which the system notifies me of the success or failure of the
operations that I perform on the system.
1
2
3
4
5
I am easily discouraged from using the system when it reports negative results for a task I am trying
to accomplish.
1
2
3
4
5
I can easily adjust my plans when the system reports a negative result for a task I was trying to
complete.
1
2
3
4
5
The system does not force me to change plans of how to achieve a goal after a certain amount of
time.
1
2
3
4
5
The system does not dictate the objective I can pursue while using it.
1
2
3
4

The methods in which I achieve my objectives on the system are the same methods that I have used
successfully in the past.
1
2
3
4
5
The way that I attempt to accomplish my tasks on the system is a realistic approach to achieving my
goals.
1
2
3
4
5

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

110 Morris and Marshall
Strongly Disagree
1
2

Strongly Agree
5

The system can provide feedback for varying durations based on my selection.
1
2
3
4
5
The goals I try to achieve on the system are logical consequences of the manner in which I try to
achieve these goals.
1
2
3
4
5
The system allows me to choose the order in which I attempt to achieve my goals.
1
2
3
4
5
I know all the steps required to complete common tasks on the system without instructions.
1
2
3
4
5
It requires little or no thought on my part to complete common tasks on the system.
1
2
3
4
5
The system provides many alternative ways to do a task.
1
2
3
4

Completing common tasks on the system has become an automated response for me.
1
2
3
4
5
There are parallels between this system and other systems I have operated.
1
2
3
4
5
The manner in which I try to complete tasks on the system changes to suit my environment.
1
2
3
4
5

APPENDIX B
Original Items for Each Factor
Factor
Goal Content
Goal Sequence
Goal Timeframe
Task Content
Task Sequence
Task Timeframe
Feedback Content
Feedback Sequence
Feedback Timeframe
Feedback Condition
Knowledge
Goal Realistic
Goal Stable
Task Realistic
Task Flexible
Task Organized

Item Numbers
2, 7, 44
3, 5, 13, 49
4, 10, 19,
6, 8, 34, 52
11, 18, 22
9, 23, 43
12, 14, 16, 20
15, 24, 26
17, 21, 47
27, 29, 32, 40
1, 28, 30, 31, 54
25, 33, 36, 37
35, 38, 41
45, 46, 48
39, 42, 55
50, 51, 53

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Measurement of Perceived Control in Information Systems

111

APPENDIX C
Final Factor Model
Number

Item

4
19
10
23
9

GT1
GT3
GT2
TT2
TT1

15
16
14
24

FS1
FC3
FC2
FS2

21
17

FT2
FT1

7
6

GC2
TC1

28
30
54

K2
K3
K5

Factor 1 Factor 2 Factor 3 Factor 4 Factor 5
.821
.803
.801
.727
.720
.786
.773
.621
.531
.660
.617
.735
.660
.799
.780
.680

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

112 Ma and Liu

Chapter VI

The Technology
Acceptance Model:
A Meta-Analysis of
Empirical Findings

Qingxiong Ma, Central Missouri State University, USA
Liping Liu, University of Akron, USA

ABSTRACT

The technology acceptance model proposes that perceived ease of use and
perceived usefulness predict the acceptance of information technology.
Since its inception, the model has been tested with various applications in
tens of studies and has become a most widely applied model of user
acceptance and usage. Nevertheless, the reported findings on the model
are mixed in terms of statistical significance, direction, and magnitude. In
this study, we conducted a meta-analysis based on 26 selected empirical
studies in order to synthesize the empirical evidence. The results suggest
that both the correlation between usefulness and acceptance and between
usefulness and ease of use are somewhat strong. However, the relationship
between ease of use and acceptance is weak, and its significance does not
pass the fail-safe test.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Technology Acceptance Model 113

INTRODUCTION

Information technology (IT) acceptance or adoption has received considerable attention in the last decade. Several theoretical models have been proposed
to explain end users’ acceptance behavior. Among them, the technology
acceptance model (TAM) proposed by Davis (1989) is widely applied and
empirically tested. There have been tens of empirical studies conducted on TAM
since its inception. Compared with its competing models, TAM is believed to be
more parsimonious, predictive, and robust (Venkatesh & Davis, 2000).
Despite the plethora of literature on TAM, the empirical tests so far have
produced mixed and inconclusive results that vary considerably in terms of
statistical significance, direction, or magnitude. Although they are not uncommon
in social sciences where human behavior is difficult and complex to explain, the
mixed findings not only undermine the precision of TAM, but also complicate
efforts for IT practitioners and academicians to identify the antecedents to user
acceptance behavior.
The goal of this study is to understand to what extent the existing body of
literature reflects substantial and cumulative validity of TAM. In particular, we
synthesize the existing findings on TAM by conducting a meta-analysis. We hope
that by integrating existing empirical findings, we can better understand how
TAM is applicable to different technologies as a whole. We will be able to
examine the relationships between the constructs of TAM with a larger sample
of subjects than any individual studies. We hope that the results of this study can
be used as a benchmark for future tests of TAM.
Besides its potential theoretical contributions, a meta-analysis on TAM also
is significant to IT management practice. By understanding the substantive
antecedents to user acceptance, IT managers can take more effective interventions to achieve greater technology acceptance or usage. As Robey and Marus
(1998) and Benbasat and Zmud (1999) noted, IT management needs prescriptions. IT researchers should not only apply rigorous methodology best suited to
their research objectives, but also produce relevant and consumable research for
practitioners. There can be many possible ways for academic research to
contribute to practice. Benbasat and Zmud (1999) noted as a successful
example, “IT research based on Theory of Reasoned Action and its extensions,
such as the Theory of Planned Behavior, to the study of IT adoption, implementation, and use” (p. 9). They suggested that once a sizable body of literature
exists regarding a phenomenon, “it does become possible to synthesize this
literature” (p. 9). Thus, they recommended that the “IS research community
produce cumulative, theory-based, context-rich bodies of research” (p. 9). In a
sense, the current study answers this rigor and relevance research call.
The outline of this chapter is as follows. We first review the literature on
TAM and indicate major inconsistencies and discrepancies in the existing
findings. Then, we describe how we collected and recorded the sample of
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

114 Ma and Liu

empirical findings and report the results of our meta-analysis based on 26
selected empirical studies. Finally, we conclude this study by including a
discussion on its limitations and some suggestions for future research.

LITERATURE REVIEW

The Technology Acceptance Model (TAM) introduced by Davis (1986) is
one of the most widely used models to explain user acceptance behavior. This
model is grounded in social psychology theory in general and the Theory of
Reasoned Action (TRA) in particular (Fishbein & Azjen, 1975). TRA asserts
that beliefs influence attitudes, which lead to intentions and, therefore, generate
behavior. Correspondingly, Davis (1986, 1989) introduced the constructs in the
original TAM (see Figure 1) as follows: perceived usefulness (PU), perceived
ease of use (PEOU), attitude, and behavioral intention to use. Among the
constructs, PU and PEOU form an end user’s beliefs on a technology and,
therefore, predict his or her attitude toward the technology, which in turn predicts
its acceptance.
Davis (1989) conducted numerous experiments to validate TAM by using
PEOU and PU as two independent variables, and system usage as the dependent
variable. He found that PU was correlated significantly with both self-reported
current usage and self-predicted future usage. PEOU also was correlated
significantly with current usage and future usage. Overall, he found that PU had
a significantly greater correlation with system usage than did PEOU. Further
regression analysis suggested that PEOU might be an antecedent of PU rather
than a direct determinant of system usage; that is, PEOU affects technology
acceptance (TA) indirectly through PU. Figure 2 shows the validated TAM.
In the last decade, TAM has received considerable attention and empirical
support (Davis, 1989; Mathieson 1991; Taylor & Todd, 1995a). We estimate that
there were about 100 studies, published in journals, proceedings, or technical
reports, that related to TAM between 1989 and 2001. In these studies, TAM was
extensively tested using different sample sizes and user groups within or among
organizations, analyzed with different statistical tools, and compared with
competing models (Gefen, 2000). It was applied to many different end-user
Figure 1. The original technology acceptance model
Perceived
Usefulness
External
Variables

Attitude
Toward Use

Behavioral
Intention to Use

System
Usage

Perceived
Ease of Use

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Technology Acceptance Model 115

Figure 2. A validated technology acceptance model
Perceived
Ease of Use

Perceived
Usefulness

Technology
Acceptance

technologies such as e-mail (Adams, Nelson, & Todd, 1992; Davis, 1989), word
processors (Adams, Nelson, & Todd, 1992; Davis, Bagozzi, & Warshaw, 1989),
groupware (Taylor & Todd, 1995b), spreadsheets (Agarwal, Sambamurthy, &
Stair, 2000; Mathieson, 1991), and the World Wide Web (Lederer, Maupin, Sena,
& Zhuang, 2000). Some studies also extended TAM by including additional
predictors such as gender, culture, experience, and self-efficacy. Overall,
researchers tend to suggest that TAM is valid, parsimonious, and robust
(Venkatesh & Davis, 2000).
Davis (1989) developed and validated the scales for PEOU and PU and
found six highly reliable items for each construct with a Cronbach’s alpha of .98
for PU and .94 for PEOU, respectively. In succeeding studies, the measurement
items for these constructs varied from researcher to researcher (Adams,
Nelson, & Todd, 1992). As a result, the cumulative number of items for
measuring PU has increased from the original six to currently about 50, and for
PEOU has increased from six to 38. The Appendix shows nine different
instruments for PU and PEOU that were used in the existing studies1. Upon
closer scrutiny of the list, we found that the differences in measurement items
among the studies tend to be the result of adapting TAM to different technologies. The essential definitions of the constructs to be measured are still the same.
Therefore, we conclude that the empirical findings on the relationships between
the constructs in TAM are not much affected by how the constructs are
measured.
Existing empirical findings on TAM are not consistent and conclusive
(Moore & Benbasat, 1991). For instance, some studies indicated that PEOU has
no significant impact on TA, while others found that such an impact is significant
(Hendrickson & Collins, 1996; Subramanian, 1994; Venkatesh & Davis, 1996).
Many studies found that the impact of PEOU on PU is stronger than that of
PEOU on TA, whereas others found a much larger effect of PEOU on TA than
PU (Lim, 2001). More perplexing is the fact that even in the same study, when
the subjects were tested with different applications, PEOU was negatively
related to TA in some cases, but positively in others (Adams, Nelson, & Todd,
1992).
With regard to the divergent findings, many possible explanations have been
provided. However, they tend to be qualitative and subjective. What is needed,
we believe, is to integrate these findings and generate a quantitative and
objective synthesis.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

116 Ma and Liu

RESEARCH METHODOLOGY

Meta-analysis is defined as the “statistical analysis of a collection of
analysis results from individual studies for the purpose of integrating the
findings” (DerSimonian & Laird, 1986). An individual test typically provides
summary statistics that indicate the significance of the test results. In metaanalysis, we need to convert the statistics into a common metric called effect
size, which is usually in the form of the Pearson Product Moment Correlation.
Essentially, an effect size represents the degree to which the phenomenon is
present in the population (Cohen, 1977). In this section, we explain how we select
individual studies for our meta-analysis and how we estimate the effect size for
each sample study.

Selection of Individual Studies

To be included in our meta-analysis, a study had to meet four requirements:
(1) It involved empirical testing of TAM directly or indirectly; (2) It reported a
sample size; (3) It reported correlation coefficients between the constructs of
TAM or other values that can be converted to correlations; and (4) It was
published or dated after 1989, which is the year TAM was first published.
One widely documented concern on selecting sample studies for metaanalysis is the publication bias. It is well known that journals are more likely to
publish research results that are statistically significant. Therefore, effect sizes
in published journals are larger than those that have not been published. Because
one of the procedures in meta-analysis is to average the effect sizes of individual
studies, the result may be inflated if only published studies are reviewed
(Schafer, 1999). To avoid this so-called file drawer problem, we searched for
related studies at all levels of publications including refereed journals, unpublished dissertations, and conference proceedings. In this study, our source paper
collection was inclusive. We included all the related studies in major journals
such as MIS Quarterly, Information Systems Research, Information and
Management, and so forth. We searched digital libraries of Association for
Computing Machinery and Association for Information Systems and major
international conference proceedings. We also searched academic databases
such as ProQuest, EBSCO, and ResearchIndex at Google.
One of the important assumptions in meta-analysis is the independence of
individual findings; effect sizes such as correlations in different studies are
statistically independent. This assumption is frequently violated because some
studies often report more than one correlation or effect size based on the same
sample (Martinussen & Bjjornstad, 1999). To observe this assumption, when we
selected studies and calculated the effect sizes, we carefully checked the sample
to make sure it was not based on the same data. If multiple tests based on the
same sample were conducted, we selected only one of them and recorded its
statistics.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Technology Acceptance Model 117

In this project, initially we found a total of 91 empirical studies. Among them,
65 studies did not report correlation coefficients or other statistics that we could
convert into correlation coefficients. Thus, we dropped those studies and
selected the remaining ones. Among the 26 selected studies, seven of them are
working papers or published in conference proceedings. Since some studies
reported test results based on multiple samples (Davis, 1989; Subramanian,
1994), we obtained 102 correlation coefficients in total from the 26 selected studies.

Estimation of Effect Sizes

It is natural that different studies may report different statistics such as
correlation coefficients, F, t, or chi-square values. Consequently, the results may
not be comparable enough to provide an insight into the strength of a relationship
or the effect of interest (Wolf, 1986). Therefore, it is necessary to convert
different statistics into a common metric before conducting a meta-analysis. In
this study, the Pearson Product Moment Correlation is used as the index of effect
size to represent the empirical strength of a relationship between each pair of the
constructs in TAM. We selected the statistic because of its ease of interpretation
and the availability of formulae for converting other test statistics into correlation
coefficients (Lipsey & Wilson, 2000). In addition to effect size, we also encoded
the sample size for each study and determined whether an effect size is positive
and statistically significant.
For each pair of the constructs in TAM: PU, PEOU, and TA, we calculated
the effect sizes as follows: an effect size is simply a correlation coefficient, if it
was reported; otherwise, it is obtained through a conversion by using a formula.
For example, if a t-value is reported, we converted it into a correlation using the
formula r =

t2
, where df is the degree of freedom. Wolf (1986) provides
t 2 + df

guidelines for converting the most common test statistics to r. Cohen (1965,
1977), Friedman (1968), Glass, McGaw, and Smith (1981), and Rosenthal (1984)
discuss the conversion process and provide guidelines for transforming some less
common statistics. This procedure is widely used in many other studies (Szymanski
& Henard 2001).

DATA ANALYSIS

In this section, we analyze the data and report findings in two steps. First,
we describe the correlations in terms of range, direction, statistical significance,
and sample size. These data will reflect the nature and diversity of the existing
findings on TAM. Second, we present the findings from the univariate analysis
of the correlations. The purpose here is to show the central tendencies of the
existing findings and their statistical significance.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

118 Ma and Liu

Table 1. A summary of selected correlations

Link

# of
# of
Studies Correlations

Range of
Correlation
Coefficients
From
To

Positive
Significant
Correlations
#
%

Positive
Insignificant
Correlations
#
%

Range of
Sample Sizes

Cumulative
Sample Size

From

Average

PU-TA

.09

.91

62.2

5.4

1370

179

6058

PEOU-TA

.07

.59

53.0

37.5

1370

194

5744

PEOU-PU

.003

.92

63.6

18.2

1370

169

5421

Descriptive Statistics

Based on the 26 selected studies, we obtained 102 correlations as summarized in Table 1. Note that not all studies reported all three correlations or
equivalents. Among the 102 correlations, 37 PU-TA correlations were obtained
from 21 studies, 32 PEOU-TA correlations from 20 studies, and 33 PEOU-PU
correlations from 21 studies. The number of studies for each of the three
relationships is approximately the same. According to the range of correlation
coefficients, it is easy to see that the strength of each relationship varies greatly
from insignificant to strongly significant. For instance, the correlation between
PEOU and PU changes from 0.003 to 0.92. In addition, the correlation
coefficient between PU and TA was insignificant in some instances, although
most studies found otherwise.
As expected, most studies reported positive significant findings and few
non-significant or negative ones. According to Table 1, the percentage of
positive significant correlations of PEOU-PU is the highest among the three
relationships, while PEOU-TA has the highest percentage of positive nonsignificant correlation. The sample size varies from study to study. In some
studies, the sample size is as low as 36; while in others, it is as high as 1,370. Of
course, the extreme cases are few in number. The average sample size indicates
that the number of subjects used in the selected studies is very close across all
three relationships.

The Analysis of Direct Effect

In Table 2, we reported the findings on the mean effect sizes using three
methods: simple mean, sample size adjusted-mean (Mosteller & Bush 1954), and
the Fisher r to Z transformation (Fisher 1932). A simple mean is simply the
average of all individual effect sizes. A sample size adjusted-mean is the sample
size-weighted average of individual effect sizes, or r = ∑ (N i ri ) ∑ N i , where Ni
and ri are the sample size and the effect size of test i, respectively. To use the
Fisher r to Z transformation, three steps are followed (Wolf 1986). First, each
correlation is transformed into Fisher’s Z score using the formula Z = 0.5 *

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Technology Acceptance Model 119

Table 2. Means and variances of correlations
Link

ln (

Sample-SizeAdjusted Correlation

Simple Mean
Correlation

Correlation
from Zr

Sample
Variance

Fail Safe
Nfs.05

Confidence
Interval

PU-TA

0.4113

0.49

0.54

0.0323

131

(.41, .57)

PEOU-TA

0.2759

0.27

0.28

0.0172

-0.7

(.21, .33)

PEOU-PU

0.4679

0.50

0.54

0.0380

(.41, .59)

1+ r
) , where r is an individual correlation coefficient. Next, we compute the
1− r

sample size weighted average of the individual Z scores for each pair of the
constructs in TAM. Finally, we convert the weighted average Z score back into
a correlation coefficient.
There are some discussions regarding these methods. Some suggest the
necessity of using the Fisher r to Z transformation in meta-analysis, while others
feel there is not much difference between simple mean and the Fisher r to Z
transformation (Wolf, 1986). Schmidt, Gast-Rosenberg, and Hunter (1980)
discussed the issue and reported a study based the Fisher transformation. In the
current study, we employed both techniques.
It is also commonly believed that correlations estimated from larger samples
and more reliable data sources can produce a mean correlation closer to the
population mean, all else being equal (Hunter & Schmidt, 1990; Szymanki &
Henard, 2001). Thus, it is desirable to calculate reliability-adjusted mean.
However, we found that it is difficult to do so due to the fact that many source
studies failed to report reliability data. Therefore, in this study, we chose samplesize-weighted mean instead of reliability-adjusted mean. Szymanki and Henard
(2001) did the same in their recent meta-analysis on customer satisfaction.
Table 2 shows some differences that result from using different methods.
First, we see that the Fisher r to Z transformation consistently results in larger
means than the other two methods. This inflation phenomenon has been reported
previously (Hunter, Schmidt, & Jackson, 1982; Schmidt, Gast-Rosenberg, &
Hunter, 1980). Second, we found that the mean effect sizes obtained using the
Fisher r to Z transformation and simple average are almost identical, while the
results from the sample-size-adjusted method are smaller. We rechecked sample
sizes and recalculated the means, and found that some extreme sample sizes
have an apparent effect on the means. For example, the correlations for a study
with a sample size of 1,370 are all relatively small (between 0.31 and 0.37). When
it is removed from the meta-analysis, the averages become larger and comparable with those obtained from the other two methods. It indicates that the sample
size adjusted method may not be appropriate for the current study. Thus, we will
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

120 Ma and Liu

interpret the results of this study based on the Fisher r to Z transformation
method.
According to Cohen (1977), the magnitude of an effect size is small when
it is close to 0.10, medium when it is close to 0.30, and large when it is close 0.50.
By this rough guideline, our meta-analysis suggests a medium-sized effect for the
relationship between PEOU and TA and large effect sizes for PU-TA and
PEOU-PU. Also, note that the effect sizes for PU-TA and PEOU-PU are
almost identical to each other. It is different from the general perception; our
study does not suggest that the PU-TA relationship is stronger than the PEOUPU relationship.
To show the statistical significance of the mean effects, we computed the
99% confidence intervals for each mean estimate, based on the assumption that
individual correlations are normally distributed. These intervals portray the range
of effects that might exist in the true population, given the presence of errors and
variation in the calculation of sample effects. According to the results in Table
2, no interval contains zero, which, therefore, suggests that all three mean effects
are significantly different from zero.
To further test the significance of the findings, given the possibility that we
may miss the studies that report null effects (r = 0), we calculated the fail-safe
N for p = 0.05 using the formula N fs.05 = (ΣZ 1.645) 2 − N , where SZ is the sum of

individual Z scores and N is the number of tests. A fail-safe N represents the
number of additional studies confirming the null hypothesis (r = 0) that would be
needed to reverse a conclusion that a significant relationship exists (Cooper,
1979). Table 2 shows that the mean correlations for PEOU-PU and PU-TA are
significantly different from zero to the extent that 71-131 of null effects would
have to exist to bring the respective mean estimates down to a level not
considered statistically significant. However, the mean correlation for PEOUTA does not pass the fail-safe test as indicated by the negative Nfs.05.
As we pointed out before in this study, we selected the individual correlations reported for the model rather than the average of the correlations reported
within a study. The former is often referred to as individual-level analysis, while
the latter is considered study-level analysis. Hunter and Schmidt (1990) raised
the possibility that an individual-level analysis might underestimate the samplingerror variance and the generalizability of the estimates. To address this concern,
we computed the variance due to sampling error and standard deviation for each
relationship. The results show that the variances of sampling-error are very close
to each other and, therefore, suggest that an individual level analysis is appropriate within the context of this meta-analysis.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Technology Acceptance Model 121

CONCLUSION

This meta-analysis was designed to synthesize and analyze the existing
empirical findings on Technology Acceptance Model (TAM). It examined the
relationships in TAM with a larger sample size, which is impossible to achieve
in a traditional empirical study. In general, the results of our study confirm Davis’
original findings: Among the three constructs in TAM, both the relationships
between PEOU and PU, and between PU and TA are strong, while the
relationship between PEOU and TA is weak. Here, we measured the strengths
of the relationship from three perspectives. First, with respect to the magnitude
of a mean effect, we found the mean effects for PEOU-PU and PU-TA are
large, while for PEOU-TA is medium. Second, with respect to the statistical
significance of a mean effect, we found that all three mean effects are
significantly positive at the level a = 0.01. Finally, with respect to the fail-safe test
significance, we found that between 71 and 131 null effects would have to be
hidden away in file drawers for the mean correlations between PEOU and PU
and between PU and TA to be non-significant, which seems unlikely. However,
the mean effect for PEOU and TA does not pass the fail-safe test in the sense
that one additional study reporting a null effect would lead to the effect being nonsignificant.
This study contributes to information technology (IT) management in
several important ways. First, usefulness is shown to be critical for IT adoption,
based on the accumulated evidence. It implies that developers should focus on
system functionalities and features to improve the acceptance of a system to be
developed. Second, the relationship between ease of use and usefulness cannot
be ignored. Therefore, when IT professionals develop, test, or adopt a new
system, they should keep in mind that the ease of use of the system has a strong
impact on the end user’s perception of its usefulness.
Of course, when interpreting or applying the results of this research, some
caution is advised. As with any other research methodology, meta-analysis has
its assumptions and limitations. One of the major difficulties of applying metaanalysis to the studies on TAM is that the findings of many previous researchers
are generated by multivariate analyses such as multiple regression, factor
analysis, and structural equation modeling. Meta-analysts have not yet developed effect size statistics that adequately represent this form of research
findings (Lipsey & Wilson, 2000). Consequently, many sample studies were
dropped from our list. Otherwise, the results of this study would be more
accurate.
A second limitation of this study is the lack of the analysis of potential
moderators. Besides documenting the distribution, central tendencies, and
magnitudes of correlations, a meta-analysis can identify whether the variation of
correlations is due to chance or measurement and method factors. Although we
planned to do the analysis on moderating effects and collected data on potential
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

122 Ma and Liu

moderators, including the technologies applied, the types of participants, and
whether the researchers use an experimental or survey approach, the small size
of the selected studies prevented us from running multiple regressions to
investigate their possible effects.
Our finding on the relationship between PEOU and TA is uncertain. More
future studies are needed to resolve the uncertainty. One possible direction is to
investigate whether the relationship is moderated by a third variable such as
gender, culture, experience, self-efficacy, complexity of a technology, or the
state of knowledge on a technology. Several existing studies do suggest the
possible existence of moderating effects. Gefen and Straub (1997) found that
although women tend to perceive the PEOU of e-mail to be higher than men, their
usage of e-mail is actually less than that of men. In their more recent study with
Internet technology, Gefen and Straub (2000) found that PEOU influences TA
when a Web site involves inquiries but does not influence TA when a Web site
is used for a purchasing task. Similarly, in a study of examining PEOU with
objected-oriented analysis technique, Liu and Grandon (2002) found that compared to the subjects with only partial training in structured technique, those
without training felt more positive about the PEOU of object-oriented techniques
but performed worse in object-oriented analysis tasks; those with full training
also felt more positive about its PEOU, but also performed better. Finally, Chircu,
Davis, and Kauffman (2000) studied the role of expertise in the adoption of
electronic commerce intermediaries and found that transaction complexity is an
important moderator for the relationships between PEOU and PU and between
PU and TA. By synthesizing these exiting findings, we noticed a common
phenomenon that individual and task characteristics moderate the PEOU-TA
relationship. Gefen and Straub (1997) and Liu and Grandon (2002) emphasized
individual differences such as gender and experience, whereas Gefen and Straub
(2000) and Chircu, Davis, and Kauffman (2000) stressed task characteristics.
Therefore, we suggest that future research should look into these characteristics
in order to better understand the weak PEOU-TA contingency.

REFERENCES

(*Studies whose findings are included in the meta-analyses.)
*Adams, D.A., Nelson, R.R., & Todd, P.A. (1992). Perceived usefulness, ease
of use, and usage of information. MIS Quarterly, 16(2), 227-250.
Agarwal, R., & Prasad, J. (1999). Are individual differences germane to the
acceptance of new information technologies? Decision Sciences, 30(2),
361-391.
Agarwal, R., Sambamurthy, V., & Stair, R.M. (2000). The evolving relationship
between general and specific computer self-efficacy—An empirical assessment. Information Systems Research, 1(4), 418-430.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Technology Acceptance Model 123

Benbasat, I., & Zmud, R. (1999). Empirical research in information systems: The
practice of relevance. MIS Quarterly, 23(1), 3-16.
*Chau, P.Y.K. (1996). An empirical assessment of a modified technology
acceptance model. Journal of Management Information Systems, 13(2),
185-204.
*Chircu, A.M., Davis, G.B., & Kauffman, R.J. (2000). The role of trust and
expertise in the adoption of electronic commerce intermediaries.
Technical Report, WP 00-07, Carlson School of Management, University
of Minnesota, Minneapolis, MN.
Cohen, J. (1965). Some statistical issues in psychological research. In B.
Wolman (Ed.), Handbook of clinical psychology. New York: Academic
Press.
Cohen, J. (1977). Statistical power analysis for the behavioral sciences.
New York: Academic Press.
Cooper, H.M. (1979). Statistically combining independent studies: A metaanalysis of sex differences in conformity research. Journal of Personality
and Social Psychology, 37, 131-146.
Davis, F.D. (1986). A technology acceptance model for empirically testing
new end-user information systems: Theory and results [Doctoral Dissertation]. Cambridge, MA: MIT Sloan School of Management.
*Davis, F.D. (1989). Perceived usefulness, perceived ease of use, and user
acceptance. MIS Quarterly, 13(3), 319-340.
*Davis, F.D. (1993). User acceptance of information technology: System
characteristics, user perceptions and behavioral impacts. International
Journal of Man-Machine Studies, 38(3), 475-487.
Davis, F.D., Bagozzi, R.P., & Warshaw, P.R. (1989). User acceptance of
computer technology: A comparison of two. Management Science, 35(8),
982-1001.
*Davis, F.D., & Venkatesh, V. (1996). A critical assessment of potential
measurement biases in the technology acceptance model: Three experiments. International Journal of Human-Computer Studies, 45(1), 1945.
DerSimonian, R., & Laird, N.M. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177-188.
Fishbein, M., & Azjen, I. (1975). Belief, attitude, intention and behavior.
Reading, MA: Addison-Wesley.
Fisher, R.A. (1932). Statistical methods for research workers. London: Oliver
and Boyd.
Friedman, H. (1968). Magnitude of experimental effect and a table for its rapid
estimation. Psychological Bulletin, 70, 245-251.
Gefen, D. (2000). Structural equation modeling and regression: Guidelines for
research practice. CAIS, 4(7).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

124 Ma and Liu

*Gefen, D., & Keil, M. (1998). The impact of developer responsiveness on
perceptions of usefulness and ease of use: An extension of the technology
acceptance model. The Data Base for Advances in Information Systems,
29(2), 35-49.
Gefen, D., & Straub, D.W. (1997). Gender differences in the perception and use
of e-mail: An extension to the technology acceptance model. MIS Quarterly, 21(4), 389-400.
*Gefen, D., & Straub, D. (2000). The relative importance of perceived ease-ofuse in IS adoption: A study of e-commerce adoption. Journal of the
Association for Information Systems, 1(8).
Glass, G.V., McGaw, B., & Smith, M.L. (1981). Meta-analysis in social
research. Beverly Hills, CA: Sage Publications.
Heijden, H. (2000). Using the technology acceptance model to predict
website usage: Extensions and empirical test. Technical Report, No.
0025, Retrieved, from http://ideas.uqam.ca/ideas/data/Papers/
dgrvuarem2000-25.html
*Hendrickson, A.R., & Collins, M.R. (1996). An assessment of structure and
causation of IS usage. The DATA BASE for Advances in Information
Systems, 27(2), 61-67.
Hunter, J.E., & Schmidt, F.L. (1990). Methods of meta analysis. Newbury
Park, CA: Sage Publications.
Hunter, J.E., Schmidt, F.L., & Jackson, G.B. (1982). Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage Publications.
*Igbaria, M., Guimaraes, T., & Davis, G.B. (1995). Testing the determinants of
microcomputer usage via a structural equation model. Journal of Management Information Systems, 11(4), 87-114.
*Igbaria, M., Iivari, J., & Maragahh, H. (1995). Why do individuals use computer
technology? A Finnish case study. Information and Management, 29(5),
227-238.
*Igbaria, M., Parasuraman, S., & Baroudi, J. (1996). A motivational model of
microcomputer usage. Journal of Management Information Systems,
13(1), 127-143.
*Igbaria, M., Zinatelli, N., Cragg, P., & Cavaye, A.L.M. (1997). Personal
computing acceptance factors in small firms: A structural equation model.
MIS Quarterly, 21(3), 279-305.
*Jackson, C.M., Chow, S., & Leitch, R.A. (1997). Towards an understanding of
the behavioral intention to use an iInformation system. Decision Sciences,
28(2), 357-389.
*Lederer, A.L., Maupin, D.J., Sena, M.P., & Zhuang, Y. (2000). The technology
acceptance model and the World Wide Web. Decision Support Systems,
29(3), 269-292.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Technology Acceptance Model 125

Lim, K-S. (2001). An empirical test of the technology acceptance model.
Proceedings of Decision Science Institute Annual Meeting, San Francisco, CA.
Lipsey, M.W., & Wilson,D.B. (2000). Practical meta-analysis: Applied social
research methods series, volume 49, Thousand Oaks, CA: Sage Publications.
Liu, L., & Grandon, E.E. (2002). An empirical study of how prior training in
structured analysis affects the perceptions on object orientation
[Technical Report]. Akron, OH: Department of Management, University
of Akron.
*Malhotra, Y., & Galletta, D.F. (1999). Extending the technology acceptance
model to account for social influence: Theoretical bases and empirical
validation. Proceedings of the 32nd Hawaii International Conference
on System Sciences, Honolulu, HI.
Martinussen, M., & Bjornstad, J.F. (1999). Meta-analysis calculations based on
independent and nonindependent cases. Educational and Psychological
Measurement, 59(6), 928-950.
*Mathieson, K. (1991). Predicting user intentions: Comparing the technology
acceptance model with the theory of planned behavior. Information
Systems Research, 2(3), 173-191.
Moore, G.C., & Benbasat, I. (1991). Development of an instrument to measure
the perceptions of adopting an information technology innovation. Information Systems Research, 2, 192-222.
Mosteller, F.M., & Bush, R.R. (1954). Selected quantitative techniques. In G.
Lindzey (Ed.), Handbook of social psychology, Volume 1. Cambridge,
MA: Addison-Wesley.
Robey, D., & Markus, L. M. (1998). Beyond rigor and relevance: Producing
consumable research about information systems. Information Resources
Management Journal, 11(1), 7-15.
*Rose, G., Straub, D.W. (1998). Predicting general IT use: Applying TAM to the
Arabic world. Journal of Global Information Management, 6(3), 39-47.
Rosenthal, R. (1984). Meta-analytical procedures for social research.
Beverly Hills, CA: Sage.
Schafer, W.D. (1999). Methods, plainly speaking—An overview of metaanalysis. Measurement and Evaluation in Counseling and Development, 32(1), 43-61.
Schmidt, F.L., Gast-Rosenberg, I., & Hunter, J.E. (1980). Validity generalization: Results for computer programmers. Journal of Applied Psychology,
65(6), 643-661.
*Segars, A.H., & Grover, V. (1993). Re-examining perceived ease of use and
usefulness: A confirmatory factor analysis. MIS Quarterly, 17(4), 517-525.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

126 Ma and Liu

*Straub, D.W., Keil, M., & Brennan, W. (1997). Testing the technology
acceptance model across cultures: A three country study. Information
and Management, 33(1), 1-11.
Straub, D.W., Limayem, M., & Karahanna, E. (1995). Measuring system usage:
Implications for IS theory testing. Management Science, 41(8), 13281342.
*Subramanian, G.H. (1994). A replication of perceived usefulness and perceived ease of use measurement. Decision Sciences, 25(5-6), 863-872.
*Szajna, B. (1996). Empirical evaluation of the revised technology acceptance
model. Management Science, 42(1), 85-92.
Szymanki, D.M., & Henard, D.H. (2001). Customer satisfaction: A metaanalysis of the empirical evidence. Journal of the Academy of Marketing
Science, 29(1), 16-35.
*Taylor, S., & Todd, P. (1995a). Understanding information technology usage:
A test of competing models. Information Systems Research, 6(2), 144173.
*Taylor, S., & Todd, P. (1995b). Assessing IT usage: The role of prior
experience. MIS Quarterly, 19(4), 561-570.
*Teo, T.S.H., Lim V.K.G., & Lai, R.Y.C. (1999). Intrinsic and extrinsic
motivation in Internet usage. Omega, 27(1), 25-37.
*Venkatesh, V., & Davis, F.D. (1996). A model of the antecedents of perceived
ease of use: Development and test. Decision Sciences, 27(3), 451-480.
*Venkatesh, V., & Davis, F.D. (2000). A theoretical extension of the technology
acceptance model: Four longitudinal field studies. Management Science,
46(2), 186-204.
Wolf, F.M. (1986). Meta-analysis—Quantitative methods for research synthesis. Newbury Park, CA: Sage Publications.

ENDNOTE

Many studies used one of these instruments with no modification. For
example, Adams, Nelson, and Todd (1992), Gefen and Straub (1997), and
Chircu, Davis, and Kauffman (2000) used the same instrument as in Davis
(1989). Similarly, Jackson, Chow, and Leitch (1997), Subramanian (1994),
and Szajna (1996) used the same instrument as Davis, Bagozzi, and
Warshaw (1989). Some other studies did not report their instruments but
mentioned that they adopted an existing one in the list.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

The Technology Acceptance Model 127

APPENDIX
Sample Measurement Scales for PU and PEOU
Study

Davis (1989)

Gefen and Keil (1998)

Heijden (2000)

Agarwal & Prasad
(1999)

Davis, Bagozzi, &
Warshaw (1989)

Gefen & Straub
(2000).

Igbaria, Iivari, &
Maragahh (1995)

Venkatesh & Davis
(1996)

Straub, Limayem, &
Karahanna (1995)

-Work more quickly
-Job performance
-Increase productivity
-Effectiveness
-Makes job easier
-Useful
-Using … enables me to accomplish
configuration tasks more quickly
-Using … improves the quality of the
work I do
-Using … improves my job performance
-Overall, I find … to be advantageous in
my job
-Using … increases my sales
productivity
-I find … primarily a useful site
-The information on the site is interesting
to me
-I find this a site that adds value
-Accomplish tasks more quickly
-Improve my job performance
-Give me greater control over my work
-Improve the quality of the work I do
-Improve my productivity
-Make it easier to do my job
-Is useful in my job
-Using … would improve my
performance
-Using … would enhance my
effectiveness
-Using … would increase my
productivity
-I would find … useful
-ABB improves my performance in book
searching
-ABC enables me to search and buy
books faster
-ABC enhances my effectiveness in book
searching and buying
--ABC makes it easier to search for and
purchase books
-ABC increases my productivity in
searching and purchasing books
-Using … improves my job performance
-Using … increases my productivity
-I find … useful in my job
-Using … enhances my effectiveness in
the job
-Using … provides me with information
that would lead to better decisions
-Using … would improve my
performance in my degree program
-Using … in my degree program would
increase my productivity
-Using … would enhance my
effectiveness in my degree program
-I find … would be useful in my degree
program
-Voicemail is very important in
performing my job;
-My decision-making is more effective

PEOU

-Easy to learn
-Clear and understandable
-Easy to become skillful
-Easy to use
-Controllable
-Flexible
-Learning to operate … was easy for me
-Using … is clear and understandable
-I believe that … is easy to use

Technology

Lab experiment with e-mail
and graphics

Field study with expert
system CONFIG

-It is easy to navigate around the site
-I can quickly find the information that I
need
-I think it is a user-friendly site
-It is easy for me to remember how to
perform tasks
-It is easy to get … to do what I want it
to do
-My interaction with … is clear and
untreatable
-Overall it is easy to use
-Learning to operate … would be easy
for me
-I would find it easy to get … to do what
I want it to do
-It would be easy for me to become
skillful at using …
-I would find … easy to use
-ABC is easy to learn
-My interaction with ABC is clear and
understandable
-It is easy to become skillful at using
ABC
-Learning to operate ABC is easy
-It is easy to interact with ABC
-ABC is flexible to interact with

Field study with a Web site

-Learning to use … would be easy for
me
-I would find it easy to get … to do what
I want it to do
-It would be easy for me to become
skillful at using …
-I would find … easy to use
-My interaction with … is clear and
understandable
-Interacting with … does not require a lot
of my mental effort
-I find … easy to use
-I find it easy to get … to do what I want
it to do

Field study with
microcomputers

-I find it easy to get voicemail to do what
I want it to do
-I feel very comfortable using voicemail

Field study with voicemail

Field study with GUI
environment

Lab experiment with a word
processor

Lab experiment with an
online bookstore

Lab experiment with PC and
word processor

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

128 Ma and Liu

Section II:
Collaborative
Technologies
and
Implementation
Issues

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Success Factors in the Implementation of a Collaborative Technology

129

Chapter VII

Success Factors in the
Implementation of a
Collaborative Technology
and Resulting Productivity
Improvements in
a Small Business:
An Exploratory Study
Nory B. Jones, University of Maine, USA
Thomas R. Kochtanek, University of Missouri in Columbia, USA

ABSTRACT

Practitioners and academics often assume that investments in technology
will lead to productivity improvements. While the literature provides many
examples of performance improvements resulting from adoption of different
technologies, there is little evidence demonstrating specific, generalizable
factors that contribute to these improvements. Furthermore, investment in
technology does not guarantee effective implementation. This qualitative
study examined the relationship between four classes of potential success

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

130

Jones and Kochtanek

factors on the adoption of a collaborative technology and whether they
were related to performance improvements in a small service company.
Users of a newly adopted collaborative technology were interviewed to
explore which factors contributed to their initial adoption and subsequent
effective use of this technology. The results show that several factors were
strongly related to adoption and effective implementation. The impact on
performance improvements was further explored. Results showed a
qualitative link to several performance improvements including timesavings
and improved decision-making. These results are discussed in terms of
generalizability as well as suggestions for future research.

INTRODUCTION

The importance of knowledge sharing and the ability to tap into an
organization’s vast reservoir of creative intellect have been acknowledged as
possibly the greatest strategic competency an organization can achieve (Davenport 1999; Pan et al. 1999). By enabling associates to share their ideas, expertise,
and wisdom, problems can be solved more easily, processes can be improved,
and productivity can be increased. As business environments become more
turbulent and technologies become increasingly dynamic, the pace of change and
competitive pressures spiral more steeply upward. As this pace continues,
organizations require technologies, capabilities, and a culture that enables them
to keep up with these changes (Rumizen 1998; Senge 1997).
Furthermore, in an era that is becoming predominantly digital, the ability to
share knowledge is becoming easier, cheaper, and more widely accepted. Many
organizations recognize that collaborative technologies, supported by distributed
electronic networks, can reduce barriers to communication and facilitate knowledge sharing within the organization (Ciborra et al. 1996). Collaborative technologies can enable people in distributed environments to work together seamlessly,
irrespective of location, time, or functional area. By sharing a common goal in a
networked environment, virtual teams can create synergistic relationships and
quality output via collaborative knowledge sharing. In addition, the communication patterns that develop in electronic collaborative environments are equally
applicable to people sharing knowledge in the same building or even in the same
room as those who are divided by continents (Barbar et al. 1998).
While causal relationships between knowledge sharing and specific quantifiable performance improvements to achieve competitive advantages have
been scarce, researchers have qualitatively documented some organizational
performance improvements. For example, the adoption of one particular collaborative technology (Lotus Notes) to facilitate knowledge sharing increased
productivity and efficiency in a software company when they created a
knowledge sharing repository to prevent duplication of research efforts (Orlikowski

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Success Factors in the Implementation of a Collaborative Technology

131

1999). The research literature acknowledges this relationship while seeking to
validate it with additional empirical studies. While most research in knowledge
management and collaborative technologies has focused on large organizations,
few studies have examined its impact on small businesses. In addition, few
studies have explored the specific success factors that contribute to the adoption
and diffusion of technologies that facilitate knowledge sharing and resulting
performance improvements.
This article describes the experiences of a scientific contract research
organization that works in the pharmaceutical industry in its attempt to improve
organizational performance by adopting a collaborative technology to facilitate
knowledge sharing within the organization. It explores four classes of potential
success factors to facilitate this process. The literature on knowledge sharing
and collaborative technologies suggests a number of factors considered to be
instrumental in achieving successful knowledge information among people in
organizations.
In addition, researchers in the field of performance measurement and
collaborative technologies have found tentative relationships between performance improvement and knowledge sharing using different collaborative technologies in different contexts. However, organizations often introduce new
technologies using a forced adoption approach without fully understanding the
specific factors required for their continued and effective use. This represents
a prerequisite requirement, if the technology is to be used to fulfill the goals for
which it was intended.
Therefore, this study explored how and why certain variables contribute or
fail to contribute to the effective use of a CSCW (computer-supported collaborative work) system as well as their influence on knowledge sharing using a
CSCW system. The relationship between knowledge sharing, facilitated by a
collaborative technology, and resulting performance improvements in this small
business was also examined. Finally, this chapter reflects on the lessons learned
from this experience and whether the findings may be generalizable to other
organizations.

REVIEW OF THE LITERATURE
Adoption of Technology Innovations and Success
Factors

Pan and Scarbrough (1998, 1999) studied specific factors relating to the
successful implementation of a knowledge sharing system. The model outlined
serves as the framework for the initial study model (Figure 1). They introduced
a theory that said knowledge management should contain three components to
be successful:
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

132

Jones and Kochtanek

Infrastructure: “The hardware/software that enables the physical/communicational contact between network members; provides the means to share
knowledge” (e.g., H. Saint-Onge, a senior vice president at a Canadian
financial company, described the necessity of infrastructure as “connectivity-building, a seamless railroad that can carry the knowledge freight
around the organization” (Informationweek, 1999, p. 7ER).
Infostructure: “The formal rules which govern the exchange between the
participants in the network, providing a set of cognitive resources (metaphors, common language) whereby people make sense of events on the
network.”
Infoculture: “The stock of background knowledge which actors take for
granted and which is embedded in the social relations surrounding work
group processes; core values and attitudes, reflected in employees and
managers’ willingness to exchange knowledge to solve company problems.” This would also be known as the organizational culture. A common
theory among researchers was that organizational culture played a crucial
role in the effective adoption and use of both collaborative technologies and
knowledge sharing.

The importance of culture in the adoption and implementation was exemplified by the CEO of Buckman Laboratories. “The core values and attitudes of
Buckman employees are reflected in their willingness to exchange knowledge
simply to solve company problems, without the usual political baggage and
ulterior motives.” He further asserted, “What happened at Buckman was 90%
cultural change. At the heart of knowledge-sharing activities at Buckman is a
climate of continuity and trust.” (Pan & Scarbrough, 1999, p.369). Saint-Onge
also stated, “[Y]ou need a culture that fosters interdependence—that has a
sense that everyone is creating the future of the firm through everything they’re
doing” (Informationweek, 1999, p.7ER)
Scheraga (1998) contended that “putting knowledge management solutions
in place can prove useless unless a company encourages its workforce to
contribute its knowledge to the cycle. This is one of management’s greatest
challenges, as workers are often reluctant to share information. The modern
business climate inherently rewards people for what they know, which discourages people from sharing their knowledge.” However, he suggests that the
answer to this is to reward employees for sharing information and knowledge.
Pan and Scarbrough (1998, 1999) also emphasized the importance of top
management involvement. As mentioned above, the CEO of Buckman Laboratories acted as the visionary and the champion in the effort to create a
knowledge-sharing environment within the company. Not only did he invest
heavily in the infrastructure (the technology to provide the vehicle for sharing
knowledge), but he also created unique reward and recognition systems to
actively promote knowledge sharing, contending that employees who share their
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Success Factors in the Implementation of a Collaborative Technology

133

knowledge would become influential throughout the organization. In addition, he
modeled the culture by sharing knowledge and empowering associates to also
share theirs. Thus, creating a successful knowledge sharing culture is a blend of
technology and sociology, creating both the mechanisms to facilitate knowledge
sharing and the culture to encourage it in practice.

Collaborative Technologies, Knowledge Sharing, and
Performance Improvements

What lessons can be learned from the literature on knowledge sharing and
collaborative technologies that can help a small business? By creating the
capability to capture, organize, and disseminate knowledge, a small business
potentially can improve decision-making, processes, quality, customer satisfaction, and can reduce costs. This premise is based on capturing and sharing the
experience and knowledge of employees to facilitate creativity and innovation.
This assertion is based on the research found in the knowledge management
literature. Karl Wigg (1999) described the benefits of a knowledge management
system as reducing costs due to benchmarking and sharing best practices
between different groups inside and outside the organization, decreasing timein-process, reducing rework, and increasing customer satisfaction and quality by
increasing people’s knowledge of and improvement of processes. Other benefits
include an increase in innovation in products, services, and processes due to
sharing of knowledge among different functional areas, and increased knowledge of customers resulting in the ability to better satisfy their needs, resulting
in increased market penetration and increased profit margins.
Reisenberger (1999) further asserts that “the rate of employee turnover and
the speed of change requires us to place greater emphasis on capturing,
disseminating, and rescuing our precious intellectual capital” (p. 96). He takes
this one step further in his contention that “[t]oday’s fast-paced business
environment is characterized by chaotic markets with constantly evolving global
customers, competitors and suppliers. Tomorrow’s winners will be determined
by these few firms that create the ability to develop constant and continuous
innovation and transformation. This ability will be manifested successfully by
those enterprises that understand, properly harness, and exploit global learning
and the use of the organization’s intellectual capital” (p. 94). Even Peter Drucker
(1995) mirrors this view, asserting that in a knowledge society, “the basic
economic resource is no longer capital or natural resources or labor, but is and
will be knowledge, and where knowledge workers will pay a central role”
(Reisenberger, 1999, p. 94).
In terms of the relationship between knowledge sharing and performance
improvement, most researchers admit that while there are many conceptual
articles supporting the relationship, there is little empirical evidence to validate
it. Davenport (1999) suggests that it is extremely difficult to establish a causal
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

134

Jones and Kochtanek

link between knowledge, strategy and organizational performance. However, in
an empirical study on large pharmaceutical firms that compete on the speed and
effectiveness of the drug development process, it was found that those firms
using knowledge management developed drugs more quickly. Furthermore, they
found that firms using knowledge management strategies, including knowledge
creation, idea generation and knowledge sharing, tended to be more profitable
than those that did not use these practices.
Davenport also suggests that one way to establish credibility in relating
knowledge management to improved performance is to use intermediate measures. For example, he suggested measuring the number of hits to a knowledge
repository or the satisfaction measures of employees using a knowledge
management system. By correlating these types of indicators with knowledge
management practices, a case can be built for the credibility of these processes.
Orlikowski (1996) studied the users of a digital collaboration software
system in a technical customer support division of a software company. Her
observations demonstrated a non-quantified increase in productivity. The creation of a knowledge repository allowed associates to share processes and
document problem-solving methods. This collective knowledge contributed to
better solutions to customer problems and improved efficiency and productivity,
since associates did not have to start from ground zero to research customer
problems. It also increased accountability and decision-making, because information entered into a repository was signed by the author, and users were aware
of the credibility of the sources. As the knowledge base grew, it shifted from
being simply a knowledge repository to being a training mechanism, as well. She
attributed the success of the groupware in this situation to a departmental culture,
which was open to change, and to using new technologies as well as adequate
training and expectations. The collaboration software was also user-centered,
emphasized a specific functionality, and was phased in gradually.
Failla (1996) similarly found that a team-oriented collaborative culture was
necessary for the successful adoption of collaboration technology tools as well
as commitment by top management and the users. He also identified interesting
criteria for the success of a collaborative database as a useful information
filtering system.1 He found that if no one took ownership of the system and
filtered data for relevance and usefulness, then it was not deemed to be valid by
the users. Consistent with this was his observation that users needed to take
personal satisfaction in the input they made into the system, inputting valuable
knowledge that would make a significant contribution to the organizational
knowledge.
In a consumer products manufacturer, Ciborra and Patriotta (1996) found
that the effectiveness2 of the new technology depended on the perceived
benefits of the new system (i.e., relative advantage) as well as the willingness
of the users to act collectively. They also found that resistance to the tool by new

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Success Factors in the Implementation of a Collaborative Technology

135

users depended on how closely it matched pre-existing work practices as well
as the presence of alternative communication tools with which users were
already familiar and comfortable. Adoption depended on organizational rewards
and incentives to use and actively contribute to the system. This particular
organization needed to change its culture to a more collaborative one and to
implement a reward scheme to encourage contributions to the system. These
findings are consistent with the adoption and diffusion literature (Rogers, 1995)
regarding the importance of relative advantage and compatibility in the continued
and effective use of a new technology.
In terms of knowledge sharing and contract research organizations, Mancini
(1998) suggested that using document sharing technologies like CSCW tools are
becoming more important as “time to market” in the pharmaceutical industry, is
more critical than ever. In addition, the reasons for using a document sharing
technology also translates into huge time savings where delays in the research
process can cause delayed approvals by the FDA (Food & Drug Administration). This potentially can result in millions of dollars in lost or delayed revenues
to the companies. Therefore, if a CSCW technology can allow people within this
industry to share information (documents and otherwise) and knowledge more
effectively, thereby reducing review, editing, and process time, it can give both the
contract research organizations and their clients, pharmaceutical and chemical
companies, a competitive advantage by reducing time to market.

THE STUDY
Purpose

The overarching purpose of this research was primarily to identify and
understand success factors that influence the continued and effective use of a
CSCW system, enabling knowledge sharing. Secondarily, the purpose was to
examine the resulting consequences of its use in several dimensions, including
performance improvements in time, productivity, and quality.

Research Questions

The literature suggests that factors associated with infrastructure,
infostructure, and infoculture are related to the successful adoption and implementation of a knowledge-sharing system, facilitated by a collaborative technology. The literature also suggests a relationship between knowledge sharing and
performance improvements in the organization. Based on this analysis of the
literature, the following research questions were posed:

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

136

Jones and Kochtanek

Which of the variables involved with (1) infrastructure, (2) infostructure,
(3) infoculture, and (4) individual concerns exert an influence (positive or
negative) on the effective use of a CSCW technology and knowledge
sharing, and in what ways do they exert their influence?
How does the use of a CSCW technology to facilitate knowledge sharing
influence the performance dimensions of time, productivity, and quality
assurance?

A study model (Figure 1) was developed to explore the influence that
different potential success factors associated with the four major categories
might exert on the adoption and diffusion of a knowledge sharing system enabled
by a collaborative technology.
The model attempts to incorporate the success factors from the literature
into a comprehensive array of potential contributors to adoption and diffusion of
a knowledge-sharing system. Specifically, the work of Pan and Scarbrough
(1998, 1999) discussed in the literature review serves as the major framework
for this study model, incorporating their socio-technical view of the firm with
their infrastructure, infostructure, and infoculture variables. This study model
added a fourth major variable called individual concerns.
In this model, the infrastructure components incorporate the following
factors:

•
•
•

Relative advantage – user-friendly technologies that users perceive as
superior to existing technologies, providing more or better benefits
Training and time to learn and use the system effectively
Compatibility with existing work routines and norms

The infostructure components dealt with rules governing the use of the
system including:

•
•

Recency and relevancy of information – how the knowledge was managed
to ensure that contributions were both recent and relevant, thus motivating
its continued use.
Rules governing the system – did users perceive that the system was
managed well with clear and consistent rules for usage?

The infoculture component integrated many variables from the literature
including:

•
•

Influence of leadership on initial and continued use of the system
Influence of reward and compensation structures

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Success Factors in the Implementation of a Collaborative Technology

•
•

137

Influence of peers and social networks
Influence of trust and communication for effective knowledge sharing

Finally, the last component—individual concerns—explored potential success factors related to an individual’s personal agenda, including:

•
•
•

Prior experience with technologies
Personality variables, identified as adopter categories
Security concerns with using a collaborative technology and sharing
knowledge.

METHODOLOGY

In January of 2000, a contract service business made the decision to adopt
a Web-based collaborative technology for the purposes of enabling knowledge
sharing within the firm. The business was composed of three divisions with about
250 employees. The population of interest for this study consisted of the entire
population of users of a CSCW system within a contract research organization.
When the study was initiated in April of 2000, there were approximately 10-15
users. By completion of the study in December of 2000, there were approximately 47 users. Special emphasis was made to interview the leadership team
(top executives), who represented the heaviest users of the system. These
individuals included the president/CEO, three of the four vice presidents, and the
chief financial officer. In addition, five of the six business development (marketing) managers, who represented low-moderate users, were interviewed as well
as the director of information systems. Finally, from the remaining pool of
approximately 37 occasional-moderate users, 20 were selected by using a quota
system3 to represent the remaining functional areas—eight managers, four
quality-assurance/compliance, and eight data-entry people agreed to be interviewed. During the interview, questions were asked using a survey instrument
designing to elicit their perceptions and attitudes toward this collaborative
technology and knowledge sharing in terms of the factors described in the study
model. Respondents were encouraged to answer freely and openly and were
prompted only to keep responses focused on the variables of interest, if the
conversation began to stray from the topic under discussion.
In this organization, the collaborative system used was a Web-based
document sharing technology called BSCW (Basic Support for Cooperative
Work) (http://bscw.gmd.de/). Its primary application was to enable top management and mid-level managers to collaborate on reports, share information about
sales and budgets, and share information about customer problems and complaints with the quality assurance personnel for quality improvement purposes.
This Internet-based software is platform independent, requiring only an Internet
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

138

Jones and Kochtanek

Figure 1. Study model
High relative advantage

Infrastructure

High compatibility
Adequate training and time

Infostructure

Information is managed for recency, relevancy, and
security
Clear rules governing use of the system

Leaders (change agents) motivated and committed to
BSCW use

Continued and effective
use of a CSCW
technology to facilitate
knowledge sharing

Reward/compensation structure and incentives

Infoculture

Positive peer influence (opinion leaders) within a social
network in favor of BSCW
Trust, good communication among colleagues

Prior experience and knowledge of technology

Individual
Concerns

Adopter category
Security concerns

Effective use of a CSCW technology to
facilitate knowledge sharing

Performance improvement in time,
processes, and/or innovation

connection, a login/password sequence, and a current version of a Web browser
such as Netscape Navigator or Internet Explorer. BSCW may be considered a
groupware product similar to Lotus Notes, in that it allows multiple users to work
on documents from a central repository, version the documents, and demonstrate
accountability in terms of who created changes and when they were made. The
system is organized within shared folders such as “Sales Forecast: Division A”
folder. The owner of that folder then invites those people that he or she wishes
to share information with, thus controlling access. Each invited member then can
retrieve documents, edit them, version them, and resubmit them. The versioning
capability allows members to see what changes each member made and keep
track of different versions. Its greatest benefit may be that employees can

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Success Factors in the Implementation of a Collaborative Technology

139

access needed information anywhere in the world at any time as long as they
have Web access.
In this qualitative study, metrics represented indicators of employee perceptions and attitudes regarding the collaborative technology. The number of times
that respondents discussed each factor was counted from the interview transcripts and analyzed using SPSS for frequency distributions and correlations with
actual usage. For example, if respondent #12 discussed relative advantage four
times, it was documented. In contrast, this respondent might not have discussed
compatibility at all as a factor in his or her usage of the collaborative technology.
Usage was documented on a daily basis from a log file that recorded the number
of hits to the system per user per day. All data were analyzed qualitatively from
the recorded transcripts as well as quantitatively using frequency distributions
and correlation analyses on the SPSS software program.

RESULTS
Influence of Major Variables

Figure 2 shows the total number of responses for each major variable in the
study. It may be inferred that the number of responses correlates with how
strongly people felt about each topic, as they would tend to elaborate more on
topics of interest to them. Because the responses focused on infrastructure and
infoculture variables, those factors are discussed in this chapter. In this case
study, the users did not express indications that the factors associated with
infostructure or individual concerns influenced their usage of this collaborative
technology or their willingness to share knowledge and information.
A correlation analysis (Table 1) between the four major variables of interest
and average BSCW use per day per person was run. Infrastructure showed a
significant correlation with usage at the 0.01 level. While not statistically
significant, infoculture did show a higher correlation than the other variables.
Based on these results, the individual components associated with infrastructure and infoculture were examined. In terms of the subcomponents
components within infrastructure, results showed that relative advantage emerged
as the major factor of influence.
For the infoculture subcomponents, the rich interview responses suggested
that leadership exerted the greatest influence on use of the collaborative
technology to support knowledge sharing. A reward/compensation structure to
support knowledge sharing was also seen as important. The following responses
exemplify this.
“Basically, [President/CEO] told us to use this system, so we are. He can
look to see who uses it to update the forecast and other information.”

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

140

Jones and Kochtanek

Figure 2. Total responses for major variables (n=30)

Percent of total

Responses by Category
33%

32%

40%
30%
20%
10%
0%

24%

11%

e
ur
ct
ru
t
s
fo
In

..
c.
tru
s
fra
In

u
oc

ltu

re
In

u
id

...

Figure 3. Total responses relating to performance improvements (n=30)

25%

30%
25%
20%
15%
10%
5%
0%

g
.l

Q
ua
lit
y

sa
nt
lie

rg
O

is
ec
D

14%

15%

e
m
Ti

24%

23%

% responses

Performance Improvement
Potential

Table 1. Correlation analyses: Major independent variables (n=30)
Independent Variables
Infrastructure
Infostructure
Infoculture
Individual Concerns

Avg. Usage per day per person*
.527**
-.202
.299
-.088

* Average usage per day was measured in terms of the number of hits to the system per person
** Correlation is significant at the 0.01 level (1-tailed).

Table 2. Correlation analysis: Subelements of infrastructure (n=30)
Independent Variables
Relative Advantage
Compatibility and Training

Avg. Usage per day per person
.515**
.279

** Correlation is significant at the 0.01 level (1-tailed).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Success Factors in the Implementation of a Collaborative Technology

141

“Well, I guess to be real honest, [President/CEO] did. He said, ‘You are
going to use it!’ End of discussion.”
“Oh yeah – [President/CEO]! He goes in there and looks—who’s been
reading it, who’s been revising it, yeah. It was the greatest motivation to begin
with.”
Figure 3 shows the major perceptions of performance improvements from
knowledge sharing, enabled by a collaborative technology.
The following sections demonstrate the qualitative link between knowledge
sharing, enabled by a collaborative technology and performance improvements.

RELATIVE ADVANTAGE
Time and Efficiency

Time is one of the most valuable and most scarce resources in a scientific
contract research organization due to the time-to-market pressures for its
clients. Thus, time-to-market and the lifespan of patents in the highly competitive
pharmaceutical and agrochemical industries presented the driving force in
achieving rapid turnaround time. In addition, many companies enjoy significant
profits from achieving a “first-mover” advantage, if they can be the first to get
their products to the marketplace.
As stated by one of the associates, “It cuts down on the number of meetings
that we need to have because we can take care of a lot of business just over the
network.” Another associate documented time savings in this statement: “You
have everybody who’s doing invoicing sitting in a room for 2 hours projecting
what you’re going to make for next month and that was a waste of time. I can
be in and out of BSCW in 10 minutes.”
By sharing information via BSCW, valuable knowledge could be entered
quickly and people could make decisions, initiate studies, and make corrections
much more quickly. In addition, simply by eliminating manual processes such as
copying and distributing information manually saved time in the research
process. One associate stated, “Unless you want to print up separate documents
and hand it to each person, and then integrate all those documents at once, this
is a better alternative because you have a single document. People at their leisure
can make changes and there’s no integration of the document left and there’s
nothing to transcribe error-wise.”
In this company, the leadership team traveled extensively, primarily to
attend conferences or visit clients. In addition, some executives were located at
the European site. Thus, the ability to share information in a distributed
environment on a timely basis was greatly facilitated by this collaborative
technology. Executives could simply input their current information about
strategic plans, budgets, forecasts, industry developments, or competitive acCopyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

142

Jones and Kochtanek

tions any time and from any place in the world. This allowed the president to save
time in face-to-face meetings and make better decisions based on the most
current information available from within the organization.

Decision Support

This collaborative technology enabled managers throughout the company to
input needed information such as sales forecasts, budgets, and client and
competitor information on a real time basis, which was accessible to the
leadership team. This allowed executives and managers to make decisions based
on the most recent and most relevant information and knowledge. This was
exemplified by the following response: “It allows managers and top executives
to make better decisions by having access to more timely information.”

Quality Assurance and Compliance

In terms of better client service, the president concluded, “This offered us
a better solution at prioritizing. We can identify if there’s going to be a conflict
with resources ahead of time, resolve it, communicate to the clients, adjust their
expectation, and meet them.” In addition, the ability to identify trends to improve
quality in processes as well as client satisfaction was demonstrated by this
associate’s statement: “The biggest advantage to this is being able to use that as
a tool to teach ourselves what we need to do.” By sharing client comments and
concerns, the quality assurance and compliance people could identify recurring
problems or trends and take corrective actions to improve quality or client
satisfaction.

Leadership Influence

In the interviews, the importance of leadership appeared to support the
literature. From the emphatic responses in the interviews, it became clear that
probably the most important influence on the initial adoption and use of the
technology was the President/CEO of the company. A typical response to “who
influenced your use of the system” was “Oh- absolutely—our CEO! He initiated
that BSCW is what we would use.” In this particular organization, the forced
adoption of a new technology appeared to have a great influence on adoption and
continued, effective use of the BSCW technology to facilitate knowledge
sharing. The results from the quantitative study appeared to support this view.
Approximately 70% of the respondents indicated that leadership or managers
exerted an influence on their use of BSCW to facilitate knowledge sharing.
Interestingly, 72% of the respondents also indicated that a perceived need
(relative advantage) in sharing knowledge also influenced their use of this
technology to facilitate knowledge sharing. Thus, this finding appears to support
further the importance of the relative advantage variable.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Success Factors in the Implementation of a Collaborative Technology

143

Potential Performance Improvements

In addition to the performance improvements in time, efficiency, decision
support, and client satisfaction, several other potential improvements surfaced.
This research was conducted while this technology was relatively new to the
company, and it was revealing to see how people were beginning to find new uses
for it that could result in additional performance improvements. For example, one
associate referenced its potential use in benchmarking best practices: “A lot of
times somebody will have a problem with the analysis or they’re trying to develop
a method and something’s not working. If there’s a way to encourage people,
especially the sharper minds in the company, to look at these and offer solutions,
then I think that that could be extremely beneficial to the company.”
Similarly, performance improvement potential was recognized in the ability
to share other information, as found in this statement by a manager: “A good use
for it would be to input agency guidelines. It took about 45 minutes to search for
an EPA guideline, but it would be great to have the EPA guidelines available
when they needed them and not have to search for them—that would save a lot
of time.”
Finally, some managers recognized the potential for better and more
efficient communication with clients by providing a Web-accessible place to
share information on study status, reports, or other valuable information. As one
manager stated, “The real benefit of using BSCW is that it allows the client to
have access to it 24 hours a day. We work with clients that are spread out all over
the United States and even in Europe. If they log on, and get on the Web, they
can access that document, download it themselves, make changes in the
document.” Table 3 summarizes these performance improvements.

DISCUSSION AND CONCLUSIONS

Corporate practitioners and academic researchers recognize the increasing
rate of change in virtually all industries, often driven by dynamic technological
changes. Technologies that provide ways to reduce time and costs or improve
quality, efficiency, or customer satisfaction are readily embraced.
What have we learned that may be generalized from this one small case
study that may prove beneficial to the managers and researchers considering
technology in support of distributed knowledge management?
First, we learned that there was a clear need, a pressing business reason,
for investing in this collaborative technology as well as for initiating the
knowledge sharing process—relative advantage. The president had recognized
the need for a mechanism to share information across the organization. This need
arose from identified redundancies in processes and the need for more up-todate sales and operational information accessible in a distributable format.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

144

Jones and Kochtanek

Table 3. Summary of performance improvements attributed to BSCW and
knowledge sharing at a contract research organization
Stakeholder
Leadership Team

Performance Improvements
•
•
•

Marketing

•
•
•

Quality
Assurance/Compliance

•
•
•

Other Performance
Improvements

•
•
•
•
•
•

Time savings due to fewer meetings, less tracking of reports and data
Better decision making due to availability of the most recent information
available
Improved quality and performance due to monitoring of projects on realtime basis
Improved efficiency on client monitoring and prospect identification
Improved responsiveness to managements needs for current sales
forecasts
Availability of marketing information on and off site to improve
marketing efforts and responsiveness to clients
More responsive to client needs by trending patterns in shared
information
Improved efficiency with fewer meetings and ability to prioritize
projects
Improved quality and problem solving by centralizing master schedules
and detecting common problems
Sharing of best practices within functional areas and among divisions
Sharing information throughout company such as new government
regulations
Improved responsiveness to client needs by sharing information on
study status and reports on a real-time basis with 24/7 availability
Improving organizational learning by sharing knowledge throughout the
company
Increased innovativeness by sharing ideas
Improved responsiveness to stakeholders including employees and
shareholders by sharing reports and other information on a timely basis

The perceived relative advantage of the system was the driving force
behind the subsequent diffusion and effective use of this collaborative technology at this company. This is consistent with the work of several researchers
including Beckman (1999) and Pan and Scarbrough (1999), who asserted that the
perceived relative advantage of knowledge sharing (e.g., time-savings, increased customer satisfaction, improved decision making, etc.) would provide a
motivating influence on employee behavior to share knowledge.
In this study, relative advantage emerged as the primary determinant
influencing use of the CSCW technology to facilitate knowledge sharing as
discussed above, but relative advantage also emerged as being context specific.
For example, the leadership in this case study perceived relative advantage in the
ability to acquire the most recent information and knowledge to make the best
decisions and monitor the financial status and health of the company, and to use
it as a control mechanism to monitor employee productivity. Leadership also
clearly perceived relative advantage in using this type of technology to facilitate
organizational learning for continuous improvement. They recognized potential
improvements in client satisfaction by increasing turnaround time and solving
client concerns.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Success Factors in the Implementation of a Collaborative Technology

145

Second, we learned leadership influence was a powerful factor in the initial
adoption and subsequent effective use of a collaborative technology and
knowledge sharing. In this situation, the initial forced adoption of the collaborative technology to facilitate knowledge sharing by the president was a very
effective means of jump-starting the system. The issue of accountability proved
to be a very powerful motivator in employee behavior in adopting and implementing the systems. When employees, from clerks to top management, understood
that their behaviors and actions were being monitored and that they were being
held accountable for using the collaborative technology to input valuable and
needed information and knowledge, they responded quickly and positively.
Based on the emerging themes from the results of this study, a new model
that we call the “Motivation-Maintenance Technology Implementation Model”
is proposed (Figure 4). The foundation for this model lies in respondent
perceptions. Most respondents indicated that only certain factors (independent
variables) truly motivated them to effectively and continually use this CSCW
technology to share information and knowledge. The major factors included the
relative advantage of the system and the influence of leadership. Secondarily, the
reward/compensation structures associated with using the technology and
sharing knowledge were also mentioned as somewhat important. In contrast, the
other factors from the original study model were expected or assumed to be
available for them.
These results were analogous to a classic management theory called
“Herzberg’s two-factor model” (Hellriegel & Slocum, 1996). In this theory,
Herzberg suggests that there are two separate and distinct factors that influence
satisfaction or dissatisfaction. The factors associated with positive factors are
called motivational influences. In contrast, factors that were associated with
dissatisfaction were called hygiene factors. Herzberg contends that these
hygiene factors are necessary to maintain satisfaction but are expected by
associates. In and of themselves, they do not contribute to increased satisfaction.
On the other hand, the factors called motivation factors do motivate employees.
The major finding was that perceived relative advantage appeared to be the
major influence on CSCW usage to facilitate knowledge sharing as well as the
perceived performance improvements resulting from knowledge sharing. However, the results from the interviews also support findings that a strong leadership
that supports knowledge sharing, enabled by a collaborative technology, was also
a very important influence on these users. Since leadership support also included
some form of reward/incentive structures to motivate individuals to share their
knowledge and use this system to do so, this was also considered a motivating
factor. Therefore, using Herzberg’s theory as a framework, these factors would
be considered the motivational factors.
In contrast, the factors considered to be maintenance (hygiene) factors
include compatibility and time/training (subcomponents of infrastructure);

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

146

Jones and Kochtanek

Figure 4. Maintenance-motivation technology implementation model
Relative
Advantage

Motivational Factors

Leadership

Reward /
Incentive
Systems

Continued &
Effective Use of
CSCW
Technology to
Facilitate
Knowledge Sharing

Maintenance Factors

Infostructure
(Recency/
relevancy)
Compatibility

Technology
Experience

Training &
Time

Trust /
Communication

Peer influence
(Opinion
Leaders)

Continued & effective use of
CSCW technology to
facilitate knowledge

Security
Concerns

Attitudes
(Adopter
Categories)

Performance
improvements

infostructure (including rules for managing the system for recency and relevancy); trust/communication and peer influence (subcomponents of infoculture);
and all of the subcomponents of individual concerns (prior technology experience, security concerns, and adopter category/attitudes toward technology and
change). The results from this study demonstrated that while each of these
factors was considered important by the respondents, they did not appear to
motivate users to continually and effectively use the CSCW technology for the
purpose of sharing their information or knowledge. Rather, they were expected
to be at a certain level. If not, they were considered to be dissatisfiers, or a
hindrance, but did not truly influence use or the sharing of knowledge. For
example, the computer hardware and software systems were expected to be
compatible. If they were not, users assumed that the IT (information technology)
department would correct the problem. They similarly assumed that they could
get the training and time they needed from the IT department or their managers.
In terms of infostructure, they normally assumed that the rules for management
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Success Factors in the Implementation of a Collaborative Technology

147

of the information (recency and relevancy) were controlled either by management or by project or department needs and requirements.
Trust and communication were deemed to be adequate to use this system
as well as the level of perceived security to share information within the
organization. However, in the future evolution of the system, when dealing with
stakeholders outside the firm (clients, suppliers), most respondents felt there
should be additional training and communication about the CSCW technology, its
level of security, and management of the information on the system. Again,
however, this was expected rather than motivational. In terms of prior technology experience and user attitudes, those respondents who had more positive
attitudes toward technology and change as well as those who had used different
technologies did appear to be more confident and comfortable using the
technology. However, these factors did not seem to motivate them to use the
system. On the other hand, the issue of incentives (a motivational factor) may
become relevant when dealing with those people who are resistant to using
technologies or sharing information for different reasons.

IMPLICATIONS FOR ORGANIZATIONS

In this study, relative advantage emerged as the primary determinant
influencing use of the CSCW technology to facilitate knowledge sharing, but it
also emerged as being context specific. For example, the leadership in this case
study perceived relative advantage as the ability to acquire the most recent
information and knowledge in order to make the best decisions, monitor the
financial status and health of the company, and use it as a control mechanism to
monitor employee productivity. In contrast to the leadership group, relative
advantage was perceived by the marketing/business development associates as
improving their efficiency by creating a repository of shared client information.
However, these associates as well as data entry and quality assurance associates perceived a unique attribute of relative advantage quite differently than the
leaders. Specifically, they perceived personal relative advantage in terms of
potential rewards or punishments for their effective use (or lack of) in providing
information and knowledge required by their bosses.
We would theorize that leaders have the power and authority to shape the
organization and to develop reward/compensation structures needed to support
the implementation of new technologies. This idea may be supported by the
classic management theory that people tend to do what they are rewarded for
or to avoid punishment. In this case study, using this CSCW technology and
sharing knowledge represented a forced adoption. This may be true in many
other organizations. However, once introduced, this research suggests that
developing rewards, incentives, or ties to performance appraisals may overcome
initial resistance. This may lead to more effective implementation of the new
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

148

Jones and Kochtanek

technology and knowledge sharing. It may also facilitate the routinization of use.
Again, the specific rewards, incentive, or ties to performance appraisals would
depend on the priorities of the leaders, the specific needs of the associates, and
the organizational culture.
Inherent in the above factors is the associated issue of accountability.
Results from this study indicate that accountability played a large role in
influencing effective use of a CSCW system and knowledge sharing. Most
associates acknowledged that accountability was a driving motivation in their
effective use of the technology and sharing valuable information on a timely
basis, especially when perceived to be tied to their performance appraisals.
Thus, despite the complexity of different organizations in different industries and cultures, using specific definitions of relative advantage, along with
supportive leadership that introduces effective reward/compensation structures
and accountability in the process, may significantly improve the successful
implementation of a CSCW technology to facilitate knowledge sharing in any
organization.

SUGGESTIONS FOR FUTURE RESEARCH

In terms of the correlation between knowledge management enabled by a
CSCW technology and resulting organizational performance improvements, it
would be helpful to establish quantitative measures to validate and confirm our
results and recommendations. If researchers were granted access to study timein-processes within an organization for specific tasks before and after, or with
and without the use of a knowledge sharing collaborative technology, this would
also help establish a more quantifiable relationship. Similarly, if researchers
measured the level of innovation in terms of new products or new processes
before and after implementation, or with versus without a CSCW technology, this
also would help to strengthen and validate the relationship. Quality could similarly
be measured by factors such as errors in reports or data. Customer satisfaction
could be measured by documented customer complaints with versus without a
CSCW system or before and after implementation, or by conducting customer
satisfaction surveys. The different success factors studied may be context
specific. Therefore, similar research in different types of organizations, industries, and situations would be interesting.
While this case study focused on a small business, the potential for
performance improvement extends to many other businesses including large,
small, or non-profit organizations. Given the potential improvements in time/
efficiency, customer satisfaction, and innovation enabled by Internet and ecommerce technologies, there exists an array of potential research explorations
involving collaborative technologies and their ability to support corporate knowledge sharing. Some of the key questions that we might need to ask include:

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Success Factors in the Implementation of a Collaborative Technology

•
•
•
•

149

Is the context of relative advantage determined by role, industry, or other
factors?
Is knowledge sharing heavily influenced by corporate culture, industry
norms, or other factors?
Is the willingness to share knowledge via a collaborative technology
influenced by prior technology experience, leadership influences, or other
factors?
Are there other factors that significantly contribute to knowledge sharing
and the use of collaborative technologies (i.e., personal agendas, reward/
compensation systems) that should be considered in management decisions
to adopt knowledge sharing?

REFERENCES

Anonymous. (1999). Knowledge-sharing roundtable. Informationweek, 731,
6ER-12ER.
Barker, R.T., & Camarata, M.R. (1998). The role of communication in creating
and maintaining a learning organization: Preconditions, indicators, and
disciplines. Journal of Business Communication, 35(4), 443-467.
Beckman, J. (1999). The current state of knowledge management. In J. Liebowitz
(Ed.), Knowledge management handbook. Boca Raton: CRC Press.
Ciborra, C.U. (1996a). Mission critical: Challenges for groupware in a pharmaceutical company. In C.U. Ciborra (Ed.), Groupware & teamwork. New
York: John Wiley & Sons, Ltd.
Ciborra, C.U. (1996b). What does groupware mean for the organizations hosting
it? In C.U. Ciborra (Ed.), Groupware & teamwork. New York: John
Wiley & Sons, Ltd.
Ciborra, C.U., & Patriotta, G. (1996). Groupware & teamwork in new product
development: The case of a consumer goods multinational. In C.U. Ciborra
(Ed.), Groupware & teamwork. New York: John Wiley & Sons, Ltd.
Davenport, T.H. (1999). Knowledge management & the broader firm: Strategy,
advantage & performance. In J. Liebowitz (Ed.), Knowledge management handbook. Boca Raton: CRC Press.
Drucker, P.F. (1995). Rethinking work. Executive Excellence, 12(2), 5-9.
Failla, A. (1996). Technologies for coordinating in a software factory. In C.U.
Ciborra (Ed.), Groupware & teamwork. New York: John Wiley & Sons, Ltd.
Mancini, J. (1998). $500 million just doesn’t seem to go very far anymore.
Inform, 12(3), 6-8.
Orlikowski, W. (1996). Evolving with notes: Organizational change and groupware
technology. In C.U. Ciborra (Ed.), Groupware & teamwork. New York:
John Wiley & Sons, Ltd.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

150

Jones and Kochtanek

Pan, S.L., & Scarbrough, H. (1998). A socio-technical view of knowledgesharing at Buckman Laboratories. Journal of Knowledge Management,
2(1), 55-66.
Pan, S. L., & Scarbrough, H. (1999). Knowledge management in practice: An
exploratory case study. Technology Analysis & Strategic Management,
11(3), 359-374.
Riesenberger, J.R. (1998). Executive insights: Knowledge—The source of
sustainable competitive advantage. Journal of International Marketing,
6(3), 94-107.
Rogers, E.M. (1995). Diffusion of innovations (4th ed.). New York: The Free
Press.
Rumizen, M.C. (1998). Report of the second comparative study of knowledge
creation conference. Journal of Knowledge Management, 2(1), 77-81.
Scheraga, D. (1998). Knowledge management competitive advantages become
a key issue. Chemical Market Reporter, 254(17), 3-6.
Senge, P.M. (1997). Creating learning communities. Executive Excellence,
14(3), 17-18.
Solomon, C. (1998). Sharing information across borders & time zones. Workforce,
3(2-Supplemental), 12-18.
Townsend, A.M., DeMarie, S.M., & Hendrickson, A.R. (1998). Virtual teams:
Technology & the workplace of the future. Academy of Management
Executive, 12(3), 17-29.
Wigg, K.M. (1999). Introducing knowledge management into the enterprise. In
J. Liebowitz (Ed.), Knowledge management handbook. Boca Raton, FL:
CRC Press.

ENDNOTES

These criteria included a great organizational emphasis on teamwork, a
need for experts who can filter and select contributions to the repository
based upon their usefulness, and forums where contributions represent a
source of personal satisfaction. The organizational culture should also
embrace and reward the use of groupware and teamwork.
Effectiveness in this study was measured by the actual usage of the
technology and the meaningful contributions to the system.
The quota system used in this study represented an attempt to interview a
proportionate number of users from each of the major functional groups
using this collaborative technology. This included middle management, data
entry, and quality assurance/compliance.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 151

Chapter VIII

Supporting the
JAD Facilitator with the
Nominal Group Technique
Evan W. Duggan, University of Alabama, USA
Cherian S. Thachenkary, Georgia State University, USA

ABSTRACT

Joint Application Development (JAD) was introduced in the late 1970s to
solve many of the problems system users experienced with the conventional
methods used in systems requirements determination (SRD) and has produced
noteworthy improvements over these methods. However, a JAD session is
conducted with freely interacting groups, which makes it susceptible to the
problems that have curtailed the effectiveness of groups. JAD outcomes are
also critically dependent on excellent facilitation for minimizing dysfunctional
group behaviors. Many JAD efforts are not contemplated (and some fail)
because such a person is often unavailable. The nominal group technique
(NGT) was designed to reduce the impact of negative group dynamics. An
integration of JAD and NGT is proposed here as a crutch to reduce the
burden of the JAD facilitator in controlling group sessions during SRD.
This approach, which was tested empirically in a laboratory experiment,
appeared to outperform JAD alone in the areas tested and seemed to
contribute to excellent group outcomes even without excellent facilitation.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

152 Duggan and Thachenkary

INTRODUCTION

There is widespread support for the belief that systems requirements
determination (SRD)—discovering and documenting the features that an information system should deliver—is an extremely important but very difficult aspect
of software development (Borovits et al., 1990; Byrd et al., 1992; Cheng, 1996;
Holtzblatt & Beyer, 1995; Raghaven, et al, 1994). This difficulty often leads to
systems failures due to both development shortcomings (failure to establish the
required features in the required time) and usage factors (abandonment by its
intended beneficiaries) (Lyytinen, 1988). Several factors account for this
difficulty, but the nature of the interaction among system developers, users, and
stakeholders is the prime contributor (Antunes, 1999; Holtzblatt & Beyer, 1995).
User-developer communication and stakeholder negotiations assume greatest importance at the requirements determination phase of the systems development life cycle (SDLC). Here, the specific details of the problem to be solved
and the needs to be satisfied are clarified. It is here, however, that poor
communication is most pervasive (Dieckmann, 1996; Holtzblatt & Beyer, 1995).
JAD is a team-oriented approach that has been widely used to (1) confront the
communication barriers to effective information elicitation and (2) increase
users’ contribution to this key systems development activity (Byrd et al., 1992).
JAD assembles a diverse group of users, analysts, and managers from various
sectors of an organization to jointly specify requirements in a face-to-face
workshop.
Despite its success in comparison to conventional SRD methods, JAD has
failed somewhat to deliver on its initial promise to forge the team rapport
necessary to alleviate known communication impediments to effective SRD, and
has introduced other group-related problems (Dean et al., 1997; Kettelhut,
1993). A major reason for this failure is that JAD workshops are conducted
under the freely interacting meeting structure where spontaneous communication occurs among group members with minimal control imposed by the communication structure (Van de Ven & Delbecq, 1974). Groups that deliberate in this
manner typically experience many of the problems in which social and emotional
dynamics obstruct the accomplishment of the objectives of the meeting (Kettelhut,
1993). The success of a JAD session often is dependent on the extent to which
these problems are alleviated. This places a very high premium on excellent
facilitation (Carmel et al., 1995; Davidson, 1999; Wood & Silver, 1995).
Facilitators have been offered several prescriptions for minimizing these
problems (Andrews, 1991; Carmel et al., 1995; Davidson, 1999; Kettelhut, 1993;
Wood & Silver, 1995). Many of these are contained within the NGT, a facilitated
technique that focuses on alleviating negative group dynamics in meetings where
participants interact in a highly structured manner. This technique could be
applied in the decision-making stages of a JAD workshop to provide a comprehensive set of procedures for increasing the group’s effectiveness. NGT

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 153

reputedly increases the effectiveness of creative problem-solving groups (Delbecq
et al., 1986). Its easy-to-apply protocol supports facilitators in producing results
that fairly accurately reflect the combined judgment of groups engaged in
problem-solving meetings (Zuech, 1992).
Our thesis is that the application of NGT in the JAD workshop will help to
reduce the criticality of excellent facilitation for high-quality JAD results, and
that this integrated communication structure will induce more acceptable results
from less than excellent facilitation. This presumption is very important because
excellent JAD facilitation is a scarce commodity (Carmel et al., 1995), despite
several years of fairly extensive JAD practice (Davidson, 1999) and increasingly
common usage (Dennis et al., 1999; Kettelhut, 1997). In this study, we examine
the effects of the integration of NGT and JAD structures on the communication
problems that typically beset user-developer interactions in SRD, when JAD
alone is used. The major objective is to determine whether NGT, in combination
with JAD, reduces the facilitator’s burden in curbing dysfunctional group
behaviors and thereby contributes to improved performance.

REVIEW OF RELEVANT LITERATURE

The prevailing viewpoint is that SRD, which is a complex process incorporating a variety of features and often conflicting stakeholder interests (Vessey
& Conger, 1994), is a critical determinant of system development success or
failure (Byrd et al., 1992; Cheng, 1996; Raghaven et al., 1994). Unfortunately,
the dominant experience is that inadequate interaction and poor communication
among system developers and users characterize this process (Holtzblatt &
Beyer, 1995). A variety of SRD approaches have been used to elicit information
from knowledgeable managers and users. These include but are not limited to
interviewing, survey by questionnaire, JAD, focus group meetings, brainstorming, prototyping, goal- and scenario-based techniques, critical success factor and
task and protocol analyses, and ethnographic techniques.
Interviewing has been the most prevalent and best-known technique
(Raghaven et al., 1994; Watson & Frolick, 1993), but this approach has proven
inadequate for resolving competing requirements and securing stakeholder
agreement (Dennis et al., 1999; Dieckmann, 1996). It is also difficult to
determine whether all problems are unearthed and all requirements captured,
and also to gauge the adequacy of the participation of those interviewed. These
deficiencies seriously challenge the accuracy, completeness, consistency, and
clarity of the resulting requirements (Dean et al., 1997). Therefore, JAD was
designed to correct these problems (Dennis et al., 1999; Purvis & Sambamurthy,
1997).
While JAD is not as widely practiced as interviewing (Purvis &
Sambamurthy, 1997), it has gained in popularity (Jackson & Engles, 1996) and
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

154 Duggan and Thachenkary

is increasing in common usage (Dennis et al., 1999; Kettelhut, 1997). Researchers (Dean et al., 1997-98) and practitioners (Jones, 1996; Spina & Rolonda,
2002) consider JAD best practice for structuring group interaction in participatory environments and for operationalizing user involvement (Uden, 1995). It has
been used increasingly with rapid application development (RAD) projects (Rist,
2001) and dynamic systems development method (DSDM), a RAD-based
technique used extensively in the UK (Barrow & Mayhew, 2000; BeynonDavies et al., 2000).
JAD is known by several other names, including facilitated technique,
facilitated workshop, joint application review, accelerated design, and usercentered design (Carmel, et al., 1995; Dean et al., 1997-98). Several derivatives
exist (Asario 2000), and some organizations have made their own modifications
to the formal JAD structure (Davidson, 1999). In addition to its application in
systems development, JAD (under any of these names) also has been used in
several other organizational decision-making contexts (Davidson, 1999; Kettelhut,
1997).
JAD places significant emphasis on the communication aspects of requirements elicitation (Liou & Chen, 1993-94; Purvis & Sambamurthy, 1997). System
developers, users, and managers assemble in a synchronous, three-to-five-day
workshop to specify information requirements and make system decisions under
the guidance of a trained facilitator (Andrews, 1991; Wood & Silver, 1995). One
of JAD’s important intentions is to develop the necessary team rapport in order
to bridge the communication gap and exploit potential synergistic opportunities
to produce higher quality system requirements (Dean et al., 1997; Purvis &
Sambamurthy, 1997). The steps in the JAD process, as described by Wood &
Silver (1995), are highlighted in Table 1.
Agile development methods are being used increasingly in systems development paradigms such as extreme programming (XP), features-driven development, adaptive software development, and DSDM (Highsmith & Cockburn,
Table 1. The five phases of JAD
JAD Phases

Key Activities

1. Project Definition

Agree scope and objectives; secure management commitment and
willingness to release experts (Liou & Chen, 1993-94)

2. Background Research

Acquire knowledge about existing business processes and problem
domain

3. Preparation for Workshop

Finalize meeting logistics and facilities

4. The Workshop (or Session)

Offsite meeting to minimize potential interruption; facilitator
demonstrates excellent interpersonal relationship skills and
understanding of group dynamics (Liou & Chen, 1993-94); the
facilitator should be neutral and objective (Anson et al., 1995;
Schuman, 1996) and should strive for broad user participation and
focus on agenda (Carmel et al., 1995)

5. Preparation of the Final Document

Review document is in the presence of participants and sponsor(s);
confirm, and get approval

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 155

2001). These methods typically collapse several life cycle stages for speed of
delivery and produce deployment-ready modules iteratively and/or incrementally. Traditional methods, however, still account for a large percentage of
development efforts1, especially for large systems with several stakeholders,
and in environments with fairly stable business processes. JAD also may be used
with newer methods, even when requirements are not completely pre-specified.
For example, it is often used with RAD to help generate use cases in objectoriented development. JAD conceivably could be used in short bursts at the
commitment stage of XP’s planning game to prioritize story cards and identify
implementation risks.
The performance of JAD facilitators during the critical interactions of the
JAD workshop is pivotal to the success of the meeting (Carmel et al., 1995;
Davidson, 1999; van Murik, 1994). Without a great deal of assistance from the
JAD communication structure, they bear the responsibility of guiding the session
toward the attainment of the desired objectives (Dean et al., 1997) and securing
decision outcomes that reflect the combined judgement of the group (Wood &
Silver, 1995). Anson et al. (1995) and Dowling and St. Louis (2000) found that
the quality of facilitation significantly influenced relationships and moderated
process outcomes. Some of the potential group problems that challenge JAD
facilitators are listed in Table 2.

Table 2. Potential JAD problems
Problem

Description

Search Behavior

Inadequate diagnosis and premature specification of solutions (Delbecq et al.,
1986)

Destructive Dominance

Less desirable contributions of powerful participants overwhelm useful ideas from
others (Wood & Silver, 1995)

Anchoring

Excessive focus on tangential issues raised by influential participants causes
digression from the main objective (Ven de Ven & Delbecq, 1974)

Groupthink

Over commitment to group harmony such that group cohesion becomes the de
facto decision criterion (Kettelhut, 1993)

Risky-shift Behavior

Empirically observed phenomenon where the group shifts from the risk profiles of
its individual members (Kettelhut, 1993)

Elective Participation and
Free Loading

Group members contribute of their own volition, and some may not contribute at
all (Carmel et al., 1995)

Commitment Errors

The group arbitrarily enlists the resources of its organization to unattainable
objectives (Kettelhut, 1993)

Goal-setting Errors

Scheduling that reflects unrealistic group aspirations (Kettelhut, 1993)

The Abilene Paradox

Conflict avoidance that permits group decisions that are contrary to the desires of
the individual members (Kettelhut, 1993)

Conforming Behavior

Participants acquiesce to the emergent group norm (Delbecq et al.., 1986)

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

156 Duggan and Thachenkary

Meeting structures may influence the conditions responsible for process
loss or gain, but outcomes are determined more often by the extent to which the
intended structure is appropriately invoked (Bostrom et al., 1993; Gopal et al.,
1992-1993; Poole & DeSanctis, 1990). Groups that faithfully apply the intended
communication structure outperform those that do not (Anson et al., 1995). The
crux of our argument is that it is the facilitator’s responsibility to inspire faithful
appropriation of the adopted meeting structure (Bostrom et al., 1993; Schuman,
1996, van Murik, 1994).
While some meeting techniques provide a challenge for facilitators, NGT is
considered to be conducive to faithful appropriation (Ho et al., 1999), and groups
react very positively to the technique (Wood & Silver, 1995). Many of the
recommendations for offsetting the dysfunctional effects of the freely interacting group technique used in JAD workshops (i.e., brainstorming, anonymity;
prescriptions for reducing destructive dominance and increasing participation,
strategies for precipitating consensus) (Wood & Silver, 1995) and proposals for
overcoming groupthink (Kettelhut, 1993) seem to be standard features of NGT.
NGT is used in problem-solving situations to elicit individual knowledge,
views, and opinions (Zuech, 1992). It is particularly useful in situations where
group members must pool their judgements to determine a particular course of
action from a large number of alternatives (Hornsby et al., 1994; Zuech, 1992).
NGT combines the effects of two factors—conveyance (uninhibited idea
generation during which free interactions are restricted) and convergence
(precipitation toward consensus). These are accommodated in the five steps
(Table 3) that help to downplay the social and emotional dynamics that affect the
performance of freely interacting groups (Delbecq et al., 1986; Ho et al., 1999).
NGT’s superiority over the interacting group technique has been demonstrated in creative, problem-solving situations (Delbecq et al., 1975; Ven de Ven
& Delbecq, 1974) and with heterogeneous groups working with complex
problems (Stephenson et al., 1982). Several other noteworthy attributes that
have been verified empirically include participants’ satisfaction with the process
(Korhonen, 1990), usefulness in identifying different problem dimensions and
error reduction (Frankel, 1987), and its adaptability to a variety of problem
domains (Chapman, 1998). NGT has been successfully combined with techniques such as multidimensional scaling (Frankel, 1987), multi-attribute utility in
decision analysis (Thomas et al., 1989), quality function deployment (Ho et al.,
1999), and the analytical hierarchy process (Teltumbde, 2000). It also has been
used to identify potential problems in information systems deployment (Henrich
& Greene, 1991).
Researchers also have evaluated the effect of using group support systems
(GSS) to improve negative JAD outcomes (Carmel et al., 1995; Dennis et al.,
1999; Liou & Chen, 1993-94). The need to design structures to improve the
conveyance of information and convergence toward consensus for effective
group decision making are common objectives of NGT and GSS (Beruvides,
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 157

Table 3. NGT process
Step

Activity

Description

Idea
Generation

Participants independently and silently generate ideas regarding goals and problem
solutions in writing. This separation of creative thinking from idea evaluation reduces
emotional attachment to an idea and contributes to greater objectivity (Delbecq et al.,
1986; Van de Ven & Delbecq, 1974).

Idea
Recording

The facilitator records one idea at a time from group members in a round-robin format
until all participants have completed their list of ideas. This accommodates increased
participation (Stephenson et al., 1982; Ven de Ven & Delbecq, 1974).

Discussion
and
Clarification

Each idea is discussed for clarification and subsequent evaluation, without either
critical evaluation or lobbying, which reduces conformance pressure on lower ranking
group members.

Ranking

Participants independently rate and rank all the ideas.

Decisionmaking

The final decision-making on the priority ordering of the alternatives (if necessary) is
based on voting and mathematical pooling of the individual rankings.

1995). NGT is primarily a manual technique, whose deliberations are usually not
automatically documented (Liou & Chen, 1993-94), while GSS uses networked
computers and specialized software to allow group members to interact anonymously but fully aware of the responses of others (Beruvides, 1995; Dennis et
al., 1999). Thus, a GSS is a more sophisticated and efficient application of the
basic NGT technique.
Carmel et al. (1995) used case studies to compare JAD in its traditional
application with JAD using GSS (EJAD) and found that EJAD sessions were
better with respect to both the degree of participation they induced and decision
time, but not as effective for conflict resolution. They recommended a more
active facilitator role for EJAD. In comparing group performances under NGTlike support and GSS support, Watson et al. (1988) and Liou and Chen (19931994) found no significant difference in task effectiveness under the two
structures. Since then, Dennis et al. (1999) have concluded that GSS-enabled
JAD has helped to reduce model-building time by approximately 75%. NGT, in
combination with JAD, however, offers an alternative medium for conducting
group research, particularly where the deployment of GSS is not yet pervasive.

RESEARCH MODEL AND HYPOTHESES

The research model (Figure 1) was adapted from several GSS process
models (Nunamaker et al., 1993; Ocker et al., 1995-96; Pinsonneault &
Kraemer, 1989) to abstract the relevant relationships among the variables of
interest in our proposed process. It depicts the transformation that occurs in a
facilitated group session as a result of the interaction of the contextual charac-

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

158 Duggan and Thachenkary

teristics of the group, task, facilitation, and meeting structure to contrive some
instrumental outcome.
The nature of the interactions is determined by this dynamic interplay, which
affects process effectiveness (the degree of process gain or loss experienced).
The meeting structure (the protocol that governs the pattern of interaction) and
the complexity of the task (the activities required to accomplish the group’s
objectives) also influence group dynamics. The quality of facilitation impacts and
is impacted by group factors, and a similar, reciprocal relationship exists between
the meeting structure and the effect of facilitation. The degree of adherence that
the structure permits and the skill of the facilitator demarcate this effect. The
effectiveness of the process influences both participants’ satisfaction with the
meeting and its results, and the quality of the decisions.
We believe that facilitators using integrated JAD and NGT in SRD will
achieve superior results to those using JAD alone. The nature of the stakeholder
issues that typify SRD precludes effective results based on conventional JAD
facilitation or makes success attainable by only the very best facilitators.
Because of its amenity to faithful appropriation, the NGT structure will help to
reduce this crucial reliance on facilitator excellence for effective interactions
(greater participation, less destructive dominance, convergence toward consensus) and successful outcomes.
The theme that underpins our propositions is that the integrated communication structure provides assistance in reducing the performance gap between
expert and novice facilitators; the structure contributes more to the expected
difference in outcomes than heroic facilitator efforts. The parts of the hypotheses that refer to the effects of facilitation are necessarily exploratory. While it
is intuitively appealing to predict that expert facilitators will contribute to more
effective results than their less skilled counterparts, there is no theoretical basis

Figure 1. General research model
Group Factors
Dynamics
Size
Effort

Task

- Effectiveness

Nature
Complexity

Facilitation

- Participants’
Satisfaction

Outcome
- Quality
- Participants’
Satisfaction

Skill Level
Control of Structure

Mtg.Structure
JAD
JAD & NGT

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 159

to predict that either will inspire more faithful application of the integrated
structure.
The first four hypotheses are concerned with impacts on process effectiveness, an intermediate indicator of successful outcomes. Process effectiveness
is concerned with the manner in which the input variables (including the
communication structure and the facilitator) interplay to generate process
conditions that are satisfying for the participants and are conducive to right
outcomes. It is partitioned into several impacts—the extent of participation; the
degree of destructive dominance, where influential members commandeer the
deliberation to the exclusion of useful contributions from other members; and the
capacity to confront conflict and converge toward consensus, so that unresolved
issues are minimized. Under JAD, it is the facilitator who must devise innovative
means to draw introspective members “out of their shells” and “chill” dominators
(Wood & Silver, 1995). But participation is involuntary within NGT; the structure
compels the involvement of all participants, which helps to reduce the facilitator’s
burden.
Hypothesis 1: The integrated communication structure will contribute to a
significantly higher level of process effectiveness, but the expert facilitator
will not.
Hypothesis 2: The integrated communication structure will induce a significantly higher level of group participation, but the expert facilitator will not.
Hypothesis 3: The integrated communication structure will contribute to less
domination, but the expert facilitator will not.
Hypothesis 4: The integrated communication structure will help groups attain
a higher degree of consensus, but the expert facilitator will not.
Satisfaction with the process may be a post-requisite measure of effectiveness. There seems to be an inverse relationship between the existence of
dysfunctional group behaviors and group members’ satisfaction with the process. If, indeed, destructive dominance is controlled, participation has increased,
and consensus is achieved, more group members should be satisfied with the
process. But usually, the disparity in background, authority, and knowledge in
SRD makes it difficult for facilitators (without the benefit of a supportive meeting
structure) to stem the tide of group behaviors that are not conducive to overall
group satisfaction. We, therefore, propose:
Hypothesis 5: The integrated communication structure will contribute to a
significantly higher level of participant’s satisfaction with the process, but
the expert facilitator will not.
Satisfaction with the decisions and high-quality requirements, which are
desired outcomes of the intervention of the integrated structure, are postulated
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

160 Duggan and Thachenkary

as important measures of the overall success of the process. We contend that
participants who use the integrated structure to generate requirements should
identify more with the results and will feel a greater sense of ownership of the
decision than JAD users. This increased affiliation derives from the greater
involvement in the decision making and satisfaction with the process that the
integrated structure supports. More effective participation in the deliberations
also is expected to result in process gain, which should help to improve the quality
of the output—the requirements.
Hypothesis 6: The integrated communication structure will contribute to a
significantly higher level of participant’s satisfaction with the outcome, but
the expert facilitator will not.
Hypothesis 7: The integrated communication structure will contribute to
significantly higher quality requirements, but the expert facilitator will not.
Because of NGT’s greater amenity to faithful appropriation, the integrated
protocol is expected to reduce the facilitator’s burden in minimizing group
behaviors inimical to good outcomes and help to reduce the reliance on excellent
facilitation for high-quality results. The reduction in dysfunctional group behavior
(i.e., destructive dominance is used as a surrogate measure) exhibited in sessions
conducted under the integrated structure and under JAD should, therefore, be
greater for the unskilled facilitator than for the skilled facilitator.
Hypothesis 8: The difference in destructive dominance in JAD and the
integrated structure sessions will be greater when the facilitator is low
skilled than when he or she is highly skilled.

RESEARCH METHOD

A completely randomized design was used to conduct the laboratory
experiment. In this design, two levels of group communication structure (i.e.,
standard JAD and the integrated protocol) were crossed with two levels of
facilitation (i.e., expert and novice). The facilitated group session was the unit
of analysis.

Procedures

Twelve professional facilitators from four facilitator associations, 18 scribes,
and 144 role players (75 females and 69 males) participated in this experiment.
Each participating facilitator led two sessions—one JAD and one the integrated
structure. The 24 experimental groups consisted of a mix of role players from a
wide cross-section of IS users, systems developers, business professionals (40%

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 161

of the participants), senior undergraduate (35%) and graduate students (25%).
This diverse combination of participants enriched the study by providing the
within-group heterogeneity to maximize the influence of the manipulated variables.
The professional organizations categorized the facilitators (who were all
volunteers) into the two skill levels by years of facilitation experience and the
number of sessions conducted. The 12 facilitators (six from each category) were
randomly pre-selected to conduct the experiments. A facilitator’s packet
containing the experimental task with instructions on how to conduct the sessions
and a script for the NGT protocol was provided before the day of the session.
Before each experiment, the facilitators participated in a further one-hour
debriefing session.
Student volunteers participated for extra credit in their systems analysis and
design classes. The four organizations (7 were asked originally) who participated, expressed great interest in the outcome of the experiment and were
promised a copy of the findings. These facilitator associations also canvassed
several business participants from among their membership, while other professionals participated during training exercises with these firms. Practicing or
trainee facilitators were not allowed to participate as role players.
The role players were assigned randomly to groups of six. Each group was
randomly assigned to one of the four experimental conditions: JAD conducted by
an expert facilitator, JAD conducted by a novice facilitator, the integrated
communication structure (that combined JAD and NGT) conducted by an expert
facilitator, and the integrated communication structure conducted by a novice
facilitator. In sessions that lasted approximately two hours, each group was
asked to generate high-level requirements for an integrated order processing,
inventory management, accounts payable and receivable, and distribution management system to solve information systems problems of a fictitious chain of
owned and franchised deli-style sandwich shops. The case was developed by
Marble (1992).
The documented requirements then were typed and verified by two independent volunteers and sent to the three expert judges, who rated them along
eight quality dimensions. Participants also completed a pre-session (i.e., objective background information) as well as a post-session survey to record their
perception of the sessions and facilitator-produced reports containing both
perceptual and objective observations.

Variables and Measures

The instruments used in this study—demographic information on participants collected before the experimental sessions, a post-session survey (PSS),
the facilitator’s report (FR), and the expert judge’s quality rating sheet (QRS)—
were adapted from previous research (Anson et al., 1995; Bailey & Pearson,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

162 Duggan and Thachenkary

Table 4. Variables and measures
Variable

Process Effectiveness
- Overall

Measure

Type of Measure

Seven-item scale (from PSS)

Perceptive

Participation

1.
2.

Facilitator count (from FR)
Two-item scale (PSS)

Objective
Perceptive

Destructive
Dominance

1.
2.

Facilitator observation (FR)
Two-item scale (PSS)

Objective
Perceptive

Consensus

1.
2.

Count of unresolved items at session end (FR)
Two-item scale (PSS)

Objective
Subjective

Participants’ Satisfaction
- With the Process

Four-item scale (PSS)

Subjective

Five-item scale (PSS)

Subjective

Aggregation of judges’ scores (maximum of five points
each) for accuracy, precision, completeness, conciseness,
relevance, creativity, consistency, and feasibility (from QRS)

Expert Rating

With the Outcome

Requirements Quality
Rating

1983; Green & Taber, 1980; Gouran, 1978) and revalidated. They measured the
dependent variables (Table 4) used in tests of hypotheses.
The following data were also obtained:
1.
2.
3.

Level of experience with SRD (single item from the pre-session instrument)
Level of business experience (from the pre-session instrument)
The level of effort expended by the group (assessed by the facilitator)

DATA ANALYSIS AND RESULTS

The main statistical procedures used to test the statistical hypotheses were
factorial multiple analysis of variance (MANOVA) for multiple dependent
variables and analysis of variance (ANOVA) for tests involving a single
dependent variable. The analysis of data was conducted in three main areas:
1.

An examination of the demographic profiles of the participants was
undertaken to establish whether potentially confounding input variables
(not manipulated experimentally) should be controlled statistically as
covariates. This analysis indicated (by the failure to reject the equality
hypotheses) that there was homogeneity across groups with respect to the
level of effort they expended for each communication structure used (pvalue < .093) and for facilitation (p-value < .254). Similar results were
obtained for equivalent experience in a professional business environment
and with SRD (p-value < .758 and < .290 for communication structure and
facilitation, respectively).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 163

The revalidated instruments indicated satisfactory evidence of internal
consistency (Cronbach’s Alpha) with reliability ratings of .9082 (postsession), .7931 (pre-session), .8287 (facilitators’ report), and .9754 (judges’
rating sheet).
The evaluations of the hypotheses follow.

Summary of Statistical Analyses

Table 5, which summarizes results from the test of the hypotheses, indicates
those hypotheses that were supported. No significant interaction effect was
found for any of the tests, nor was there any indication that facilitation made a
difference where the integrated structure did not. In all cases, it was observed
that the integrated structure outperformed JAD, regardless of the caliber of the
facilitator. The results indicate that skillful facilitators will induce greater
participation, inspire a higher level of satisfaction with the process, and contribute to higher quality requirements than unskilled ones under any of the meeting
structures. However, there was no significant difference attributable to facilitator competence for process effectiveness, destructive dominance, conflict
resolution, and satisfaction with the outcome.
The results, where both the integrated structure and facilitation competence
were found to contribute to improved group performance, may indicate that the
integrated technique makes good facilitation better in these areas. Taken
together, the other cases that indicated significant effect due to structure and the
insignificant effect due to facilitator competence are also useful results. They
provide empirical support for the conclusion that the integrated structure may be
able to achieve objectives (reducing destructive dominance, precipitating consensus, and contributing to satisfaction with the requirements) that have eluded
even highly skilled facilitators under JAD.
Hypothesis 8 (which examined the relative performances of novice and
expert facilitators under the integrated structure and JAD, respectively) was
also supported in the test in which two dependent variables were used in a twofactor MANOVA (Table 5). This is an important result, especially as we accept
that excellent facilitation is a scarce commodity. It implies that the integrated
structure solution can overcome potential deficiencies imposed by less than
perfect JAD facilitation. Although only destructive dominance was used in the
test, the graphs in Figures 2(a) to 2(d) demonstrate that this phenomenon may be
true for other important determinants of process effectiveness and other
desirable outcomes.
These figures demonstrate the disproportionate improvements in performance by unskilled facilitators compared to their more competent counterparts,
as both groups switch between JAD and the integrated structure. The mean
scores (where bigger scores are more desirable) for satisfaction with the
outcome, destructive dominance, satisfaction with the process, and overall

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

164 Duggan and Thachenkary

Table 5. Summary of results from the tests of hypotheses
Hypothesis

Type of Test

Difference Due to
Structure

Difference Due
to Facilitation

1. Process Effectiveness

ANOVA

Y (Sig. = .001)

N (Sig. = .070)

2. Level of Participation

MANOVA

Y (Sig. = .001)

Y (Sig. = .027)

3. Conflict Resolution

MANOVA

Y (Sig. = .001)

N (Sig. = .539)

4. Destructive Dominance

MANOVA

Y (Sig. < .000)

N (Sig. = .306)

5. Satisfaction with Process

ANOVA

Y (Sig. = .034)

Y (Sig. = .042)

6. Satisfaction with Outcome

ANOVA

Y (Sig. = .017)

N (Sig. = .202)

7. Quality of Requirements

ANOVA

Y (Sig. = .011)

Y (Sig. = .027)

8. Difference in Level of
Dysfunctional Behavior

MANOVA
(equivalent of
Hotelling’s T2)

Y (sig. = .032)

Not Applicable

Figure 2(a). Satisfaction with outcome
180

170

Mean Satisfaction with Outcome

160

150

140

Facilitator Level
130

Skilled

120

Unskilled

JAD

NJAD

Group Structure

Figure 2(b). Destructive dominance
80

Mean Destructive Dominance

Facilitator Level
Skilled
50

Unskilled

JAD

NJAD

Group Structure

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 165

Figure 2(c). Satisfaction with process
150

Mean Satisfaction With Process

140

130

120

Facilitator Level

110

Skilled
100

Unskilled

JAD

NJAD

Group Structure

Figure 2(d). Process effectiveness
260
250
240

Mean Process Effectiveness

230
220
210
200

Facilitator Level

190

Skilled

180

Unskilled

JAD

NJAD

Group Structure

process effectiveness, respectively, increased as skillful facilitators switched
between JAD and the integrated structure, respectively. However, unskilled
facilitators experienced a much larger increase for a similar shift. These all signal
the potentially beneficial effects of the integrated structure in making facilitation
less complex.

DISCUSSION AND IMPLICATIONS

This research effort was justified on the general acknowledgement that the
freely interacting meeting structure often contributes to dysfunctional group
behaviors that curtail the effectiveness of JAD. These behaviors impede
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

166 Duggan and Thachenkary

communication; contribute to process loss, which prevents groups from realizing
their true potential; and obstruct the realization of synergy. JAD, therefore, is
critically dependent on excellent facilitation (a scarce resource) to overcome
these potential problems. The study was designed to evaluate the legitimacy of
the claim that the integration of NGT and JAD could enhance facilitation
effectiveness and obviate this prerequisite of facilitation excellence for SRD
success.
The findings support this expectation in the dimensions tested. However,
caution is advised in the general interpretation of these results. They were
obtained under experimental conditions where the task was simulated, and the
time for its accomplishment was compressed. Additionally, the high-level
requirements generated in the experiments lacked the details that typify normal
systems specifications. Further, role playing under experimental conditions
cannot capture realistically the effects of power asymmetry, intensity, emotiveness,
and political turf issues that characterize the natural process. On the contrary,
participants in the experiments seemed far more willing to make concessions
than is typical in natural settings. This research effort, therefore, should be
viewed as a laboratory model that requires replication in the field.
A further limitation of this study was the difficulty in establishing incontrovertible criteria for classifying facilitators into the two skill levels of expert or
novice. Facilitators perform at least three distinctly different but complementary
functions (Wood & Silver, 1995). They carry out environmental analysis—highlevel data collection to circumscribe the problem domain and the solution goals.
They plan and organize the meeting agenda, tools, techniques, and resources for
the session; and they facilitate communication during the meeting. Only the latter
was required in our study, but success in all three areas characterizes the
“goodness” of facilitation. Effective performance at the earlier levels can
simplify the effort at the communication level. Even more, good facilitators
overall may not be competent at all levels.
Although years of experience as a facilitator and the number of sessions
conducted were suggested criteria, participating organizations were free to
consider other means of classification. Discriminant analysis, however, indicated that the resulting categorization seemed reasonable. It is recognized that
facilitation competence levels exist along a continuum, but for this experiment
only the levels at the extremes of the continuum were used, which allowed this
manipulated variable the maximum potential for exhibiting relative influence.
Two other limitations also could have impacted the results. The facilitators
were given the freedom to decide the order in which they conducted their two
sessions. In hindsight, the claim to any systematic effect of facilitator learning
might have been nullified, if we had included an experimental condition that
required randomization of the order in which each facilitator conducted his or her
two sessions. Additionally, the facilitators were not paid; they volunteered

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 167

because of their interest in the outcome. Non-payment conceivably could have
induced differences in performance levels for some facilitators and not others,
although it would not necessarily affect the manner in which each conducted his
or her two sessions.
The insignificant effects of facilitation competence for process effectiveness, conflict resolution, destructive dominance, and satisfaction with the
outcome seem to bear out the suggestion that for these, the integrated technique
had the capability to equalize facilitator performance across the skill level
continuum. This inference seems consistent with the accepted role of facilitation
as the guardian of the faithful appropriation of the meeting structure. It is
precisely the integrated structure’s amenity to faithful appropriation that gave
rise to the proposition that even otherwise incompetent facilitators may experience successful results (induced by the structure) that facilitation competence
could not by itself enable.
Despite JAD’s success in some areas, several authors emphasized the
imperative for facilitators to “chill the dominators” and promote consensus. Yet
the experience has been that even excellent facilitators have not always
alleviated the deleterious impact of dominance by both powerful and extroverted
group members in group sessions generally, and in JAD workshops in particular.
Similarly, the desire by less influential group members to avoid conflict has tacitly
contributed to JAD outcomes that reflect the decision preferences of dominators
rather than group consensus. The indication that highly competent facilitators are
no more likely to reduce these problems than their less skillful counterparts
demonstrates the facilitators’ inability to curb dysfunctional behaviors and
enhances the value of the integrated structure.
For the other three measures—level of participation, satisfaction with the
process, and quality of the requirements—the significant result due to both
communication structure and facilitator competence suggests that these effects
may be additive; the integrated approach also helps good facilitators produce
better results. It may appear somewhat inconsistent, though, that facilitator
competence was shown to have a significant effect on the level of participation—one of the indicators of process effectiveness, but not on process
effectiveness itself. Similarly, there is an apparently conflicting indication of
significant facilitator effect on participants’ satisfaction with the process and not
with the outcome.
One credible explanation for the former may be that excellent facilitators
have a larger repertoire of artifices to cajole introspective group members
toward more active participation, as one of the authors observed during the
experiments. But these innovations were not enough to rectify more pernicious
problems like destructive dominance, anchoring, or subjective defense of opinions (especially those expressed by influential group members) that are not
supported by the group. This result is a good one in the larger scheme of the
integration of JAD and NGT; facilitation is still an important component in the
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

168 Duggan and Thachenkary

idea clarification stage. Ultimately, the objective is not necessarily to equalize
participation, but to ensure breadth of participation and influence proportional to
the level of knowledge of the participants. The problem is the dysfunctional
consequences of inequality of influence and participation due to power or
personality that does not contribute to consensus. It seems that the communication structure and the facilitator can play mutually supportive roles to establish
this objective.
The significant facilitator effect for process satisfaction (but not satisfaction with the outcome) also may be due to the ability of more skillful facilitators
to draw on a larger set of tools to keep the sessions fluid, interesting, and more
efficient, which helps participants feel better about the process itself. However,
it is possible that the memory of people-related problems (prior to and even
despite facilitator intervention) lingered and affected participants’ evaluation of
the actual outcome.
The results also supported the proposition that the difference in facilitator
effectiveness under the integrated structure and JAD was greater on the
average for low-skilled facilitators than it was for the highly skilled facilitators.
Another way to view this is that the integrated structure induces greater
improvements in the results produced by groups directed by unskilled facilitators
than in those directed by highly skilled facilitators in comparison to similar results
under JAD. It reduces the onus for excellent process outcomes on the competence of the facilitator. This is an important finding, which could have potentially
far-reaching implications for practitioners. It suggests that superb facilitation is
still desirable, but it is not a critical success factor under the integrated structure.

CONCLUSION

The integrated approach appears to be able to preserve the benefits
commonly attributable to JAD (i.e., speed of SRD and user involvement) and
treat the group behavioral problems that mitigated its success. If this occurs,
several benefits could accrue to practitioners. First, some JAD efforts are not
contemplated because excellent facilitation is not always available; researchers
and practitioners agree that highly skilled JAD session leaders are in short
supply. The indications from this research could provide the confidence to
reduce this quandary (to apply JAD poorly or not at all) and make the technique
generally available to address the pervasive problems with systems requirements.
In addition, higher quality requirements permit early detection of potential
specification errors, which helps to reduce scope creep, prevent post-design
alterations, and ultimately contribute to better information systems. These
benefits would contribute to reduced systems development and maintenance
costs. Greater user satisfaction with both the SRD process and its decisions

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 169

could also promote user ownership—a sense of responsibility for the realization
of the system benefits—and help deflect usage-related systems failures.
These findings suggest some follow-up objectives for future research. An
interesting one is to attempt to replicate these results in the field to incorporate
the realism of true SRD environments and reduce many of the other limitations
of this study. It also would be useful to evaluate the performance of the
integrated structure by varying some of the factors that were controlled in this
experiment (i.e., group size and task complexity). Another possible research
undertaking would be to study the potential areas of applicability of the combined
JAD/NGT approach with newer systems development paradigms. The latter
objectives could provide valuable insights that could lead to the establishment of
contingency strategies for deploying this technique in several contexts.

REFERENCES

Andrews, D.C. (1991). JAD: A crucial dimension for rapid applications development. Journal of Systems Management, 42(3), 23-27, 31.
Anson, R., Bostrom, R., & Wynne, B. (1995). An experiment assessing group
support system and facilitator effects on meeting outcomes. Management
Science, 41(2), 189-208.
Antunes, P. (1999). On the design of group decision processes for electronic
meeting rooms. CLEI Electronic Journal (Special Issue) 2(1). Special
issue of best papers from CRIWG’98, held at Buzios, Brazil, September
1998.
Asaro, P.M. (2000). Transforming society by transforming technology: The
science and politics of participatory design. Accounting Mmanagement
and Information Technologies, 38, 257-290.
Bailey, J.E., & Pearson, S.W. (1983). Development of a tool for measuring and
analyzing computer user satisfaction. Management Science, 29(5), 530545.
Barrow, P.D.M., & Mayhew, P.J. (2000) Investigating principles of stakeholder
evaluation in a modern IS development approach. Journal of Systems and
Software, 52(2-3), 95-103.
Beynon-Davies, P., Mackay, H., & Tudhope D. (2000). It’s lots of bits of paper
and ticks and post-it notes and things: A case study of a rapid application
development project. Information Systems Journal, 10(3), 195-216.
Borovits, I., Ellis, S., & Yeheskel, O. (1990). Group processes and the development of information systems. Information & Management, 19, 65-72.
Bostrom, R.P., Anson, R., & Clawson, V.K. (1993). Group facilitation and group
support systems. In L.M. Jessup & J.S. Valacich (Eds.), Group support
systems: New perspectives (pp. 146-168). New York: Macmillian.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

170 Duggan and Thachenkary

Byrd, T.A., Cossick, K.L., & Zmud, R.W. (1992). A synthesis of research on
requirements analysis and knowledge acquisition techniques. MIS Quarterly, 16(3), 117-138.
Carmel, E., George, J.F., & Nunamaker, J.F. (1995). Examining the process of
electronic-JAD. Journal of End User Computing, 7(1), 13-22.
Chapman, R.J. (1998). The effectiveness of working group risk identification
and assessment techniques. International Journal of Project Management, 16(6), 333-343.
Cheng, K.E. (1996). A requirements definition and assessment framework for
SDL tools. Computer Networks and ISDN Systems, 28, 1703-1715.
Davidson, E. J. (1999). Joint application design (JAD) in practice. The Journal
of Systems and Software, 45(3), 215-223.
Dean, D.L., Lee, J.D. Pendergast, M.O., Hickey, A.M., & Nunamaker J.F.
(1997-98). Enabling the effective involvement of multiple users: Methods
and tools for effective software engineering. Journal of Management
Information Systems, 14(3), 179-222.
Dekleva, S.M. (1992). The influence of the information systems development
approach on maintenance. MIS Quarterly, 16(3), 355-372.
Delbecq, A.L., Van de Ven, A.H., & Gustafson, D.H. (1986). Group techniques for program planning: A guide to nominal group and Delphi
processes. Middleton, WI: Greenbriar.
Dennis, A. R., Hayes, G.S., & Daniels Jr., R.M. (1999). Business process
modeling with group support systems. Journal of Management Information Systems, 15(4), 115-142.
Dieckmann, M. (1996). Making new technology investments pay off. Managing
Office Technology, 41(7), 14-16.
Dowling, K.L., & St. Louis, R.D. (2000). Asynchronous implementation of the
nominal group technique: Is it effective? Decision Support Systems,
29(3), 229-248.
Frankel, S. (1987). NGT + MDS: An adaptation of the nominal group technique
for ill-structured problems. Journal of Applied Behavioral Science,
23(4), 543-551.
Gopal, A., Bostrom, R.P. & Chin, W. (1992-1993). Applying adaptive structuration
theory to investigate the process of group support systems use. Journal of
Management Information Systems, 9(3), 45-69.
Gouran, D.S., Brown, C., & Henry, D.R. (1978). Behavioral correlates of
perceptions of quality in decision making discussions. Communication
Monographs, 45, 51-63.
Green, S.G., & Taber, T.D. (1980). The effects of three social decision schemes
on decision group process. Organizational Behavior and Human Performance, 25, 97-106.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 171

Henrich, T.R., & Greene, T.J. (1991). Using the nominal group technique to elicit
roadblocks to MRP II implementation. Computers & Industrial Engineering, 21(1-4), 335-338.
Highsmith, J., & Cockburn, A. (2001). Agile software development: The
business of innovation. Computer, 34(9), 120-127.
Ho, E.S., Lai, Y.J., & Chang, S.I. (1999). An integrated group decision-making
approach to quality function deployment. IIE Transactions, 31(6), 553567.
Holtzblatt, K., & Beyer, H.R. (1995). Requirements gathering: the human
factor. Communications of the ACM, 38(5), 30-32.
Hornsby, J.S., Smith, B.N., & Gupta, J.N. (1994). The impact of decisionmaking methodology on job evaluation outcomes. Group & Organization
Management, 19(1), 112-128.
Huber, G.P. (1980). Managerial decision making. Glenview, Ill: Scott,
Foresman.
Jackson, R.B., & Embley, D.W. (1996). Using joint application design to develop
readable formal specifications. Information and Software Technology,
38(10), 615-631.
Jones, C. (1996). Patterns of software system failure and success. Boston:
International Thomson Computer Press.
Kettelhut, M.C. (1993). JAD methodology and group dynamics. Information
Systems Management, 10(1), 29-36.
Kettelhut, M.C. (1997). Using JAD for strategic initiatives. Information Systems Management, 14(3), 46-53.
Korhonen, L.J. (1990). Nominal group technique. In M.W. Galbraith (Ed.),
Adult Learning Methods (pp. 247-259). Malabar, Florida: Krieger Publishing Company.
Liou, Y.I., & Chen, M. (1993-94). Using group support systems in joint
application development for requirements specifications. Journal of Management Information Systems, 8(10), 805-815.
Lyytinen, K. (1988). Expectation failure concept and systems analysts’ view of
information system failures: Results of an exploratory study. Information
& Management, 14(1), 45-56.
Marble, R. (1992). Casebook for systems analysis and design: FSS., Inc.
New York, NY: Mitchell-McGraw Hill.
Nunamaker, J.F., Dennis, A.R., Valacich, J.S., Vogel, D.R., & George, J.F.
(1993). Group support systems research: Experience from the Lab and
field. In L.M. Jessup & J.S. Valacich (Eds.), Group support systems:
New perspectives (pp. 125-145). New York: Macmillian.
Ocker, R., Hiltz, S.R., Turoff, M., & Fjermestad J. (1995-96). The effects of
distributed group support and process structuring on software requirements
development teams: Results on creativity and quality. Journal of Management Information Systems, 12(3), 127-153.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

172 Duggan and Thachenkary

Pinsonneault, A., & Kraemer, K.L. (1989). The impact of technological support
on groups: an assessment of the empirical research. Decision Support
Systems, 5(2), 197-216.
Poole M.S., & DeSanctis, G. (1990). Understanding the use of group decision
support systems: The theory of adaptive structuration. In C. Steinfield & J.
Fulk (Eds.), Organization and communication technology (pp. 173179). Beverly Hills, CA: Sage Publications.
Purvis, R., & Sambamurthy, V. (1997). An examination of designer and user
perceptions of JAD and the traditional IS design methodology. Information
& Management, 32(3), 123-135.
Raghaven, S., Zelesnik, G., & Ford, G. (1994). Lecture notes in requirements
elicitation [SEI-94-EM-010]. Pittsburgh, PA: Carnegie Mellon University.
Rist, O. (2001). Reinventing e-business. Network Computing, 12(20), 41-47.
Schuman, S.P. (1996). What to look for in a group facilitator. Quality Progress,
29(6), 69-72.
Spina, M. J., & Rolando, J.A. (2002, July). JAD on a shoestring budget.
CrossTalk The Journal of Defense Software Engineering, 26-28.
Stephenson, B.Y., Michaelsen, L.K., & Franklin, S.G. (1982). An empirical test
of the nominal group technique in state solar energy planning. Group &
Organization Studies, 7(3), 320-334.
Teltumbde, A. (2000). A framework for evaluating ERP projects. International
Journal of Production Research, 38(16), 4507-4520.
Thomas, J.B., McDaniel Jr., R.R., & Dooris, M.J. (1989). Strategic issue
analysis: NGT + decision analysis for resolving strategic issues. Journal of
Applied Behavioral Science, 25(2), 189-200.
Uden, L. (1995). Design and evaluation of human centered CIM systems.
Computer Integrated Manufacturing Systems, 8(2), 83-92.
Van de Ven, A.H., & Delbecq, A.L. (1974). The effectiveness of nominal,
Delphi, and interacting group decision making processes. Academy of
Management Journal, 17(4), 605-621.
van Maurik, J. (1994). Facilitating excellence: Styles and processes of facilitation. Leadership & Organizational Development Journal, 15(8), 30-34.
Vessey, I., & Conger S.A. (1994). Requirements specification: Learning object,
process, and data methodologies. Communications of the ACM, 37(5),
102-112.
Watson, H.J., & Frolick, M.N. (1993). Determining information requirements
for an EIS. MIS Quarterly, 17(3), 255-269.
Watson, R.T., DeSantis, G., & Poole, M.S. (1988). Using a GDSS to facilitate
group consensus: Some intended and unintended consequences. MIS
Quarterly, 12(3), 463-478.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Supporting the JAD Facilitator with the Nominal Group Technique 173

Wood, J., & Silver, D. (1995). Joint application development. New York: John
Wiley & Sons.
Zuech N. (1992). Identifying and ranking opportunities for machine vision in a
facility. Industrial Engineering, 24, 42-44.

ENDNOTE

The International Software Benchmarking Group survey on worldwide
business systems projects (June 2001) indicated that 66% of these projects
used traditional systems techniques. They also found that RAD/JAD was
used in 28% of the overall projects.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

174 Adams, Berner and Wyatt

Chapter IX

Applying Strategies to
Overcome User Resistance
in a Group of Clinical
Managers to a Business
Software Application:
A Case Study

Barbara Adams, Cyrus Medical Systems, USA
Eta S. Berner, University of Alabama at Birmingham, USA
Joni Rousse Wyatt, Norwood Clinic, USA

ABSTRACT

User resistance is a common occurrence when new information systems are
implemented within health care organizations. Individuals responsible for
overseeing implementation of these systems in the health care environment
may encounter more resistance than trainers in other environments. It is
important to be aware of methods to reduce resistance in end users. Proper
training of end users is an important strategy for minimizing resistance.
This article reviews the literature on the reasons for user resistance to
health care information systems and the implications of this literature for
designing training programs. The other principles for reducing resistance—
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Applying Strategies to Overcome User Resistance

175

communication, user involvement, strategic use of consultants—are
illustrated with a case study involving training clinical managers on
business applications. Individuals responsible for health care information
system implementations should recognize that end user resistance can lead
to system failure and should employ these best practices when embarking
on new implementations.

INTRODUCTION

Traditionally, health care has lagged significantly behind other industries in
the use of information technology (Parton & Glaser, 2002). Until recently, the
use of computers in health care primarily has been to automate the business and
administrative functions. Today, a variety of pressures are forcing the health
care industry to invest more money and effort into using information technology
in clinical settings. New legislation to protect privacy and confidentiality of
medical information encourages the development of electronic medical records
(U.S. Department of Health and Human Services, 2002). Concerns over medical
errors have led to an increased interest in clinical decision support systems and
computer-based physician order entry (Bates et al., 1999; Leapfrog Group,
2000). These developments will lead to more need for direct use of computers
by health care providers who are used to manual processes for the same tasks.
In addition, many clinicians now are assuming managerial positions in health
care, where they will be expected to use traditional business applications as well
(Merry, 1999). Not only are these new managers not used to automating some
of these tasks, but, as clinicians, they also have not seen use of the computer as
part of their professional role. As a chairman of a clinical department once said,
“What do I have a secretary for?”
The reluctance to use the new systems may be perceived as resistance, or,
in fact, there may be real resistance to the changes that information technology
makes in the clinical work processes (Worthley, 2000). In either case, administrators, information technology personnel, or clinicians charged with promoting
the use of information technology in the health care environment may encounter
more resistance than is found in other environments; therefore, added to the
issues of training end users that are common across a variety of settings, health
care project managers also need to be aware of methods to reduce resistance
in end users (Kaplan, 1997).
User resistance is a common occurrence when new information systems
are implemented within a health care organization. There is also a sizable amount
of literature on health care system implementations to explain and give insight
into some of the reasons behind this resistance and to suggest strategies for
overcoming it (Ash et al., 2000; Jiang et al., 2000; Lauer et al., 2000; Lorenzi &

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

176 Adams, Berner and Wyatt

Riley, 1995; Lorenzi et al., 1997, 2000, 2004; McNurlin & Sprague, 1998;
Worthley, 2000). The principles advocated in these studies can be used to
develop a variety of end-user training programs in health care settings. We will
discuss this literature and the implications for design of training programs and will
illustrate the application of these principles with a case study that involved
training clinical managers on business applications.

POSSIBLE EFFECTS OF USER RESISTANCE

When computers were first introduced into the health care setting, the
technical issues were the most important to address, but as Lorenzi et al. (1997,
2004) discuss, now that many technical issues have been solved, the managerial,
organizational, and people issues are equally, if not more, important. Worthley
(2000) attributes the failure of many system implementations to the failure to
properly address user resistance. He discusses five different forms that resistance can take: sabotage of computer equipment; employees being absent or late
to work; badmouthing the system, not using the new system, and continuing to
use the old system; and data tampering (Worthley, 2000).

REASONS FOR USER RESISTANCE

Kaplan (1997) discusses some of the causes of resistance to information
systems. She mentions user-centered theories, which “consider resistance to be
due to factors inherent in users, such as their lack of knowledge or their
reluctance to change” (p. 95). Other researchers also have identified resistance
to change as a significant source of implementation problems (Ash et al., 2000;
Brown & Coney, 1994; Jiang, Muhanna, & Klein, 2000; Lorenzi & Riley, 2000;
Yaghmaie, Jayasuriya, & Rawstorne, 1998).
In addition to resistance to change, both Lorenzi and Riley (1995) and
Worthley (2000) discuss some other reasons for user resistance:

•
•
•
•
•
•

Fear of loss of prestige and status in the organization due to not knowing the
new information systems (Lorenzi & Riley, 1995; Worthley, 2000)
Pressure to develop new skills (Lorenzi & Riley, 1995)
Pressure of higher performance expectations (Lorenzi & Riley, 1995)
Fear of loss of social interaction with other workers (Worthley, 2000)
Historical reasons, such as a previous bad experience with an information
technology effort (Worthley, 2000)
Benefits may not be clear to the user (Ash et al., 2000).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Applying Strategies to Overcome User Resistance

177

STRATEGIES FOR A SUCCESSFUL
INFORMATION SYSTEMS IMPLEMENTATION

Based on this literature review, the following were recurrent themes as
strategies to overcome user resistance:
•

•

Communication – Several researchers name communication as one of the
most important strategies for a successful systems implementation (Jiang
et al., 2000; Krishnan, 1999; Lauer et al., 2000; Lorenzi & Riley, 1995;
Worthley, 2000).
User involvement – Ives and Olson (1984) found as a result of reviewing
over 20 articles that “participation leads to increased user acceptance and
use by encouraging realistic expectations, facilitating the user’s system
ownership, decreasing resistance to change, and committing users to the
system” (p. 587-588).
Clarification of benefits – A strategy discussed by Ash et al. (2000,
p.128) “is to make sure the system provides immediate benefits to users.”
Ash et al. suggest that telling the user what the short- and long-term
benefits will be for that individual user will motivate a person to use the
system more than telling the user what the overall benefits of the system
will be for the organization.
Role of consultants – Consultants can be beneficial in information
systems implementation by filling in the experience and knowledge gaps of
their clients (Bauman, 2001). As an outsider, they are not involved in office
politics, and their decisions are based on what is in the business’ best
interest, not the best interest politically (Bauman, 2001). Very often, a
credible outside consultant can get a user to adopt new practices that
internal managers have tried to implement unsuccessfully.
Training – Training has been one of the main topics researchers have
emphasized as essential to successful information systems implementation
(Ash et al., 2000; Jiang et al., 2000; Lauer et al., 2000; Lorenzi & Riley,
1995; Lorenzi et al., 1997, 2000; McNurlin & Sprague, 1998; Worthley,
2000). Both Lorenzi and Riley (1995) and Worthley (2000) advise the use
of the just-in-time training concept when training users on a new information system. This means that training should occur just prior to implementation. They also suggest training the users in the order in which they are
going to use the system. There is a danger of training users too early and
then finding that the users have forgotten much of what they learned and/
or are not as familiar with the product when the actual implementation
occurs (Lorenzi & Riley, 1995). This means that the implementation
schedule should be monitored carefully, and the training plans should be
revised if there is significant delay.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

178 Adams, Berner and Wyatt

Lorenzi and Riley (1995) suggest that training addresses both technical
content and attitudes. “Any training needs to be a combination of educating
people in how to use the system plus building their enthusiasm for doing so”
(Lorenzi & Riley, 2004, p.260). Lorenzi and Riley (1995) and Worthley (2000)
discuss the importance of training manuals and online help. Complex systems
may require special training such as training in stages (Lorenzi & Riley, 1995).
Breaking up the training will allow the users to get comfortable with one part of
the system and may help to build confidence in learning the more complex parts
(Lorenzi & Riley, 1995). Researchers have mentioned that physicians may
require specialized training (Ash et al., 2000). Physicians dislike spending any
time in training (Lorenzi & Riley, 2004) and “frequently want to be trained by
other physicians” (Ash et al., 2000, p.129). One method that Lorenzi et al. (1997)
suggest when training physicians on a new system is to design a training program
that adapts to their current work styles. One idea is to use “training aids that are
prepared on 3x5 index cards because most physicians are accustomed to keeping
pertinent information in this manner” (Lorenzi et al., 1997, p.84).

CASE STUDY

We applied these principles in designing a training program in the use of the
Microsoft Excel application to clinical managers of an outpatient medical clinic.
Although all of the principles discussed above in regard to major systems
implementation are not applicable to the small case study, many of them are. In
addition, many of the same reasons for resistance can be found with small
business applications as with large informational technology implementations.
We will describe the background to the request for assistance, the application of
the principles in the design of the training program, and the outcomes.

BACKGROUND

Southern Medical Clinic (not its real name) is a multi-specialty group
practice started in 1926. Currently, the clinic has been home to several of the
physicians for more than 20 years. One particular physician has been there for
more than 40 years, over half of the entire life of Southern Medical Clinic.
Likewise, the support staff boasts many that have been with the facility for 25,
30, and even 40 years. In health care today, such statistics are rare. Southern
Medical Clinic still operates as a multi-specialty, physician-owned clinic. They
still use paper medical records; they still house a completely handwritten master
patient index; they have a paper card catalogue of all the patient charts; and they
even make paper copies of all insurance remittances. For such a facility to
continue to thrive, it not only must let technology in the door, but it also must

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Applying Strategies to Overcome User Resistance

179

embrace what technology has to offer;, yet it also is to be expected that with such
a stable staff, there may be reluctance to doing things differently from the way
they have always been done. Southern Medical Clinic has overcome much
resistance already. All of the practice’s management, accounting, HR, claims
filing, and payroll are now electronic. Internet classes are held, and spreadsheets
have replaced old handwritten forms. They soon will be scanning all remittance
reports to a server, and one department is considering the idea of electronic
medical records. The board meetings now have physicians with pocket PCs, and
the preferred means of mass communication is e-mail.
The problem that formed the focus for the case study was the reluctance
of the clinical department managers to learn to use Excel. The managers had
been encouraged to use Excel for a variety of tasks including documenting
employee leave, which they currently monitored manually. The human resources
department provided them with spreadsheets that they were expected to use.
Although a few were interested in learning how to use the spreadsheets, most
of the managers were not extremely computer literate and appeared to prefer to
calculate the leave manually and ask their secretaries to enter the data into the
spreadsheet. Obviously, such a practice has a greater opportunity for introducing
error and is less efficient than directly entering and calculating the leave with the
spreadsheet, yet the managers were not eager to change the system with which
they were familiar. It was with this background that the clinic manager requested
outside consultation and training for her staff.

APPLICATION OF PRINCIPLES
FOR OVERCOMING RESISTANCE

The strategies in the literature review were applied to developing the
training program. A strategy discussed by Bauman (2001) was to bring in outside
consultants. The trainer did not have any affiliation with the health care
organization and did not know any of the clinical managers in the training classes.
When preparing the training program, an effort was made to make sure that the
users would see the personal benefits of using the program as advocated by Ash
et al. (2000). To accomplish this, the training program included examples of some
current processes that the managers performed manually, and the managers
were told that the goal of the training was to work on automating them. Some of
the benefits of using the software were emphasized, and the managers were told
how they would be able to use the software as soon as they went back to work.
Another strategy that was incorporated into the training program was based on
Lorenzi and Riley’s (1995) suggestion to break up the training into stages. There
were three training sessions scheduled, and each session progressively covered
more complex material. Lorenzi and Riley (1995) discussed how breaking up the
training into stages would allow users to get comfortable with one part of the
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

180 Adams, Berner and Wyatt

system and help to build confidence in the users when they were faced with the
more complex parts.
Prior to taking the Excel training, the department managers were required
to pass a proficiency test in basic computing and Microsoft Word. Out of the 18
health care managers in this organization who were eligible to take the Excel
class, 12 enrolled, and 11 completed the class. There were three sessions, each
two hours long, completed over a three-week period, with multiple sessions given
each week to accommodate small groups of the managers. The training groups
were mixed according to level of computer experience. An evaluation form was
developed to get feedback on the teaching and to assess any resistance that the
health care managers might have had toward learning and using Excel. The
evaluation form was distributed at the end of the final training session. The
instructor informed the managers that their employer would not see the individual
evaluation forms.

Outcomes

The 11 health care managers completed the evaluation form about the Excel
training class. On the whole, they were very positive. More than 80% responded
positively to nine of the 10 questions. The exception still was positive, in that eight
of the managers felt the class was less difficult than they thought it would be, and
only one thought it was more difficult. What was particularly interesting was that
all of the managers agreed that their interest in using Excel was increased (73%
responded with agree, and 27% responded with strongly agree), and that they
would use Excel in their work (55% strongly agree; 45% agree). The respondents were very positive about the instructor, and all of them were interested in
taking another class with her. Approximately nine months after the training
session, the actual extent of use of the software was assessed. Results showed
more than half of the trained managers now were using the software. Of those
who were not, 18% did not have access to a computer. Twenty-seven percent
of the original group of trainees was still resisting using the software.

DISCUSSION

Given the expectation of encountering resistance, it was surprising that
during the class as well on the final evaluation, this resistance did not appear at
all. The training program was designed and developed in an effort to minimize
user resistance and appeared to have done so. When designing the Excel training
program, the one overall goal from the perspective of the organization was for
the managers to produce a sick and vacation report in Excel. This is a report that
all of the managers were required to do, and all were performing this report
manually. In the first class, managers were told that the sick and vacation report
was the overall goal, and they responded positively. During the classes, there
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Applying Strategies to Overcome User Resistance

181

were conversations about reports and processes that they perform manually and
discussions about how they could use Excel to automate some of those other
processes, as well. The basics of Excel were covered in the first class. In the
second and third classes, the focus was to work on the leave report. Showing the
managers how Excel could help automate a time-consuming manual report
helped to motivate them to use and learn Excel. As an outsider to the organization, the instructor was not encumbered by any relationships with the users. The
users knew that how they did in the training classes was not going to affect how
they were evaluated or perceived in their jobs. Thus, using outside consultants
to provide a different perspective, increase credibility, design the program to
illustrate personal benefits, and focus the training on gradually building up
confidence in use, resulted in very positive evaluations from a group of users
perceived to be resistant to learning new technologies.
Although it would be gratifying to think that the attention to evidence-based
good practices in instructional design were solely responsible for the positive
outcomes, there are other explanations. It is possible that the most resistant users
did not take the class, since a third of the eligible managers did not take advantage
of the training opportunity. In addition, sometimes users’ lack of confidence using
a computer system can be perceived as resistance to the system. It actually may
have been the clinical managers’ lack of confidence in using Excel that was
perceived as resistance. The training classes may have helped to build their
confidence, and then they realized that it was not as difficult as they thought it
would be.

CONCLUSION

As health care systems begin to implement new information technology, end
user resistance likely will become increasingly common. There are a variety of
reasons for the resistance, but there are also strategies that have been shown to
minimize it. These strategies include communication, user involvement, and
training, with specific focus on increasing user confidence. These strategies can
be used with large-scale implementations and can be incorporated within the
training programs themselves. Individuals responsible for health care information system implementations should recognize that end-user resistance can lead
to system failure and that they should employ these best practices when
embarking on new implementations.

REFERENCES

Ash, J.S., et al. (2000). Managing change: An analysis of a hypothetical case.
Journal of the American Medical Informatics Association, 7, 125-134.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

182 Adams, Berner and Wyatt

Bates, D.W., et al. (1999). Impact of computerized physician order entry on
medication error prevention. Journal of the American Medical Informatics
Association, 6(4), 313-321.
Bauman, C. (2001). Readying your organization for new technology. IT Health
Care Strategist, 3(7), 1-3.
Brown, S.H., & Coney, R.D. (1994). Changes in physicians’ computer anxiety
and attitudes related to clinical information system use. Journal of the
American Medical Informatics Association, 1, 381-394.
Ives, B., & Olson, H. (1984). User involvement and MIS success: A review of
research. Management Science, 30, 586-603.
Jiang, J.J., Muhanna, W.A., & Klein, G. (2000). User resistance and strategies
for promoting acceptance across system types. Information Management, 37, 25-36.
Kaplan, B. (1997). Addressing organizational issues into the evaluation of
medical systems. Journal of the American Medical Informatics Association, 4, 94-101.
Krishnan, R.S. (1999). Without nurses, inventory control impossible. Hospital
Materials Management, 24, 10-14.
Lauer, T.W., Joshi, K., & Browdy, T. (2000). Use of the equity implementation
model to review clinical system implementation efforts. Journal of the
American Medical Informatics Association, 7, 91-102.
Leapfrog Group for Patient Safety: Rewarding Higher Standards (2000). The
Leapfrog Group. Retrieved February 21, 2005 from http://www.leapfrog
group.org/
Lorenzi, N., Smith, J.B., Conner, S.R., & Campion, T.R. (2004). The success
factor profile© for clinical computer innovation. Proceedings, Medinfo,
2004, San Francisco, California.
Lorenzi, N.M., & Riley, R.T. (1995). Organizational aspects of health
informatics: Managing technological change. New York: SpringerVerlag.
Lorenzi, N.M., & Riley, R.T. (2000). Managing change. Journal of the
American Medical Informatics Association, 7, 116-124.
Lorenzi, N.M., & Riley, R.T. (2004). Managing technological change:
Organizational aspects of health informatics (2nd ed.). New York:
Springer.
Lorenzi, N.M., Riley, R.T., Blyth, A.J.C., Southon, G., & Dixon, B.T. (1997).
Antecedents of the people and organizational aspects of medical informatics:
Review of the literature. Journal of the American Medical Informatics
Association, 4, 79-93.
McNurlin, B.C., & Sprague, Jr., R.H., Jr. (1998). Information xystems management in practice. New Jersey: Prentice Hall.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Applying Strategies to Overcome User Resistance

183

Merry, M. (1999). Wanted: A new breed of physician drivers for healthcare’s
nitroglycerin trucks. Frontiers of Health Services Management, 15(4),
29-35.
Parton, C., & Glaser, J.P. (2002, July). Myths about IT spending. Healthcare
Informatics. Retrieved February 21, 2005 from http://www.healthcareinformatics.com/issues/2002/07_02/myths.htm
U.S. Department of Health and Human Services (2002). Administrative simplification. Retrieved February 21, 2005 from http://aspe.hhs.gov/admn
simp/
Worthley, J.A. (2000). Managing information in healthcare: Concepts and
cases. Chicago, IL: Health Administration Press.
Yaghmaie, F., Jayasuriya, R., & Rawstorne, P. (1998). Computer experience
and computer attitude: A model to predict the use of computerised
information systems. Medinfo’ 98, Proceedings of the Ninth World
Congress on Medical Informatics, Amsterdam, Netherlands.

APPENDIX

Lesson Plan for Excel Training Session 1
Introduction
•
Define what Microsoft Excel is and what it is used for.
•
Excel is a spreadsheet application that allows you to perform quick and
accurate calculations on data that are entered into a worksheet. Using
Excel also helps to avoid errors and present data in a professional format.
•
Discuss how Excel can be beneficial to them in their daily jobs.
Objectives for Session 1
•
Learn how to open and create an Excel workbook.
•
Be able to identify the components of the Excel screen.
•
Learn how to use the menus and toolbars to perform commands in Excel.
•
Be able to enter text and values in a worksheet.
•
Learn how to close and save an Excel file.
Assignment
•
In class, we will create a calorie counter worksheet. This will involve the
students entering text and data. They will have a brief introduction to simple
formulas. We will use the auto-sum function and average function in this
assignment. Next session will cover formulas in more detail.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

184 Adams, Berner and Wyatt

Lesson Plan for Excel Training Session 2
Introduction
•
Question and answer section from last session
•
Review of what we covered in the last session
•
Discussion of formulas in Excel
Objectives for Session 2
•
Learn how to perform calculations in Excel by the using shortcut formulas
and by entering formulas.
•
Be able to edit and delete data in Excel.
•
Be able to cut, copy, and paste in Excel.
•
Learn how to move cells around in Excel using the drag and drop method.
Assignment
•
In class, we will begin work on a sick and vacation report that will allow
them to apply skills learned from the previous lessons. The report will
include entering text, values, and formulas. The objective is for them to use
the worksheet in their jobs to keep up with the sick and vacation hours for
each of the employees in their department.

Lesson Plan for Excel Training Session 3
Introduction
•
Question and answer section from last session
•
Review of what we covered in the last session
•
Discuss number formatting in Excel and how that can change the way
values are displayed.
Objectives for Session 3
•
Learn how to apply number formats.
•
Learn how to format cells using the formatting toolbar.
•
Be able to delete and insert columns and rows.
•
Be able to change the width of a column and the height of a row.
•
Learn how to use the undo function to undo your mistakes and the redo
function in case you change your mind and want to redo the action.
•
Be able to move between worksheets.
•
Learn how to add, delete, and rename worksheets in a workbook.
•
Learn how to get help from the office assistant and other resources in
Excel.
Assignment
•
We will continue to work on the sick and vacation report.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Applying Strategies to Overcome User Resistance

Excel Training Class Evaluation Form
Please circle the response for each that best reflects your opinion. Please write any additional
comments or suggestions on the back of this form.
The instructor covered too much material in each session.
Strongly
Agree

Agree

Neutral

Disagree

Strongly
Disagree

The sequence of course content facilitated learning about the subject matter.
Strongly
Agree

Agree

Neutral

Disagree

Strongly
Disagree

The Excel class was less difficult than I thought it would be.
Strongly
Agree

Agree

Neutral

Disagree

Strongly
Disagree

The instructor was friendly and easy to talk to.
Student Comment: Helped above class subject for me.
Strongly
Agree
Neutral
Disagree Strongly
Agree
Disagree

The instructor delivered the course content clearly.
Strongly
Agree

Strongly
Disagree

Agree

Neutral

Disagree

Strongly
Disagree

Agree

Neutral

Disagree

Strongly
Disagree

Agree

Neutral

Disagree

Strongly
Disagree

Neutral

Disagree

Strongly
Disagree

I will use Excel in my work.
Strongly
Agree

10.

Disagree

This course increased your interest in using Excel.
Strongly
Agree

Neutral

I learned more about Excel than I expected.
Strongly
Agree

Agree

This class met your objective for what you wanted to accomplish in this training.
Strongly
Agree

185

Agree

If you had the opportunity, would you take another class from the instructor?
Yes
No

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

186 Stahl

Chapter X

Responsibility for
Information Assurance
and Privacy:

A Problem of Individual Ethics?
Bernd Carsten Stahl, De Montfort University, UK

ABSTRACT

Decisions regarding information assurance and IT security can affect
individuals’ rights and obligations and thereby acquire a moral quality.
The same can be said for questions of privacy. This chapter starts by
showing how and why information assurance and privacy can become
problems worthy of ethical consideration. It demonstrates that there is no
simple and linear relationship between ethics and information assurance or
between ethics and privacy. Many decisions in the area of IT, however,
affect not only one, but both of these subjects. The ethical evaluation of
decisions and actions in the area of privacy and security, therefore, is
highly complex. This chapter explores the question whether individual
responsibility is a useful construct to address ethical issues of this complexity.
After introducing a theory of responsibility, this chapter discusses the
conditions that a subject of responsibility typically is assumed to fulfill.
This chapter will argue that individual human beings lack some of the

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Responsibility for Information Assurance and Privacy

187

essential preconditions necessary to be ascribed responsibility. Individuals
have neither the power, the knowledge, nor the intellectual capacities to
deal successfully with the ethical challenges in the tension of privacy and
information assurance. This chapter ends by suggesting that the concept of
responsibility, nevertheless, may be useful in this setting, but it would have
to be expanded to allow collective entities as subjects.

INTRODUCTION

Proponents of information assurance aim at meeting the security testing,
evaluation, and assessment needs of IT consumers and producers. They are
mostly interested in eliminating security threats and, in the long run, want to
increase the levels of trust that users and consumers have in IT and networks.
While most users support these goals of information assurance, they also have
other objectives when using IT; among them is the preservation of privacy. To
a certain degree, these two objectives are contradictory. In order to facilitate
security, it would be helpful to eliminate privacy, because this would allow an
easier detection and elimination of security risks. Privacy, on the other hand,
requires security, because the protection of private data relies on the assumption
that no unauthorized access is possible. Privacy and information assurance thus
also can be complementary.
Further complicating this relationship, both terms also have an ethical side
to them. Trust, as the ultimate aim of information assurance, is at least partly a
moral notion. Security is necessary to facilitate a free and equal exchange of
ideas. At the same time, an excess of security can stifle the exchange of ideas
and the greater good. Privacy generally is recognized as a moral good, but it is
debatable how this good can be justified and where its limits are. The individual
user, who must make decisions concerning the weighting of privacy and
information assurance, therefore finds himself in a situation where, despite an
ethical quality of the choices, it is less than clear how decisions are to be made.
This is where the concept of responsibility enters the picture. This chapter
will describe a theory of responsibility and put a special emphasis on the question
of who can be the subject of responsibility. This theory of responsibility then will
be applied to the complex problem of privacy and information assurance. The
theory and conditions of responsibility will be used to demonstrate that while
individual responsibility can play an important role in such ethical decisions, it also
runs into severe problems. It will be argued that due to the lack of fulfillment of
the basic conditions of responsibility, the individual end user is not able to
shoulder the burdens required in order to make an ethical decision. As a
consequence, questions of privacy and information assurance require a wider
context and frame in which they can be answered. Only in such a frame does
individual responsibility make sense and can it achieve its objectives.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

188 Stahl

How should the individual end user deal with this dilemma? The conclusion
of the chapter will argue that the content of this chapter is of high relevance for
the individual end user, because it allows him or her to recognize the limits of his
or her capacities. The very fact that individual humans quickly reach their
fundamental limits when they are ascribed responsibility in the context of
information assurance and privacy will allow them to overcome their limitations.
By pointing out why they cannot accept such responsibility ascriptions, they
should be able to transcend the ascription and open discourses that will include
other subjects, which, in turn, might be able to solve the problem. Briefly, the
arguments presented in this chapter can be used to protect the individual end user
from responsibility ascription that he or she is incapable of satisfying. At the
same time, they should help avoid situations where responsibility is wrongly
ascribed to individuals.

INFORMATION ASSURANCE AND PRIVACY:
AN ETHICAL CHALLENGE

As indicated in the introduction, a brief look at the concepts of information
assurance and privacy could suggest that the two can be contradictory, but the
opposite interpretation is just as possible. Since it is the purpose of this chapter
to analyze the role that individual responsibility can play with regard to the
realization of information assurance and privacy, this section will be dedicated
to a discussion and definition of the concepts. In both cases, the focus of the
discussion will be their ethical content and the ethical challenges they pose.
We will leave the definition of ethics as open as possible and work with a
common sense notion of ethics. While this may not be very satisfactory from a
philosophical point of view, one reason to support this approach is that most end
users who make moral decisions concerning the use of ICT do not have a formal
education in philosophical ethics. At the same time, most users probably can be
considered to be ethical in the sense that they want to do the right thing. In order
to do so, they need some sort of private understanding of ethics, and it is this that
we will work with here. Briefly, ethics here will be understood as having to do
with doing good. Ethical behavior aims at improving the circumstances of oneself
and, more importantly, of others. Part of this is to respect other people’s rights
and interests, and in order to determine these, one has to assume a basic similarity
of rights and interests between different people. A generally shared rule of pretheoretical ethics, which is also reflected in ethical theories, is to treat the other
in such a way as one wants to be treated oneself. This rough sketch of ethics will
suffice for the identification of ethical problems posed by information assurance
and privacy. The reader who is not satisfied with this concept of ethics is referred
to Stahl (2004) for a detailed discussion of the relationship of philosophical ethics
and the idea of reflective responsibility.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Responsibility for Information Assurance and Privacy

189

The Ethics of Information Assurance

Information assurance is a term that denotes the idea of security of
information in technical systems. The U.S. National Information Assurance
Partnership, for example, is a government initiative “designed to meet the
security testing, evaluation, and assessment needs of both information technology (IT) producers and consumers” (NIAP, 2003). Information assurance tends
to focus on security in information systems using a relatively wide framework.
As the above definition shows, it is a collaborative effort of private, public, and
often academic institutions that endeavors to consider all of the relevant
stakeholders. While not a completely new idea, the information assurance
movement seems to have been strengthened by the September 11, 2000, terrorist
attack and the subsequent attention by governments, particularly the U.S.
federal government, to security threats. This is shown by the fact that the U.S.
military plays a prominent role in information assurance (IASE 2003). This close
connection of information assurance to terrorism, government, and the military
carries ethical implications by itself that this chapter will not be able to discuss.
For the purposes of this chapter, we will understand information assurance as
concerned with the security in information systems and, therefore, concentrate
on the ethical implications of security.
Threats and Solutions to Security Problems
Threats to security and information assurance can be seen in many different
areas. They can result from intentional attacks but also from unintended
misbehavior or from technical or organizational mistakes. A central problem in
rectifying these mistakes is that they not only are unknown but, by definition,
unknowable. Human beings who make decisions regarding information technology deal with future states and, therefore, have to accept uncertainty and risks
in their decisions (Grabner-Kräuter 2002).
Nevertheless, the knowledge of uncertainties and possible dangers compels
us to act in order to avoid damages. In the case of threats to IT security, there
are several areas where security efforts can be fruitful. Among them, one can
find malicious attacks such as worms, viruses, and the like (Eisenberg et al.,
1995) or hacking (Johnson 2001). The biggest fear at the moment is probably
terrorism. It is interesting to note, however, that it is not a new fear and that
connections between the use of IT and terrorist attacks have been predicted
years before the attack on the World Trade Center (Levy 1995).
Information assurance, as the attempt to assure the availability of IT
services, can go several routes and will usually take several of them simultaneously. On the one hand, there are organizational measures that are frequently
linked with the embedding of information systems in organizations and hierarchies (Healy & Iles 2002). On the other hand, there are technical measures that
include the setting of standards of reliability (Littlewood & Stringy 1995),

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

190 Stahl

measures to ensure authentication such as biometrics (van der Ploeg 2001), or
others such as encryption (Tavani 2000).
The Ethical Implications of Information Assurance
Given the loose definition of ethics used in this chapter, there are many
possible points of contact where information assurance and security can have an
ethical impact. If ethics has to do with doing the right thing (whatever that may
mean in a particular situation), then security in IT is something of ethical
relevance. Security questions affect who gets access to which technology. They
determine who can communicate with whom and about which topics. Security
regulations imply power relationships with all their ethical baggage. The question
of reciprocity that was introduced as central to ethical thinking is affected by
security measures. Generally, one can state that most, if not all, attempts to
secure information technology affects in some way or other the way people can
behave and interact. They thereby automatically affect moral rights and obligations. At the same time, security is also a necessary precondition for ethical
action. In order to act ethically, one needs social norms and institutions that
support and facilitate such action. This means that these institutions must be
secured. On the other hand, security considerations can be misused as excuses
for immoral behavior.
Information assurance thus can be seen as a topic of ethical importance that
has no clear and unequivocal ethical message. Security measures can have
morally positive results, but they can also become moral liabilities. There are no
simple guidelines along the lines of “the more security, the more ethics”.
Individuals making decisions regarding security are, therefore, in a difficult
position when they want to consider ethical problems. This difficult situation is
exacerbated when one widens possible decisions beyond pure security considerations and looks at other factors—in our case, privacy.

The Ethics of Privacy

Privacy is a concept that has gained prominence due to the spread of
information technology and the approaching information society. Given the
complexity of the discussion about privacy and the importance it has for the
question of how to assume responsibility for it, this section first will look at the
definitions of privacy, then at the justifications that can be found in the literature
in order to analyze the limits of the concept.
Definitions of Privacy
There is much debate about privacy from different academic disciplines.
One of the fields where privacy is most hotly debated is that of computer and
information ethics. Most authors agree that privacy is a moral value. However,
it often is not clear what exactly privacy is (Weckert & Adeney, 1997) and why

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Responsibility for Information Assurance and Privacy

191

it must be considered valuable. However, the amount of protection we believe
to be appropriate and the outcome of conflicts between privacy and other
interests depend on our answer to these questions, which is why it is necessary
in this chapter to give an overview of definition, justification, and limits of privacy.
Unlike information assurance, which is a term that only makes sense in a
modern society and with the use of information technology in mind, privacy is an
old concept that can be traced back to the beginning of modern civilization among
the ancient Greeks (Rotenberg, 1998). The modern meaning and importance of
the term privacy, nevertheless, are linked closely to technology. Also, the legal
protection of privacy is a relatively new phenomenon dating back to the late
1800s, which coincides with the growth of cities and the migration from rural
environments (Sipior & Ward, 1995). It is interesting to note that the legal
protection of privacy was a reaction to a technological development; namely,
photography. Warren and Brandeis (1890) wrote a seminal chapter that started
the modern discussion about privacy and led to its legal codification because of
the fact that through the use of technology, it became possible for the first time
to make accurate pictures of someone without their consent. Warren and
Brandeis (1890) also put forward the definition of privacy that to this day is used
most frequently; namely, the definition of privacy as the right “to be let alone”
(p. 205). They saw this right to privacy as one part of a larger right to be let alone,
which includes such rights as not to be assaulted, beaten, imprisoned, maliciously
prosecuted, and so forth. Although such privacy is a basic right that many authors
today still recognize (Britz, 1999; Velasquez, 1998), others argue that it is not a
basic right protected by the U.S. Constitution. While this argument was
developed in the U.S., similar views of privacy can be found in most countries,
usually codified by law. In some areas, notably the European Union, the
recognition of privacy goes beyond the negative right to merely be left alone and
extends to the positive right of informational self-determination. In the European
Union, privacy is recognized expressly as a fundamental human right (European
Union, 2000, Article 7). For now, however, we will continue to use the definition
of privacy as the right to be left alone, because this can be viewed as a generally
acceptable minimum standard.
Albeit widely recognized and generally understandable, this definition
produces several problems. First, it is based on a natural rights approach that is
hard to justify. Second, it is too broad to be of much practical use and does not
capture all of the meanings of the term that can be found today. The meanings
of privacy cover a diverse range from privacy as a situation, a right, a claim, to
privacy as a form of control or a value (Gavison 1995). Privacy is interlinked with
freedom and property. It can be defined in part as “the freedom to do things away
from the eyes and ears of others” (Weckert & Adeney 1997, p. 76). Privacy is
invaded when “individuals are unable to control their interactions with the social
and physical environment” (Culnan 1993, p. 344).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

192 Stahl

There are different reasons why privacy has gained importance over the
last decades. The first one, which is probably the most important in the context
of this chapter, is the impact that information and communication technology
(ICT) has. ICT has not changed the fact that data about persons are collected,
processed, and exchanged. However, it has changed fundamentally the speed
and scope in which this can be done. It has changed the mobility of data; as Moor
(2000) puts it, ICT has “greased” the data. The result of this is that on the one
hand, more data than ever before are collected on individuals, while on the other
hand, these individuals have less control over what happens with these data than
ever before. The availability of ICT also facilitates the creation of synergies and
the exchange of data in ways that have the potential to affect people’s lives
deeply. One example might be genetic data that can be used to create information
about health risks. This could be linked to employers’ or insurance companies’
databases with the effect that individuals lose their employment or are unable to
acquire insurance coverage. Such examples easily can be continued, and they
show that privacy has a deep impact on the role of individuals in society, on rights
and obligations, and on the way we interact on ethical matters. This is why
privacy is such a frequently discussed topic in computer and information ethics
(Anderson et al. 1993; Johnson, 2001; Mason, 1986; Robison, 2000; Straub &
Collins 1990).
The technological development is linked and propelled by economic interests, and, as the previous example shows, the two combine to exacerbate the
privacy problem. While traditionally the state has been seen as the greatest threat
to privacy, especially in totalitarian states, today many authors see a bigger threat
coming from private enterprises that have the technical means and the economic
incentive to collect data and that are not regulated in the way actions by state and
government bodies and representatives often are (Himanen, 2001; Tavani,
2000). Again, this is not an entirely new development. The original American
legal codification of privacy by Warren and Brandeis already aimed at curbing
commercial interests that promoted the technical threats to privacy. However,
in modern, information-based economies, the incentives to collect data on
individuals are great, and regulations are diverse and frequently contradictory.
One other reason why the question of privacy is so complex is that it runs across
several societal fault-lines; it interferes with the grand discourses of liberalism
versus collectivism, freedom, and autonomy; and our societies do not seem to be
able to agree on a position with regard to these questions (van den Hoven, 1999).
Justifications of Privacy
Among the different strategies used to defend a right to privacy, one can
distinguish between absolute and relative ones. Privacy as an absolute right or
value is based on the assumption that it is a basic right comparable to human
rights, which must be defended independent of specific circumstances. Privacy

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Responsibility for Information Assurance and Privacy

193

is thus afforded the same status that human rights generally have, and some
authors believe that it, indeed, is a basic human right (Rogerson, 1998; Spinello,
1997). Moor (2000) offers a similar distinction by introducing intrinsic and
instrumental values. Intrinsic values are those that deserve to be defended,
whereas instrumental values are only valuable with regard to something else,
ultimately with regards to intrinsic values. One example of privacy as an intrinsic
value is put forward by Milberg et al. (1995), who see it as a “hypernorm”—a
moral rule that seems to be a human universal and that is generally recognized
and does not need further justification. (For a description of the idea of
hypernorms, see Donaldson & Dunfee, 1999)
Those authors who do not see privacy as an intrinsic value or as a basic
human right and who still agree that it is something worth protecting must show
reasons why it is, nevertheless, important. There are again two possible
strategies that can be found in the literature for doing so. On the one hand,
privacy is defended as important for the individual; on the other hand, it is
portrayed as crucial for society. To some extent, these strategies reflect a
deontological and a teleological ethical argument, respectively.
The individual approach emphasizes the importance of privacy for the
development and maintenance of the individual. Privacy has been described as
the “basis for self-determination, which is the basis for self-identity as we
understand it” (Severson, 1997, p. 65). That privacy is important for individual
development is in little doubt. Just why and how privacy is needed to become a
fully developed individual or person is less clear. There seem to be different
functions that privacy has in the process of developing individuality. Johnson
(2001), drawing on Fried, believes that friendship, intimacy, and trust need
privacy in order to develop. The generally shared assumption is that in order to
develop satisfactory relationships with others, one must have a place where the
other cannot follow, where one is sure to be alone. Without this type of control
over who has access to us and who knows what about us, we have difficulties
developing meaningful relationships (Rachels, 1995). One problem of the
intrusions of privacy through ICT can be that others not only have access to areas
that one may believe to be intimate, but they also have more information about
individuals than the individuals have themselves (Robison, 2000). In extreme
cases, the access that others can have to one’s information is presumed to be
enough not only to undermine one’s relationships with others, but also, in fact, to
jeopardize the identity or the inner self of the person in question (Brown, 2000).
Another aspect of privacy and the individual is that respect for the privacy of
others can be interpreted as respect for the other, per se; or, put negatively, a lack
of respect for privacy equals a lack of respect for the other (Elgesiem, 1996). For
proponents of these arguments, privacy is an instrumental value. However, its
importance for the intrinsic value behind it (i.e., personhood) is such that it seems
to become an intrinsic value itself. This means that it deserves being protected
as a basic right (Introna, 2000).
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

194 Stahl

The other strand of arguments used to defend privacy as an instrumental
value aims at its social utility. In this line of reasoning, privacy is seen as valuable
for something that is useful from a social or societal point of view. Classically,
one can find the contention here that privacy is necessary for the collective
deliberation process that determines decision making in democracies. Johnson
(2001), for example, states the case that individuals who are constantly observed
are incapable of the essential processes necessary for democracy to work. In a
more general sense, privacy can be seen as part of the values that characterize
democratic societies. Privacy, in this respect, is an instrumental value, because
it contributes to the success of democracies (Gavison, 1995).
Limits and Problems of Privacy
Independent of the arguments used to defend privacy, most authors agree
that there are limits to the right and the protection of privacy (Britz, 1999).
“Privacy is a relative concept. It is a continuum” (Introna, 2000, p. 190). This can
be explained easily by looking at the two possible extremes of privacy—complete
privacy and complete lack of privacy. A complete lack of privacy would lead to
social and psychological problems explained by our apparently natural need for
an undisturbed space. However, if this space is brought to the opposite
extreme—complete privacy—the results would be equally negative. Social
institutions rely on information about the members of society. Complete privacy
would bring about a collapse of many social institutions (Gavison, 1995),
especially those institutions that exert constraints on people, such as military
conscription and the tax system. The resulting collapse of much of what defines
our societies would be hard to justify on the grounds of privacy protection.
Basically, we find ourselves now in a situation where the value of privacy
is recognized, but where it is quite unclear how this translates into specific
privacy protection measures. In the context of this chapter, it is important to note
that privacy is a moral notion that can be defended from most ethical viewpoints.
Many of the defenses of privacy discussed are based on utilitarian grounds.
Privacy protection is supposed to increase the individual’s utility by allowing the
individual to develop his or her personality to the maximum and to engage in
meaningful social interaction. At the same time, privacy maximizes social utility,
first by increasing individual utilities and second, by facilitating social interaction.
However, privacy can be justified just as easily by deontological arguments. One
can see privacy as an intrinsic value, which could be translated into the duty to
respect it. Another strategy would be to stress the importance privacy has for
allowing the development of individual autonomy. Personal autonomy is important from a deontological perspective, because it is the basis of setting one’s own
norms, which, in turn, is a central idea in Kantian (Kant, 1995) morality. Privacy
has a moral value, because it can be seen as a sign of respect for persons, which
again can be justified by teleological (i.e., aims-base, consequentialist) as well as
deontological (i.e., duty-based) ethical theories. Furthermore, respect for priCopyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Responsibility for Information Assurance and Privacy

195

vacy also can be seen as a virtue, which puts it squarely into the realm of virtue
ethics.
Privacy is thus a moral value or a moral right, and it is worth defending.
However, like most moral rights and values, it is not absolute. Philosophical
ethics have always had to deal with the question of how to limit moral rights and
what to do in case of a conflict of different rights. In many cases, moral rights
deemed significant within society are transformed into legal rights. Privacy is no
exception. The question of the limits of a moral right then becomes the question
of the limit of a legal right. While this sort of question often is solved more easily
in practical terms, due to the existence of a hierarchy of courts that gives us
practical solutions, privacy still raises a lot of questions when one looks for its
exact extent and limits.
Some of the limits of privacy are fundamental and closely related to the
justification of the term. If privacy gives individuals freedom to interact and to
do what they want without detection, then it also gives them freedom to do
undesirable things (i.e., commit crimes, do terrorist acts, etc.). Levy (1995),
therefore, can ask whether we can protect our privacy in an age of computers
“without also protecting the dark forces in society” (p. 652). A somewhat less
apocalyptic problem that can be raised by privacy is that of intellectual property.
Some authors suggest the framing of privacy in terms of property. That means
that information about a person is seen as that person’s property, and, consequently, the person gets to decide what to do with it (Hunter, 1995). Unfortunately this does not solve the problem either, because it remains questionable
whether personal information can be described usefully in terms of property, and,
even if this were agreed uppon, it would change the problem to the question of
the limits of personal property.
Apart from these fundamental problems that most individuals would be
hard-pressed to address, there are also many practical problems with privacy
protection. We have seen that privacy can be described as a right, but the status
of the right remains open. Is privacy a moral or a legal right, and what is the
relationship between the two? In most societies, a right to privacy is recognized
by the legal system, either explicitly or implicitly. However, as soon as we enter
the sphere of legal rights, we are confronted with a whole new set of problems.
Given the international nature of the information technology that threatens
privacy, we find a host of international questions such as jurisdiction and cultural
and language difficulties. One problem in this regard results from the different
perceptions of privacy between the U.S. and Europe. These differences go deep
enough to endanger data exchange between the two areas (Langford, 1999;
Tavani, 2000). The EU follows a strong privacy protection regime and outlaws
transborder data exchange with countries that do not guarantee the same level
of protection. Given that the U.S. has weaker privacy protection, complicated
arrangements have been set up to facilitate data exchange.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

196 Stahl

Having now discussed the ethical aspects of information assurance, privacy, and their justifications and limits, we can proceed to the difficult question
of their relationship with regard to ethics.

The Relationship of Information Assurance and Privacy
from and Ethical Viewpoint

Information assurance and privacy can reinforce one another, but they can
also come into conflict. This section will discuss these two possibilities while
emphasizing the way in which this may affect moral rights or obligations.
Ethical Correspondences of Information Assurance and Privacy
Information assurance and privacy can correspond and thereby jointly
protect or even constitute moral values. Both terms can refer to the concept of
security. Information assurance in this text is understood as the attempt to
increase the security of the use of ICT on different levels. This more technical
view of security can be seen as a precondition of a more personal psychological
concept of security. Human beings need security in an emotional sense, which
means that they need to feel secure. Technical and personal security can support
this feeling of security, but they do not have to be. One can feel secure because
one is not aware of the danger, and one can feel insecure despite an “objective”
lack of threats. However, objective technical security can be translated easily
into personal security. Most people will feel more secure in a modern car with
all sorts of active and passive security measures than in an older one that lacks
these measures. Similarly, information assurance is a precondition for feeling
secure when interacting with computers (Spafford, 1995). It is this feeling of
security, this psychological security, that was cited as one of the main reasons
that privacy should be defended. Security, therefore, is probably the most
important area where information assurance and privacy overlap, and where, at
the same time, they protect an important moral value—the individual and its
formation (Moor, 2000). Drawing on Giddens and Goffman, Brown calls this a
“‘protective cocoon’ which allows individuals to deal with life on a daily basis and
protect the inner self they know from exposure to outside scrutiny. This ‘veil’
separates self from those things that are external and therefore not self, in this
manner providing the most basic sense of ontological security” (Brown, 2000, p.
63) Technical security is thus necessary to produce privacy. An example would
be so-called privacy-enhancing technologies (PETs), and the most fundamental
of these is encryption, which simultaneously promotes privacy and security
(Tavani & Moor, 2001).
A related area where privacy and security work to achieve a shared goal,
which is generally recognized as an ethical value, is that of trust. Many authors
emphasize that in order to develop trust, an individual needs physical security as
well as privacy. Again, this is based on assumptions about the working of the
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Responsibility for Information Assurance and Privacy

197

human mind, which needs a feeling of security and a feeling of individuality in
order to interact meaningfully with others (Koehn, 2001). The dependence of
trust on privacy and security is a prevalent topic of discussion in e-commerce.
A lack of trust has been identified as one of the major impediments to the
continued success of online trading. Many individual customers are reluctant to
buy or sell online because they do not trust the procedures. The reasons for this
are manifold, but most authors agree that they can be found at least partly in the
areas of security and privacy (Hoffman, Novak, & Peralta, 1999; Nikander &
Karvonen, 2000; Khare & Rifkin, 1998).
Another angle under which the affinity of privacy and security can be
captured is that of control. For individuals to be in control of their lives, a certain
amount of control over what is happening to them and with them is indispensable.
This control can be interpreted in terms of security, which means that the
individual can prevent unwanted intrusion. This sort of control then automatically
entails a degree of privacy (Camp, 2001). Again, control in this sense can be seen
as a moral value, because it is a sign of the autonomy of the individual that we
have identified as ethically important.
Ethical Contradictions of Information Assurance and Privacy
While security and privacy can overlap, as we have seen in the last section,
they also can be contradictory. As a general point, one can note that security
requires openness, clarity, and accountability, whereas privacy often means the
opposite (Beu & Buckley, 2001). Privacy and security can come into conflict in
different areas. One example that can be used to demonstrate the possible
contradiction is trust. While we have argued that trust requires privacy and
security at the same time, the relationship also can be contradictory. One
argument along this line is that security can limit the exchange of ideas, and that
this free exchange is one of the basic building blocks of the computer community.
An increase in security thus means a decrease in free speech and consequentially a decrease in trust (Rotenberg, 1995). Other authors argue that the entire
trust issue is misleading, because it is based on fallacious premises. A large part
of the literature on trust in e-commerce, for example, tries to show ways that can
be followed in order to build trustworthy information systems. Critics of this
approach argue that one cannot trust technology; one can only trust people. More
security, therefore, cannot induce trust (Rutter, 2001; Corbato, 1995).
Another area where security and privacy may be contradictory is that of
power. Especially with regard to hierarchical organizations, both of these terms
can be seen as an expression of power of particular groups or individuals. In this
setting, security usually means control over access and use of systems, and in
many cases, this will be in contradiction to the power to protect one’s privacy
(Forester & Morrison, 1994). The expression of such conflicts often can be seen
in more technical issues that are needed to implement security or privacy

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

198 Stahl

concerns. Examples of this are biometrics or encryption. Both of these technologies increasingly are used to secure data and information systems, and both also
can be interpreted as tools that have the potential to decrease privacy. While
biometrical protection of one’s personal data and files may increase one’s
privacy, it also can serve to identify a person more clearly and to use this data
for other purposes (van der Ploeg, 2001).
The most important area of conflict between security and privacy, however,
is that of surveillance, especially the surveillance of employees by organizations.
All of the arguments mentioned in this section apply to this, but the topic acquires
its importance due to the sheer magnitude of surveillance in today’s societies.
While there are important national differences in this area, it is probably true to
say that most commercial organizations have rather strong incentives to subject
their employees to surveillance, and the practice is widespread (Hartman, 2001;
Schulman, 2000; Bowie, 1999). The reasons for this differ among companies,
industries, and countries, but most of them have something to do with information
assurance. The rationale for surveillance usually is to produce security for the
company in some sense, be it for legal problems (Brown, 2000), competition, or
other economic considerations such as making sure that employees do their job
(Boncella, 2001; Posner, 1995). The price to be paid is a decrease in employee
privacy. It is in this situation—the surveillance of employees in their workplaces—that the problematic relationship between security and privacy becomes
most clear, and the question of ethical problems and of individual decision making
is most salient. It is, therefore, a good starting point for the discussion of how
these questions can be addressed. Since the proposed solution in this chapter is
the concept of responsibility, we now will look at how individuals can make
responsible decisions regarding the tension of privacy and information assurance.

RESPONSIBILITY AS AN ANSWER TO
ETHICAL PROBLEMS

So far, this chapter has argued that security and information assurance and
privacy are terms with important ethical aspects, but that their relationship is
problematic. Individual and organizational decisions with regard to the two
concepts will affect people’s rights and obligations in most cases, but it is usually
unclear exactly how this will be the case. The ethical evaluation of such decisions
depends on a highly opaque muddle of ethical theories, moral practices, empirical
consequences, legal frameworks, international negotiations, and so forth. One
ethical concept that is used frequently to address such muddles is responsibility.
This section will describe briefly the concept of responsibility, emphasizing the
individual perspective. By concentrating on the conditions of responsibility, it will

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Responsibility for Information Assurance and Privacy

199

be shown that responsibility, while in principle very useful for this sort of
situation, requires more than an individual is able to deliver.

The Conditions of Responsibility

It will not be possible in this chapter to present a comprehensive overview
of the concept of responsibility and its use in moral philosophy. (For more
exhaustive theories of responsibility, see Bayertz, 1995; Fischer, 1999; French,
1992; Lenk, 1998; May & Hoffman, 1991; Neuberg, 1997; Paul et al., 1999;
Sänger, 1991; Stahl, 2004.) The discussion, therefore, will concentrate on the
subject of responsibility and the conditions a potential subject needs to fulfill in
order to be admissible as a subject. In order to do this, it will be necessary to give
a brief definition of the concept of responsibility.
In this chapter, responsibility will be understood as a social construct that
results in the ascription of an object to a subject, usually before an authority of
some kind. The most accessible example of this is the case of legal responsibility,
where the object is the crime, the subject is the accused, and the authority is
represented by the law and the judge who interprets it. Responsibility, understood
in this way, is a complex notion that, in order to be practically relevant, requires
attention to many details such as the acceptability of the underlying rules, the
temporal aspect of ascription, and the admissibility of the object. There are
several advantages to the concept of responsibility over other normative
constructs that let it appear as a suitable candidate to address the problems raised
by information assurance and privacy. Responsibility is a formal process that can
transport different meanings. That means that it is capable of addressing the legal
questions at the same time it deals with the moral and ethical ones. Furthermore,
responsibility, at least in its legal form, is well established. It also has a positive
image, and responsibility statements often are more acceptable than other moral
assertions.
The most important component of the responsibility ascription is the subject.
Traditionally, the subject has been the individual adult and rational human
being—the person. In order to be ascribed responsibility, the subject needs to
fulfill several conditions. The one that probably is most frequently named is that
of causality. In order to be ascribed responsibility for an object, the subject must
have a causal relationship with it, must have caused it or at least have been
capable of changing the course of events that led to the object (Bechtel, 1985;
Birnbacher, 1995; Etchegoyen, 1993; Goldman, 1999; Jonas, 1984; Zimmerli,
1987). This causality assumes several other conditions. In order to be able to
influence events, the subject must have knowledge of the consequences of
actions (Rötzer, 1998) and must have the power to change them (Lenk & Maring,
1990; Nida-Rümelin, 1998; Staddon, 1999). In order to be able to do this, the
subject needs to fulfill some more implicit conditions such as freedom (Höffe,
1995; Frankfurt, 1997; Wunenburger, 1993) and personal qualities such as

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

200 Stahl

emotions, empathy, a certain amount of rationality, a certain state of mind, and
so forth (Bierhoff, 1995; Hart, 1968; Stocker, 1999). Some of these assumptions
and conditions are quite difficult to come to terms with, as the example of the
concept of freedom shows. Freedom is a highly contentious topic in philosophy,
and it is not clear exactly what it is, whether it is possible, and how it relates to
responsibility (Wallace, 1996).

The Limits of Individual Responsibility in Privacy and
Information Assurance

The previous discussion of the conditions of responsibility should have made
it clear that it is very hard for an individual, as the traditional subject of
responsibility, to live up to the expectations, even in rather simple cases. While
humans usually fulfill the personal qualities such as awareness, intentionality, and
emotions, they are hard pressed fulfilling all of them simultaneously. In the
context of complex sociotechnical systems that are the center of attention in this
chapter, individuals generally fail to have the necessary knowledge, freedom,
and power to change events. They may play a role in the causality but are often
unable to change causalities, even if they are aware of them. This has led to a
weakening of the role of the individual as the subject of responsibility, and even
to the perception that there is a loss of the subject (Hubig, 1995a; Hubig, 1995b;
Kaufmann, 1992).
In order to clarify the difficulties that individuals have as subjects of
responsibility, let us return to the problem at hand, which is ethical responsibility
for information systems with regard to security and privacy. Let us take a look
at one typical example of this—a manager who has to make a decision regarding
the introduction of a system that monitors employees’ Internet use. In order for
the social construct of responsibility ascription to be successful and acceptable,
the manager would have to fulfill the conditions. First, the manager would have
to play a part in the causal chain of events. This condition is met due to the
realities of the case. Second, the manager would have to possess the freedom
to influence events according to his or her own free will. Disregarding the
philosophical problems of freedom and free will, we will grant this. By definition,
the manager also does fulfill the third set of conditions; namely, those referring
to personhood. Unfortunately, however, the manager lacks the remaining
capabilities that would render him or her a suitable responsibility subject. In order
for the manager to be a proper subject, the manager would have to know the
situation well enough to estimate all of the relevant factors and developments.
This is practically impossible since the manager does not know all of the
stakeholders’ views, which is principally impossible because the development
stretches into the unknown future.
Furthermore, the manager not only would have to know the objective facts,
but also would have to know their normative evaluation. That means the manager
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Responsibility for Information Assurance and Privacy

201

would have to know not only the laws but also the moral norms of the affected
parties, and the manager would have to know how to interpret and reconcile
these in case of a conflict. This discussion of the complex ethical relationship
between security and privacy has shown that the knowledge necessary to be a
competent subject is complex and distributed. It seems quite impossible for any
one subject to have it at his or disposal. But even if it were available, it would be
impossible for an individual to reconcile all of the potential contradictions and
feedback loops and to come to a decision that is responsible in the sense that it
is sustainable and attributable to the individual. If this is true, and if individuals,
even under the best of circumstances, will not be able to act as subjects of
responsibility in light of the ethical problems of security and privacy, where does
this leave us?

WHAT ABOUT THE END USER?

The purpose of this chapter is to show that information assurance and
privacy are concepts of high ethical importance; their relationship with each
other and with ethics is highly complex. The chapter analyzed the theory of
responsibility and argued that, despite some advantages of the notion, it runs into
serious difficulties when applied to the complex ethical questions relating to
privacy and information assurance. The one specific problem that was discussed
in more depth was that of the individual subject of responsibility. It was argued
that an individual lacks some of the crucial conditions necessary for a successful
ascription of responsibility. This allows us to give an answer to the question posed
in the title of the chapter: The combination of information assurance and privacy
is something that cannot be addressed successfully by individual ethics, at least
not by individual ethics as expressed by the concept of responsibility.
Where does all of this leave the end user who reads this chapter? There are
different possible answers to this. The message of the chapter should not be
misconstrued as meaning that there is no place for individual responsibility in the
normative muddle of security and privacy, and it also should not be misunderstood as a blanket excuse for individuals to do as they like. Given the severity of
the problem and its importance for organizational and social life, however, the
chapter leads to the conclusion that it would be irresponsible to rely on the
individual, who is not equipped to deal with questions of this sort. The argument
in this chapter, therefore, can be used by end users as an explanation of their
incapacity to accept individual responsibility for the complex social relationships
within privacy and information assurance. Going back to the example of the
manager who is asked to introduce a surveillance system for Internet use, a
manager could use this chapter to demonstrate his or her limits in the capacity
as a responsibility subject. This does not mean that the manager should go back
to his or her manager and refuse to do the job. Instead, the reflection on the
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

202 Stahl

concept of responsibility would allow the manager to clearly state what he or she
is capable of doing and not doing. Furthermore, it would allow the manager to
indicate a way of dealing with the problem. Realizing the manager’s lack of
information about the affected people’s views of norms regarding Internet use,
the concept of responsibility introduced here would provide a framework for
determining how to get this information. This means that the manager might
devise specific or new processes that would allow the retrieval of the information. In this way, the manager might meet the expectations or requirements of
information assurance as well as privacy considerations.
This raises several other issues. If the analysis of the problem is correct, and
if ethics is involved in our topic, then the individual is not a promising starting point
from which to look for solutions. Thus, we have to ask how the problems can be
addressed. This question aims at directing the reader’s attention to the idea of
collective responsibility. I believe that the concept of responsibility offers the
promise to be able to address the problem. This would be possible by transcending the traditional definition of the individual human being as the sole possible
subject and by allowing more complex and collective ascriptions. This approach
raises several new questions (e.g., whether collectives can be moral subjects),
but it allows for the extension of the term. While individuals generally lack power
and knowledge in complex situations, the same is not necessarily true for
collectives. This argument for the extension of the concept of responsibility has
been made before (French, 1992; Werhane, 1985) and, in some respects, is
generally recognized already. If we were to agree that it is possible to address
the ethical problems of security and privacy by extending the concept of
responsibility, this again would raise many new questions and require us to
collectively agree on the applicable norms, the problems in question, institutions
of ascription, and sanctions.
Again, this is an area where the individual end user is concerned. If
collective responsibility is to be successful, then even a superficial view indicates
that it is to be connected with individual responsibility. It will be the responsibility
of individuals to participate in the discourses and processes that define suitable
collectives as subjects as well as potential sanctions, or mechanisms of attribution. Furthermore, there must be a structure that allows the drawing of
conclusions from individual to collective responsibility and vice versa. All of
these are aspects that individuals should keep in mind, and where end users can
and must play a central role.
The essential message of the chapter is, thus, that individual responsibility
plays an important role in the relationship of information assurance and privacy.
At the same time, it cautions that concentrating exclusively on the individual as
the responsible agent is easier than going down the long and stony way to
collective responsibility but, as I have tried to show in this chapter, is not very
promising and, thus, irresponsible.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Responsibility for Information Assurance and Privacy

203

REFERENCES

Anderson, R.E., Johnson, D.G., Gotterbarn, D., & Perrolle, J. (1993). Using the
new ACM code of ethics in decision making. Communications of the
ACM, 36(2), 98-106.
Bayertz, K. (Ed.) (1995). Verantwortung: Prinzip oder Problem? Darmstadt,
Germany: Wissenschaftliche Buchgesellschaft.
Bechtel, W. (1985, October). Attributing responsibility to computer systems.
Metaphilosophy, 16(4), 296-305.
Beu, D., & Buckley, M.R. (2001). The hypothesized relationship between
accountability and ethical behavior. Journal of Business Ethics, 34, 5773.
Bierhoff, H.W. (1995). Verantwortungsbereitschaft, Verantwortungsabwehr
und Verantwortungszuschreibung—Sozialpsychologische Perspektiven. In
K. Bayertz (Ed.), Verantwortung: Prinzip oder Problem? (pp. 217-240).
Darmstadt, Germany: Wissenschaftliche Buchgesellschaft.
Birnbacher, D. (1995). Grenzen der Verantwortung. In K. Bayertz (Ed.),
Verantwortung: Prinzip oder Problem? (pp. 143-183). Darmstadt, Germany: Wissenschaftliche Buchgesellschaft.
Boncella, R.J. (2001). Internet privacy—At home and at work. Communications of the Association for Information Systems, 7, 1-28.
Bowie, N.E. (1999). Business ethics—A Kantian perspective. Oxford:
Blackwell.
Britz, J.J. (1999). Ethical guidelines for meeting the challenges of the information
age. In L.J. Pourciau (Ed.), Ethics and electronic information in the 21st
century (pp. 9-28). West Lafayette, IN: Purdue University Press.
Brown, W.S. (2000). Ontological security, existential anxiety and workplace
privacy. Journal of Business Ethics, 23, 61-65.
Camp, L.J. (2001). Web security and privacy: An American perspective. In
R.A. Spinello, & H.T. Tavani (Eds.), Readings in cyberethics (pp. 474486). Sudbury, MA: Jones and Bartlett.
Culnan, M.J. (1993). How did they get my name? An exploratory investigation
of consumer attitudes toward secondary information use. MIS Quarterly,
17(3), 341-363.
Donaldson, T., & Dunfee, T.W. (1999). Ties that bind: A social contracts
approach to business ethics. Boston: Harvard Business School Press.
Eisenberg, T. et al. (1995). The computer worm: A report to the provost of
Cornell University on an investigation conducted by the Commission of
Preliminary Enquiry. In D.G. Johnson, & H. Nissenbaum (Eds.), Computers,
ethics & social values (pp. 60-89). Upper Saddle River, NJ: Prentice Hall.
Elgesiem, D. (1996). Privacy, respect for persons, and risk. In C. Ess (Ed.),
Philosophical perspectives on computer-mediated communication (pp.
45-66). Albany, NY: State University of New York Press.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

204 Stahl

Etchegoyen, A. (1993). Le temps des responsables. Paris: Editions Julliard.
European Union (2000). Charter of fundamental rights of the European Union
(2000/C 364/01). Europarl. Retrieved January 13, 2004, from http://
www.europarl.eu.int/charter/default_en.htm
Fischer, J.M. (1999). Recent work on moral responsibility. Ethics, 110(1), 93-139.
Forester, T., & Morrison, P. (1994). Computer ethics—Cautionary tales and
ethical dilemmas in computing. Cambridge, MA/London: MIT Press.
Frankfurt, H.C. (1997). Partis contraires et responsabilité morale. In M.
Neuberg (Ed.), La responsabilité—Questions philosophiques (pp. 5564). Paris: Presses Universitaires de France.
Frankfurt, H.G. (1969). Alternate possibilities and moral responsibility. Journal
of Philosophy, LXVI, 828-839.
French, P.A. (1992). Responsibility matters. Lawrence, Kansas: University
Press of Kansas.
Gavison, R. (1995). Privacy and limits of law. In D.G. Johnson, & H. Nissenbaum
(Eds.), Computers, ethics & social values (pp. 332-351). Upper Saddle
River, NJ: Prentice Hall.
Goldman, A.I. (1999). Why citizens should vote: A causal responsibility approach. In E.F. Paul, F.D. Miller, & J. Paul (Eds.), Responsibility (pp. 201217). Cambridge, MA: Cambridge University Press.
Grabner-Kräuter, S. (2002). The role of consumers’ trust in online-shopping.
Journal of Business Ethics, 39, 43-50.
Hart, H.L.A. (1968). Punishment and responsibility—Essays in the philosophy of law. Oxford: Clarendon Press.
Hartman, L. (2001). Technology and ethics: Privacy in the workplace. Business
and Society Review, 106(1), 1-27.
Healy, M., & Iles, J. (2002). The impact of information and communications
technology on managerial practices: The use of codes of conduct. In I.
Alvarez et al. (Eds.), The transformation of organisations in the
information age: Social and ethical implications. Proceedings of the
Sixth ETHICOMP Conference, November 13-15, 2002, Lisbon, Portugal.
Himanen, P. (2001). The hacker ethic and the spirit of the information age.
London: Secker & Warburg.
Höffe, O. (1995). Moral als Preis der Moderne: Ein Versuch über
Wissenschaft, technik und umwelt. Frankfurt a. M.: Suhrkamp.
Hoffman, D.L., Novak, T.P., & Peralta, M. (1999). Building consumer trust
online. Communications of the ACM, 42(4), 80-87.
Hubig, C. (1995a). Verantwortung und Hochtechnologie. In K. Bayertz (Ed.),
Verantwortung: Prinzip oder Problem? (pp. 98-139). Darmstadt, Germany: Wissenschaftliche Buchgesellschaft.
Hubig, C. (1995b). Technik- und Wissenschaftsethik. Berlin/Heidelberg, NY:
Springer Verlag.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Responsibility for Information Assurance and Privacy

205

Hunter, L. (1995). Public image. In D.G. Johnson, & H. Nissenbaum (Eds.),
Computers, ethics & social values (pp. 293-299). Upper Saddle River,
NJ: Prentice Hall.
IASE (2003). Information assurance support environment. IASE. Retrieved
April 8, 2003, from http://iase.disa.mil/
Introna, L. (2000). Privacy and the computer—Why we need privacy in the
information society. In R.M. Baird, R. Ramsower, & S.E. Rosenbaum
(Eds.), Cyberethics— Social and moral issues in the computer age (pp.
188-199). New York: Prometheus Books.
Johnson, D.G. (2001). Computer ethics. Upper Saddle River, NJ: Prentice Hall.
Jonas, H. (1984). Das Prinzip Verantwortung. Frankfurt a. M.: Suhrkamp.
Kant, I. (1995). Kritik der praktischen Vernunft/Grundlegung zur Metaphysik
der Sitten. Frankfurt a. M.: Suhrkamp.
Kaufmann, F.-X. (1992). Der Ruf nach Verantwortung. Freiburg im Breisgau:
Herder.
Khare, R., & Rifkin, A. (1998). Trust management on the World Wide Web.
First Monday, 3(6). Retrieved February 21, 2005 from http://www.first
monday.org/issues/issue3_6/khare/index.html
Koehn, D. (2001). Ethical challenges confronting businesses today. Proceedings of the 11th International Symposium on Ethics, Business and
Society, Barcelona, Spain.
Langford, D. (1999). Business computer ethics. Harlow, UK: Addison-Wesley.
Lenk, H. (1998). Konkrete Humanität: Vorlesungen über Verantwortung
und Menschlichkeit. Frankfurt a. M.: Suhrkamp Verlag.
Lenk, H., & Maring, M. (1990). Verantwortung und soziale Fallen. Ethik und
Sozialwissenschaften, 1, 49.
Levy, S. (1995). Battle of the clipper chip. In D.G. Johnson, & H. Nissenbaum
(Eds.), Computers, ethics & social values (pp. 651-664). Upper Saddle
River, NJ: Prentice Hall.
Littlewood, B., & Stringy, L. (1995). The risks of software. In D.G. Johnson, &
H. Nissenbaum (Eds.), Computers, ethics & social values (pp. 432-437).
Upper Saddle River, NJ: Prentice Hall.
Mason, R.O. (1986). Four ethical issues of the information age. MIS Quarterly,
10, 5-12.
May, L., & Hoffman, S. (Eds.). Collective responsibility: Five decades of
debate in theoretical and applied ethics. Savage, MD: Rowman &
Littlefield Publishers Inc.
Milberg, S., Burke, S., Smith, H.J., & Kallman, E. (1995). Values, personal
information, privacy, and regulatory approaches. Communications of the
ACM, 38(12), 65-74.
Moor, J.H. (2000). Toward a theory of privacy in the information age. In R.M. Baird,
R. Ramsower, & S.E. Rosenbaum (Eds.), Cyberethics—Social and moral
issues in the computer age (pp. 200-212). New York: Prometheus Books.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

206 Stahl

Neuberg, M. (Ed.) (1997). La responsabilité—Questions philosophiques.
Paris: Presses Universitaires de France.
NIAP (2003). About NIAP. Retrieved April 7, 2003, from http://niap.nist.gov/
Nida-Rümelin, J. (1998). Über den Respekt vor der Eigenverantwortung des
Anderen. In B. Neubauer (Ed.), Eigenverantwortung: Positionen und
Perspektiven. Waake, Germany: Licet Verlag.
Nikander, P. & Karvonen, K. (2000). Users and trust in cyberspace. Cambridge Security Protocol Workshop 2000, Cambridge, UK.
Paul, E.F., Miller, F.D., & Paul, J. (Eds.) (1999). Responsibility. Cambridge,
MA: Cambridge University Press.
Posner, R.A. (1995). An economic theory of privacy. In D.G. Johnson, & H.
Nissenbaum (Eds.), Computers, ethics & social values (pp. 358-366).
Upper Saddle River, NJ: Prentice Hall.
Rachels, J. (1995). Why privacy is important. In D.G. Johnson, & H. Nissenbaum
(Eds.), Computers, ethics & social values (pp. 351-357). Upper Saddle
River, NJ: Prentice Hall.
Robison, W.L. (2000). Privacy and appropriation of identity. In G. Collste (Ed.),
Ethics in the age of information technology (pp. 70-86). Linköping,
Sweden: Centre for Applied Ethics.
Rogerson, S. (1998). Ethical aspects of information technology—Issues for
senior executives. London: Institute of Business Ethics.
Rotenberg, M. (1995). Computer virus legislation. In D.G. Johnson, & H.
Nissenbaum (Eds.), Computers, ethics & social values (pp. 135-147).
Upper Saddle River, NJ: Prentice Hall.
Rotenberg, M. (1998). Communications privacy: Implications for network design.
In R.N. Stichler, & R. Hauptman (Eds.), Ethics, information and technology: Readings (pp. 152-168). Jefferson, NC: MacFarland & Company.
Rötzer, F. (1998). Eigenverantwortung in komplexen Systemen und als komplexes
System. In B. Neubauer (Ed.), Eigenverantwortung: Positionen und
Perspektiven. Waake, Germany: Licet Verlag.
Rutter, J. (2001). From the sociology of trust towards a sociology of “e-trust.”
International Journal of New Product Development & Innovation
Management, 2(4), 371-385.
Sänger, M. (Ed.). Arbeitstexte für den Unterricht: Verantwortung. Stuttgart,
Germany: Philipp Reclam jun.
Schulman, M. (2000). Littlebrother is watching you. In R.M. Baird, R. Ramsower,
& S.E. Rosenbaum (Eds.), Cyberethics—Social and moral issues in the
computer age (pp. 155-161). New York: Prometheus Books.
Severson, R.J. (1997). The principles of information ethics. Armonk, NY/
London: M.E. Sharpe.
Sipior, J.C., & Ward, B.T. (1995). The ethical and legal quandary of email
privacy. Communications of the ACM, 38(12), 48-54.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Responsibility for Information Assurance and Privacy

207

Spafford, E.H. (1995). Are computer hacker break-ins ethical? In D.G. Johnson,
& H. Nissenbaum (Eds.), Computers, ethics & social values (pp. 125134). Upper Saddle River, NJ: Prentice Hall.
Spinello, R. (1997). Case studies in information and computer ethics. Upper
Saddle River, NJ: Prentice Hall.
Staddon, J. (1999). On responsibility in science and law. In E.F. Paul, F.D.
Miller, & J. Paul (Eds.), Responsibility (pp. 146-174). Cambridge, MA:
Cambridge University Press.
Stahl, B.C. (2004). Responsible management of information systems. Hershey,
PA: Idea Group Publishing.
Stocker, M. (1999). Responsibility and the abuse excuse. In E.F. Paul, F.D.
Miller, & J. Paul (Eds.), Responsibility (pp. 175-200). Cambridge, MA:
Cambridge University Press.
Straub, D.W., & Collins, R.W. (1990). Key information liability issues facing
managers: Software piracy, proprietary databases, and individual rights to
privacy. MIS Quarterly, 14.
Tavani, H. (2000). Privacy and security. In D. Langford (Ed.), Internet ethics
(pp. 65-89). London: McMillan.
Tavani, H.T., & Moor, J.T. (2001). Privacy protection, control of information,
and privacy-enhancing technologies. In R.A. Spinello, & H.T. Tavani
(Eds.), Readings in cyberethics (pp. 378-391). Sudbury, MA: Jones and
Bartlett.
van den Hoven, M.J. (1999). Privacy or informational injustice? In L.J. Pourciau
(Ed.), Ethics and electronic information in the 21st century (pp. 139150). West Lafayette, IN: Purdue University Press.
van der Ploeg, I. (2001). Written on the body: Biometrics and identity. In R.A.
Spinello, & H.T. Tavani (Eds.), Readings in cyberethics (pp. 501-514).
Sudbury, MA: Jones and Bartlett.
Velasquez, M. (1998). Business ethics: Concepts and cases. Upper Saddle
River, NJ: Prentice Hall.
Wallace, R.J. (1996). Responsibility and the moral sentiment. Cambridge,
MA/London: Harvard University Press.
Warren, S.D., & Brandeis, L.D. (1890). The right to privacy. Harvard Law
Review, 5, 193-220.
Weckert, J., & Adeney, D. (1997). Computer and information ethics. Westport,
CT/London: Greenwood Press.
Werhane, P. (1985). Persons, rights, and corporations. Englewood Cliffs, NJ:
Prentice-Hall.
Wunenburger, J.-J. (1993). Questions d’éthique. Paris: Presses Universitaires
de France, PUF.
Zimmerli, W.C. (1987). Wandelt sich die Verantwortung mit dem technischen
Wandel? In H. Lenk, & G. Ropohl (Eds.), Technik und Ethik (pp. 92-111).
Stuttgart: Philipp Reclam jun.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

208 Jones and Price

Chapter XI

Organizational Knowledge
Sharing in ERP
Implementation:
Lessons from Industry

Mary C. Jones, University of North Texas, USA
R. Leon Price, University of Oklahoma, USA

ABSTRACT

This study examines organizational knowledge sharing in enterprise
resource planning (ERP) implementation. Knowledge sharing in ERP
implementation is somewhat unique, because ERP requires end users to
have more divergent knowledge than is required in the use of traditional
systems. Because of the length of time and commitment that ERP
implementation requires, end users also are often more involved in ERP
implementations than they are in more traditional ERP implementations.
They must understand how their tasks fit into the overall process, and they
must understand how their process fits with other organizational processes.
Knowledge sharing among organizational members is one critical piece of
ERP implementation, yet it is challenging to achieve. There is often a large
gap in knowledge among ERP implementation personnel, and people do not
easily share what they know. This study presents findings about
organizational knowledge sharing during ERP implementation in three
firms. Data were collected through interviews using a multi-site case study
methodology. Findings are analyzed in an effort to provide a basis on
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

209

which practitioners can facilitate knowledge sharing more effectively
during ERP implementation.

INTRODUCTION

Enterprise resource planning (ERP) is a strategic tool that helps companies
gain a competitive edge by streamlining business processes, integrating business
units, and providing organizational members greater access to real-time information. Many firms are using ERP systems to cut costs, standardize operations, and
leverage common processes across the organization. ERP allows firms to have
a more convergent view of their information by integrating processes across
functional and divisional lines using a centralized database and integrated sets of
software modules (Scott & Kaindl, 2000; Zheng et al., 2000).
However, the convergence that ERP affords at the organizational level
often results in a divergence of the knowledge required at the individual level
(Baskerville et al., 2000). ERP imposes a new framework on the organization
(Robey et al., 2002). It requires end users to have broader knowledge than is
required in the use of traditional systems. They must understand how their tasks
fit into the overall process and how their process fits with other organizational
processes (Lee & Lee, 2000). Thus, knowledge sharing is one critical piece of
ERP implementation. During implementation, an organization begins to build the
foundation on which end users can understand enough about the ERP framework
to realize its benefits (Robey et al., 2002). Because of the time commitments and
the extensive knowledge sharing that must take place during ERP implementation, end users often are more involved in the implementation than they are in
more traditional implementations. In some cases, ERP implementations are
managed and led by end users and end-user managers, and IT staff serves
primarily as technical advisors (Jones, 2001). Unfortunately, there is usually a
significant gap in knowledge among these implementation personnel, and people
do not easily share what they know (Constant et al., 1994; Jarvenpaa & Staples,
2000; Osterloh & Frey, 2000; Soh et al., 2000).
This study was undertaken to examine how firms ensure that organizational
knowledge is shared during ERP implementations. One objective is to identify
facilitators of organizational knowledge sharing. Another is to synthesize findings into lessons about knowledge sharing during implementation that other firms
can apply in their own ERP implementations.

THEORETICAL BACKGROUND

Knowledge sharing in ERP implementation is somewhat unique, because
ERP redefines jobs and blurs traditional intraorganizational boundaries (Lee &
Lee, 2000). Knowledge must be shared across functional and divisional boundCopyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

210 Jones and Price

aries, and the knowledge required during ERP implementation entails a wider
variety of experiences, perspectives, and abilities than traditional information
systems implementations do (Baskerville et al., 2000; Robey et al., 2002).
Knowledge sharing is challenging, because much knowledge is embedded into
organizational processes (Davenport, 1998). The way people actually do their
jobs is often different from the formal procedures specified for even the most
routine tasks (Brown & Duguid, 2000). It is also challenging because there are
gaps between what people do and what they think they do (Brown & Duguid,
2000). Some tasks are so routine, and people have done them for so long, that
many of the steps involved are subconscious (Leonard & Sensiper, 1998).
However, there is a variety of factors that can facilitate knowledge sharing
during ERP implementation.
In order to present a coherent and logical view of knowledge sharing, we
identify factors that influence knowledge sharing that are linked by a common
conceptual underpinning which allows individuals to share observations and
experiences across traditional boundaries. Most ERP implementation activities
center around the ERP implementation team (Baskerville et al., 2000). ERP
implementation teams typically consist of organizational members from a variety
of functional areas and organizational divisions. Each team member must
understand what the others do in order to effectively map processes during the
implementation (Baskerville et al, 2000). Team members must work to achieve
this level of understanding. The knowledge sharing required does not come
automatically with team membership; it must be facilitated. Thus, facilitation of
knowledge sharing on the team is one factor examined.
The team must also interact with end users to gather relevant information
about processes and to keep end users and user managers informed about
changes to expect when the ERP is implemented (Robey, Ross, & Boudreau,
2002). Ideally, there is an intensive exchange of knowledge between the team
and these users that they represent (Baskerville et al., 2000). Inadequate
knowledge sharing between these two groups leads to unsuccessful implementation (Soh et al., 2000). One key to a smooth ERP implementation is effective
change management (Andriola, 1999; Harari, 1996). Because of the complexity
and cost of ERP, it must be planned and implemented visibly (Hammer, 1990).
One way to communicate plans, share knowledge with end users, and gather
knowledge from end users is through careful change management (Clement,
1994). Therefore, change management is another knowledge-sharing factor
examined.
A large part of change management is training. Those affected by the
implementation should receive training to develop new and improved skills to deal
with new challenges brought about by the change (Andriola, 1999). Users must
gain knowledge about the business rules and processes embedded in the ERP
software (Lee & Lee, 2000). They also must understand the integrative nature
of ERP in order to use it effectively. ERP requires end users to understand that
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

211

they are no longer working in silos, and whatever they do now impacts someone
else (Welti, 1999). Entire departments must be retrained with this in mind (AlMashari & Zairi, 2002; Caldwell & Stein, 1998). Training on transactions and on
the integrative nature of ERP is another factor examined.
Most firms hire external consultants (i.e., integration partners) that know
the ERP software in order to help them through the implementation (Soh et al.,
2000). This involves knowledge sharing because the organizational implementation team seeks ways for the know-how and skills possessed by integration
partner staff (IPS) to be shared with them so that they are not lost when the IPS
leaves (Al-Mashari & Zairi, 2000). This goes beyond written documentation and
training manuals. For example, consultants are assigned to work side-by-side
with organizational team members so that the members can learn what the
consultants know about the package that cannot easily be written down
(Osterloh & Frey, 2000). One source of failure in ERP implementation is IPS
who work alone and fail to share knowledge with organizational members (Welti,
1999). When the IPS fails to share what they know, the firm often has trouble
supporting the ERP after they leave. Thus, it is important that the firm capture
as much of the IPS’ knowledge as possible before they transition off the team.
Transition of IPS knowledge is another knowledge-sharing factor examined. In
summary, several factors that may influence knowledge sharing are examined.
These are facilitation of knowledge sharing on the implementation team, change
management activities, type of training end users receive (i.e., transactional or
integrative), and use of formal knowledge transfer from integration partner staff
when they leave the organization.
Finally, the extent to which a firm is beginning to alter its core knowledge
competency after SAP implementation is examined. The active sharing of
organizational members’ knowledge is linked to a firm’s ability alter its core
knowledge competencies (Grant, 1996; Hine & Goul, 1998; Kogut & Zander,
1992). Altering knowledge competency involves sharing knowledge across the
organization in a way that preserves existing knowledge competencies and, at the
same time, absorbs new knowledge that expands and strengthens those competencies (Stein & Vandenbosch, 1996). An innovation that impacts the entire
organization and facilitates major changes in a firm’s processes, as ERP does,
provides an opportunity for firms to do this (Brown & Vessey, 1999). Evidence
of this alteration is found in fundamental changes in the way a firm performs its
core processes. ERP benefits are the result of ongoing efforts to continuously
improve processes (Ross, 1999). At the time of data collection, these firms were
still too early in their use of ERP to have realized extensive change. However,
they were making efforts to integrate processes and thereby alter core knowledge competency. Thus, change in core knowledge competency in this study is
assessed as the extent to which processes were being changed as a result of
ERP, rather than the extent to which they had changed.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

212 Jones and Price

METHODOLOGY

Data were collected as part of a larger study using a multiple case study of
firms in the petroleum industry that had implemented SAP R/3. Focusing on a
single package helps minimize bias that might be introduced into findings across
packages. However, because the focus is on knowledge sharing rather than on
technical aspects of the package itself, findings should be generalizable to
implementation of other ERP software in other industries. The CIO or top IS
executive of 10 firms in the industry were contacted to determine if they had
implemented or were implementing SAP, and if so, whether they would agree to
participate in the study. In some cases, a division of the firm was included rather
than the whole firm. Because of size, structure, or geographic dispersion, some
firms have conducted completely separate implementations in divisions around
the world, with little or no communication between the implementation teams. In
those cases, the division seemed to be a more appropriate case site than the entire
organization. We collected data from those that agreed to participate and that
met two other criteria. We eliminated firms that had implemented only one or two
modules with no plans to implement more. We also eliminated firms that had not
implemented across the organization or the specific division in which we were
interested. Each firm in the study implemented the major modules of SAP,
including FI/CO (financial accounting and controlling), AM (assets management), PS (project systems), PM (plant maintenance), SD (sales and distribution), MM (materials management), and PP (production planning). These criteria
helped to ensure that the case sites were comparable and that differences in
findings were not due to the scale of implementation.
In order to minimize bias that the researchers might introduce into the
process of analyzing findings, a rigorous and structured approach to analysis was
followed (Yin, 1989). For example, the interviewer took notes and taped each
interview. Tapes and notes were transcribed by a third party and reviewed by
the interviewer; respondents were asked for clarification on points that seemed
vague or missing. The transcriptions then were summarized and reviewed by
another researcher to help ensure that the transcriptions flowed well and made
sense. Finally, the primary contacts in each firm reviewed summaries to help
ensure that what was recorded represented actual events and perceptions. A
case study database consisting of interview notes, documentation provided by
respondents, tables summarizing findings, and an exact narrative transcription of
all interviews were used. The questions from the interview guide are provided
in Appendix A. A within-case analysis was performed, where data were
extracted using the interview questions as a guide to get a clearer picture of
knowledge sharing in each firm. Then, a cross-case analysis was performed in
which knowledge sharing across the firms was compared.
Because of the size of the project teams, interviewing a sample of key
members was deemed more manageable than attempting to interview each

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

213

member. In addition, many members had left the firm or moved out of the areas
in which they had originally worked. Thus, we asked each of the top IS
executives to identify key members of their SAP project team that were still
involved with SAP in some way, including support and post implementation
process redesign. This method of identifying respondents has been demonstrated
to be acceptable, because professionals in a field have been shown to be capable
of nominating key respondents that have a consistent set of attributes appropriate
for a study such as this (Nelson et al., 2000). A series of semi-structured
interviews were conducted with eight to 10 members of each firm. The number
of interviewees was chosen based on the concept of theoretical saturation where
“incremental learning is minimal because the researchers are observing phenomena seen before” (Eisenhardt, 1989, p. 545). In these interviews, the researcher
often heard the same examples from most of the respondents in a site, regardless
of functional background, when they came on the team, or what their job was at
the time of the interview. In these interviews, the researcher often heard the
same examples from most of the respondents in a site, and they often used the
same phrases to express their perceptions. This was true of respondents who
were not located at the same physical locations at a site, or who were not all on
the team at the same time. Thus, it was deemed that additional interviews would
not yield significantly different insights. For example, all the respondents at
USWhole used the phrase psychological effort when referring to how they
approached the project. They indicated one guiding tenet of their project was that
the implementation was as much a psychological effort as a technical effort. In
another example, the phrase the accountants always cleaned up after
everyone came up in most interviews at each case site.
The interviews lasted between one and two hours each over a period of
seven months between July of 2000 and February of 2001. Each person was
interviewed once in person for one to two hours, and then was contacted by email or by telephone for additional information or clarification. In addition to the
face-to-face interviews, the researchers also preceded and followed up the
interviews with e-mail and telephone calls for background information, clarification, and points not covered in the interviews.
Respondents included both information systems staff and business/functional staff. Some had been on the team from the beginning, while others joined
at various points in the project. These people represented a variety of perspectives on SAP, including some who were pleased with it, some who hated it, and
others who were indifferent. They also represented a variety of levels in the firm
ranging from CIO and/or project manager to lower level employees, and included
people from such functional areas as accounting, purchasing, refineries, sales
and distribution, and a variety of engineering functions (Table 1).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

214 Jones and Price

Table 1. Profile of respondents
Company
USWhole
(Multiple roles
for many
members)
E&P

Chemicals

SAP Team Role for Each Respondent (1 respondent per line)
Responsible for SAP configuration; reengineering processes; managed quality assurance and testing; change
management
IT team lead; applications development lead; general leadership with three others of Chemical and
Downstream implementations; managed configuration and upgrades throughout the company
Project Manager
Project Manager
Service Delivery Manager; managed transition plan from production to operations and oversight of the
conversion
Technical leader for FI/CO; was also on HR design team
Functional expert in project systems and asset management
Team member; worked with conversion of legacy systems and investment management data to SAP
Leader for transition from development to support
Site Implementation Manager
Logistics team lead
Team lead of all financial modules of SAP
Director of the order-to-cash process; dealt with customer service, accounts receivable, credit, and some sales
accounting.
Team lead for sales and operations planning
Change management lead; responsible for communications and training materials.
Business implementation lead
Manager of the support group
Co-project manager
Co-project manager

PROFILE OF COMPANIES

USWhole is the U.S. division of one of the world’s leading oil companies.
It includes upstream (i.e., exploration and production), downstream (i.e., marketing, refining, and transportation), and chemical segments. The firm has exploration and production interests in many countries, with a large concentration in the
U.S., and it markets its products worldwide. USWhole performed five SAP
implementations for each of its major business units, including a small pilot test
site and corporate headquarters. It began its SAP project in early 1995 and
completed its first implementation in March of 1996. The final two implementations were completed simultaneously in July of 1998. There are approximately
15,000 SAP users in USWhole.
E&P is the North American exploration and production division of an
international petroleum company that has annual revenues in excess of $90
billion dollars U.S.. This particular division is engaged in the exploration and
production of crude oil and natural gas worldwide and accounts for approximately $6.8 billion dollars U.S. of the corporation’s revenue. Although SAP has
been implemented in various units of the parent company throughout the world,
each project has been a separate activity from all the others. The teams, scope,
budget, and timelines have been managed separately, and SAP has been
designed and configured differently for each with very little or no collaboration
among the units. Therefore, focusing only on E&P’s application in this firm
seems to provide a unit of analysis that is comparable to that in the other sites.
E&P began its project in 1996, using a big bang implementation where all modules
were implemented at one time, and finished the implementation for approximately 3,000 users in mid-1998.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

215

Table 2. Corporate profile
Corporate
Identity
USWhole
E&P
Chemicals

Revenue

Began SAP

Implementation
Date

Number of Users

(U.S. $)
*
$6.8 Billion
$4 Billion

1995
1996
1996

1996-1998
1998
1998-1999

15,000
3,000
5,000

* USWhole requested that this not be revealed.

Chemicals is the chemical division of an international petroleum company
with annual revenues of approximately $16 billion dollars U.S.. Chemicals
accounts for approximately one-fourth of its parent company’s revenue, with
annual revenues of approximately $4 billion dollars U.S. It is a leading chemical
manufacturer with interests in basic chemicals, vinyls, petrochemicals, and
specialty products. Its products are largely commodity in nature, in that they are
equivalent of products manufactured by others and are generally available in the
marketplace. They are produced and sold in large volumes, primarily to industrial
customers for use as raw materials. Chemicals began its SAP project in late
1996, with the first of nine implementations in January 1998. The implementations occurred approximately every two to three months, until all implementations were finished in December 1999. There are approximately 5,000 SAP users
in Chemicals. A summary of profiles is provided in Table 2.

DATA ANALYSIS

In the sections that follow is a description of knowledge sharing factors in
each firm, including facilitation of knowledge sharing on the team, change
management/training, and transition of IPS knowledge. The extent to which
firms had changed or were beginning to change their core knowledge competencies through changes in processes as a result of the SAP implementation is also
discussed. A summary of points covered is provided in Tables 3a, 3b, and 3c.
Table 3a provides a summary of facilitation of knowledge sharing on the team.
Table 3b provides a summary of change management and training activities, and
IPS knowledge transition activities. Table 3c provides a summary of changes in
core knowledge competency.

USWHOLE
Facilitation of Knowledge Sharing on the Team

Teams at USWhole had a negative connotation prior to the SAP project.
They often were used as dumping grounds for weak employees. This was a
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

216 Jones and Price

Table 3a. Summary of facilitation of knowledge sharing on the team
Company
USWhole

Facilitation of knowledge sharing on the team
deemphasized titles, rank, and seniority on the team

E&P

emphasis on codifying how things worked and comparing written descriptions
lots of socialization after work
team members got to know each other and were supportive of each other
viewed each other as experts in their respective areas
focused on a common purpose
some tension between IT and integration partner, yet it subsided as the project
required heavy time and energy commitments

Chemicals

proactively sought ways to minimize the impact of the tension
team organized by process
deemphasized seniority and rank by providing the same bonus to all on the team
actively
involved a variety of key users early in the process to ensure that they gathered
knowledge from the right people

Table 3b. Summary of change management/training for end users and
transition of IPS knowledge
Company
USWhole

Change Management

Training

Transition of IPS Knowledge

Team communicated
with end users about
how SAP would change
their jobs

Identified power users
among end users to
train

Worked with IPS throughout
the project

Identified end users to
be change agents within
the units
Relied on change agents
to communicate as well
E&P

Team went to change
management training
Followed a change
management strategy
Focused on
communicating project
status to the company

Chemicals

Made sure end users
who were not directly
part of the team had
input into the project
Focused on helping end
users understand how
their jobs would change
after SAP
Focused on how end
users would use SAP

Power users helped
train other users
Focused largely on
transactions
Limited focus on
integration
Identified power users
among end users to
train
Power users helped
train other users
Focused largely on
transactions

Documented lessons learned at
the end of each go-live
Used no formal transfer
process at the end

Used formal transfer process
with checklists on how to
configure and on which things
triggered what
Transferred knowledge from
IPS to 3rd party consultant,
then from that consultant to
E&P support team

Limited focus on
integration
Identified power users
among end users to
train
Power users helped
train other users
Focused on integration
in addition to
transactions

Built knowledge transfer into
the contract with the IPS
Focused on how they solved
problems and where they
looked for answers
Team members gradually took
on more responsibility so they
could learn what the integration
partner knew

major obstacle to overcome in facilitating knowledge sharing on the SAP
implementation team. Top management strongly supported SAP, so the project
managers were able to ask for and get the best people in most cases for the
implementation team. They sent people back to their units if they did not work
out. Thus, they put together team members that had reasonably good knowledge
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

217

Table 3c. Summary of changes in core knowledge competency
Company
USWhole

Changes
Gradually eliminating silo behavior
Some units adapted better than others and, thus, have seen more changes than others

E&P

Adaptation across the firm seems to be occurring; “it’s not like I do a job anymore, but I perform a
step in a process”
Slowly moving away from silo behavior
People are beginning to understand the integration points better, particularly in the financials area
Adaptation limited by corporate budget cuts unrelated to SAP

Chemicals

Are still in the learning cycle, but changes are ongoing
Majority of Chemicals units have embraced the concept of common processes, particularly in
financials and purchasing
Have completed development of a common master file for parts, and units are designing
purchasing around families of parts
Processes in general are now more well defined and better understood across functions within and
across divisions

about their own processes. USWhole facilitated knowledge sharing on the team
by eliminating seniority and functional distinctions. For example, senior people
worked alongside hourly workers on the team, and if the lower level employees
had an idea or wanted to try something, the senior people listened to them and,
in some cases, took direction from them. As one person said, before this project,
“a lower level person wouldn’t say what they thought in front of a more senior
person. But with the shared goal of getting the project done quickly, they did.”
Lower level people also challenged senior people if they didn’t agree or thought
there was a better way of doing something. USWhole provided a structure to the
team that allowed people to share knowledge openly and freely. This helped to
resolve conflicts and to map processes to SAP effectively.
As one person said, “The bad thing was to have an idea and not express it.”
USWhole also relied heavily on codifying knowledge and writing down how
processes worked. For example, “if someone said we can’t do it this way, we
said, ‘Why can’t you? Is it really unique?’ We’d get them to list what they do and
to look at what others have listed, and identify the commonalities.” USWhole
used several approaches to facilitate knowledge sharing on the team, including
codifying knowledge, structuring the team to remove barriers to knowledge
sharing, and proactively seeking to overcome the stigma associated with teams.

Change Management/Training

From the beginning of the project, USWhole had a strong change management team to communicate with the rest of the organization about project status,
issues, ideas, managing expectations, and training. As one said, “It’s all about
change management. That’s the name of the game.” Another person indicated,
“We had to break down cultural barriers (to common processes) through
communication.” The team members shared their knowledge about SAP with
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

218 Jones and Price

the users in order to do so. They used several verbal and written communication
means to reach users at all levels of the organization. The change management
team helped users and managers understand how SAP would impact them;
gathered feedback on user perceptions, concerns, and issues; and helped
overcome resistance to change.
USWhole used a power user concept for training users. They identified
users in each of the business units that were influential in their units and that were
interested in SAP, and trained them extensively in how to do transaction
processing as well as in how processes were changing and being integrated.
However, there was more emphasis on the how-to than on process changes.
Users largely learned the latter on the job as they began to use the system. As
power users shared their knowledge with other users, knowledge about how to
use SAP began to permeate the organization. However, this was more difficult
in some streams than in others. For example, one unit had old technology and
went from “1960’s technology to 1990’s technology in one fell swoop. Some had
never used a mouse before, and one guy was moving his mouse over the screen
to choose an icon.” Thus, it was harder for them to learn how to use the new
system even at the most basic level.

Transition of IPS Knowledge

Because of the sheer size of the project, USWhole had several integration
partners. They did not use a formal knowledge transfer process when IPS left,
but they did document how to configure and perform all major activities, and they
documented lessons learned with each implementation. USWhole people worked
with each integration partner throughout the project so that the knowledge
transfer took place over time. In addition, although integration partners may have
been different for each business unit, the core team from USWhole was the same
throughout. Thus, knowledge gained in one implementation was not lost but
rather enhanced as the project progressed.

Changes in Core Knowledge Competency

As a result of SAP, team members gained knowledge about the organization
as they learned about the “linkages and inefficiencies between processes.”
However, the organization has had mixed results in altering core knowledge
competencies to change the way they perform processes. “Different streams
have adapted differently.” The downstream operations are the most complex to
do in SAP, and this stream had experienced the least change in the past. In the
beginning, it had the greatest difficulty in adapting to integrated, common
processes. “Downstream adapted very poorly early on.” The chemicals division
was used to change because it operates in an “acquisition and trade environment.” It also was running SAP R/2, so it was more familiar with the integrated
process approach. Thus, it has had an easier time adapting. Similarly, “upstream
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

219

is primarily accounting based, so with the changing economy they grew used to
change,” and this stream has adapted to the changes more readily. Thus,
USWhole has experienced mixed results in its efforts to alter core knowledge
competencies, but it is continually working toward change.
One explanation for this is that USWhole did not recognize early enough the
differences in the streams’ abilities to adapt to change. Its change management
approach was not tailored to each stream, and even though they received
feedback from each, if a stream was resistant, it may not have shared enough
of what it knew so that the team could make the transition more effectively.
Although USWhole worked hard to ensure that effective knowledge sharing took
place on the team, its efforts to ensure knowledge sharing between the team and
the rest of the organization may not have been strong enough to impact change
in core processes.

E&P
Facilitation of Knowledge Sharing on the Team

E&P used informal team building activities to help solidify team member
relationships in an effort to foster knowledge sharing. Team members frequently
socialized together after work, and at the end of major milestones, the company
treated the entire team to various dinners, parties, and other outings. The team
was also solidified, because team members “knew the legacy systems on the
business and technical side, and they were highly capable and credible in their
areas.” They viewed each other as experts in their areas and, thus, were willing
to listen to and learn from each other.
However, E&P had a somewhat unique obstacle to knowledge sharing to
overcome in its implementation. The information technology (IT) division of the
parent company is managed as a separate company and must contract with E&P
and in competition with other outsourcing vendors for jobs. The IT company bid
to be the integration partner on the SAP project, yet the E&P project lead chose
another firm to be the primary integration partner because it had more experience
with SAP. However, the IT staff had extensive knowledge about and experience
with the E&P legacy systems that SAP replaced. In some cases, the IT staff had
as much or more knowledge about how processes worked than the E&P business
unit employees. Thus, they were selected to be part of the SAP team in order not
to lose their knowledge. At first, there was some tension between IT staff and
the IPS, because the IT staff felt that they should have been chosen as the
primary integration partner. However, there was a strong corporate culture of
working in teams; thus, this tension was minimized, and team members focused
primarily on the common purpose of completing the project rather than on
themselves. In addition, as new people came on the team throughout the project,
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

220 Jones and Price

they were not aware of the earlier tension, which also helped to dissipate it. As
one said, “We didn’t have time to draw lines in the sand. We were concerned with
meeting deadlines, and we all had the same goal—making SAP work.”

Change Management/Training

E&P had a change management team in place whose responsibility was to
make sure that current project status was communicated to all company
employees and to make sure that people not directly tied to the project felt like
they also had some ownership. There was particular emphasis on this communication because “our experience on these large implementations has a very
checkered past.” E&P had implemented fairly well conceived large systems in
the past, where the change management people did not handle the change well.
As a result, organizational members did not like the systems, and sometimes the
systems were perceived as becoming the “butt of a lot of jokes.” Thus, senior
management placed a high priority on managing change in the SAP project, and
a large piece of the budget was devoted to it. The change management team
went through change management training classes, and the integration partner
“brought in a very strong change management plan.” The “”change management
piece was very mature, very well thought out, very strong.” The change
management team handled all communication using a variety of written and
verbal communication techniques ranging from e-mail to town-hall meetings.
“We got some good input [through this communication] that helped us restore
some things that may have caused trouble later on.”
Training was done using the power user concept . The emphasis in training
was more on how to perform transaction processing than on the way processes
were changing or the integrative nature of processes. The project budget
provided for the latter aspect of training after implementation to give the users
a chance first to understand how to use the system for basic transactions.
However, the budget for all training at E&P, not just SAP, was cut, and they did
not get to do as much of that as they wanted. This hurt the change management
team’s ability to share knowledge with the organization. One site was able to do
more training, because they had some additional resources that they could use.
Even though “it wasn’t much more training, you can really see the difference in
how much better they are able to take advantage of SAP than other locations
are.”

Transition of IPS Knowledge

When it came time to transition the IPS off the team, the original tension
regarding choice of partner began to resurface. Although most of the team
members either had gotten past it or were unaware of it throughout the project,
the project manager still had reservations about the firm’s internal IT capabilities
to support SAP after implementation. He wanted to hire the integration partner
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

221

to continue working with the firm indefinitely as the SAP support team, even
though this was a more expensive long-run option. Because of the expense and
the tension the decision created, senior management overrode the project
manager’s decision and hired the IT division to do long-term support. To ensure
that the transition was smooth, they removed the project manager from the
project and transferred him laterally to another part of the organization that had
nothing to do with SAP. They appointed an experienced, senior IT manager to
oversee the transition of knowledge from the integration partner to the team and
to manage the establishment of the support team. This was a strong, proactive
attempt to overcome an obstacle that could have negatively impacted the rest of
the project.
Another choice that helped minimize effects of this situation is that E&P
hired another consulting firm with experience in SAP to help transition the IPS
off the team and ensure that their knowledge was not lost to E&P. Because the
support team members had worked throughout the SAP project, they already
understood the processes quite well but were missing technical information such
as how to configure particular processes or where to look for certain technical
or operational information. The integration partner transferred its knowledge to
the third party consultant, and then the third party consultant transferred that
knowledge to the E&P support team. In their knowledge transfer model, “it was
transferring SAP knowledge from one SAP experienced group to another SAP
experienced group, then that group transitioned the knowledge to us in a way we
could understand.” While some knowledge was surely lost because of the
varying perceptions, experiences, and communication barriers involved in getting
second-hand or third-hand knowledge, this may have been the best way E&P
could gain integration partner knowledge, given the situation in which they were
working. Thus, E&P took strong steps to minimize knowledge loss when it
recognized a potential problem with knowledge sharing.

Changes in Core Knowledge Competency

The extent to which E&P has integrated the results of its knowledge sharing
to alter core knowledge and processes is somewhat lacking in consistency. One
person indicated that for a long time “people didn’t really try to exploit SAP; they
just tried to get their jobs done.” However, several months after implementation,
that began to change. The support team is “getting more requests from people
looking at how to use SAP to change the business.” Part of that is because budget
cuts and layoffs that occurred about the time SAP was implemented (not SAP
related) created a strain on employees’ time and motivation to learn something
new. The pressure on the budget has eased, yet the emphasis on cost cutting
remains. Thus, end users have renewed their efforts to find opportunities to run
the business more cost effectively. They are asking the SAP support team
questions about how to identify and make use of these opportunities in SAP. They

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

222 Jones and Price

also have begun to understand that what they do in their process now affects
someone else in another process and are looking for ways to take advantage of
that. The SAP support team continues to encourage people to exploit SAP
opportunities. E&P has continued sharing knowledge and seeking ways to
exploit old certainties and explore new possibilities long after implementation.
Thus, changes to the core knowledge competency are ongoing. Users are now
trying to use SAP to change the way they perform processes and, thus, are
beginning to alter core knowledge competencies. One explanation for this may
be the efforts E&P made throughout the implementation to facilitate knowledge
sharing on the team and with the rest of the organization. These knowledgesharing efforts helped make the organization ready to facilitate change in
processes, and that readiness lasted through corporate budget cuts.

CHEMICALS
Facilitation of Knowledge Sharing on the Team

Chemicals’ project manager said that “one of the things I always tell my
folks is that SAP is a team sport. If you don’t play as a team you can’t win.” One
of the things in place to discourage individual hoarding of knowledge was that
each member of the team received the same bonus at the end of an implementation regardless of rank in the organization, and the bonus was based on the
quality of the work and how well the implementation deadlines were met. Thus,
there was an incentive for each member to work with others to accomplish a
common goal. “We had a foxhole mentality,” whereby team members were
united around a common cause.
The team was also organized by process rather than by function or SAP
module to facilitate knowledge sharing. Chemicals built overlap between modules and functions into the project, and often two or more groups worked together
on a particular piece. For example, logistics is in the SD (sales and distribution)
module, but Chemicals broke it out and had a subteam manage the logistics
process separately from the sales and distribution people. Much of that data was
also in the order-to-cash process performed by the customer service area. Thus,
the logistics group had to work closely with the order-to-cash group to make sure
that the logistics pieces fit. They also did cross-team training to help ensure that
people working on one piece understood how their piece impacted others.
Although this approach required more effort in many cases than a moduleoriented approach, they believed that “if you get too module oriented, you get too
focused on the modules you’re working on” and lose sight of the big picture,
which is the processes. Thus, the SAP team was organized to focus on the
transfer of knowledge across functions, processes, and units, and to eliminate silo
behavior within the team and between the team and the organization. Although
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

223

they did not use formal team building activities, “everyone on the team had to rely
on everyone else,” because no one person or group knew all the things it took to
do the project.
The SAP team also decided to bring the key end users across plants into
implementation planning meetings, where each team gave a basic overview of
how each process would work in SAP, which included SAP terminology and
basic concepts. They went through an exhaustive set of detailed questions about
how processes worked and how they did their jobs. They built these questions
over time, based on the integration partner’s experience and on what they
learned with each implementation. Thus, by the last few implementations, they
had developed a set of questions that allowed them to cover almost every
conceivable part of the business processes. “We’d talk about the pros and cons
of each decision these plant people made. And we’d try to make people
understand what it actually meant and document the decision. We’d distribute
minutes of the meeting and have people either agree or not with what we’d
decided on.” This allowed the business people in multiple plants to share
knowledge and make decisions about common processes across the plants. As
a result, there was more uniformity of processes across plants, and there was a
better understanding of how to handle exceptions or things that had traditionally
been “workarounds” in the legacy system. “We had some consultants who said
our method was non-standard and shouldn’t be used, but it worked well for us.”

Change Management/Training

Chemicals had a very strong change management process in place, and
although they had a formal team in place for this, much of the change
management was an overall SAP team responsibility rather than that of just one
subgroup. “We had never worked so hard on cultural readiness,” one person
said. “We worked really hard on communications through e-mail, memos, ‘lunch
and learns,’ and television monitors with an animated video presentation that ran
continually in the cafeterias in plants.” The on-site planning meetings were also
viewed as an important part of change management. “We had decision makers
from every functional group in the plants in each design and implementation,”
which went a long way toward the cultural readiness on which change management was focused.
Although there was some use of the power user concept for training, the
team members who implemented also trained the users on site during the
implementation using materials the change management group had developed.
Training involved the transactional-based skills and a “heavy focus on the
integration points” to help people understand “where they fit in the chain of
events and why their piece was important and how it had downstream processes.” They originally thought that the training would be more focused on the
how-to, transactional skills, yet when they realized that “if we were going to get

TEAM LinG

224 Jones and Price

the wins we hoped were there, it was predicated on everybody doing their job.”
Thus, the training role changed considerably. “We spent more time and money
around training than we originally planned, and we had planned to train much
heavier than we had in any previous system.”

Transition of IPS Knowledge

Chemicals built knowledge sharing into their contract with the integration
partner. Team members focused both on how the partner solved problems and
on where they looked for answers. This provided them not only with how-to
knowledge, but also with more experiential knowledge about how to solve
problems. Another way that Chemicals ensured knowledge sharing with its
integration partner is that team members took on more responsibility as the
implementation progressed, so they could learn what the integration partner
knew, and so they could develop shared SAP experiences with the partner.

Changes in Core Knowledge Competency

Chemicals has made substantial progress toward integrating what it learned
through the knowledge sharing in the SAP project into its core knowledge, and
its processes have begun to change. “A lot more people are aware of the
integration and dependencies among processes.” “Our business processes have
become much more well-defined and understood.” They are beginning to see
substantial financial savings from leveraging common processes across units.
For example, the purchasing process is now uniform throughout all the plants, and
Chemicals has negotiated better prices on parts by buying the same part for all
plants through fewer suppliers. To do this, the plants had to work together to
change their nomenclature for parts to create a common master file of parts
across plants, which was a major hurdle to cross because of the vast number of
parts involved. Chemicals hired a consulting firm that had experience with this
type of task to help them. The firm now has a uniform online catalog of parts and
vendors. Buyers are now called alliance owners who “negotiate contracts,
approve changes from vendors, and monitor the business flow with the vendors”
across Chemicals for a particular family of parts rather than buying all the parts
for a given plant. “We’re still not over the hump on all of these [standards]. The
process works, but there’s room for improvement.”
One explanation for this is the strong knowledge sharing facilitators that
Chemicals built throughout the implementation. It began from a process focus in
which functional boundaries were removed, and team members from a variety
of processes had to work together and share their knowledge. This process focus
also engaged end users from across the organization to ensure that their
knowledge about processes was incorporated into the implementation. In
addition, users who were not directly involved in the implementation were trained
not only on transactions, but also on the integrative nature of performing
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

225

processes in ERP. Based on this evidence, Chemicals had the strongest
knowledge sharing during implementation and seems to have been able to move
more quickly than the other two firms in altering core knowledge competencies
by changing the way they perform processes.

LESSONS LEARNED

The firms that have had success with knowledge sharing during the
implementation process are making great strides toward taking advantage of
ERP in order to change the way they perform key processes. Although it is too
early for these companies to have realized substantial benefits, they have
mechanisms in place that put them well on the road to doing so. They have formed
cross-functional, cross-unit networks of employees to alter core knowledge
competencies by standardizing nomenclature, leveraging common processes,
and eliminating silo behavior between units. These networks have arisen out of
the knowledge sharing that took place during the implementation project among
team members, other organizational members, and integration partners. Thus,
there are several valuable lessons from these findings (see Table 4).
One lesson learned is that when firms start to implement an ERP, they
should identify organizational facilitators of and obstacles to knowledge sharing
and proactively seek to overcome the obstacles. For example, team members at
E&P recognized a potential problem in the tension between two organizational
units and made a conscious decision to minimize it, thus successfully ensuring it
was not passed on to new team members. One of Chemicals’ goals was to
engage a large number of the appropriate end users in the implementation to
ensure that they captured the right knowledge. Chemicals and USWhole both

Table 4. Summary of lessons learned
Identify and eliminate obstacles to success
e.g., cultural barriers such as stigma associated with teamwork or tensions between units;
structural barriers that promote silo behavior or inhibit knowledge sharing between levels
Focus on integration from the beginning of the project
e.g., implement by process rather than by module;
focus training on integration points in addition to how to process transactions
Focus on finding the best solutions to problems
e.g., don’t sweep problems under the carpet and hope to fix them later;
resist pressures to meet deadlines simply to mark milestones
Build organizational knowledge sharing throughout the project
e.g., foster knowledge sharing among team members with formal and informal activities;
encourage knowledge sharing between team and other organizational members;
minimize knowledge lost when consultants or other team members leave through formal roll- off
procedures
Learn from the past
e.g., acknowledge prior project weaknesses and look for ways to do better;
recognize prior strengths and build on those

TEAM LinG

226 Jones and Price

worked to overcome traditional barriers to knowledge sharing such as rank,
seniority, titles, and physical workspace. Barriers or obstacles that a firm has had
in the past on large-scale projects will not disappear by themselves, and ignoring
them in an ERP project may magnify them. These firms took actions that were
different from, if not counter to, organizational norms and patterns in order to
ensure that the ERP implementation could successfully integrate processes and
eliminate silos.
Lesson two is that firms should focus on integration from the beginning of
the ERP project. Because ERP requires integration of processes in the end, the
transition from silos is easier if the entire implementation effort is built around this
integration. Implementing by module may feel more natural, because its how
organizational members are used to working. However, it only prolongs the
inevitable change to integrated processes necessary to realize significant ERP
benefits. For example, Chemicals sought to overcome the divisions among
functional areas and business units from the very beginning of its project.
Chemicals’ employees were educated about integration of processes from the
beginning of the project, because they were involved in integrated groups as they
worked with the SAP team to map out processes, and as they were trained to use
SAP. Furthermore, the firms that primarily focused on how-to training said that
they regret not having realized the importance of focusing on integration points
with users earlier.
Lesson three is that firms should learn from the past and not be afraid to
acknowledge prior project weaknesses or failures. For example, E&P recognized that they had not been good at change management in the past and took
steps to correct this weakness. USWhole recognized that its management of
teams in the past was not good and took deliberate steps to build a strong SAP
team.
Lesson four is that firms should focus on knowledge sharing both on the
team and with the rest of the organization. For example, USWhole may not have
recognized the differences in the ability of its streams to adapt to changes
brought about by SAP early enough, because it did not tailor its change
management activities to the different streams. As a result, different streams
adapted differently, and the team had to work harder with some than with others
to begin to effect change in processes. On the other hand, E&P and Chemicals
worked hard to facilitate knowledge sharing among all relevant stakeholders in
their implementations. Both organizations have begun to change core processes
and alter core knowledge competencies.
Thus, the findings from this study provide several lessons that firms may
apply in their own ERP implementations. Even firms that already have implementations in progress or that are struggling to make ERP work after the initial
implementation can apply these lessons to their own situations. ERP is a longterm solution, and, once implemented, it is difficult, if not impossible, to go back

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

227

to the way things were prior to ERP. Thus, it is never too late to look at other
firms’ success stories to find out what we can learn from them.

LIMITATIONS OF THE STUDY AND
DIRECTIONS FOR FUTURE RESEARCH

One limitation of the study is that only one industry and one package were
examined. Although this helps to minimize bias that could be introduced across
industries and packages, there is a trade-off between generalizability of findings
and minimizing bias. Minimizing bias helps eliminate many factors that might
confound the results and provides a clearer view of the phenomenon of interest.
Selection of appropriate case sites controls extraneous variation and helps define
the limits for generalizing findings (Eisenhardt, 1989). If consistent results are
found across similar case sites, then we can be more sure that the theory that led
to the case study originally will also help identify other cases to which results are
analytically generalizable (Eisenhardt, 1989; Yin, 1989). However, using one
industry ignores difficulties or challenges in implementation that may be unique
to a given industry. One avenue for future research is to examine these
constructs in this study across industries and using different ERP packages to
determine whether industry or package mediate the findings in this study.
Another limitation is the number of respondents interviewed. Although
many of the same phrases were heard from respondents, indicating that
theoretical saturation had been reached, one direction for future research is to
examine the phenomena of interest in this study using a larger sample size in
order to be more sure that the responses obtained in this study do represent the
broader views and perceptions of the project team. In addition, the unit of
analysis in this study is restricted to the implementation team. Although most
knowledge sharing during the implementation revolved around this team, future
research that explores the perceptions of other organizational members or of the
integration partner staff could be useful. One avenue for this future research is
to compare responses to determine whether knowledge sharing is perceived
differently among team members, other organizational members, and the integration partner staff.

CONTRIBUTIONS OF THE STUDY

This study contributes in several ways to what is known about knowledge
sharing in ERP implementations. First, it identifies, categorizes, and discusses
several factors that facilitate knowledge sharing during ERP implementation.
Second, it links knowledge sharing to attempts to change core knowledge
competencies. Third, it provides several lessons for practitioners that they can

TEAM LinG

228 Jones and Price

use in their own ERP implementations. Practitioners engaged in ERP implementation can use these findings both to determine what may work best for them and
to identify their own facilitators of knowledge sharing. Fourth, this study provides
directions for future research by identifying limitations of the current study and
suggesting ways that future research could examine those limitations to further
extend what we know about knowledge sharing in ERP implementation.

REFERENCES

Al-Mashari, M., & Zairi, M. (2000). The effective application of SAP R/3: A
proposed model of best practice. Logistics Information Management,
13(3), 156-166.
Andriola, T. (1999). Information technology—The driver of change. Hospital
Material Management, 21(2), 52-58.
Baskerville, R., Pawlowski, S., & McLean, E. (2000). Enterprise resource
planning and organizational knowledge: Patterns of convergence and
divergence. Proceedings of the 21st ICIS Conference, Brisbane, Australia.
Brown, C., & Vessey, I. (1999). ERP implementation approaches: Toward and
contingency framework. Proceedings of the 20th Annual International
Conference on Information Systems, Charlotte, NC.
Brown, J.S., & Duguid, P. (2000, May-June). Balancing act: How to capture
knowledge without killing it. Harvard Business Review, 73-80.
Caldwell, B., & Stein, T. (1998, November). Beyond ERP—New IT agenda—
A second wave of ERP activity promises to increase efficiency and
transform ways of doing business. InformationWeek, 34-35.
Clement, R.W. (1994). Culture, leadership, and power: The keys to organizational change. Business Horizons, 37(1),33-39.
Constant, D., Kiesler, S., & Sproull, L. (1994). What’s mine is ours, or is it? A
study of attitudes about information sharing. Information Systems Research, 5(4), 400-421.
Davenport, T.H. (1998, July-August). Putting the enterprise in the enterprise
system. Harvard Business Review, 121-131.
Eisenhardt, K.M. (1989). Building theories form case research. Academy of
Management Review, 14(4), 532-550.
Grant, R.M. (1996). Prospering in dynamically competitive environments: Organizational capability as knowledge integration. Organization Science,
7(4), 375-387.
Hammer, M. (1990). Reengineering work: Don’t automate, obliterate. Harvard
Business Review, 68(4), 104-112.
Harari, O. (1996). Why did reengineering die. Management Review, 85(6), 49-

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

229

52.
Hine, M.J., & Goul, M. (1998). The design, development, and validation of a
knowledge-based organizational learning support system. Journal of
Management Information Systems, 15(2), 119-152.
Jarvenpaa, S.L., & Staples, D.S. (2000). The use of collaborative electronic
media for information sharing: An exploratory study of determinants.
Journal of Strategic Information Systems, 9, 129-154.
Jones, M.C. (2001). The role of organizational knowledge sharing in ERP
implementation. Final Report to the National Science Foundation
Grant SES 0001998.
Kogut, B., & Zander, U. (1992). Knowledge of the firm, combinative capabilities, and the replication of technology. Organization Science, 3(3), 383397.
Leonard, D., & Sensiper, S. (1998). The role of tacit knowledge in group
innovation. California Management Review, 40(3), 112-132.
Nelson, K.M., Nadkarni, S., Narayanan, V.K., & Ghods, M. (2000). Understanding software operations support expertise: A revealed causal mapping
approach. MIS Quarterly, 24(3), 475-507.
Osterloh, M., & Frey, B.S. (2000). Motivation, knowledge transfer, and organizational forms. Organization Science, 11(5), 538-550.
Robey, D., Ross, J.W., & Boudreau, M.-C. (2002). Learning to implement
enterprises systems: An exploratory study of the dialectics of change.
Journal of Management Information Systems, 19(1), 17-46.
Ross, J. (1999, July/August). Surprising facts about implementing ERP. IT Pro,
65-68.
Scott, J.E., & Kaindl, L. (2000). Enhancing functionality in an enterprise
software package. Information and Management, 37, 111-122.
Soh, C., Kien, S.S., & Tay-Yap, J. (2000). Cultural fits and misfits: Is ERP a
universal solution? Communications of the ACM, 43(4), 47-51.
Stein, E.W., & Vandenbosch, B. (1996). Organizational learning during advanced systems development: Opportunities and obstacles. Journal of
Management Information Systems, 13(2), 115-136.
Welti, N. (1999). Successful SAP R/3 implementation: Practical management
of ERP projects. Reading, MA: Addison-Wesley.
Yin, R.K. (1989). Case study research: Design and methods. Newbury Park,
CA: Sage Publications.
Zheng, S., Yen, D.C., & Tarn, J.M. (2000, Fall). The new spectrum of the crossenterprise solution: The integration of supply chain management and
enterprise resources planning systems. Journal of Computer Information
Systems, 84-93.

TEAM LinG

230 Jones and Price

APPENDIX A
Semi-Structured Interview Guide
Team vs. Individual Efforts
1. Do you usually work on a project team, or do you primarily work alone on
projects?
2. Do you think you are more rewarded for individual activities or for work on
teams? How important is project teamwork to your company?
3. Are teams primarily made up of people from the same functional areas or
from across functions?
4. How would you describe the culture of the firm?
Process vs. Product (Deadline) Orientation
1. How much focus was there on meeting deadlines and finishing the project
under budget?
2. How well were deadlines met?
3. When deadlines weren’t met, what was the reason?
4. How did your team determine whether the goals were valid and being met?
5. How did your team learn about opportunities SAP could provide your firm?
6. Do you think this learning process occurred throughout the implementation?
Organizational Knowledge Sharing During the Project
1. How were the SAP project team members selected?
2. How were differences in perspectives melded together?
3. Was this easy or difficult?
4. Was there ever a time when differences couldn’t be resolved? (If so, how
was that handled?)
5. How did your team seek input from others in the company on areas where
you were uncertain?
6. How did your team seek to keep others in the company informed about
company goals and progress on SAP?
7. Do you think this was ever seen as simply another IT project?
8. How much did your group rely on outside consultant expertise?
9. How did you make sure that you had learned enough from them so that you
could carry on after they left?
10. Was there much transition off your SAP team? How was it managed?
11. How were new people coming on the team brought up to speed?
12. During SAP team meetings, were people encouraged to express their ideas,
even if they weren’t fully formed yet? And did they express these ideas?
Can you give some examples?

TEAM LinG

Organizational Knowledge Sharing in ERP Implementation

231

13. Was there ever anything in the implementation process you felt just wasn’t
right, but couldn’t exactly explain why? If so, did you express this? Why or
why not?
14. Was there anything you assumed to be true about SAP that you later
changed your mind about?
Incorporation of New Knowledge Into Core Knowledge Competencies
1. Do you believe that the organization is different now than before SAP
implementation? If not, why not? If so, how?
2. Have the processes changed, or are they being changed because of SAP?
3. How has SAP changed the way you think about your job or the company?
4. What are some things that you learned about the business processes at the
company that you didn’t know before the SAP implementation?

TEAM LinG

232 Jones and Price

Section III:
E-Commerce
Processes
and
Practices

TEAM LinG

Electronic Banking and Information Assurance Issues

233

Chapter XII

Electronic Banking
and Information
Assurance Issues:
Survey and Synthesis

Manish Gupta, State University of New York, USA
Raghav Rao, State University of New York, USA
Shambhu Upadhyaya, State University of New York, USA

ABSTRACT

Information assurance is a key component in e-banking services. This
article investigates the information assurance issues and tenets of ebanking security that would be needed for design, development, and
assessment of an adequate electronic security infrastructure. The technology
terminology and frameworks presented in the article are with the view to
equip the reader with a glimpse of the state-of-art technologies that may help
toward learning and better decisions regarding electronic security.

INTRODUCTION

The Internet has emerged as the dominant medium in enabling banking
transactions. Adoption of e-banking has witnessed an unprecedented increase
over the last few years. Twenty percent of Internet users now access online
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

234 Gupta, Rao and Upadhyaya

banking services, a total that will reach 33% by 2006, according to the Online
Banking Report. By 2010, more than 55 million U.S. households will use online
banking and e-payments services, which are tipped as “growth areas.” The
popularity of online banking is projected to grow from 22 million households in
2002 to 34 million in 2005, according to Financial Insite, publisher of the Online
Banking Report1 newsletter.
Electronic banking uses computer and electronic technology as a substitute
for checks and other paper transactions. E-banking is initiated through devices
such as cards or codes to gain access to an account. Many financial institutions
use an automated teller machine (ATM) card and a personal identification
number (PIN) for this purpose. Others use home banking, which involves
installing a thick client on a home PC and using a secure dial-up network to access
account information; others allow banking via the Internet. This article will
discuss the information assurance issues (Maconachy, Schou, & Ragsdale, 2002)
that are associated with e-banking infrastructure. We hope that this chapter will
allow information technology (IT) managers to understand information assurance issues in e-banking in a holistic manner, and that it will help them make
recommendations and take actions to ensure security of e-banking components.

INTERNET/WEB BANKING

A customer links to the Internet from his or her PC. The Internet connection
is made through a public Web server. When the customer brings up the desired
bank’s Web page, the customer goes through the front-end interface to the
bank’s Web server, which, in turn, interfaces with the legacy systems to pull data
out at the customer’s request. Pulling legacy data is the most difficult part of Web
banking. While connection to a direct dial access (DDA) system is fairly
straightforward, doing wire transfer transactions or loan applications requires
much more sophisticated functionality. A separate e-mail server may be used for
customer service requests and other e-mail correspondence. There are also other
middleware products that provide security to ensure that the customer’s account
information is secured, as well as products that convert information into an
HTML format. In addition, many of the Internet banking vendors provide
consulting services to assist banks with Web site design and overall architecture.
Some systems store financial information and records on client PCs but use the
Internet connections to transmit information from the bank to the customer’s PC.
For example, the Internet version of Intuit’s BankNOW runs off-line at the client
and connects to the bank via the Internet only to transmit account and transaction
information (Walsh, 1999).
In this section, we discuss some of the key nodal points in Internet banking.
The following points are the foundations and principal aspects of e-banking: Web
site and service hosting, possibly through providers; application software that

TEAM LinG

Electronic Banking and Information Assurance Issues

235

includes middleware; regulations surrounding e-banking and standards that allow
different organizations and platforms to communicate over the Internet.

Web Site and Banking Service Hosting

Banks have the option of hosting Web sites in-house or outsourcing either
to service bureaus or core processing vendors with expertise in Internet banking.
Whether outsourced or packaged, Internet banking architectures generally
consist of the following components: Web servers; transaction servers; application servers; and data storage and access servers. Vendors such as Online
Resources2 offer a package of Web banking services that includes the design
and hosting of a financial institution’s Web site and the implementation of a
transactional Web site. Online’s connection makes use of the bank’s underlying
ATM network for transactions and real-time bill payment. In addition, optional
modules are generally available for bill payment, bill presentment, brokerage,
loan application/approval, small business, and credit cards. The fact that multiple
options of Web hosting exist also brings with them issues in security and
privacy—a topic that will be considered in a later section.
The components that form a typical Internet banking initiative are shown in
Figure 1.
•

•

Internet Banking Front-End: The front-end is often the client-side
browser access to the bank’s Web server. Client-side, thin-client access to
the bank’s Web server: This model allows the customer to download a thinclient software product from the bank’s Web site and may allow storing
financial data locally. Client-side, thick-client access to the bank’s Web
server: This is the model used when supporting personal financial management packages as tools to access account data and execute transactions. It
is important to note that these models are not mutually exclusive of each
other (Starita, 1999).
Internet Banking Transaction Platforms: The Internet banking transaction platform is the technology component that supports transactional
processes and interfaces between the front-end user interface and the
back-end core processors for functions like account information retrieval,
account update, and so forth. In general, the transactional platform defines
two main things: (1) the functional capabilities of the Internet banking

Figure 1. Architectural pieces of Internet banking (Starita, 1999)
In te rn et
F ro n t E n d
B ro w ser
b ased
Th in cl ien t
Th ick clien t

W eb site
B ac k E n d

M id d le wa re an d B ac k E n d
Ap p l ic atio n se rver s
D atab a se ser ve rs
Le gac y ap plicatio n s
B u s in e ss lo gic serve rs
S ecu r it y p a ck a ge s

TEAM LinG

236 Gupta, Rao and Upadhyaya

offering (i.e., whether it offers bill payment or credit card access); and (2)
the method of access or interface between the front-end and back-end
legacy processors (Starita, 1999).

Internet Banking Platforms and Applications

Most of the Internet plumbing to present data onto Web interfaces from data
sources is offered by Internet banking application software vendors, who link
legacy systems to allow access to account data and transaction execution. Most
players position themselves as end-to-end solution providers by including a
proprietary front-end software product, integration with other front-end software, or Web design services.
Some of the solutions are middleware platforms with plug-in applications to
provide bill payment, bill presentment, brokerage, loan, small business, and/or
credit card functionality. Most vendors use Open Financial Exchange standard
(OFX) to connect to different delivery channels such as Interactive Voice
Response (IVRs), Personal Finance Managers (PFMs), and the Internet.
Middleware tools are designed to handle Internet-delivered core banking and bill
payment transactions (Walsh, 2002). Middleware platforms provide a link
between financial institutions’ legacy host systems and customers using browserbased HTML interfaces and OFX-enabled personal financial management
software (Walsh, 2002).
Middleware is designed for financial institutions that require a platform that
translates messages between collections of separate processing systems that
house core processing functions. Core processing systems include bill payment,
credit card, brokerage, loans, and insurance. Electronic bill payment and
presentment is widely believed to be the compelling application that brings large
volumes of customers to the Internet channel to handle their finances. There are
two kinds of Web sites: nontransactional and transactional. The nontransactional
sites, commonly known as promotional Web sites, publish content with information about bank products and allow customers to investigate targeted areas such
as college loans or retirement planning. These sites give basic information on
bank products and do not allow any transactions. Banks can collect information
to start to develop customer profiles by recording where a customer visits on the
Web site and comparing it with demographic information to develop personalized
marketing strategies.
Transactional sites link to back-end processing systems and include basic
functionality such as the ability to view recent transactions and account histories,
download information into PFM software, and transfer funds between existing
accounts. As banks become more sophisticated with transactional capabilities,
such things as electronic bill payment or moving of funds outside of the bank
become possible. Integrating with a third-party processor such as Checkfree or
Travelers Express most often does this. Bill presentment is also part of

TEAM LinG

Electronic Banking and Information Assurance Issues

237

transactional capability; however, it is being done on a limited basis through a
small number of pilots. Some banks allow customers to apply for loans,
mortgages, and other products online, although much of the back-end process is
still done manually. In transactional Web sites, every page must be composed
dynamically and must offer continual updates on products and pricing.

Standards Compliance

Standards play a vital role in seamless flow and integration of information
across channels and help to reduce risk emanating from diverse platforms and
standards. In addition to the challenge of integrating Internet banking products
into the bank’s own IT environment, many Internet banking functions involve
third-party participation. This poses a significant integration question: What is the
best way to combine separate technology systems with third parties in a costeffective way in order to enable each participant to maintain control over its data
and maintain autonomy from other participants? The response from the technology marketplace has been to establish Internet banking standards to define
interactions and the transfer of information between multiple parties (Bohle,
2001). The premise of a standard is that everyone would use it in the same
consistent fashion; unfortunately, that is not the scenario in the current Internet
banking environment. One of the problems for the lackluster performance of ebanking arguably is the industry’s failure to attend to the payments infrastructure
(Orr, 2002). One initiative that does show promise is by the National Institute of
Standards and Technology, which has developed a proposed standard—Security
Requirements for Cryptographic Modules—that will require role-based authentication and authorization (FIPS, 1992). Some of the standards pervasive in
current e-banking models are the ADMS standard, the GOLD standard, and the
OFX standard.

INFORMATION ASSURANCE

Web banking sites include financial calculators; e-mail addresses/customer
service information; new account applications; transactions such as account
balance checks, transfers, and bill payment; bill presentment/payment; cash
management; loan applications; small business; credit card; and so forth. The
modes by which they can be accessed include online service provider or portal
site, direct-dial PC banking program, Internet-bank Web sites, WebTV, and
personal financial manager. Depending on the functionality of the Web sites,
different information assurance requirements are found.
Some examples of exploitation of information assurance issues in the
Web-banking arena include the following:

TEAM LinG

238 Gupta, Rao and Upadhyaya

•
•

Many ATMs of Bank of America were made unavailable in January 2003
by the SQL Slammer worm, which also affected other financial services
like Washington Mutual3,4.
Barclays suffered an embarrassing incident when it was discovered that
after logging out of its online service, an account immediately could be reaccessed using the back button on a Web browser. If a customer accessed
their Barclays account on a public terminal, the next user could thereby view
banking details of the previous customer. According to the bank, when
customers join the online banking service, they are given a booklet that tells
them to clear the cache to prevent this from happening. However, this
procedure shifts the responsibility for security to the end user5.

Security and Privacy Issues
In their annual joint study in April 2002, the FBI and the Computer Security
Institute noted that the combined financial losses for 223 of 503 companies that
responded to their survey (Computer Crime and Security Survey) was $455
million for year 2002 (Junnarkar, 2002). Security and integrity of online transactions are the most important technical issues that a bank offering Web services
will need to tackle. The Internet bank Web sites handle security in different
ways. They can choose either public or private networks. The Integrion
consortium, for example, uses the private IBM/AT&T Global Network for all
Internet network traffic (Walsh, 1999). Server security is another important
issue, usually accomplished by server certificates and SSL authentication. Banks
must look at three kinds of security (Walsh, 1999): communications security;
systems security, from the applications/authorization server; and information
security.
Table 1. Standards in e-banking models
•

The ADMS Standard: Access Device Messaging System (ADMS) is a proprietary standard developed and
supported by Visa Interactive. From September 1998, this standard has been made obsolete for GOLD
standard.

•

The GOLD Standard: The GOLD standard is an electronic banking standard developed and supported by
Integrion to facilitate the exchange of information between participants in electronic banking transactions. Integrion is a PC direct-dial and Internet banking vendor developed as a consortium with 16
member banks, IBM and Visa Interactive (through acquisition) in an equal equity partnership. IBM is the
technology provider for the Integrion consortium.

•

The OFX Standard: Open Financial Exchange (OFX) is a standard developed cooperatively by Microsoft,
Intuit and Checkfree. Recently, Microsoft launched its OFX version 2.0 without the involvement of its
partners, Checkfree and Intuit. OFX v.2.0 is developed with XML to enable OFX to be used for bill
presentment. Though OFX can be considered as a much better solution for inter-operability needs of
banks, it imposes problems of incompatibility between older OFX versions.

•

The IFX Standard: Interactive Financial Exchange (IFX) initiative was launched in early 1998 by BITS
(the Banking Industry Technology Secretariat) in order to ensure convergence between OFX and another
proposed specification, GOLD, propounded by Integrion Financial Network. According to the IFX
forum, IFX specification provides a robust and scalable framework for the exchange of financial data and
instructions independent of a particular network technology or computing platform.

•

XML as standard: XML language is often perceived as a solution to the problem of standards incompatibility. XML appears as an ideal tool for multi-banking, multi-service Internet banking applications.

TEAM LinG

Electronic Banking and Information Assurance Issues

239

From a user’s perspective, security must accomplish privacy, integrity
authentication, access control, and non-repudiation. Security becomes an even
more important issue when dealing with international banks, since only up to 128K
encryption is licensed for export. Currently, most Internet bank Web sites use a
combination of encryption, firewalls, and communications lines to ensure security. The basic level of security starts with an SSL-compliant browser. The SSL
protocol provides data security between a Web browser and the Web server, and
is based on public key cryptography licensed from security systems. Security has
been one of the biggest roadblocks that have kept consumers from fully
embracing Internet banking. Even after the advent of highly secure sites with the
aid of 128K encryption, a virtually invulnerable encryption technology, the
perception among some consumers is that Internet banking is unsafe. They
apprehend privacy violations, as the bank keeps track of all transactions, and they
are unsure of who has access to privileged data about their personal net worth.
The basic security concerns that face financial institutions offering banking
services and products through the Internet are summarized in Figure 2 and are
discussed next.
Authentication
Authentication relates to assurance of identity of person or originator of
data. Reliable customer authentication is imperative for financial institutions
engaging in any form of electronic banking or commerce. Strong customer

Figure 2. E-banking security infrastructure

S ta n d a rd s
C om p lia n ce
(Inter-operability
and acceptance)

E -b a n k in g
S ecu rity
In fr as tru ctu r e

R egu la to ry
C om p lia n ce
(Legal enforcement
and credibility)

In fo rm ation A ssu ra n ce Issu es an d C o n cern s

(Questions and problems)
Authentication
Access Control
Non-repudiation
Integrity
Confidentiality
Availability
Perimeter Defense
Intrusion Detection
Malicious Content
Incident Response
Administration
Social Engineering
Security Event Detection

S ecu rity S erv ice s, M e ch a n is m s
a n d P rotectio n

(Answers and solutions)

Encryption
Security Protocols
Kerberos
Firewalls, IDS
Passwords, PINs
Tokens
Biometrics
PKI, Certificates
HSDs
Crypto algorithms
Industry Standards
DR and CP Plans
Training and Awareness

TEAM LinG

240 Gupta, Rao and Upadhyaya

authentication practices are necessary to enforce anti-money laundering measures and help financial institutions detect and reduce identity theft. Customer
interaction with financial institutions is migrating from physical recognition and
paper-based documentation to remote electronic access and transaction initiation. The risks of doing business with unauthorized or masquerading individuals
in an electronic banking environment could be devastating, which can result in
financial loss and intangible losses like reputation damage, disclosure of confidential information, corruption of data, or unenforceable agreements.
There is a gamut of authentication tools and methodologies that financial
institutions use to authenticate customers. These include the use of passwords
and personal identification numbers (PINs), digital certificates using a public key
infrastructure (PKI), physical devices such as smart cards or other types of
tokens, database comparisons, and biometric identifiers. The level of risk
protection afforded by each of these tools varies and is evolving as technology
changes. Multi-factor authentication methods are more difficult to compromise
than single factor systems. Properly designed and implemented multifactor
authentication methods are more reliable indicators of authentication and stronger fraud deterrents. Broadly, the authentication methodologies can be classified,
based on what a user knows (passwords, PINs), what a user has (smart card,
magnetic card), and what a user is (fingerprint, retina, voiceprint, signature).
The issues that face banks using the Internet as a channel are the risks and
risk management controls of a number of existing and emerging authentication
tools necessary to initially verify the identity of new customers and authenticate
existing customers that access electronic banking services. Besides, effective
authentication framework and implementation provides banks with a foundation
to enforce electronic transactions and agreements.
•

•

Account Origination and Customer Verification: With the growth in
electronic banking and commerce, financial institutions need to deploy
reliable methods of originating new customer accounts online. Customer
identity verification during account origination is important in reducing the
risk of identity theft, fraudulent account applications, and unenforceable
account agreements or transactions. There are significant risks when
financial institutions accept new customers through the Internet or other
electronic channels because of the absence of the tangible cues that banks
traditionally use to identify individuals (FDIC, 2001).
Monitoring and Reporting: Monitoring systems play a vital role in
detecting unauthorized access to computer systems and customer accounts. A sound authentication system should include audit features that can
assist in the detection of fraud, unusual activities (e.g., money laundering),
compromised passwords, or other unauthorized activities (FDIC, 2001. In
addition, financial institutions are required to report suspicious activities to

TEAM LinG

Electronic Banking and Information Assurance Issues

241

appropriate regulatory and law enforcement agencies as required by 31
CFR 103.18.
Access Control
Access control refers to the regulating of access to critical business assets.
Access control provides a policy-based control of who can access specific
systems, what they can do within them, and when and from where they are
allowed access. One of the primary modes of access control is based on roles.
A role can be thought of as a set of transactions that a user or set of users can
perform within the context of an organization. For example, the roles in a bank
include teller, loan officer, and accountant, each of whom can perform different
functions. Role-based access control (RBAC) policy bases access control
decisions on the functions that a user is allowed to perform within an organization. In many applications, RBAC is concerned more with access to functions
and information than strictly with access to information.
The applicability of RBAC to commercial systems is apparent from its
widespread use. Nash and Poland (1990) discuss the application of role-based
access control to cryptographic authentication devices commonly used in the
banking industry. Even the Federal Information Processing Standard (FIPS) has
provisions for support for role-based access and administration.
Non-Repudiation
Non-repudiation refers to the need for each party involved in a transaction
to not go back on their word; that is, not break the electronic contract (Pfleeger,
1997). Authentication forms the basis for non-repudiation. It requires strong and
substantial evidence of the identity of the signer of a message and of message
integrity sufficient to prevent a party from successfully denying the origin,
submission, or delivery of the message and the integrity of its contents. This is
important for an e-banking environment where, in all electronic transactions,
including ATMs (cash machines), all parties to a transaction must be confident
that the transaction is secure; that the parties are who they say they are
(authentication), and that the transaction is verified as final. Essentially, banks
must have mechanisms that ensure that a party cannot subsequently repudiate
(reject) a transaction. There are several ways to ensure non-repudiation, which
include digital signatures, which not only validate the sender but also “time
stamps” the transaction, so it cannot be claimed subsequently that the transaction
was not authorized or not valid.
Integrity
Ensuring integrity means maintaining data consistency and protecting from
unauthorized data alteration (Pfleeger, 1997). Integrity is very critical for
Internet banking applications, as transactions have information that is consumer
and business sensitive. To achieve integrity, data integrity mechanisms can be
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

242 Gupta, Rao and Upadhyaya

used. These typically involve the use of secret-keyor public-key-based algorithms
that allow the recipient a piece of protected data to verify that the data have not been
modified in transit. The mechanisms are presented further in a later section.
Confidentiality and Privacy
Privacy and security concerns are not unique to banking systems. Privacy
and confidentiality are related but are distinct concepts. Protection of personally
identifiable information like banking records must be ensured for consumers.
Information Privacy (NIIAC, 1995) is the ability of an individual to control the use
and dissemination of information that relates to him or her. Confidentiality
(NIIAC, 1995) is a tool for protecting privacy. Sensitive information is accorded
a confidential status that mandates specific controls, including strict limitations
on access and disclosure. Those handling the information must adhere to these
controls. Information confidentiality refers to ensuring that customer information
is secured and hidden as it is transported through the Internet environment.
Information not only must be protected wherever it is stored (e.g., on computer
disks, backup tape, and printed form), but also in transit through the Internet.
Availability
Availability in this context means that legitimate users have access when
they need it. With Internet banking, one of the strongest selling propositions is 24/
7 availability; therefore, it becomes even more critical for e-banks. Availability
applies both to data and to services. Expectations of availability include presence
of a service in usable form, capacity to meet service needs, timeliness of service,
fair allocation, fault tolerance, controlled concurrency, and deadlock management. One example where availability is compromised is the denial of service
attack. On the Internet, a denial of service (DoS) attack is an incident in which
a user or organization is deprived of the services of a resource they would
normally expect to have. When there are enormous transactions on the Internet
bank’s Web site, the losses that may arise owing to unavailability are severe in
terms of financial losses and reputation losses. Typically, the loss of service is the
inability of a particular network service, such as email, to be available or the
temporary loss of all network connectivity and services. It becomes imperative
and crucial for IT managers in the Internet banking world to better understand the
kind of denial of attacks possible. Some of the common and well-known types of
denial of service attacks (IESAC, 2003) are the following:
•

SYN Attack: It floods the server with open SYN connections, without
completing the TCP handshake. TCP handshake is a three-step process to
negotiate connection between two computers. The first step is for initiating
the computer to send SYN (for synchronize) packet.

TEAM LinG

Electronic Banking and Information Assurance Issues

•

243

Teardrop Attack: It exploits the way that the Internet Protocol (IP)
requires a packet that is too large for the next router to handle be divided into
fragments. Here, the attacker’s IP puts a confusing offset value in the
second or later fragment of the packet. It can cause the system to crash.
Smurf Attack: In this attack, the perpetrator spoofs the source IP address
and broadcasts ping requests to a multitude of machines to overwhelm the
victim.

Perimeter Defense
Perimeter defense refers to the separation of an organization’s computer
systems from the outside world (IETF, 2000). This must allow free sharing of
certain information with clients, partners, suppliers, and so on, while also
protecting critical data from them. A security bulwark around network and
information assets of any bank can be achieved to a certain extent by implementing firewalls and correctly performing tuning and configuration of firewalls.
Today, with the kind of traffic generated toward Web-banking sites for all
kinds of purposes, from balance enquiries to interbank fund transfers, implementation of screening routers to ensure incoming and outgoing traffic would add
another layer of security. In this age of systems being hijacked for cyber-attacks,
it is also important that screen routers detect and prevent outgoing traffic that
attempts to gain entry to systems like spoofing IP addresses. Further, the
periphery of the corporate computer infrastructure can be bolstered by implementing VPN solutions to ensure privacy of data flowing through the firewall into
the public domain.
Probes and scans often are used techniques that are exploited to learn about
exposures and vulnerabilities on the network systems. A probe is characterized
by unusual attempts to gain access to a system or to discover information about
the system. Probes are sometimes followed by a more serious security event, but
often they are the result of curiosity or confusion. A scan is simply a large number
of probes done using an automated tool. Scans can sometimes be the result of a
misconfiguration or other error, but they are often a prelude to a more directed
attack on systems that the intruder has found to be vulnerable.
Intrusion Detection
Intrusion detection refers to the ability to identify an attempt to access
systems and networks in a fashion that breaches security policies. The Internet
banking scenario, where most of business these days is carried out over public
domain Internet and where a banking Web site becomes a single point interface
for information as well as transactions, gives hackers enough motivation to
intrude into Internet banks’ systems. To safeguard from such unwanted activities, organizations need to be able to recognize and distinguish, at a minimum, the
following (Gartner, 1999): internal and external intrusion attempts; human versus

TEAM LinG

244 Gupta, Rao and Upadhyaya

automated attacks; unauthorized hosts connecting to the network from inside and
outside the perimeter; unauthorized software being loaded on systems; and all
access points into the corporate network.
Intrusion detection systems (IDS) allow organizations to protect their
systems from the threats that come with increasing network connectivity and
reliance on information systems. Given the level and nature of modern network
security threats, the question for security professionals should not be whether to
use intrusion detection, but which intrusion detection features and capabilities to
use. IDSs have gained acceptance as a necessary addition to every organization’s
security infrastructure. IDS products can provide worthwhile indications of
malicious activity and spotlight security vulnerabilities, thus providing an additional layer of protection. Without them, network administrators have little
chance of knowing about, much less assessing and responding to, malicious and
invalid activity. Properly configured, IDSs are especially useful for monitoring
the network perimeter for attacks originating from outside and for monitoring
host systems for unacceptable insider activity.
Security Event Detection
Security event detection refers to the use of logs and other audit mechanisms
to capture information about system and application access, types of access,
network events, intrusion attempts, viruses, and so forth. Logging is an important
link in the analysis of attack and real-time alerts of any kind of suspicious activity
on the Internet bank Web site. For proper tracking of unusual events and attempts
of intrusion, the following logs should be collected: basic security logs, network
event logging, log authentication failures, log access violations, log attempts to
implant viruses and other malicious code, and log abnormal activity. This strongly
implies that the technical department that is analyzing logs to identify unusual
behavior must be aware of business initiatives. In addition, it has to be ensured
that audit logs are retained long enough to satisfy legal requirements. Also, at a
minimum, investigation of security breaches should be allowed for up to 14 days
after any given attack (IETF, 2000). Today, data mining techniques can interpret
millions of items of log data and reveal any unobserved attempts to breach an
ebank’s Web site. For this, it has to be ensured that logs do not overwrite
themselves causing loss of data. For analysis of events at a site, documentation
of automated systems that identify what the logs mean should be maintained.
Understanding the nature of attempts such as whether an attack was from within
the organization or from outside or whether it was just a false alarm is critical to
security.
Malicious Content
Malicious content refers to programs of any type that are introduced into a
system to cause damage or steal information. Malicious content includes viruses,
Trojan horses, hacker tools, and network sniffers. While common in multiple
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Electronic Banking and Information Assurance Issues

245

domains, this is as important in the e-banking world, as well. Malicious code
brings with it the potential to create serious technical and economic impact by
crashing e-mail servers and networks, causing millions of dollars of damage in lost
productivity.
Some of the common forms of malicious contents are the following:
•

•
•

Virus: A virus is a computer program that runs on a system without being
asked to do so, created to infect other computer programs with copies of
itself. Pioneer virus researcher Fred Cohen has defined a virus as “a
program that can ‘infect’ other programs by modifying them to include a,
possibly evolved, copy of it.”
Worm: A worm has the ability to spread over a network and, thus, can take
advantage of the Internet to do its work. Worms reside in memory and
duplicate themselves throughout the network without user intervention.
Trojan Horse: A Trojan horse is the name applied to a malicious computer
program disguised as a seemingly innocent activity such as initiating a
screen saver, accessing an e-mail attachment, or downloading executable
files from an untrusted Web site. Some of the widely manifested malicious
codes are Stoned, Yankee, Michelangelo, Joshi, Lehigh, Jerusalem, MBDF
(for Macintosh), Melissa, Concept, LoveBug (ILOVEYOU), ShapeShift,
Fusion, Accessiv, Emporer, Sircam, Nimda, and Badtrans.

Protection against malicious codes like viruses, worms, Trojan horses, and
so forth could be effectively dealt with by installing security protection software
that thwarts and mitigates the effects of codes. However, such software provides
only a level of defense and is not by itself sufficient. Recommendations for ebanking IT infrastructure include (Noakes, 2001):
•
•
•
•

Install detection and protection solutions for all forms of malicious code, not
just an antivirus solution.
Ensure that all users are aware of and follow safe behavior practices—do
not open attachments that have not been scanned, do not visit untrusted Web
sites, and so forth.
Ensure that users are aware of how easy data may be stolen automatically
just by visiting a Web site. Install an effective solution. Keep it current with
the latest signatures as new forms of malicious code are identified.
Use anti-spammers, harden operating systems, configure stricter firewall
rules, and so forth.

Security Services, Mechanisms, and Security Protection

Security risks are unlike privacy risks; they originate outside the Financial
Service Provider (FSP) and change rapidly with advances in technology (DeLotto,

TEAM LinG

246 Gupta, Rao and Upadhyaya

1999). In December 2000, IATF released guidelines that require all covered
institutions to secure their clients’ personal information against any reasonably
foreseeable internal or external threats to their security, confidentiality, and
integrity. By July 1, 2001, FSPs were expected to develop customer information
security programs that ensured the security and confidentiality of customer
information, protected against any anticipated threats or hazards to the security
or integrity of customer information, and protected against unauthorized access
to or use of customer information that could result in substantial harm or
inconvenience to customers.
The services and mechanisms that are prevalent in an e-banking environment are presented below in order to provide an understanding of key issues and
terms involved.
Encryption
Encryption is the process of using a key to scramble readable text into
unreadable cyphertext. Encryption on the Internet, in general, and e-banking, in
particular, have many uses, from the secure transmission of credit card numbers
via the Web to protecting the privacy of personal e-mail messages. Authentication also uses encryption by using a key or key pair to verify the integrity of a
document and its origin. The Data Encryption Standard (DES) has been endorsed
by the National Institute of Standards and Technology (NIST) since 1975 and is
the most readily available encryption standard. Rivest, Shamir, and Adleman
(RSA) encryption is a public-key encryption system; it is a patented technology
in the United States and, thus, is not available without a license. RSA encryption
is growing in popularity and is considered quite secure from brute force attacks.
Another encryption mechanism is Pretty Good Privacy (PGP), which allows
users to encrypt information stored on their system as well as to send and receive
encrypted e-mail. Encryption mechanisms rely on keys or passwords. The longer
the password, the more difficult the encryption is to break. VPNs employ
encryption to provide secure transmissions over public networks such as the
Internet.
Security Protocol Services
The Internet is viewed as an insecure place. Many of the protocols used in
the Internet do not provide any security. Today’s businesses, particularly the
banking sector, must integrate security protocols into their e-commerce infrastructure to protect customer information and privacy. Some of the most common
protocols are discussed briefly in Appendix A.
Firewalls and Intrusion Detection Systems
A firewall is a collection of hardware and software designed to examine a
stream of network traffic and service requests. Its purpose is to eliminate from

TEAM LinG

Electronic Banking and Information Assurance Issues

247

the stream those packets or requests that fail to meet the security criteria
established by the organization. A simple firewall may consist of a filtering router,
configured to discard packets that arrive from unauthorized addresses or that
represent attempts to connect to unauthorized service ports. Firewalls can filter
packets based on their source and destination addresses and port numbers. This
is known as address filtering. Firewalls also can filter specific types of network
traffic. This is also known as protocol filtering, because the decision to forward
or reject traffic is dependent upon the protocol used (e.g., HTTP, ftp, or telnet).
Firewalls also can filter traffic by packet attribute or state. But a firewall cannot
prevent individual users with modems from dialing into or out of the network,
bypassing the firewall altogether (Odyssey, 2001). In this age of systems being
hijacked, it is also important that firewalls and screen routers detect and prevent
outgoing traffic that attempts to compromise the integrity of the systems. A
Network Intrusion Detection System (NIDS) analyzes network traffic for
attacks. It examines individual packets within the data stream to identify threats
from authorized users, backdoor attacks, and hackers who have thwarted the
control systems to exploit network connections and access valuable data. NIDS
adds a new level of visibility into the nature and characteristics of the network.
They provide information about the use and usage of the network. Host Based
IDS/Event Log Viewers are a kind of IDS that monitors event logs from multiple
sources for suspicious activity. Host IDS is best placed to detect computer misuse
from trusted insiders and those who have infiltrated the network. The technology
and logical schemes used by these systems often are based on knowledge-based
misuse detection (Allan, 2002). Knowledge-based detection methods use
information about known security policy, known vulnerabilities, and known
attacks on the systems they monitor. This approach compares network activity or
system audit data to a database of known attack signatures or other misuse
indicators, and pattern matches produce alarms of various sorts. Behavior-based
detection (Allan, 2002) methods use information about repetitive and usual
behavior on the systems they monitor. Also called anomaly detection, this
approach notes events that diverge from expected (based on repetitive and usual)
usage patterns. One technique is threshold detection (Allan, 2002) in which
certain attributes of user and system behavior are expressed in terms of counts,
with some level established as permissible. Another technique is to perform
statistical analysis (Allan, 2002) on the information, build statistical models of the
environment, and look for patterns of anomalous activity.

Passwords and Personal Identification Numbers (PINs)

The most common authentication method for existing customers requesting
access to electronic banking systems is the entry of a user name and a secret
string of characters such as a password or PIN. User IDs combined with
passwords or PINs are considered a single-factor authentication technique.

TEAM LinG

248 Gupta, Rao and Upadhyaya

There are three aspects of passwords that contribute to the security they provide:
secrecy, length and composition, and system controls. In the present Internet
banking scenario, there are policies, for customers as well as employees, set by
banks for passwords to ensure effective authentication, like prohibiting using
public e-mail IDs as user IDs, ensuring that there are no user IDs with no
password, ensuring that policies exist and can be automatically enforced
concerning minimum password length, password format (i.e., which characters
make up a valid password), expiration and renewal of passwords, uniqueness of
passwords, not allowing the use of real words for passwords, and so forth.
Tokens
The use of a token represents authentication using something the customer
possesses. Typically, a token is part of a two-factor authentication process,
complemented by a password as the other factor. There are many benefits to the
use of tokens. The authentication process cannot be completed unless the device
is present. Static passwords or biometric identifiers used to activate the token
may be authenticated locally by the device itself. This process avoids the
transmission of shared secrets over an open network such as the Internet.
Digital Certificates and Public Key Infrastructure (PKI)
A financial institution may use a PKI system to authenticate customers to
their own electronic banking product. Institutions may also use the infrastructure
to provide authentication services to customers who wish to transact business
over the Internet with other entities or to identify employees and commercial
partners seeking access to the business’ internal systems. A properly implemented and maintained PKI may provide a strong means of customer identification over open networks such as the Internet. By combining a variety of hardware
components, system software, policies, practices, and standards, PKI can
provide for authentication, data integrity, and defenses against customer repudiation, and confidentiality (Odyssey, 2001). The certificate authority (CA), which
may be the financial institution or its service provider, plays a key role by attesting
with a digital certificate that a particular public key and the corresponding private
key belong to a specific individual or system. It is important when issuing a digital
certificate that the registration process for initially verifying the identity of
customers is adequately controlled. The CA attests to the individual’s identity by
signing the digital certificate with its own private key, known as the root key.
Each time the customer establishes a communication link with the financial
institution, a digital signature is transmitted with a digital certificate. These
electronic credentials enable the institution to determine that the digital certificate
is valid, identify the individual as a customer, and confirm that transactions
entered into the institution’s computer system were performed by that customer.
PKI, as the most reliable model for security and trust on the Internet, offers a

TEAM LinG

Electronic Banking and Information Assurance Issues

249

comprehensive e-security solution for Internet banking. Unlike the other security
models, PKI is a standards compliant, most credible trust framework, highly
scalable and modular. PKI comprehensively satisfies the security requirements
of e-banking (Odyssey, 2001).
A brief discussion on the processes and mechanisms used in PKI to address
common security concerns follows:
•

•

Authentication: The customer requests the Registration Authority (RA)
for a certificate. The Registration Authority validates the customer’s
credentials. After valid credentials are ensured, the RA passes the certificate request to the Certification Authority (CA). CA then issues the
certificates. A digital certificate can be stored on the browser on the user’s
computer, on a floppy disk, on a smart card, or on other hardware tokens.
Confidentiality: The customer generates a random session key at his or her
end. The session key is sent to the bank, encrypting it with the bank’s public
key. The bank decrypts the encrypted session key with its private key. The
session key is employed for further transactions.
Integrity: The message is passed through a suitable hashing algorithm to
obtain a message digest or hash. The hash, encrypted with the sender’s
private key, is appended to the message. The receiver, upon receiving the
message, passes it through the same hashing algorithm. The digest the
receiver obtains is compared with the received and decrypted digest. If the
digests are the same, it implies that the data have not been tampered with
in transit.
Non-Repudiation: The hash is encrypted with the sender’s private key to
yield the sender’s digital signature. Since the hash is encrypted with the
sender’s private key (which is accessible only to the sender), it provides an
indisputable means of non-repudiation.
The use of digital signatures and certificates in Internet banking has
provided the trust and security needed to carry out banking transactions
across open networks like the Internet. PKI, being a universally accepted
standards compliant security model, provides for the establishment of a
global trust chain. (Odyssey, 2001).

Biometrics
A biometric identifier measures an individual’s unique physical characteristic or behavior and compares it to a stored digital template to authenticate that
individual. A biometric identifier representing “something the user is” can be
created from sources such as a customer’s voice, fingerprints, hand or face
geometry, the iris or retina in an eye, or the way a customer signs a document or
enters keyboard strokes (FDIC, 2001). The success of a biometric identifier rests
on the ability of the digitally stored characteristic to relate typically to only one

TEAM LinG

250 Gupta, Rao and Upadhyaya

individual in a defined population. Although not yet in widespread use by financial
institutions for authenticating existing customers, biometric identifiers are being
used in some cases for physical access control.
Banks could use a biometric identifier for a single or multi-factor authentication process. ATMs that implement biometrics like iris-scan technologies are
examples of the use of a biometric identifier to authenticate users. The biometric
identifier may be used for authentication instead of the PIN. A customer can use
a PIN or password to supplement the biometric identifier, making it part of a more
secure two-factor authentication process. Financial institutions also may use
biometric identifiers for automating existing processes. Another application
would be a financial institution that allows customer to reset a password over the
telephone with voice-recognition software that authenticates the customer. An
authentication process that relies on a single biometric identifier may not work
for everyone in a financial institution’s customer base. Introducing a biometric
method of authentication requires physical contact with each customer to
capture initially the physical identifier, which further buttresses the initial
customer verification process. But this process may increase the deployment
costs.
Hardware Security Devices (HSDs)
This mechanism is an extension to usage of tokens for authentication. Using
hardware devices for authentication provides “hacker-resistant” and “snoopingproof” two-factor authentication, which results in easy-to-use, effective user
identification (Grand, 2001). To access protected resources, the user simply
combines his or her secret PIN (something the user knows) with the code
generated by the user’s token (something the user has). The result is a unique,
one-time-use code that is used to positively identify, or authenticate, the user
(Grand, 2001). Some central server validates the code. The goal is to provide
acceleration, secure key management.
A hardware security module is a hardware-based security device that
generates, stores, and protects cryptographic keys. There are universal criteria
for rating these devices. The criteria are documented in a Federal Information
Processing Standard (FIPS) called FIPS 140-1 to 140-4—Security for Cryptographic Modules. Such hardware devices generate tokens that are dynamic, onetime passwords through the use of a mathematical function. Passwords generated by tokens are different each time the user requests one, so an intercepted
password is useless, as it will never be used again. Acceptance and credibility of
the devices is reflected in the increasing number of devices in use.
Industry Standards and Frameworks
Industry standards for financial transactions over the Internet are an
absolute necessity for ensuring various security aspects of business as well as
consumer confidence. There have been a constant search and a development of
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Electronic Banking and Information Assurance Issues

251

Table 2. End-user involvement with the security issues

standards for e-banking infrastructural tenets like authentication, access control,
non-repudiation, and so forth. Some of the standards developed and advocated
by different industry players and their proponents are briefly discussed in
Appendix B, which will provide an overall understanding of the evolution and
prevalence of some of the standards.

User and E-Banking Focus on Security Issues

To summarize, Table 2 presents issues over which the user has direct control
or with which the user has involvement, and issues that are commonly left for the
systems to handle.

CONCLUSIONS

It should be noted that the discussion of e-banking information assurance
(IA) issues also has included several generic IA issues. To illustrate this, Table
3 briefly categorizes e-banking-specific information assurance issues and generic issues separately. Some issues may be more significant than in other areas.
We have made an attempt to comprehensively discuss all the areas in the article.
Security for financial transactions is of vital importance to financial institutions providing or planning to provide service delivery to customers over the
Internet, as well as to suppliers of products, services, and solutions for Internetbased e-commerce. The actual and perceived threats to Internet-based banking
define the need for a set of interrelated security services to provide protection to
all parties who can benefit from Web banking in a secure environment. Such
services may be specific to counter particular threats or may be pervasive
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

252 Gupta, Rao and Upadhyaya

Table 3. IA issues

throughout an Internet-based environment to provide the levels of protection
needed.
There are also requirements that the entire e-commerce environment be
constructed from components that recognize the need for security services and
provide means for overall security integration, administration, and management.
These services that offer the security from an infrastructure standpoint are found
throughout the e-commerce network and computing infrastructure. Financial
institutions should carry out, as a matter of corporate security policy, identification of likely targets, which should include all systems that are open to the public
network, such as routers, firewalls, Web servers, modem banks’ Web sites, and
internal unsecured systems such as desktops. They should regularly revise and
update their policies on auditing, risk assessment, standards, and key management. Vulnerability assessment and identification of likely targets and the
recognition of systems most vulnerable to attack are critical in the e-banking
arena. Accurate identification of vulnerable and attractive systems will contribute to prioritization when addressing problem areas.

ACKNOWLEDGMENTS

The authors would like to thank John Walp and Shamik Banerjee for their
contributions and help with this chapter, and the anonymous referees for their
comments that have improved this chapter. We would also like to thank the NSA
for the Center for Information Assurance recognition and the Department of
Defense for two student fellowships. The research of the second author was
supported in part by National Science Foundation (NSF) under grant 990735, and
the research of the third author was supported in part by the U.S. Air Force
Research Lab, Rome, New York, under Contract F30602-00-10505.

REFERENCES

Allan, A. (2002). Technology overview. Intrusion detection systems (IDSs):
Perspective. Gartner Research Report (DPRO-95367).
Basel Committee (2001). Risk management Principles for electronic banking. Basel Committee Publication No. 82.

TEAM LinG

Electronic Banking and Information Assurance Issues

253

Bohle, K. (2001). Integration of Internet payment systems—What’s the problem? ePSO (E-payments systems Observatory)—–Newsletter. Retrieved
March 1, 2003, from http://epso.jrc.es/newsletter/vol11/ 5.html
Burt, S. (2002). Online banking: Striving for compliance in cyberspace. Bankers
Systems Inc. Retrieved September 5, 2002, from http://www.bankers
systems.com/compliance/article13.html
DeLotto, R. (1999). Competitive intelligence for the e-financial service
provider. Gartner Group Research Report.
Dittrich, D. (1999). Incident response steps. Lecture series at University of
Washington.
FDIC (Federal Deposit Insurance Corporation) (2001). Authentication in electronic banking. Financial Institution Letters.
FIPS (Federal Information Processing Standard). (1992). Security requirements for cryptographic modules. Federal Information Processing Standard 140-1. National Institute of Standards and Technology.
GartnerGroup RAS Services. (1999). Intrusion detection systems. R-08-7031.
Glaessner, T., Kellermann, T., & McNevin, V. (2002). Electronic security: Risk
mitigation in financial transactions. Public policy issues. The World
Bank.
Grand, J. (2001). Authentication tokens: Balancing the security risks with
business requirements. Cambridge, MA: Stake, Inc.
IESAC (2003). Transactional security. Institution of Engineers, Saudi Arabian Center. Retrieved January 12, 2003, from http://www.iepsac.org/
papers/p04c04a.htm
Internet Security Task Force (2000). Initial recommendations for conducting
secure ebusiness. Retrieved January 12, 2003, from http://www.ca.com/
ISTF/recomme ndations.htm
Junnarkar, S. (2002). Online banks: Prime targets for attacks. e-Business
ZDTech News Update.
Maconachy, W.V., Schou, C.D., Ragsdale, D., & Welch, D. (2001, June 5-6). A
model for information assurance: An integrated approach. Proceedings of
the 2001 IEEE Workshop on Information Assurance and Security,
United States Military Academy, West Point, NY.
Marchany, R. (1998). Internet security & incident response: Scenarios &
tactics. Retrieved February 2, 2003, from https://courseware.vt.edu/
marchany/InternetSecurity/Class
NIIAC (The National Information Infrastructure Advisory Council) (1995).
Common ground: Fundamental principles for the national information
infrastructure.
Noakes, K. (2001). Virus and malicious code protection products: Perspective. Fry Technology Overview, Gartner Research Group, DPRO-90840.
OCC (Office of the Comptroller of the Currency) (1998). OCC bulletin 98-3
Technology risk management. PC Banking.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

254 Gupta, Rao and Upadhyaya

OCC (Office of the Comptroller of the Currency) (2001). AL 2001-4 OCC
advisory letter.
Odyssey Technologies (2001). PKI for Internet banking. Retrieved August 23,
2002, from http://www.odysseytec.com
Orr, B. (2002). Infrastructure, not innovation. ABA Banking Online Journal.
Retrieved August 8, 2002, from http://www.banking.com/aba/infra
structure.asp
Pfleeger, C.P. (1997). Security in computing. Upper Saddle River, NJ: Prentice
Hall.
Poland, K.R., & Nash, M.J. (1990). Some conundrums concerning separation of
duty. IEEE Symposium on Computer Security and Privacy.
Starita, L. (1999). Online banking: A strategic perspective. Context Overview Report (R-08-7031-Gartner).
United States Senate (2002). Financial services modernization act: Provisions of
GLB act. The United States Senate. Retrieved August 8, 2002, from http:/
/www.senate.gov/~banking/conf/grmleach.htm
Walsh, E. (1999). Technology overview: Internet banking: Perspective.
DPRO-90293, Gartner.
Walsh, E. (2002). Product report: S1 corporate suite e-banking software.
DPRO-95913 Gartner Research Group.

ENDNOTES

1
2
3

http://www.epaynews.com/statistics/bankstats.html
http://www.orcc.com
Robert Lemos, Staff Writer, CNET news.com, Counting the Cost of
Slammer, Retrieved March 31, 2003, from http://news.com.com/21001001-982955.html
Reuters, Seattle (Washington), CNN.com, Technology news, Feb 5,
2003. Retrieved March 8, 2003, from http://www.cnn.com/2003/TECH/
internet/02/05/virus.spread.reut/
Atomic Tangarine Inc, NPV: Information Security¸Retrieved March 21,
2003, from www.ttivanguard.com/risk/netpresentvalue.pdf
The latest version of the specifications, EMV 2000 version 4.0, published
December 2000, http://www. emvco.com/).
CEN/ISSS was created in mid-1997 by CEN (European Committee for
Standardization) and ISSS (Information Society Standardization) to provide
a comprehensive and integrated range of standardization-oriented services
and products.

TEAM LinG

Electronic Banking and Information Assurance Issues

255

APPENDIX A:
COMMON SECURITY PROTOCOL
SERVICES
Protocol
Secure Sockets Layer
(SSL)

Secure Shell (SSH)
AS1 and AS2

Digital Certificates

Pretty Good Privacy (PGP)

Secure Multipurpose
Internet Mail Extension
(S/MIME)

Secure HTTP
HTTP)

(S-

Simple Key management
for Internet Protocols
(SKIP)
Encapsulating Security
Payload (ESP)

Authentication Header
(AH)

Description
Originally developed by Netscape, the SSL security protocol provides data encryption, server authentication, message
integrity, and optional client authentication for a TCP/IP connection. SSL has been universally accepted on the World
Wide Web for authenticated and encrypted communication between clients and servers. However, SSL consumes large
amounts of the Web server's processing power due to the massive cryptographic computations that take place when a
secure session is initiated. If many secure sessions are initiated simultaneously, then the Web server quickly becomes
overburdened. The results are slow response times, dropped connections, and failed transactions.
SSH Secure Shell is the de facto standard for remote logins. It solves an important security problem on the Internet of
password hacking. Typical applications include secure use of networked applications, remote system administration,
automated file transfers, and access to corporate resources over the Internet.
AS1 provides S/MIME encryption and security over SMTP (Simple Mail Transfer Protocol) through object signature
and object encryption technology. AS2 goes a step further than AS1 by supporting S/MIME over HTTP and HTTPS.
Both AS1 and AS2 provide data authentication, proving that the sender and receiver are indeed the people or company
that they claim to be.
Digital certificates are used to authenticate the identity of trading partners, ensuring partners are really who they say
they are. In addition to data authentication, digital signatures support non-repudiation, proving that a specific message
did come from a known sender at a specific time. A digital signature is a digital code that can be sent with electronically
transmitted message and it uniquely identifies the sender. It is based on digital certificates. This prevents partners from
claiming that they didn’t send or receive a particular message or transaction.
PGP is a freely available encryption program that uses public key cryptography to ensure privacy over FTP, HTTP and
other protocols. PGP is the de-facto standard software for the encryption of e-mail and works on virtually every
platform. But PGP suffers from absence of Trust management and it is not standards compliant though it could provide
for integrity, authentication, non-repudiation and confidentiality.. PGP also provides tools and utilities for creating,
certifying, and managing keys.
S/MIME addresses security concerns such as privacy, integrity, authentication and non-repudiation, through the use of
signed receipts. S/MIME provides a consistent way to send and receive secure MIME data. Based on the MIME
standard, S/MIME provides authentication, message integrity, non-repudiation of origin (using digital signatures) and
data confidentiality (using encryption) for electronic messaging applications. Since its development by RSA in 1996,
S/MIME has been widely recognized and widely used standard for messaging. The technology for S/MIME is primarily
built on the Public Key Cryptographic Standard, which provides cryptographic interoperability. Two key features of
S/MIME are the digital signature and the digital envelope. Digital signatures ensure that a message has not been
tampered with during transit. Digital signatures also provide non-repudiation so senders can’t deny that they sent the
message.
S-HTTP is an extension to HTTP, which provides a number of security features, including Client/Server
Authentication, Spontaneous Encryption and Request/Response Non-repudiation. S-HTTP allows the secure exchange
of files on the World Wide Web. Each S-HTTP file is either encrypted, contains a digital certificate, or both. For a
given document, S-HTTP is an alternative to another well-known security protocol, Secure Sockets Layer (SSL). A
major difference is that S-HTTP allows the client to send a certificate to authenticate the user whereas, using SSL, only
the server can be authenticated. S-HTTP is more likely to be used in situations where the server represents a bank and
requires authentication from the user that is more secure than a userid and password.
It is a manifestation of IP-Level Cryptography that secures the network at the IP packet level. Any networked
application gains the benefits of encryption, without requiring modification. SKIP is unique in that an Internet host can
send an encrypted packet to another host without requiring a prior message exchange to set up a secure channel. SKIP is
particularly well suited to IP networks, as both are stateless protocols.
ESP is security protocol that provides data confidentiality and protection with optional authentication and replaydetection services. ESP completely encapsulates user data. ESP can be used either by itself or in conjunction with AH.
ESP may be implemented with AH, as discussed in next paragraph, in a nested fashion through the use of tunnel mode.
Security services can be provided between a pair of communicating hosts, between a pair of communicating security
gateways, or between a security gateway and a host, depending on the implementation. ESP may be used to provide the
same security services, and it also provides a confidentiality (encryption) service. Specifically, ESP does not protect any
IP header fields unless those fields are encapsulated by ESP (tunnel mode).
A security protocol that provides authentication and optional replay-detection services. AH is embedded in the data to
be protected (a full IP datagram, for example). AH can be used either by itself or with Encryption Service Payload
(ESP). The IP Authentication Header is used to provide connectionless integrity and data origin authentication for IP
datagrams, and to provide protection against replays. AH provides authentication for as much of the IP header as
possible, as well as for upper level protocol data. However, some IP header fields may change in transit and the value of
these fields, when the packet arrives at the receiver, may not be predictable by the sender. The values of such fields
cannot be protected by AH. Thus the protection provided to the IP header by AH is somewhat piecemeal and not
complete.

TEAM LinG

256 Gupta, Rao and Upadhyaya

APPENDIX B:
SOME INDUSTRY STANDARDS AND
FRAMEWORKS IN E-BANKING
Standard
SET

HBCI
EMV1
CEPS

XMLPay

ECML
W3C standard on
micropayments
Passport

eWallet project of
CEN/ISSS2
SEMPER

IOTP
SEPP

STT

JEPI

Description
Secure Electronic Transaction (SET) is a system for ensuring the security of financial transactions on the Internet. It was
supported initially by Mastercard, Visa, Microsoft, Netscape, and others. With SET, a user is given an electronic wallet
(digital certificate) and a transaction is conducted and verified using a combination of digital certificates and digital
signatures among the purchaser, a merchant, and the purchaser's bank in a way that ensures privacy and confidentiality. SET
makes use of Netscape's Secure Sockets Layer (SSL), Microsoft's Secure Transaction Technology (STT), and Terisa System's
Secure Hypertext Transfer Protocol (S-HTTP). SET uses some but not all aspects of a public key infrastructure (PKI). SET
provides authentication, integrity, non-repudiation and confidentiality.
HBCI is a specification for the communication between intelligent customer systems and the corresponding computing
centers for the exchange of home banking transactions. The transmission of data is done by a net data interface, which is
based on flexible delimiter syntax.
Specifications by Europay, MasterCard and Visa that define a set of requirements to ensure interoperability between chip
cards and terminals on a global basis, regardless of the manufacturer, the financial institution, or where the card is used.
The Common Electronic Purse Specifications (CEPS) define requirements for all components needed by an organization to
implement a globally interoperable electronic purse program, while maintaining full accountability and auditability. CEPS,
which were made available in March of 1999, outline overall system security, certification and migration. CEPS have paved
the way for the creation of an open, de facto, global electronic purse standard (http://www.cepsco.com/).
XMLPay is a standard proposed/developed by Ariba and Verisign. It defines an XML syntax for payment transaction
requests, responses and receipts in a payment processing network. The intended users are Internet merchants and merchant
aggregators who need to deal with multiple electronic payment mechanisms (credit/debit card, purchase card, electronic
cheque and automated clearing house payment). The supported operations include funds authorization and capture, sales and
repeat sales, and voiding of transactions.
The Electronic Commerce Modeling Language ECML is a specification that describes the format for data fields that need to
be filled at checkout in an online transaction. The fields defined include shipping information, billing information, recipient
information, payment card information and reference fields. Version 2.0 describes these fields in XML syntax.
The W3C standard on micropayments has originated from IBM’s standardization efforts. It covers the payment function for
payment of digital goods. The Micropayment initiative specifies how to provide in a Web page all the information necessary
to initialize a micropayment and transfer this information to the wallet for processing. The W3C Ecommerce/Micropayment
Activity is now closed.
Microsoft Passport is an online user-authentication service. Passport’s primary service is user authentication, referred to as
the Passport single sign-in (SSI) service. Passport also offers two other optional services: Passport express purchase (EP),
which lets users store credit card and billing/shipping address information in their optional Passport wallet profiles to
expedite checkout at participating e-commerce sites, and Kids Passport (source: Microsoft Passport Technical White Paper).
CEN/ISSS Electronic Commerce Workshop initiated the eWallet project in mid-2001 assuming a need for standardization in
the field. CEN/ISSS has chosen a flexible working definition considering an eWallet as "a collection of confidential data of a
personal nature or relating to a role carried our by an individual, managed so as to facilitate completion of electronic
transactions".
Secure Electronic Market Place for Europe (SEMPER) was produced by an EU supported project under a special program,
undertaken by a 20 partner consortium led by IBM. It is a definition of an open and system independent architecture for
Electronic Commerce. The project was concluded in 1999. Based on access via a browser, the architecture specifies common
functions to be supported by applications which include Exchange of certificates, Exchange of signed offer/order, Fair
contract signing, Fair payment for receipt, and Provision of delivery information.
The Internet Open Trading Protocol (IOTP) is defined as an interoperable framework for Internet commerce. It is optimized
for the case where the buyer and the merchant do not have a prior acquaintance. IOTP is payment system independent. It can
encapsulate and support several of leading payment systems.
Secure Electronic Payment Process is a protocol developed by MasterCard and Netscape to provide authentication, integrity
and payment confidentiality. It uses DES for confidentiality and 512, 768, 1024 or 2048 bit RSA and 128 bit MD5 hashing.
RSA encrypts DES key to encrypt hash of account numbers. It uses up to three public keys, one for signing, one for key
exchange, one for certificate renewal. Besides, SEPP uses X.509 certificates with CMS at top of hierarchy[ 26].
Secure Transaction Technology was developed by Visa and Microsoft to provide authentication, integrity and confidentiality
to the Internet based transactions. It is based on 64 bit DES or 64 bit RC4 (24-bit salt) for confidentiality and 512, 768, 1024
or 2048 bit RSA for encryption with 160 bit SHA hashing. It uses two public keys, one for signing, one for key exchange. It
has credentials similar to certificates but with account details and higher level signatures, though they are not certificates.
(Joint Electronic Payment Initiative) CommerceNet and the W3 Consortium are jointly initiating a multi-industry project to
develop an Internet payment negotiation protocol. The project explores the technology required to provide negotiation over
multiple payment instruments, protocols and transports. Examples of payment instruments include credit cards, debit cards,
electronic cash and checks. Payment protocols include STT and SEPP (amongst others). Payment transport encompasses the
message transmission mechanism: S-HTTP, SSL, SMTP, and TCP/IP are all categorized as transport technologies that can be
used for payment.

The latest version of the specifications, EMV 2000 version 4.0, was published in December 2000 (http://
www.emvco.com/).
CEN/ISSS was created in mid-1997 by CEN (European Committee for Standardization) and ISSS(Information
Society Standardization) to provide with a comprehensive and integrated range of standardization-oriented
services and products

TEAM LinG

Computer Security and Risky Computing Practices

257

Chapter XIII

Computer Security and
Risky Computing Practices:
A Rational Choice Perspective
Kregg Aytes, Idaho State University, USA
Terry Connolly, University of Arizona, USA

ABSTRACT

Despite rapid technological advances in computer hardware and software,
insecure behavior by individual computer users continues to be a significant
source of direct cost and productivity loss. Why do individuals, many of
whom are aware of the possible grave consequences of low-level insecure
behaviors such as failure to backup work and disclosing passwords,
continue to engage in unsafe computing practices? In this chapter we
propose a conceptual model of this behavior as the outcome of a boundedly
rational choice process. We explore this model in a survey of undergraduate
students (N = 167) at two large public universities. We asked about the
frequency with which they engaged in five commonplace but unsafe
computing practices, and probed their decision processes with regard to
these practices. Although our respondents saw themselves as knowledgeable,
competent users and were broadly aware that serious consequences were
quite likely to result, they reported frequent unsafe computing behaviors.
We discuss the implications of these findings both for further research on
risky computing practices and for training and enforcement policies that
will be needed in the organizations that these students will be entering
shortly.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

258 Aytes and Connolly

INTRODUCTION

Over the past few years, the public has become increasingly aware of
computer security issues, as incidents have been covered in the popular news
media. Computer viruses, denial of service attacks, and cases of intruders
hacking into corporate systems and stealing confidential information are becoming more commonplace. Information technology (IT) professionals seem to be
waging a constant battle to maintain control over corporate technology and
information assets.
The costs of security breaches are enormous and widespread. The most
recent survey of 503 corporate and government organizations conducted by the
Computer Security Institute and the FBI includes these sobering facts (Power,
2002):

•
•
•
•

40% report intrusion into information systems from outside the organization
85% were hit by worms or computer viruses
80% acknowledged financial losses due to computer security breaches
While only 40% quantified their losses, those that did reported a total of
almost $455 million in financial losses in 2001, mostly through the theft of
proprietary information and financial fraud.

More important than just the magnitude of these numbers is the fact that
they have gotten worse during the seven years in which the survey has been
conducted. Financial losses have climbed each year, and most categories of
attacks either have gotten worse or remain substantially unchanged from
previous years.
Although there are technological solutions to counteract the many security
threats, most security professionals realize that technology alone is insufficient
to adequately protect a firm’s assets. Because information systems involve
human users, and people do not always act the way they are supposed to, users
are now considered one of the major chinks in the armor of computer security
countermeasures (Rhodes, 2001; Tuesday, 2001). User-related risks include
such low-level insecure behaviors as sharing passwords, creating and using
weak passwords that easily can be guessed, and opening e-mail attachments
without checking for viruses. In addition to these risky behaviors, users pose a
serious threat to computer security because hackers have learned to manipulate
them into divulging confidential information (Adams & Sasse, 1999), a technique
referred to as “social engineering.”
To counter the risks that users pose, security professionals propose security
training and awareness programs for users (Gips, 2001; Peltier, 2000; Tuesday,
2001). The primary goals of such programs are to make users aware of the
various computer security risks and how they could affect the organization, and
to get users to understand the importance of engaging in safe computing behavior
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Computer Security and Risky Computing Practices

259

(Peltier, 2000). Fear of negative consequences is a common theme of these
programs. Many of the security standards that have developed over the last 20
or 30 years originated in the federal government, often in the Department of
Defense, where compliance can be mandated with more success than in private
industry. Some authors suggest that security can be increased also by implementing positive motivators for users (Parker, 1999; Tuesday, 2001).
Unfortunately, these training and remediation efforts are designed largely
in the absence of reliable knowledge about the behaviors they are seeking to
change. We know very little about why computer users choose to engage in
unsafe computing behaviors. Are they unaware that they are doing so? Do they
know about the safer behaviors they could choose, and do they have the training
to implement those behaviors effectively? Do they misjudge the likelihood that
their unsafe behaviors will lead to bad consequences or believe that the
consequences will not, in fact, be very serious? Are their behaviors simply a
matter of knowing better but doing worse, of succumbing to the temptations of
the moment instead of doing the prudent thing? Our hope is that a better
understanding of the individual’s decision process relating to safe or unsafe
computing behaviors will provide a better basis for strategies aimed at influencing the process.
Viewing the practice of safe computing behaviors as a rational decision
process is consistent with several well-researched theories related to the use of
information technology. Fishbein and Ajzen’s (1975) Theory of Reasoned
Action (TRA) and Davis, Borgozzi, and Warshaw’s (1989) Technology Acceptance Model (TAM) both view the use or non-use of an information system to
be based on, among other things, behavioral intentions. Those behavioral
intentions are the result of a choice the users make based on their attitudes and
their perceptions of the norms concerning the behavior. For example, the TAM
model posits that a person’s intention to use a system is determined by the
person’s attitude toward a system and the person’s beliefs about the probability
that the system will increase his or her job performance (Jackson et. al., 1997).
That is, a person makes a rational choice either to use or not to use an information
system based on several decision criteria. This intention to use an information
system is, then, a major determinant of a person’s actual behavior. Put in the
context of safe computing behaviors, we believe that a person’s intention to
employ safe computing behaviors (e.g., scan for viruses, change passwords,
etc.) is also a rational choice based on the person’s perceptions about the
usefulness of the safe behaviors and the consequences of not engaging in safe
behavior.
This study had two main goals. First, we wanted to document the prevalence
of unsafe computing practices in one population. For all the concern noted above
about unsafe computing practices, we were unable to find any systematic
evidence of which practices are prevalent, to what extent, and in which
populations. Our population here is undergraduate students at two large U.S.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

260 Aytes and Connolly

universities, a population chosen primarily for its convenience, but also as
representing a group of active, non-specialist computer users, rather than a
technological elite. These are young adults who have grown up with computing
as part of their everyday work and professional lives, who are likely to remain
active users in the foreseeable future, and who soon will be entering the work
force. Thus, their computing habits will be of concern to employers assessing
their potential vulnerability to unsafe practices and the extent of retraining or
other remediation that will be required.
Our second goal was to explore a tentative theoretical model of risky
computing, both as a guide for the present study and as a framework for future
work. When viewing the use of countermeasures as a choice made by the user,
we have available to us a substantial body of research investigating the
perceptions of risk and decision making under uncertainty. This research is
particularly relevant, as choices about safe computing practices are quite similar
to choices studied in this referent literature, such as general technological risk
(Fischhoff et al., 1978) and seatbelt use (Slovic et. al., 1978). Here, we present
a model based on concepts from this referent literature. It assumes that risky
computing behavior is a result of individual choices at least weakly guided by
considerations of the probability and desirability of choice consequences. We, of
course, are not postulating a highly rational, utility-maximizing user; a huge
empirical literature (Connolly, Arkes, & Hammond, 2002; Goldstein & Hogarth,
1997) attests to the implausibility of such an assumption. We do, however,
propose that conscious thought about consequences plays some role in guiding
risky behavior. (If the evidence fails to support even this weak assumption,
alternative theoretical frameworks such as habit formation, peer pressure, or
simple impulsivity would have to be considered.) The basic outline of our model
is presented in Figure 1.
The core of our model (see Figure 1) is the assumption that risky computing
behavior is a result of individual choices at least weakly guided by considerations
of the probability and desirability of choice consequences, consistent with the
assumptions of the Theory of Reasoned Action (Fishbein & Ajzer, 1975) and the
Technology Acceptance Model (Davis et. al., 1989). We assume, then, that the
user faces a choice between two courses of action. In the simplest case, Option
1—safe practice—leads to a certainty1 of no negative consequences, but at the
cost of some additional time or effort. Option 2—risky practice—involves no
additional costs but leads with some probability (p) to negative consequences
and, with probability (1-p), to no negative consequences. Some implications of
this highly simplified choice model are summarized in Figure 1.
The central implication is that several conditions must be met in order for
the user to choose Option 1—safe practice (i.e., use of security countermeasures)—rather than Option 2—risky practice. These conditions are similar to the
predictive model for information use founded on TRA and TAM and proposed

TEAM LinG

Computer Security and Risky Computing Practices

261

Figure 1. Rational choice model
Training
News/Media
Friends &
Coworkers
Personal
Experience
Policies &
Procedures

a. Awareness
of safe
practice
b. Awareness
of negative
consequences

c. Availability
of safe practice
option

d. Perception of
probability of
negative
consequences

Option 1:
Safe
Practice
Choice
Process
Option 2:
Risky
Practice

e. Perception of
severity of
negative
consequences

by Jackson, Chow, and Leitch (1997). In that model, the perceived usefulness
and perceived ease of use are considered predictors of information system use.
Specific factors affecting the decision include the following:
a.

b.
c.

Is the user aware of the safe practice options that may be used as
countermeasures against threats? For example, is the user aware of a
practice called “regular backup”? Is an appropriate backup technology
available, and does the user know how to use this technology with
reasonable time and effort? No choice exists unless this option is realistically available.
Is the user aware of the potential negative consequences of not using safe
practices? For example, is the user aware that by not backing up data
regularly, significant data loss will occur in the event of a disk crash?
Is the user aware that safe practice options (i.e., countermeasures) are
readily available for his or her use and that countermeasures easily can be
employed? For example, the capability of backing up data (i.e., CD burner,
tape drive, etc.) must be available when needed, and the user needs to know
how to use it properly.
Does the user believe that, with a non-zero probability, risky practices may
lead to negative consequences? An extensive literature (Slovic, Fischhoff,
& Lichtenstein, 1979; Zeckhauser & Viscusi, 1990) shows that humans
estimate and react to small probabilities in non-normative ways, either
exaggerating or ignoring the risks involved. A related literature (Fischhoff,
Bostrum, & Quadrel, 1993) suggests that we are insensitive to the
compounding of small probabilities with repeated exposure. It seems likely
that the risks involved in many unsafe computing practices will be small in
individual instances and compounded by repetition, so both types of
distortions may be found.

TEAM LinG

262 Aytes and Connolly

Does the user believe that the negative consequences that may follow
unsafe practice are both substantial in scope and personally significant? For
example, the user may believe that modern software restricts data loss only
to recent documents or that the loss of a data file, while significant for the
organization, is not a matter of personal importance. In some cases of virus
propagation, the costs may be borne mainly by others who receive infected
files rather than by the user, who may remain unaware of the problem.

Figure 1 summarizes these relationships. It also suggests some of the
antecedents of these decision-related elements. For example, formal training,
which makes users aware of safe practices and instructs them in using them
efficiently (and thus, at low personal cost), also offers an opportunity to raise the
user’s awareness of the probability and significance of the negative consequences of unsafe practices. A formally identical model could guide research on
computer security issues in general, where safe practice would be understood to
include using appropriate countermeasures to offset the efforts of an active
opponent. A rational choice framework applies whether the situation is conceived as a game against nature (e.g., the possible failure of a hard drive) or as
a game against an opponent (e.g., the possible theft of financial data), though the
analysis is obviously more complex in the latter case.
In summary, the study reported here has two goals: (1) to assess the
prevalence of risky computing practices in a university population and (2) to
explore the value of a rational choice model as a theoretical framework for
understanding these practices. The design of the study is described in the
following section. The remainder of the chapter presents the findings and their
implications.

STUDY DESIGN
Practices Considered

We chose three areas of potentially risky behaviors: password usage, e-mail
usage, and backing up data. (We originally included a fourth area, financial
transactions over the Web, but the issues involved in this area emerged as
significantly different from the first three, and we do not discuss them further
here). Obviously, this is not a complete list of risk issues, but we judged them to
be a reasonable sampling of everyday practices that carry some element of risk.
Within password usage, we asked how frequently users share their login
passwords with others and how frequently they voluntarily change their passwords. No matter how well a computer system is protected from unauthorized
intrusion using technical countermeasures, all is for naught if authorized users
share their passwords with others. Even if they only share passwords with other
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

Computer Security and Risky Computing Practices

263

authorized users, audit trails and accountability are compromised. Because
passwords may be undetectably cracked or revealed to outsiders or unauthorized
personnel, frequently changing one’s password helps prevent long-term usage of
an account by an intruder.
We investigated e-mail usage primarily because e-mail-borne viruses are
one of the major threats to the security of computer systems today (Consumer
Reports, 2002). We asked our respondents how often they opened e-mail
attachments, both from known and unknown sources, without first checking
them for viruses. We also asked about practices concerning backing up files to
protect from loss of data. Regularly backing up data to a second device (i.e.,
network drive, diskette, CD, etc.) offers some protection to users from computer
hardware problems and viruses. It cannot, of course, prevent the problems, but
it does reduce the risk of losing valuable information.

Questionnaire

The five practices were included in a questionnaire (see Appendix 1).
Within each practice, a standard sequence of questions asked the respondent to
indicate how frequently he or she engaged in the practice, how frequently others
do so, the probability and severity of bad consequences from engaging in unsafe
practices, and whether the respondent or someone he or she knew had
experienced these bad consequences. The questionnaire also included several
items concerning the respondent’s training in and knowledge about computer
security issues, and some brief demographic questions. Responses were anonymous, and respondents were asked not to put their names on the questionnaires.

Sample and Administration

Respondents were recruited from undergraduate business classes in two
large public universities and were given offers of extra course credit and
discharge of a lab participation obligation. The questionnaires were distributed
and completed in class; completion required approximately 30 minutes. Completed questionnaires were received from a total of 167 respondents, who were
surveyed during the fall semester of 2001. The average age of the respondents
was 23.5 years; 60% were male; 94% were juniors and seniors, and the
remainder was sophomores and graduate students.

RESULTS

Overall, the respondents considered themselves to be a highly knowledgeable group, with the vast majority of them describing themselves as “knowledgeable/comfortable,” “very knowledgeable/comfortable,” or “expert” at using email and protecting themselves from viruses and computer crashes. Half of the

TEAM LinG

264 Aytes and Connolly

Table 1. Self-rated knowledge of four areas of computer use
Activity
E-mail
Protecting against viruses
Protecting against computer crash
Protecting against interception of
financial information

% Rating Themselves
Knowledgeable, Very
Knowledgeable, or Expert
93%
69%
70%
50%

subjects were similarly confident that they knew how to protect themselves from
unauthorized interception of financial information (see Table 1).
This self-image of sophisticated, security-savvy users does not track very
well with their training and actual behaviors. Nearly half of them (79/167: 47%)
report having never received any information or tutoring on computer securityrelated matters. Of those that have received information, only 19% received it
through formal training or education. The most common sources of computer
security information were friends and co-workers (cited by 52%), and personal
experience (cited by 42%).
A surprising number of subjects reports that they do not follow basic
computer-security procedures (Table 2). Only 22% (36/167) report never
sharing their passwords with others, and a majority (85/166: 51%) reports that
they had rarely or never voluntarily changed their passwords after establishing
an online account. Almost a quarter (38/158: 24%) reports having opened e-mail
attachments from unknown sources without checking for viruses, and more than
half (89/159: 56%) reports having done so when the source appeared to be a
friend. Well under half (64/167: 38%) reports that they backup their work
frequently or all the time, and over a quarter (45/167: 27%) rarely or never
backup their work.
Table 2. Self-reported frequency of engaging in five security-related
practices
Activity

Never

Rarely Occasionally Frequently

Share passwords
22%
53%
20%
Voluntarily change
27%
24%
29%
password*
Open e-mail attachments
58%
18%
14.5%
without virus checking—
unknown source
Open e-mail attachments
25%
19%
18%
without virus checking—
known source
Backup regularly*
6%
21%
35%
* = Higher frequency on these items indicates more secure behavior

3%
9%

All the
Time
2%
11%

23%

25%

24%

14%

TEAM LinG

Computer Security and Risky Computing Practices

265

Given these percentages, 49% of the subjects engaged in risky computing
behavior at least occasionally, with 28% doing so frequently or all the time.
Additionally, in each of these cases, subjects report that their peers engage in
these insecure behaviors more frequently than they do themselves. (It is not clear
whether this reflects accurate observations of their friends or a way of disclosing
their own socially-disapproved bad behavior. In either reading, these peer
reports suggest that insecure behavior is even more widespread than the selfreport rates would indicate.)
The subjects seemed to be aware of the possibility that negative consequences may result from engaging in these risky behaviors. When asked if there
were any negative consequences to these various activities, an average of less
than 12% felt there were no negative consequences for all of these behaviors.
The activity most widely recognized as potentially having negative consequences
was opening e-mail attachments from unknown sources without running a virus
check; only 4% said that there were no negative consequences to doing so. The
activity least recognized as having negative consequences was not changing
passwords; 24% said that there were no negative consequences.
However, while they recognize the possibility of negative consequences,
they seem quite optimistic regarding the likelihood of personally experiencing a
negative outcome. On average, 34% felt that negative consequences were likely
to happen to them only rarely, and 6% felt that they would never occur. The
behaviors deemed least likely to result in negative consequences were sharing
passwords and not changing passwords. The behaviors most likely to result in
personal negative outcomes were deemed to be opening e-mail attachments
from either known or unknown sources without first scanning for viruses, and not
backing up files (Table 3).

Table 3. Perceptions of likelihood of negative consequences following five
security-related
Likelihood of negative
consequences from:
Sharing passwords
Not voluntarily
changing password
Opening e-mail
attachments without
virus checking –
unknown source
Opening e-mail
attachments without
virus checking – known
source
Not backing up regularly

Never

Rarely Occasionally Frequently

10%
10%

28%
48%

28%
30%

25%
10%

All the
Time
9%
2%

21%

43%

28%

48%

17%

28%

17%

26%

40%

23%

10%

TEAM LinG

266 Aytes and Connolly

These data suggest that although subjects know that there are possible
negative consequences to insecure computer practices, they feel that in many
areas, the probability of experiencing those negative consequences is fairly low.
In summary, these descriptive analyses suggest that there is a significant
gap between the respondents’ self-perceptions that they are quite expert at and
comfortable with good practice, and their reports that they actually engage in
risky behavior quite often. While the vast majority of them recognize that there
can be significant negative consequences to risky computing behavior, they also
believe that the probability of it happening to them is fairly low. They tend to:

•
•
•
•

be somewhat overconfident in their computer security knowledge.
engage in risky behavior rather frequently.
recognize that there are potential negative consequences to their risky
behaviors.
think the probability that they will experience negative consequences is
quite low.

In addition to these descriptive analyses, we also examined several correlational and predictive issues.

Safe Computing Cluster

Is there evidence of a cluster of safe computing behaviors, a tendency for
those who are careful about one area to be careful about another? Surprisingly,
there is not. The five behaviors appear to be largely independent of one another.
There is almost no evidence that students (by self-report) who do a lot of one,
Table 4. Pearson’s correlation coefficients (measure of the linearity of a
relationship) among self-reported frequency of five security-related
practices2
Shared Voluntarily Opened e-mail Opened e-mail
from known from unknown
passwords changed
password sources without sources without
checking for
checking for
viruses
viruses
1
-.074
.070
.034
1
-.063
-.172*

Shared passwords
Voluntarily changed
password
Opened e-mail from
known sources without
checking for viruses
Opened e-mail from
unknown sources
without checking for
viruses
Backed up PC
* Significant at the 0.05 level (2-tailed).
** Significant at the 0.01 level (2-tailed)

Backed up
PC

-.051
.037

.498**

.086

.059

TEAM LinG

Computer Security and Risky Computing Practices

267

do a lot another. The only exceptions are that (a) those who frequently check emails for viruses tend to do so both for known and unknown sources, and (b)
those who frequently change passwords are somewhat less likely to open e-mail
attachments from known sources without checking them for viruses. The
remaining eight correlations are all quite small and are not significantly different
from zero. With the exception of virus checking of e-mails, these behaviors
essentially are independent of one another (see Table 4).

The Role of Expertise

Four measures on the questionnaire (final section, questions 1 and 2a-2c)
asked respondents to assess aspects of their expertise. Scores on these items
were strongly correlated (mean r = .76), so we averaged them to provide a simple
index of self-rated expertise. Correlations between this expertise index and the
reported frequency of the five security-related behaviors are shown in Table 5.
As Table 5 shows, expertise is related to several of the risky behaviors we
considered. Subjects reporting higher levels of expertise are less likely to open
emails without screening them for viruses and more likely to change passwords.
There is some indication that they are less likely to have shared passwords (r =
-.14, p<.10), and more likely to do frequent backups (r = .13, p<.10). Thus, there
is some connection between self-claimed expertise in safe computing and selfreported safe practices, but the connection is not especially strong. Even among
those 91 subjects who scored at or above the median of our expertise scale, 26%
share passwords at least occasionally; 36% rarely or never change passwords;
18% open e-mail attachments from unknown sources, without virus checking;
36% open e-mail attachments from known sources, without virus checking; and
25% rarely or never backup their work. Knowledge about how to protect oneself
thus increases the rate of safe behavior but falls a long way short of ensuring it.

Perceptions of Bad Consequences

There are two aspects of negative consequences for following insecure
computing practices: how likely respondents feel the negative consequences are,

Table 5. Pearson’s correlation coefficients of expertise index with reported
frequency of five security-related practices
Voluntarily Opened email Opened email Backed up your
changed
from known from unknown
PC
password sources without sources without
checking for
checking for
viruses
viruses
-.136
.473**
-.295**
-.291**
.134
Expertise
** Significant at the 0.01 level (2-tailed).
* Significant at the 0.05 level (2-tailed).
Shared
passwords

TEAM LinG

268 Aytes and Connolly

and how significant they expect those consequences to be. The questionnaire
contained two or more items (question 4) in each behavior domain related to the
subjects’ perception of the probability of negative consequences. Within each
domain, the responses were highly correlated, so the responses were aggregated
into one measure for each. A single question within each domain (question 5)
related to perceived significance of those consequences.
The five measures of the probability of negative consequences are positively, but not strongly, correlated with one another (mean r = .34), suggesting a
weak computer paranoia cluster: Those who see bad outcomes as likely in one
area tend to see them as more likely in other areas. The two probability measures
related to opening e-mail attachments from known and unknown sources without
checking them for viruses tend to be positively, but only weakly, associated with
our expertise measure (r = .17, r = .21). None of the other probability measures
is correlated with expertise. This suggests that expertise does not significantly
change people’s belief that bad consequences are likely to happen to them if they
behave insecurely.
The probability measures are correlated with one’s (in)secure behavior
measures (Table 6). Those who see the behavior as likely to lead to bad outcomes
are less likely to share passwords, more likely to change passwords, less likely
to open unscreened e-mails from unknown and known sources, and more likely
to backup frequently. While these correlations are in the directions one would
expect, they are relatively weak.
Correlations between perceived significance of consequences and behavior
are even more modest (Table 6). Correlations are in the direction one would
expect, but are all .21 or lower, and two of them are not statistically significant.

Table 6. Pearson’s correlation coefficients between perceived probability
and significance of consequences and reported frequencies of five core
behaviors
Sharing
Passwords

Perceived probability
-.32**
of negative
consequences for each
behavior
Perceived significance
-.19*
of negative
consequences
** Significant at the 0.01 level (2-tailed)
* Significant at the 0.05 level (2-tailed)
+
Significant at the 0.10 level (2-tailed)

Changing
Passwords

.24**

.06

Frequency of Behavior
Opening e-mail
Opening e-mail
from known
from unknown
sources without sources without
checking for
checking for
viruses
viruses
-.16*
-.22**

.07

.21**

Backing
up PC

.15+

.17*

TEAM LinG

Computer Security and Risky Computing Practices

269

Table 7. Regression models predicting frequency of behavior from expertise
and rated probability and significance of consequences.
Frequency of
behavior

Expertise

Sharing
-.09
passwords
Changing
.46**
passwords
Opening e-mail
-.26**
attachments –
unknown
source
Opening e-mail
-.22**
attachments –
known source
Backing up PC
.16*
** p < 0.01 level (2-tailed)
* p < 0.05 level (2-tailed)

Beta Values
Perceived
probability of
consequences
-.28**

Significance of
consequences

Adjusted Rsquared

-.07

.10

.20**

.06

.26

-.12

.06

.08

-.17*

.18*

.12

.08

.18

.05

Predicting Behaviors

As a summary of the preceding correlational analyses, we built simple linear
regression models for each of the five core behaviors, assessing how well we
could predict the self-reported frequency of each behavior from three measures:
the respondent’s ratings of (a) his or her expertise, (b) the probability, and (c) the
significance of negative consequences that might follow from behaving unsafely.
The five regression models are summarized in Table 7.
Other than changing passwords (R-squared = .26), all the other R-squared
values are quite small. With this single exception, it seems that we cannot predict
much of the variance in how often someone engages in a particular securityrelated behavior from knowing (a) how much they know about protecting
themselves (expertise), (b) how bad they think the consequences of insecure
behavior are, or (c) how likely they think these consequences are to follow from
insecure behavior. The exception—changing passwords—is a once-and-done
event, while the remaining four behaviors require continuing safe practice. The
implication is that our respondents may approximate sensible choices when the
behavior in question is a single event, but they make much less risk-sensitive
choices when the behavior requires repeated inconvenience on an ongoing basis.

DISCUSSION

We argued earlier that reducing computer security risks to organizations
and individuals will continue to require that individual users voluntarily comply
with safe computing practices. To the extent that our sample of advanced

TEAM LinG

270 Aytes and Connolly

undergraduates represents the user population of the future, the data presented
suggest that achieving such compliance will continue to be a significant problem.
Some of those advocating training and awareness programs for users seem to
assume that simply describing the possible threats to secure computing and the
possible negative consequences of insecure behaviors will effectively gain
compliance with security procedures. These data, consistent with what is known
about risk perception in general, suggest otherwise.
Our data suggest that many users, despite having little or no formal training
in computer security, feel relatively comfortable in their ability to protect
themselves from viruses, computer crashes, and password violations. This raises
an important question: If users feel they already know how to engage in secure
computing, will they be motivated to become truly educated? Assuming our
respondents are reasonably typical of the rising generation of corporate recruits,
it seems clear that hiring organizations will need to undertake serious security
training themselves. Simply relying on new entrants’ assurances that they are
familiar with safe computing practices is not enough.
The data suggest that corporate trainers of new recruits face a difficult
challenge. Even if they can provide improved knowledge and make trainees
aware of the probabilities of negative consequences and the significance of those
consequences, the data give little confidence that this knowledge will ensure safe
computing practices. Expertise and perceptions related to consequences are
only weak predictors of secure computer practices. Better understanding of this
gap between knowledge and behavior is key to effectively altering user behavior
(Fischhoff et al., 1993). Although risk perceptions in this domain have been
studied little, they can be understood better from the perspective of what we
know about people’s risk perceptions in general. For example, there is evidence
that people underestimate the rate at which single-incident risks cumulate with
repeated exposure (Shaklee & Fischhoff, 1990). People often engage in behavior, such as driving without seatbelts, that carries some risk. However, most
people make many trips without ever experiencing negative consequences. We
know that experience affects people’s behavior and that each safe trip reinforces not using seatbelts. Conversely, people who consistently use seatbelts and
are not involved in an accident are penalized for their safe behavior, because
using a seatbelt has some cost (in comfort and time). This means that each safe
trip reinforces not using a seatbelt. Other factors, such as the knowledge that
seatbelts are not 100% effective, also may reduce people’s tendency to use
seatbelts (Slovic, 1978).
Although computer security breaches are not likely to have life and death
consequences for most people, it is likely that some of these same factors are at
work in this domain. The vast majority of the time, users can share passwords,
open e-mail attachments without checking them for viruses, and so forth, with no
negative consequences. In fact, they are rewarded in this behavior, because they

TEAM LinG

Computer Security and Risky Computing Practices

271

are seen either as helpful (in the case of sharing passwords) or as saving time
(by not scanning for viruses). Just as with seatbelts, they are usually penalized,
at least mildly, for engaging in safe behavior.
Logically, it would seem that informing users of the efficacy of safe
computing and describing in detail the potential catastrophic effects of poor
computing practices would affect behavior. However, experience with seatbelt
usage (Robertson et al., 1974) suggests that these approaches may not work in
practice. As long as the probability of an occurrence is perceived as being
negligible, the significance of the consequences seems to matter little (Slovic,
1978).
Once again, there seems to be a logical approach to addressing the problem:
Inform users about the probability of experiencing a negative outcome. While the
probability of a single event resulting in negative consequences is small, the
overall probability is increased, of course, when the risky behavior is repeated.
Research has shown that increasing people’s time horizons can induce them to
take repetitive risks more seriously (Slovic, 1978). This approach holds promise,
but there is a dearth of empirical data regarding the probabilities of negative
consequences for various computing practices. For example, we do not have
accurate data on what percentage of e-mail attachments contains viruses or how
often password files are cracked. Without credible, understandable probabilities,
we cannot expect users to be swayed in their beliefs that they are unlikely to
suffer from their behaviors.
A final complication is the fact that negative consequences may not be
personally significant to the offending user even when they do occur. Many email-borne computer viruses, for example, do no damage to the infected user.
Instead, they wreak havoc on the larger computing community by replicating
themselves to many other systems, overloading e-mail systems, and requiring
extensive work by system administrators. While peer pressure and the embarrassment of being known as the cause of others’ distress may provide some
motivation, in many cases, infected individuals may never know the trouble they
cause.

CONCLUSION AND FUTURE RESEARCH

Threats to computer security are an increasing concern to organizations.
While technological countermeasures offer some protection, computer users still
need to follow good security practices. This investigation of users’ knowledge,
perceptions, and behaviors regarding computer security issues is an attempt to
better understand the human element of computer security. Our student respondents consider themselves to be fairly knowledgeable about computer use and
safe computing practices, but, nonetheless, they continue to engage in unsafe

TEAM LinG

272 Aytes and Connolly

computing practices. They appear to be cognizant of the risks involved, but, as
in other domains of risk, this knowledge does little to curb unsafe behavior. The
assumption that users are predominantly making sensible action choices in light
of expected consequences is supported only modestly in these data. Other
mechanisms, such as unthinking behavior (Langer, 1978) or impulsivity
(Zuckerman, 2003), also may be implicated.
The findings suggest that it is unlikely that computer users will change their
behavior significantly in response to being provided simply with additional
information regarding computing risks and safe practices. While extending the
time frame that users consider when making computing choices may increase
compliance with security procedures, it also is likely that organizations will have
to enforce compliance when the risks warrant it. This can be done through
automatic means, such as blocking all e-mail attachments, but in many cases
human choice still could negate technological solutions. In these cases, personnel
policies, such as close monitoring and sanctions for violating procedures, may be
necessary to establish a culture of safe practice. These personnel procedures
reflect important organizational choices that will affect organizational climate
and culture, and will involve significant costs, financial and otherwise. They will
have to be weighed against the security risks imposed.
Extensions of this survey approach are also needed. Surveys of corporate
computer users, where policies regarding security practices are both known and
enforced, would be useful in better understanding the users’ perspectives
regarding the effectiveness of such policies. Additionally, further research into
those areas where users may experience personal financial loss (e.g., identity
theft, credit card fraud) is also necessary to better understand if there are certain
types of risks that users are less willing to take.
Finally, further development of a research model that incorporates users’
perceptions and decision-making process is needed. Rational choice models,
such as the one presented here, appear to be useful in understanding users’
behaviors, although further understanding of the factors that affect a user’s
decision process is necessary. Extensions of the model proposed here, perhaps
borrowing more from related models such as the Technology Acceptance
Model, will help us to understand better how to improve information security by
concentrating on factors most directly related to users’ acceptance of countermeasures.

ACKNOWLEDGMENT

Our thanks to Raghu Nandan for assistance in collecting and coding the data
for this study.

TEAM LinG

Computer Security and Risky Computing Practices

273

REFERENCES

Adams, A., & Sasse, M. (1999). Users are not the enemy. Communications of
the ACM, 42(12), 41-46.
Connolly, T., Arkes, H., & Hammond, K. (2002). Judgment and decision making:
An interdisciplinary reader. New York: Cambridge University Press.
Cyberspace invaders (2002, June). Consumer Reports, 67(6), 16-21.
Davis, F., Bagozzi, R.P., & Warshaw, P.R. (1989). User acceptance of
computer technology: A comparison of two theoretical models. Management Science, 35(8), 982-1003.
Fischhoff, B., Bostrum, A., & Quadrel, M. (1993). Risk perception and communication. Annual Review of Public Health, 14, 183-203.
Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An
introduction to theory and research. Reading, MA: Addison-Wesley.
Gips, M. (2001). Plugging into awareness. Security Management, 45(11), 2122.
Goldstein, W., & Hogarth, R. (1997). Research on judgment and decision
making. New York: Cambridge University Press.
Jackson, C., Chow, S., & Leitch, R. (1997). Toward and understanding of the
behavioral intention to use an information system. Decision Sciences,
28(2), 357-389.
Langer, E. (1978). Rethinking the role of thought in social interaction. In J.
Harvey, W. Ickes, & R. Kidd (Eds.), New directions in attribution
research, vol.2. Hillsdale, NJ: Erlbaum.
Peltier, T. (2000). How to build a comprehensive security awareness program.
Computer Security Journal, 16(2), 23-32.
Power, R. (2002). Computer security issues and trends. 2002 CSI/FBI Computer Crime and Security Survey, 8(1), 1-22.
Rhodes, K. (2001). Operations security awareness: The mind has no firewall.
Computer Security Journal, 18(3), 27-36.
Robertson, L., et al. (1974). A controlled study of the effect of television images
on safety belt use. American Journal of Public Health, 64, 1071-1080.
Shaklee, H., & Fischhoff, B. (1990). The psychology of contraceptive surprises:
Judging the cumulative risk of contraceptive failure. Journal of Applied
Psychology, 20, 385-403.
Slovic, P., Fischoff, B., & Lichtenstien, S. (1978). Accident probabilities and
seat belt usage: A psychological perspective. Accident Analysis and
Prevention, 10, 281-285.
Slovic, P., Fischoff, B., & Lichtenstien, S. (1979). Rating the risks. Environment, 21, 14 -39.
Tuesday, V. (2001, April 30). Human factor derails best-laid security plans.
Computerworld, 35(18), 52-55.

TEAM LinG

274 Aytes and Connolly

Zeckhauser, R., & Viscusi, K. (1990 ). Risk within reason. Science, 248, 559564.
Zuckerman, M. (2003). Biological bases of personality. In T. Millon, & M.J.
Lerner (Eds.), Handbook of psychology, volume 5 (pp. 85-116). New
York: John Wiley.

ENDNOTES

We recognize that this is not truly a certainty of no negative outcomes. In
our simplified model, however, we consider the user to perceive the
outcome to be a practical certainty.
For details on statistical terms, see: Freedman, D., Pisani, R., & Purves, R.
(1998). Statistics (3rd ed.), New York: Norton.

TEAM LinG

Computer Security and Risky Computing Practices

275

APPENDIX:
QUESTIONNAIRE USED
IN DATA COLLECTION
COMPUTER SECURITY ISSUES AND PRACTICES
DECISION ANALYSIS SURVEY
We are conducting a study of how people deal with computer security – things people do and do
not do that can make their computer use more or less safe. These activities include:

Sharing (or not sharing) your passwords with other people
Changing (or not changing) your passwords frequently
Opening (or not opening) emails with suspicious attachments – from friends; from
others
Backing up your files regularly (or not)
Performing financial transactions via the Internet (Web)

Your responses to this survey will help us better understand the issues that need to be addressed
in personal computer security habits and practices.
Prior to answering the questions in the next four (4) pages, we request you to fill out the
information below. This information will help us in keeping track of demographic data of survey
participants.
* Please note that all information that you provide will remain anonymous.

STATISTICAL DATA
Age

: _______ years

Gender

Male

Year in School
Graduate Student

Freshman Sophomore Junior Senior

Major

: ________________________________________

Full-Time Student

Yes

Female

TEAM LinG

276 Aytes and Connolly

SHARING PASSWORDS

E-mail, online membership accounts for various services, financial accounts, and even your student information at the university requires you to enter
a login and password prior to accessing your account. Logins and passwords
together form a unique combination that is theoretically supposed to prevent
unauthorized persons from accessing your information (e-mail, bank accounts,
credit card accounts, etc.). While a login name may be easily identifiable (your
first name, full name, etc.), your password is usually a set of letters and numbers
that is known only to yourself. In this section, we would like you to answer a few
questions regarding the privacy of your passwords.

1. Have you ever shared your password with friends, family, co-workers or others?
2. In your opinion, how often do your peers share passwords with family, friends and
co-workers?
3. Are there any negative consequences to sharing passwords?
4. How likely are the following consequences IF you share your password?
a. Unauthorized persons accessing and reading your e-mail
b. Unauthorized persons accessing and using your financial information
c. Unauthorized persons accessing personal information
5. If these consequences were to occur, how significant would they be to you?
(0=no significance; 1=mild inconvenience; 2=cause for concern; 3=considerable;
4=disaster)
6.
a. Have any of the above consequences ever happened to you?
b. If yes, how recently? ___ months ago
7.
a. To your knowledge, have any of the above consequences ever happened to
someone you know?
b. If yes, how recently? ___ months ago

Occas- Frequ- All the
Never Rarely -ionally -ently time

0
0
Yes
0
0
0
0

1
1
1
1
1
1

2
2
No
2
2
2
2

Yes

3
3

4
4

3
3
3
3

4
4
4
4

CHANGING PASSWORDS

Once you establish your online account, you ordinarily need not change your
password. However, most accounts have an option that allows you to voluntarily
change your password. Some accounts require you to change your password and
prompt you to do so at regular intervals. In this section, we would like to ask you
some questions that are related to your voluntary password changing habits.

TEAM LinG

Computer Security and Risky Computing Practices

1. After establishing an online account, have you ever voluntarily changed your
password(s)?
2. In your opinion, how often do your peers voluntarily change their passwords(s)?
3. Are there any negative consequences to NOT CHANGING password(s)?
4. How likely are the following consequences IF you DID NOT CHANGE your
password(s)?
a. Unauthorized persons accessing and reading your e-mail
b. Unauthorized persons accessing and using your financial information
c. Unauthorized persons accessing personal information
5. If these consequences were to occur, how significant would they be to you?
(0=no significance; 1=mild inconvenience; 2=cause for concern; 3=considerable;
4=disaster)
6.
a. Have any of the above consequences ever happened to you?
b. If yes, how recently? ___ months ago
7.
a. To your knowledge, have any of the above consequences ever happened to
someone you know?
b. If yes, how recently? ___ months ago

277

Occas- Frequ- All the
Never Rarely -ionally -ently time

1
0
0
1
Yes

2 3
2 3
No

4
4

0
0
0
0

2
2
2
2

4
4
4
4

1
1
1
1

Yes

3
3
3
3

UNKNOWN E-MAIL ATTACHMENTS/UNKNOWN
SOURCE

Most of us receive unsolicited e-mail from unknown companies, organizations, and individuals. In this section, we would like to learn more about your
behavior in handling an unknown e-mail attachment (an attachment that is
ambiguous in content or is unexpected) from an unknown or unrecognizable
source.
1.
a. Have you ever received an unknown e-mail attachment from an UNKNOWN
source?
b. If yes, have you opened the attachment without checking for computer
viruses?
2.
a. In your opinion, how often do your peers receive e-mail attachments from
unknown sources?
b. In your opinion, how often do your peers open these attachments without
checking for viruses?
3. Are there any negative consequences to opening these attachments?
4. How likely are the following consequences IF you were to open an unknown
attachment from an unknown source?
a. Attachment contains computer virus that infects files
b. Attachment contains computer virus that crashes computer
5. If these consequences were to occur, how significant would they be to you?
(0=no significance; 1=mild inconvenience; 2=cause for concern; 3=considerable;
4=disaster)
6
a. Have any of the above consequences ever happened to you?
b. If yes, how recently? ___ months ago
7.
a. To your knowledge, have any of the above consequences ever happened to
someone you know?
b. If yes, how recently? ___ months ago

Occas- Frequ- All the
Never Rarely -ionally -ently time

0
0
0

1
1
1

2
2
2

3
3
3

4
4
4

0
1
Yes

2 3
No

0
0
0

1
1
1

2
2
2

3
3
3

4
4
4

Yes

TEAM LinG

278 Aytes and Connolly

UNKNOWN E-MAIL ATTACHMENT/KNOWN
SOURCE

Many of us receive and send e-mails that contain jokes, interesting articles,
and other not-so-relevant information, such as a forwarded message to family,
friends, and co-workers. Some of these e-mails may contain attachments that
are not easily recognizable, although you are able to recognize the source (person
who sent the e-mail). In this section, we would like to learn more about your
behavior in handling an e-mail that contains an unknown attachment but comes
from a known source (friend, family, or colleague).
1

a. Have you ever received unknown e-mail attachments from a KNOWN source?
b. If yes, have you opened these attachments without checking for computer

viruses?
2.
a. In your opinion, how often do your peers receive unknown e-mail attachments
from known sources?
b. In your opinion, how often do your peers open these attachments without
checking for viruses?
3. Are there any negative consequences to opening these attachments?
4. How likely are the following consequences IF you were to open an unknown
attachment from a known source?
a. Attachment contains computer virus that infects files.
b. Attachment contains computer virus that crashes computer.
5. If these consequences were to occur, how significant would they be to you?
(0=no significance; 1=mild inconvenience; 2=cause for concern; 3=considerable;
4=disaster)
6.
a. Have any of the above consequences ever happened to you?
b. If yes, how recently? ___ months ago
7.
a. To your knowledge, have any of the above consequences ever happened to
someone you know?
b. If yes, how recently? ___ months ago

Occas- Frequ- All the
Never Rarely -ionally -ently time

0
0
0

1
1
1

2
2
2

3 4
3 4
3 4

3 4

0
1
Yes

2 3 4
No

0
0
0

2
2
2

1
1
1

Yes

3 4
3 4
3 4

REGULAR BACK-UP

This section explores your behavior in backing-up your personal computer
work. A backup copy is defined as any SECOND copy that is NOT on your hard
drive. If you were to save your work on your hard drive, and then copy it onto
a floppy disk, that would constitute a backup copy. Other traditional means of
backing up your work involve digital tape backup, CD-ROMs, backing up on the
Internet, and so forth. The following questions explore your habits in this area.

TEAM LinG

Computer Security and Risky Computing Practices

1. Do you back-up your personal computer work?
2. In your opinion, do your peers backup their personal computer work?
3. Are there any negative consequences to NOT BACKING-UP your personal work?
4. How likely are the following consequences IF you DID NOT back-up your personal
work?
a. Some work lost when computer freezes
b. No consequences at all
c. Work lost due to computer virus
5. If these consequences were to occur, how significant would they be to you?
(0=no significance; 1=mild inconvenience; 2=cause for concern; 3=considerable;
4=disaster)
6.
a. Have any of the above consequences ever happened to you?
b. If yes, how recently? ___ months ago
7.
a. To your knowledge, have any of the above consequences ever happened to
someone you know?
b. If yes, how recently? ___ months ago

279

Occas- Frequ- All the
Never Rarely -ionally -ently time

0
1
0
1
Yes

2 3 4
2 3 4
No

0
0
0
0

2
2
2
2

1
1
1
1

Yes

3
3
3
3

4
4
4
4

COMPUTER SECURITY KNOWLEDGE

This section asks you some general questions regarding your computing
habits.
0=not very comfortable/ knowledgeable; 1=somewhat comfortable/ knowledgeable; 2=comfortable/knowledgeable; 3=very comfortable/ knowledgeable;
4=expert

1. How comfortable are you using a computer for sending and receiving e-mails and performing
financial transactions via the Internet (Web)?
2. How knowledgeable are you about protecting yourself against
a. Computer virus attacks?
b. Computer crashes resulting in loss of work saved on your computer?
c. Unauthorized people intercepting your financial information via the Internet (Web)?
3. Have you ever received any sort of information or tutoring about protecting yourself against
computer viruses, computer crashes resulting in loss of information, invasion of privacy by
hackers, and other computer security-related issues?
4. If yes, which of these methods best describes the information/tutoring you have received regarding
computer security? (check all that apply)
ο Learned from friends, acquaintances or co-workers
ο Learned from personal experience
ο Heard or read about it in the media
ο Received formal computer training (class-related, job-related training, etc.)
ο Received information when purchasing computer
5. Which of the following best describes the PRIMARY computer you use? (check ONLY one)
ο Personal computer at home
ο Computer at work
ο University computer (computer labs, library)

0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
Yes

TEAM LinG

280 Warkentin, Davis and Bekkering

Chapter XIV

A TAM Analysis of an
Alternative High-Security
User Authentication
Procedure
Merrill Warkentin, Mississippi State University, USA
Kimberly Davis, Mississippi State University, USA
Ernst Bekkering, Northeastern State University, USA

ABSTRACT

The objective of information system security management is information
assurance, which means to maintain confidentiality (privacy), integrity,
and availability of information resources for authorized organizational
end users. User authentication is a foundation procedure in the overall
pursuit of these objectives, and password procedures historically have
been the primary method of user authentication. There is an inverse
relationship between the level of security provided by a password procedure
and ease of recall for users. The longer the password and the more
variability in its characters, the higher the level of security is that is
provided by the password, because it is more difficult to violate or crack.
However, such a password tends to be more difficult for an end user to
remember, particularly when the password does not spell a recognizable

TEAM LinG

A TAM Analysis of an Alternative High-Security User Authentication Procedure 281

word or when it includes non-alphanumeric characters such as punctuation
marks or other symbols. Conversely, when end users select their own more
easily remembered passwords, the passwords also may be cracked more
easily. This study presents a new approach to entering passwords that
combines a high level of security with easy recall for the end user. The
Check-Off Password System (COPS) is more secure than self-selected
passwords and high-protection, assigned-password procedures. The present
study investigates tradeoffs between using COPS and three traditional
password procedures, and provides a preliminary assessment of the efficacy
of COPS. The study offers evidence that COPS is a valid alternative to
current user authentication systems. End users perceive all tested password
procedures to have equal usefulness, but the perceived ease of use of COPS
passwords equals that of an established high-security password, and the
new interface does not negatively affect user performance compared to a
high-security password. Further research will be conducted to investigate
long-term benefits.

BACKGROUND

Despite continuing improvements in computer and network technology,
computer security continues to be a concern. One of the leading causes of
security breaches is the lack of effective user authentication, primarily due to
poor password system management (The SANS Institute, 2003), and the ease
with which certain types of passwords may be cracked by computer programs.
Yet even with today’s high-speed computers, an eight-character password can
be very secure, indeed. If a Pentium 4 processor can test 8 million combinations
per second, it would take more than 13 years on average to break an eightcharacter password (Lemos, 2002). However, the potential for password
security has not been fully realized, and a security breach can compromise
significantly the security of information systems, other computer systems, data,
and Web sites. Furthermore, the increasing degree to which confidential and
proprietary data are stored and transmitted electronically makes security a
foremost concern in today’s age of technology. This is true not only in civilian
use, but also in government and military use.
A primary objective of information system security is the maintenance of
confidentiality, which is achieved in part by limiting access to valuable information resources. Historically, user authentication has been the primary method of
protecting proprietary and/or confidential data by preventing unauthorized
access to computerized systems. User authentication is a foundation procedure
in the overall pursuit of secure systems, but in a recent e-mail to approximately
one million people, Bill Gates (chairman of Microsoft Corporation) referred to

TEAM LinG

282 Warkentin, Davis and Bekkering

passwords as “the weak link” in computer security, noting that most passwords
are either easy to guess or difficult to remember (“Gates Pledges Better
Software Security,” 2003). Gates correctly identified a classic tradeoff that
system and network administrators must face when considering various password procedures for adoption. Specifically, there is an inverse relationship
between the level of security provided by a password procedure and the ease of
recall for end users. When end users select their own easily remembered
passwords, the passwords are easier to crack than longer passwords with a
greater variety of characters. The longer the password and the more variability
in the characters, the higher the level of security provided by such a password.
However, human memory has significant limitations, and such passwords tend
to be more difficult for end users to remember. Typically, human short-term
memory can store only seven plus or minus two (7 ± 2) “chunks” of information
(Miller, 1956), and alphanumeric characters such as punctuation marks and other
symbols are not easily combined in a chunk with other characters. For example,
the letters b, a, n, and d can be stored easily together as a single chunk, but it is
difficult for humans to combine symbols such as the vertical bar ( | ) and tilde
( ~ ) with other characters to form a chunk. The problem of striking a balance
between security and ability to remember passwords will become more acute as
the number of passwords per user increases.
In a survey with 3,050 distinct respondents (Rainbow Technologies Inc.,
2003), the following picture emerged:

•
•
•
•
•
•

Respondents used, on average, almost 5½ passwords
23.9% of respondents used eight or more passwords
More than 80% were required to change passwords at work at least once
a year
54% reported writing down a password at least once
9% reported always writing down their passwords
More than half had to reset business passwords at least once a year,
because they forgot or misplaced the password

The 352 participants in the present study reported using an average of 3.9
passwords at the time of the study and 4.53 passwords in the prior six months.
Furthermore, 35.5% reported writing down at least one password. Clearly, the
use of multiple passwords constitutes a burden to users.

PASSWORD STRATEGIES

Because of these tradeoffs and because methods and technologies employed by crackers are improving constantly, new security strategies with

TEAM LinG

A TAM Analysis of an Alternative High-Security User Authentication Procedure 283

improved password procedures are required. Traditional methods include allowing users to select their own passwords and assigning passwords to them, both
of which may be subject to restrictions on password length and character
choices. The efficacy of both systems depends on the ability of end users to recall
such passwords without writing them down. The Federal Information Processing
Standards (FIPS) publication 112 includes guidelines for different levels of
password security (National Institute of Standards and Technology, 1985). At
the highest level, these guidelines include passwords with six to eight characters
composed from the full 95 printable character ASCII set. Furthermore, the
guidelines specify using an automated password generator, individual ownership
of passwords, use of non-printing keyboards, encrypted password storage, and
encrypted communications with message numbering. The theoretical number of
passwords using the FIPS procedure is approximately 6.7 x 1015 ( = 958 + 957 +
956 ). However, to utilize the full set of characters, all printable non-alphanumeric
characters must be included in the set and have an equal chance of selection as
the alphanumeric characters. But passwords with non-alphanumeric characters
can be hard to remember. Consider, for example, passwords such as “ ,swFol=; ”
or “ >_F<“Yjz .” To avoid having to use such awkward passwords, we have
devised a new password interface for user authentication. (The FIPS procedure
is one of the four procedures investigated in this study.)
When allowed to select their own password, users tend to select passwords
that may be easy to remember but also may be easy to crack. On the other hand,
when they are assigned a cryptographically strong password, users generally will
find them difficult to remember and will frequently record them in writing. To
remedy these potential security problems, various strategies currently are used.
Some organizations attempt to reduce the number of passwords needed by using
a single system sign on (SSO) (Boroditsky & Pleat, 2001). Others are researching the possibility of using graphical mechanisms (Bolande, 2000; Pinkas &
Sander, 2002) or combining passwords with keystroke dynamics (Monrose,
Reiter, & Wetzel, 1999). Organizations can instruct their members in the proper
selection of passwords to varying degrees, from simple instructions regarding the
minimum number of positions and the minimum variability of characters to
extensive instructions and even feedback mechanisms where weak passwords
are rejected immediately (Bergadano, Crispo, & Ruffo, 1998; Jianxin, 2001).
Weirich and Sasse (2001) advocate proper instruction and motivation of users,
as well as a flexible approach depending on the organization and type of work for
which the security is needed. The self-selection procedure (“Self”) is the second
of four procedures investigated in this study. Users were required to include at
least one letter and at least one number in their password and were required to
select passwords of at least six but not more than 14 characters in length. The
third procedure utilizes system-assigned passwords from the list of common

TEAM LinG

284 Warkentin, Davis and Bekkering

passwords found in Spafford’s Technical Report (1988), typically spelling
common words that are relatively easily remembered (“Spafford”). Table 1
shows some examples of passwords that might be used under the three primary
password procedures in use today.
In a study of password usage, Adams and Sasse (1999) identified the
following four factors that negatively influence the use of passwords:
1.
2.
3.
4.

The need to remember multiple passwords due to the use of different
passwords for different systems and the requirement to change passwords
at intervals;
Lack of user awareness regarding the requirements for secure password
content
Perceived lack of compatibility of passwords with work practices
Unrealistic user perception of organizational security and information
sensitivity

Though the latter three factors can be remedied with organizational
measures such as review of password policies and user education, the first factor
is grounded in the limitations of human memory. Since the number of secure
systems used by each individual is bound to increase rather than decrease over
time, resulting in the need to remember more passwords, memory limitations
must be accommodated.

Table 1. Examples of passwords used in current password procedures
Self-Selected

Spafford

FIPS

fido

academia

|iaSq/a<

101565 (birthdate)

brenda

.EUg`=M=

mustang

deluge

JVe*,UuB

corvette

garfield

1X+]HpM?

2895 (last four of SSN)

heinlein

*R(AO-3P

mary

irishman

&\qe2P\H

john

lamination

,WU$&TTW

newyork

marvin

Ua#==f9Z

godawgs81 (sports fan)

napoleon

GB<X(DE)

chevy1

oxford

_so\yor$

march241981

password

$4Y,*T6R

Jesus1

rascal

L1t_<Mv,

covergirl1

saxon

`(WCpod{

holein1 (golf fan)

william

2]^p<ngn

Bu11ocks

yellowstone

s@5A:L7>

TEAM LinG

A TAM Analysis of an Alternative High-Security User Authentication Procedure 285

HUMAN MEMORY

A heuristic for the capacity of the human short-term memory system states
that an individual can recall seven plus or minus two (7 ± 2) chunks of information
(Miller, 1956). This rule of thumb only applies to information to be recalled for
relatively brief periods without rehearsal. Information can be maintained for
longer periods of time, but elaborate rehearsal is required for transfer to longterm memory (Hewett, 1999; Newell & Simon, 1972). A recent model describes
a working memory, which is part of the larger memory system and not distinct
from long-term memory (Anderson, 1994). In this model, memory limitations also
depend on the ability to retrieve information from long-term storage to working
memory. Regardless of the cognitive model, a capacity limitation exists. The
proposed password procedure addresses this memory capacity limitation by
using a password that may be easier to remember than FIPS-compliant passwords, although the input mechanism may be more cognitively challenging.

CHECK-OFF PASSWORD SYSTEM (COPS)

Traditional password procedures either assign an ordered series (sequence)
of characters, which may or may not spell something meaningful to the user, or
allow users to select their own ordered sequence of characters. In either case,
the order of the characters is significant and must be maintained. A strength of
the Check-Off Password System (COPS), the fourth procedure in this investigation, is that the order of characters within the password is irrelevant, and,
therefore, the user can choose to remember them in many ways. A COPS
password is assigned to each user and consists of a set of eight different
characters (the “COPS password”) selected from the 16 most commonly used
letters in the American alphabet (AskOxford.com, 2002) (the “COPS Superset”),
including all five major vowels (E A R I O T N S L C U D P M H G). The user
may form any word or words by rearranging these eight characters (similar to
an anagram) and may use any of the characters repeatedly in doing so. For
example, suppose a user were issued the characters “ULATSREG,” which we
will refer to as the “Example Password.” Using the characters in the Example
Password, one user might form the compound word “STARGLUE” in order to
remember the eight characters, whereas another user may select “GLUERATS,”
“SLUGTEARS,” or “RESTGULAG.” In other words, while the Example
Password (and every COPS password) consists of a random selection of 8
alphabetic characters without repetition, users may reorder those characters
(and use characters more than once) to form their own password to facilitate
recall. The user even may use characters not found in the COPS Superset (B F
Y W K V X Z J Q) to form a memorizable password, since those characters will
not be included on the input interface (the COPS selection grid), as described

TEAM LinG

286 Warkentin, Davis and Bekkering

later. For example, by using the “B” character, a music aficionado could form the
password “GREATBLUES” from the Example Password. In addition to users
tapping their imagination to form a memorable password, an automated password generator with a facility for suggesting words from a dictionary could be
used. Table 2 shows some additional examples of COPS passwords with several
user-tailored modifications.
To authenticate the user, COPS presents an 8-by-7 grid of checkboxes,
each adjacent to a character randomly selected from the COPS Superset. The
user checks off only the boxes adjacent to characters contained in the COPS
password. Because the grid contains 56 checkboxes generated from only 16
characters (the COPS Superset), characters will typically appear more than
once. On average, each character in the COPS Superset will appear in a grid 3.5
times (56 ¸16), ranging from a minimum of zero times (even if such character
appears in the COPS password) to a maximum of 56 times, although each of
those extremes would be a rare occurrence. Thus, users must check off a given
letter in a COPS password an average of 3.5 times, as follows.
Consider the Example Password again (“ULATSREG”). To enter the
password, the user would be presented with a grid such as the one shown in
Figure 1, which demonstrates a failed attempt to enter the Example Password.
To successfully enter the Example Password in such grid, the user would need

Table 2. Examples of COPS passwords with user-tailored modifications
Original COPS Password
TRLOHASM

GDRHISTE

MROTPSCA

ENTMAOPL

RMTOCALG

SAGNTPHI

TASHMREN

User-Tailored COPS Password
HARM BOLTS
FARM SLOTH
SLAM THROW
RAM SLOTH
WED RIGHTS
DREG THIS
THE GIRDS
DR EIGHTS
CRAFTS MOP
RAMP COST
CAM SPORT
FARM PC TO
PETAL MONK
PANT MOLE
PANEL TOM
LAMB PET ON
GRAM CLOT
CART GLOM
TRACK GLOM
TRACY GLOM
SNAP BIGHT
HANG SPIT
PANT WHIGS
PAST NIGH
FARMS THEN
HARM VENTS
WHEN SMART
MARSH NEWT

HAL STORM
HALF STORM
HARM LOTS
MARSH LOT
RED SIGHT
SHRED TWIG
SHRED GIFT
BIGHTS RED
WORST CAMP
CATS PROM
CRAP MOST
CRAB STOMP
MENTAL POW
NAME PLOT
LAMP TONE
TAP MELON
CLOG MART
MALT VCR GO
TRAM CLOG
GRAM COLT
PATH SIGN
GAP HINTS
HANG PITS
ZAP NIGHTS
SMART HEN
TRASH MEN
MATH FERNS
MASH RENT

TEAM LinG

A TAM Analysis of an Alternative High-Security User Authentication Procedure 287

to check the box adjacent to every “U” appearing in the grid (i.e., three “U”
checkboxes would need to be checked), and the user would need to check the
box for every “L” appearing in the grid (i.e., four “L” checkboxes), and so forth.
In Figure 1, the user has neglected to check off the “S” box in the fifth row of
the eighth column, which will result in a failed login attempt. When the user fails
to successfully check all of the necessary boxes, he or she will be presented with
a new grid in a randomized layout, which will almost certainly be different than
the preceding layout. The user must reattempt the COPS procedure with the
same password and a new grid.
Without the ever-changing grid interface, the number of possible combinations would be no higher than C(16,8) or 12,870, because the presence of one
instance of a character would determine the result for all other instances of the
same character. In other words, if one “T” is selected, all other boxes with a “T”
on the same interface should also be selected. A cracker could manually try to
enter all 12,870 combinations, although time considerations would make this
impractical. But this assumes that the cracker is a human who is able to see the
letters next to each checkbox. A computer could run through combinations much
faster, but the lack of knowledge of which letters appear on the grid (due to the
lack of sight) causes the number of possible combinations to increase to 256 or
7.2 x 1016. For the computer, the number of possible combinations is higher than
the 12,870 combinations for humans, because the universe of possible combinations is based on 56 checkboxes rather than letters appearing on the grid. Optical
Character Recognition (OCR) could overcome this limitation of computers and
is currently effectively counteracted online by major organizations such as
AltaVista, PayPal, Yahoo (Pinkas & Sander, 2002) and Ticketmaster
Figure 1. Representative COPS selection grid

TEAM LinG

288 Warkentin, Davis and Bekkering

(Ticketmaster, 2003) by using a Reverse Turing Test (Coates, Baird, &
Fateman, 2001). As long as the layouts are randomly generated and OCR cannot
be used effectively, the number of possible combinations with 56 check-off
boxes either selected or not selected will remain 2 56 or 7.2 x 10 16. Unlike selfselected, Spafford, and FIPS passwords, COPS passwords achieve both a high
level of security and a high degree of memorability—neither goal of password
procedures is made paramount at the expense of the other. In other words, the
COPS paradigm achieves the “best of both worlds” in terms of security and
memorability.

RESEARCH QUESTIONS

Given these issues, an investigation into the COPS password system was
undertaken in order to assess its efficacy in an end-user environment. Would the
COPS system be perceived as useful and easy to use? Would users be inclined
to use the COPS system, given the presumed relative tradeoff in its use? From
these research questions, the following specific research hypotheses were
developed.
Hypothesis 1: All password procedures are perceived to be equally useful.
Hypothesis 2: All password procedures are perceived to be equally easy to
use.
Hypothesis 3: Users will be equally inclined to use each of the password
procedures.

METHODOLOGY

For this investigation, the authors devised and executed a controlled
experiment in which the participants took a pre-survey to characterize their
perceptions about the usefulness of passwords and their preferences and
intention to use passwords in general. The participants then used one of the four
password procedures under investigation, followed by a post-survey regarding
their perceptions of the password procedure they used in the experiment. This
research methodology is based on the theoretical foundations of the Technology
Acceptance Model, from which the survey instrument was derived in a modified
form.

Applicability of TAM

In new technology implementations, the Technology Acceptance Model
(TAM) indicates that Perceived Ease of Use (PEOU) and Perceived Usefulness
(PU) are considered antecedents of Behavioral Intent to Use (BI), which in turn

TEAM LinG

A TAM Analysis of an Alternative High-Security User Authentication Procedure 289

is an antecedent to actual use (Davis, 1989). Although the semi-self-selected
passwords using COPS are relatively easy to remember, the system also
requires user input that is more cognitively challenging than traditional password
systems. If only one check-off box is erroneously missed or selected, an entirely
new check-off grid is generated and must be completed, thereby increasing the
cognitive load of the activity. This may cause the system user to become
frustrated, resulting in a low Perceived Ease of Use of the password procedure.
If the user had a choice, such frustration might generate resistance to adopting
the technology. The end user typically does not have a choice, however, of which
password procedure is employed for user authentication. The selection of user
authentication measures is the purview of system administrators, not end users.
One might query, therefore, why researchers should be interested in end users’
“intention to use” password procedures and why TAM is relevant to determining
the potential for the widespread adoption of COPS.
Although system or network administrators are responsible for the selection
and adoption of password procedures to protect the systems they manage, such
decisions are not made in a vacuum. They are not at liberty to select the most
cryptographically secure password procedure, if users are unable to efficiently
and effectively use such a system. Imagine at one extreme a password system
that is impossible to crack but also impossible to use. Obviously, implementation
of that system is infeasible. Also, an extremely secure password procedure can
generate user resistance and complaints, and lead to a high incidence of
passwords being reset. At the other extreme is a password procedure that is
extremely easy to use but also extremely easy to crack. Such a system also is
not likely to be a system administrator’s preferred choice. A system administrator must weigh the multiple objectives of a password system (including security
and usability) in determining the most suitable password procedures to use for
the protection of different computer systems. Therefore, PEOU and PU, as
measured from the user’s perspective, are indirect factors affecting the system
administrator’s adoption of a password procedure. Thus, it is imperative that
we test the PEOU and PU of COPS from the user’s perspective in order to
evaluate its potential as an alternative to current user authentication methods. In
order to preliminarily evaluate the efficacy of COPS, a controlled empirical study
comparing user perceptions of COPS and existing alternatives was conducted.

Scale Development and Modification

A formal process was observed to develop measurement scales with
reliability and validity within the TAM framework. A large inventory of existing
TAM scale items was gathered from existing literature and adjusted for use in
this password procedure study. During this process, the authors were confronted
with the contrast between password procedures and other technologies in which
the TAM model has been used to research user acceptance. In previous studies,

TEAM LinG

290 Warkentin, Davis and Bekkering

ease of use and usefulness were considered assets of the technology. For
example, if a menu clearly enables a user to attach a file to an e-mail with one
click, it might be deemed easy to use. For a password procedure, however, ease
of use may be a liability if it allows unauthorized users to easily crack the
password procedure and obtain unauthorized access to a computer system or
confidential data. Moreover, it is never easier to access a system that requires
a password than a system that does not require a password. Similarly, the
usefulness of a password protection procedure may not be apparent to the user,
unlike the functionality of e-mail or other software. This requires some modification of the traditional TAM scales.
For example, an original Davis (1989) TAM instrument item stated, “Electronic mail enables me to accomplish tasks more quickly” (p. 324). If we simply
had modified existing TAM indicators by substituting password procedures for
the technology being tested, the instrument items would not have measured
properly the Usefulness or Ease of Use in this context. For example, users
always would strongly disagree with the statement, “This password system
enables me to log on to computers more quickly,” because password systems, by
their very nature, require more time and make it more difficult to logon to
computers. It was critical, therefore, to measure the constructs in terms of the
absence of a negative, specifically the absence of inconvenience and frustration
to the user. The above question was, therefore, modified as follows: “I can
efficiently access computers even when I must use this password procedure.”
Similarly, the traditional TAM question, “Using this technology saves me time,”
was modified to read, “Using this password procedure does not take too much
time.”
Furthermore, password procedures do not facilitate end-user activity by
producing a tangible benefit such as improved communication or increased
efficiency in completing a task. The primary benefit of password procedures is
maintaining the security of confidential data or proprietary applications. Such
security is achieved by limiting access to authorized users through user authentication methods such as password procedures. Whether the purpose of such
security is to protect a user’s own data or an employer’s data, password
procedures frequently may be viewed by the user as a hindrance rather than a
facilitator to accomplishing a task. While in the long run the user may recognize
the necessity of password procedures for maintaining security, passwords often
may be viewed as a necessary evil that otherwise impairs the user’s immediate
need for legitimate and authorized access to protected data or applications.
Therefore, we determined that it was necessary to modify traditional TAM
indicators designed to measure perceived usefulness of technologies in general
to properly measure perceived usefulness in this context (see Table 3 for the
resulting instrument items).
With these differences in mind, we made appropriate adjustments to
standard TAM survey instrument items and developed a pre-test research
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

A TAM Analysis of an Alternative High-Security User Authentication Procedure 291

instrument to measure attitudes toward password procedures in general and a
post-test research instrument to measure attitudes toward the particular password procedure to which a user was exposed during the study. We generated a
list of 14 potential items to measure PU, 16 potential items to measure PEOU,
and five potential items to measure BI. Because all items were generated using
well-established scales, all participants in our study responded to all items in the
pre-survey before exposure to a password procedure, and then responded again
to all items in the post-survey after exposure to one of the four password
procedures being tested. All items were measured on a five-point Likert scale
ranging from strongly disagree to strongly agree.

Data Collection

The study was conducted as a controlled experiment in a large computer
facility with 40 stations, where 352 participants were exposed to one of the
following four password procedures: (1) self-selected passwords with limited
restrictions;, (2) system-assigned passwords from the list of common passwords
found in Spafford’s Technical Report (1988); (3) system-assigned passwords
compliant with the FIPS standard for high protection (National Institute of
Standards and Technology, 1985); and (4) system-assigned passwords in the
Check-Off Password System (COPS). The size of the groups was almost equal,
with n = 90 for COPS, n = 88 for self-selected passwords, and n = 87 for FIPS
and Spafford passwords. All participants were experienced system users who
used an average of 3.90 passwords at the time of the study and 4.53 passwords
in the six months prior.
Each study participant signed an implied consent form and then received a
disk containing a compiled program written in Visual Basic. All disks were
numbered and contained only one of four versions of the program. Each version
of the program incorporated the pre-test instrument to measure attitudes toward
password procedures in general, a login procedure using one of the four
password procedures, and the post-test instrument to measure attitudes toward
that particular password procedure. The different versions of the program were
randomly distributed to participants (with randomization accomplished by using
randomization functions in Microsoft Excel), so that participants were randomly
exposed to the different password procedures. These randomized disks with no
indication of the version were distributed sequentially as participants arrived at
the lab. Participants executed the program from the disk, and at the end of the
session, all data were automatically captured to a text file with the text files
transferred to the researcher’s hard disk after completion.
Each program started with a brief general description of the purpose of the
study. Participants were informed that they would be using only one of four
possible password procedures, and that they would have the option to stop trying
if they were unable to successfully enter the correct password within five

TEAM LinG

292 Warkentin, Davis and Bekkering

attempts, in which case the participants were presented immediately with the
post-test survey instrument. After completing the pre-test survey reflecting
general attitudes, participants received instructions for using the password
procedure contained on their disk. Instructions for each procedure were
different only to the extent necessary due to differences in the procedure to be
used. After successfully entering their password (or electing to stop trying after
five failed attempts), participants completed the post-test survey regarding the
procedure they had just used and answered general demographic questions.
Upon completion, disks were immediately returned to the researchers. Participants experienced minimal waiting time to receive their disks, and, once
received, there was no further delay.

Scale Purification

Following data collection, we used the pre-test survey responses to generate the measurement scales. First, all items hypothesized to reflect one of the
constructs were used to generate a unidimensional scale for each construct.
Items were eliminated based on changes in coefficient Alpha and their effect on
unidimensionality. A list of pre-test and post-test instrument items and their
status of inclusion or elimination is provided in Table 3.
The scale for PU retained seven items and had a coefficient Alpha of 0.91.
The scale for PEOU also retained seven items and had a coefficient Alpha of
0.80. In his classic work, Davis (1989) noted that this construct may contain three
clusters relating to physical effort, mental effort, and ease of learning. Consistent
with his findings, each cluster appears to be represented by two items in our
scale, with overall ease of use as the seventh item. Finally, the scale for BI
retained three items, with a coefficient Alpha of 0.71. All three scales were
unidimensional. Testing for discriminant validity between the three constructs
was performed with a Varimax rotation of all items in the scales, yielding clear
loading on the hypothesized construct without any cross-loadings (see Table 4).

RESULTS AND DISCUSSION

According to the TAM literature, PEOU and PU influence BI. Real-world
user authentication, however, is accomplished with password procedures that
are mandated by the organization or its technical staff, not adopted by user
choice. Password procedure usage results indirectly from users’ needs to use
computer systems. In addition, one of our procedures (COPS) is experimental
and not currently available for real-life application. Therefore, the full TAM
model, which includes actual use, cannot be evaluated. In this chapter, we will
limit our analysis to a discussion of differences between the four password
procedures in terms of PEOU, PU, and BI, but not actual use, which was
impossible to observe. To measure the effect of each procedure, we used
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

A TAM Analysis of an Alternative High-Security User Authentication Procedure 293

summated scales constructed from the responses to the individual items. Since
all items were worded positively, no reversal of negative scoring was needed.
The ANOVA model for PU revealed an absence of statistically significant
differences between the four procedures (p = 0.84).
Table 3. Pre- and post-test instrument items
Item
PU1
PU2
PU3
PU4
PU5
PU6
PU7
PU8
PU9
PU10
PU11
PU12
PU13
PU14
PEOU1
PEOU2
PEOU3
PEOU4
PEOU5
PEOU6
PEOU7
PEOU8
PEOU9
PEOU10
PEOU11
PEOU12
PEOU13
PEOU14
PEOU15
PEOU16
BI1
BI2
BI3
BI4
BI5

Pre-Test Instrument Item
Password procedures effectively protect my
confidential information.
Password procedures enable computers to limit
access to authorized users.
I find password procedures useful.
Using password procedures to maintain security
is a good idea.
Using password procedures is generally
important for using computers.
Using password procedures is important to me
for using computers.
Using password procedures enhances my
security when working with computers.
I find password procedures useful for protecting
my confidential information.
Using password procedures improves the
security of my confidential information.
Password procedures are an effective way to
maintain security.
Password procedures improve the security of
computers.
I find password procedures useful for limiting
access to confidential information.
Using password procedures for security is
important to me.
Overall, I find password procedures useful.
Using password procedures is an easy method
for maintaining security.
Using password procedures does not take too
much time.
Password procedures make it difficult to use
computers.
Password procedures make it easier for me to
maintain security.
When I use password procedures, computers
behave in unexpected ways.
Using password procedures requires a lot of
mental effort.
I can efficiently access computers even when I
must use password procedures.
I make mistakes frequently when I use
password procedures.
I find it easy to correct my mistakes while using
password procedures.
I often become confused when I use password
procedures.
It is easy for me to become skillful at using
password procedures.
Learning to use password procedures is easy for
me.
Passwords are easy to remember.
Passwords are easy to enter.
Using password procedures is frustrating.
Overall, password procedures are easy to use.
I do not mind using password procedures when
they are required.
Whenever possible, I will avoid using
computers that require password procedures.
I intend to choose password procedures for
security over other procedures if I am given a
choice.
I would prefer using computers that use
password procedures.
I intend to use password procedures.

Post-Test Instrument Item
This password procedure would effectively
protect my confidential information.
This password procedure would enable
computers to limit access to authorized users.
I would find this password procedure useful.
Using this password procedure to maintain
security would be a good idea.
Using this password procedure would be
generally important for using computers.
Using this password procedure would be
important to me for using computers.
Using this password procedure would enhance
my security when working with computers.
I would find this password procedure useful for
protecting my confidential information.
Using this password procedure would improve
the security of my confidential information.
This password procedure would be an effective
way to maintain security.
This password procedure would improve the
security of computers.
I would find this password procedure useful for
limiting access to confidential information.
Using this password procedure for security
would be important to me.
Overall, I would find this password procedure
useful.
Using this password procedure would be an easy
method for maintaining security.
Using this password procedure would not take
too much time.
This password procedure would make it difficult
to use computers.
This password procedure would make it easier
for me to maintain security.
When I used this password procedure, the
computer behaved in unexpected ways.
Using this password procedure would require a
lot of mental effort.
I could efficiently access computers even if I
must use this password procedure.
I made mistakes frequently when I used this
password procedure.
I found it easy to correct my mistakes while
using this password procedure.
I often became confused when I used this
password procedure.
It would be easy for me to become skillful at
using this password procedure.
Learning to use this password procedure was
easy for me.
This password was easy to remember.
This password was easy to enter.
Using this password procedure was frustrating.
Overall, this password procedure was easy to
use.
I would not mind using this password procedure
if it were required.
Whenever possible, I would avoid using
computers that require this password procedure.
I intend to choose this password procedure for
security over other procedures if I am given a
choice.
I would prefer using computers that use this
password procedure.
I intend to use this password procedure.

Status
Eliminated
Eliminated
Eliminated
Eliminated
Eliminated
Eliminated
Retained
Eliminated
Retained
Retained
Retained
Retained
Retained
Retained
Eliminated
Eliminated
Eliminated
Eliminated
Eliminated
Eliminated
Retained
Eliminated
Retained
Eliminated
Retained
Retained
Retained
Retained
Eliminated
Retained
Eliminated
Eliminated
Retained
Retained
Retained

TEAM LinG

294 Warkentin, Davis and Bekkering

The results show that end users do not perceive any procedure to be more
useful than another (see Table 5) but do consider passwords in general to be
useful, considering the high average mean scores for the our procedures (COPS
25.6, FIPS 26.0, Spafford 26.0, and Self 27.6 on a scale raging from 7 to 35). This
absence of differences between procedures may be related to the fact that using
passwords is generally mandated by system administrators rather than chosen
by users, but could also be related to lack of user awareness regarding the
protection that each procedure offers. Users generally are not aware of the ease
with which self-selected or Spafford passwords can be cracked. Similarly, users
generally do not reflect on the level of computing resources necessary to
compromise passwords with the number of permutations offered by FIPS and
COPS, and the amount of time required to be successful. In order not to influence
our participants and possibly skew the responses with the impressive differences
in the level of security provided by the different password procedures, we did not
provide any information regarding this issue.
There were significant differences (p = 0.00) between password procedures for PEOU, but only between the two groups with low and high levels of
security (and consequently different levels of effort needed to use them).
Considering our large sample of 352 participants, average summated scores for
Table 4. Factor analysis of PU, PEOU, and BI
PU7
PU9
PU10
PU11
PU12
PU13
PU14
PEOU7
PEOU9
PEOU11
PEOU12
PEOU13
PEOU14
PEOU16
BI3
BI4
BI5

Using password procedures enhances my
security when working with computers.
Using password procedures improves the
security of my confidential information.
Password procedures are an effective way to
maintain security.
Password procedures improve the security
of computers.
I find password procedures useful for
limiting access to confidential information.
Using password procedures for security is
important to me.
Overall, I find password procedures useful.
I can efficiently access computers even
when I must use password procedures.
I find it easy to correct my mistakes while
using password procedures.
It is easy for me to become skillful at using
password procedures.
Learning to use password procedures is easy
for me.
Passwords are easy to remember.
Passwords are easy to enter.
Overall, password procedures are easy to
use.
I intend to choose password procedures for
security over other procedures if I am given
a choice.
I would prefer using computers that use
password procedures.
I intend to use password procedures.

Component
PEOU

.799

.082

.171

.852

.028

.015

.803

.116

.003

.795

.161

.011

.790

.121

.016

.747

.117

.279

.782

.155

.204

.121

.570

.059

.046

.653

.028

.137

.711

.208

.130

.766

.181

.071
.045

.693
.530

.216
.127

.181

.626

.371

.102

.236

.620

.081

.220

.812

.113

.226

.797

TEAM LinG

A TAM Analysis of an Alternative High-Security User Authentication Procedure 295

Table 5. ANOVA for PU
Tests of Between-Subjects Effects
Dependent Variable: Perceived Usefulness
Source

Type III
Sum of Squares

200.966(a)

66.989

2.236

.084

243,442.351

8125.086

.000

2.236

.084

Corrected Model

243442.351

Intercept

200.966

Mean Square

66.989

Error

10426.713

348

29.962

Total

254071.000

352

10627.679

351

Password Procedure

Corrected Total

Sig.

R Squared = 0.019 (Adjusted R Squared = 0.010)

COPS (21.82) and FIPS (22.0) are remarkably close, and the difference is
statistically negligible (p=.995). The average scores for the lower-security
Spafford (29.38) and self-selected passwords (29.41) are much higher than their
high-security counterparts and not (p = 1.00) different from each other. Interestingly, the specific forms of the password procedures used appear to be
irrelevant (see Table 6).
The same pattern (p = 0.00) can be found in the average summated scores
for BI. Users are more inclined to use self-selected (11.1) or Spafford (10.46)
passwords than either COPS or FIPS, but have no clear preference (p = .443)
between the self-selected and Spafford passwords. Users are less inclined to use
either COPS (8.29) or FIPS (8.84), but without a clear preference (p = .591)
between those two procedures. The pattern is evident from Table 7. Of equal
interest to system administrators may be the response to item BI1 (“I would not
mind using this password procedure if it were required”), not included in the
scale. The average summated score for COPS (3.17) is not statistically
significantly (p= .056) different from FIPS (3.53).
In contrast with differences in PEOU and BI, examination of demographic
information revealed no influence of gender (p = .186), or computer ownership
Table 6. Difference of means for PEOU*
Password Procedure

COPS

21.81

FIPS

22.00

Tukey HSD Spafford
Self-selected
Significance (Alpha =. 05)

Subset 1 Subset 2

29.38

29.41
.995

1.000

* Scale of 7-35, with higher scores indicating higher PEOU.

TEAM LinG

296 Warkentin, Davis and Bekkering

Table 7. Difference of means for BI*
Password procedure

COPS

8.29

FIPS

8.84

Tukey HSD Spafford
Self-selected

Subset 1 Subset 2

10.46

Significance (Alpha=.05)

11.11
.591

.441

*Scale of 3-15, with higher scores indicating higher BI

(p = .244) on intent to use any of the procedures. Finally, performance measures
revealed that COPS and FIPS require a statistically equal (p = .29) number of
attempts to successfully enter the correct password (3.17 and 2.83 attempts,
respectively), and that users are equally likely (p = .92) to abandon the procedure
after repeated failed attempts. We did find a statistically significant difference
in completion time for each attempt, between COPS (84.0 seconds) and all other
procedures (Spafford 9.3 seconds, self-selected 10.3 seconds, FIPS 17.9
seconds). Indeed, participants required substantially more time (over a minute
longer) to login using the COPS password than any of the other passwords. This
finding is not shocking, however, in light of the fact that the Spafford, selfselected, and FIPS passwords involve the familiar task of entering characters
in a textbox using a keyboard, while COPS involves a new task of selecting
checkboxes using a mouse. We surmise that the time required to login using a
COPS password would significantly diminish over time as users acquire practice
and familiarity with the task. Moreover, it is interesting that the substantial
difference in completion time did not result in a significant difference in PEOU
between COPS and FIPS, which may indicate that the difficulty of logging in
using COPS is commensurate with the difficulty of logging in using FIPS, albeit
for different reasons. In the case of FIPS passwords, the difficulty of remembering odd characters such as the vertical bar ( | ) and tilde ( ~ ) and locating
them on the keyboard presumably reduced its PEOU scores.
Having empirically established that for high security systems COPS and
FIPS are not distinguishable from a usability point of view, we will briefly discuss
the security levels afforded by both. Initially, COPS and FIPS seem to afford a
fairly comparable level of security, considering the total number of combinations
(7.2 x 1016 vs. 6.7 x 1015). This similarity is illusory, however, if users of FIPS are
allowed to select their own passwords to make them easier to remember.
Theoretically, this applies even more if restrictions other than minimum password length are placed on password selection, since each restriction decreases
the total search space for a given password length. Character-selection restric-

TEAM LinG

A TAM Analysis of an Alternative High-Security User Authentication Procedure 297

tions reduce the search space of combinations by a very significant amount, in
some cases by more than 90%. For instance, the FAA can be considered to be
an organization with high security requirements, especially after September 11,
2001. Government documents are no longer freely posted on the World Wide
Web (WWW), but a copy of the FAA Policy on Password Administration (2002)
is cached on Google. Restrictions on password selection include using at least
three of four characteristics: two or more numeric characters, two or more
upper-case non-numeric characters, two or more lower-case non-numeric
characters, and one or more special characters. For the best combination, the
search space of combinations decreases by an astonishing 99.8% (953x33x26 4/
958 = 1.3 x 1013/6.6 x 1015 = 0.19% of original combinations left).
Computers continue to increase in speed. Soon, we may reach the point
where a reduced search space can be effectively covered in a brute force attack.
At that point, system administrators are no longer able to balance security and
usability concerns, and allowing users to select their own passwords may not be
an option. Before we reach that point, easier-to-remember passwords such as
COPS need to be researched and introduced in the field.

CONCLUSION

This study has demonstrated that end users perceive password procedures
equally useful regardless of the specific procedure used. Further, though the
study also shows that users perceive easy-to-remember passwords as easier to
use than high security passwords and, therefore, are more inclined to use them,
the mathematical and practical difficulty of hacking the high-security password
procedures makes them more attractive to system administrators. A classic
tradeoff is found to exist between ease of use and effectiveness of various
password systems. Procedures that are easier for end users likely are also easier
for hackers to compromise.
The study establishes that at least one acceptable alternative exists for the
current guidelines for high security passwords as defined in FIPS 112—namely,
the Check-Off Password System (COPS). If system administrators wish to use
password procedures with protection levels equal to or exceeding the high
security guidelines of FIPS 112, passwords from the full 95-character set should
be assigned and not chosen, or an alternative password procedure such as COPS
should be used. COPS can provide a secure alternative to FIPS passwords,
which are difficult to remember for many users. COPS is more secure than FIPS
by a factor of 10 (7.2 x 1016 vs. 6.7 x 1015)), and, despite the additional time
required to enter a COPS password, PEOU is equal for both procedures.
Moreover, as the number of passwords per user increases (because the number
of secure systems they must access multiplies), improved memorability becomes
an increasingly strong advantage of COPS passwords.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

298 Warkentin, Davis and Bekkering

Future research efforts should include conducting longitudinal studies to
establish the efficacy of COPS for more realistic applications in which passwords are used to access systems repeatedly over time. In particular, future
studies should examine whether training end users how to use COPS passwords
has an impact on its PEOU, and whether the time required to login using the
COPS password procedure significantly decreases with proper training and
repeated usage to the point where the login time is comparable to that of the FIPS
password procedure. Will users be more able to remember the COPS password
than the alternative high-security password (FIPS) for realistic periods of time?
The PEOU and PU of COPS also should be evaluated from the perspective
of system administrators, who are ultimately responsible for the selection and
adoption of password procedures to protect the systems they manage. Because
system operators are more knowledgeable about security risks and the degree
to which security measures may be compromised using available cracking tools,
we might expect system operators (as opposed to end users) to appreciate more
the increased security afforded by COPS passwords, which may lead to a
significantly higher PU for COPS than traditional password procedures. Future
studies could also examine a lower-security version of COPS by reducing the
number of check-off boxes on the interface, which would reduce the cognitive
load of the input mechanism, while still providing greater practical security than
self-selected passwords.

ACKNOWLEDGMENT

This research is supported by the Mississippi State University Center for
Computer Security Research and funded by the U.S. National Security Agency
(NSA), grant number DUE-0209869.

REFERENCES

Adams, A., & Sasse, M.A. (1999). Users are not the enemy. Communications
of the ACM, 42(12), 40-46.
Ames, B.B. (2002). PC developers worry about security. Design News, 57(16), 29.
Anderson, J.R. (1994). Cognitive psychology and its implications. New York:
W.H. Freeman.
Bergadano, F., Crispo, B., & Ruffo, G. (1998). High dictionary compression for
proactive password checking. ACM Transactions on Information and
System Security (TISSEC), 1(1), 3-25.
Bolande, H. (2000). Forget passwords, what about pictures? Retrieved September 18, 2002, from http://zdnet.com.com/2102-11-525841.html

TEAM LinG

A TAM Analysis of an Alternative High-Security User Authentication Procedure 299

Boroditsky, M., & Pleat, B. (2001). Security @ the Edge—Making security and
usability a reality with SSO. Passlogix. Retrieved September 18, 2002,
from http://www.passlogix.com/media/pdfs/security_at_the_edge.pdf
Burrows, J. H. (1985). Password usage: Federal information processing standards publication 112. National Institute of Standards and Technology.
Retrieved September 18, 2002, from http://www.itl.nist.gov/fipspubs/
fip112.htm
Coates, A.L., Baird, H.S., & Fateman, R.J. (2001). Pessimal print: A reverse
turing test. Proceedings of the Sixth International Conference on
Document Analysis and Recognition, Seattle, WA.
Davis, F.D. (1989). Perceived usefulness, perceived ease of use, and user
acceptance of information technology. MIS Quarterly, 13(3), 318.
Federal Aviation Administration (FAA), U.S. Department of Transportation
(2002). Password administration (Internatl document N 1370.38, dated 3/
25/2002, cancelled 3/24/2003). Retrieved April 8, 2003, from http://
216.239.39.100/search?q=cache:UcqXFyMXIPYC:www2.faa.gov/
aio/common/documents/HTMLfiles/N137038.htm+faa-8 (originally at
http://www2.faa.gov/aio/common/documents/N137038.pdf)
Gates pledges better software security. CNN.com/Technology. Retrieved
January 21, 2003, from http://www.cnn.com/2003/TECH/biztech/01/25/
microsoft.security.ap/
Hewett, T.T. (1999). Cognitive factors in design (tutorial session): Basic
phenomena in human memory and problem solving. Proceedings of the
Third Conference on Creativity & Cognition, Loughborough, United
Kingdom.
Jianxin, J.Y. (2001). A note on proactive password checking. Proceedings of
the 2001 Workshop on New Security Paradigms, Cloudcroft, NM.
Lemos, R. (2002). Passwords: The weakest link? Retrieved September 18, 2002
from http://news.com.com/2009-1001-916719.html
Miller, G.A. (1956). The magical number seven, plus or minus two: Some limits
on our capacity for processing information. Psychological Review, 63, 8197.
Monrose, F., Reiter, M.K., & Wetzel, S. (1999). Password hardening based on
keystroke dynamics. Proceedings of the 6th ACM Conference on
Computer and Communications Security, Singapore.
Newell, A., & Simon, H.A. (1972). Human problem solving. Englewood Cliffs,
NJ: Prentice-Hall.
Password usage survey results: June 2003 (2003). SafeNet. Retrieved August
7, 2003, from http://mktg.rainbow.com/mk/get/pwsurvey03
Pinkas, B., & Sander, T. (2002). Securing passwords against dictionary attacks.
Proceedings of the 9th ACM Conference on Computer and Communications Security, Washington, D.C.

TEAM LinG

300 Warkentin, Davis and Bekkering

Spafford, E.H. (1988). The Internet worm program: An analysis (Purdue
Technical Report CSD-TR-823). West Lafayette, IN: Purdue University.
The twenty most critical Internet security vulnerabilities (updated): The experts
consensus (2002). SANS. Retrieved May 2, 2002, from http://www.sans.org/
top20.htm
The twenty most critical Internet security vulnerabilities (updated) (2003).
SANS. Retrieved August 7, 2003, from http://www.sans.org/top20
Ticketmaster—Read this first (2003). Ticketmaster. Retrieved January 23,
2003, from https://www.ticketmaster.com/checkout/reserve
Weirich, D., & Sasse, M.A. (2001). Pretty good persuasion: A first step towards
effective password security in the real world. Proceedings of the Workshop on New Security Paradigms, Cloudcroft, NM.
What is the frequency of the letters of the alphabet in English? (2002).
AskOxford.com. Retrieved September 29, 2002, from http://www.ask
oxford.com/asktheexperts/faq/aboutwords/frequency

TEAM LinG

A Blended Approach Learning Strategy for Teacher Development 301

Chapter XV

A Blended Approach
Learning Strategy for
Teacher Development
Kalyani Chatterjea, Nanyang Technological University, Singapore

ABSTRACT

In-service upgrading has been an accepted avenue for retraining practicing
teachers in Singapore to keep abreast of changing curriculum requirements
as well as infusion of information technology (IT) in teaching and learning.
To cope with the teachers’ busy work schedules and many school
commitments, upgrading courses were offered to the teachers primarily
asynchronously, using the Internet platform with some integrated
synchronous sessions. This chapter analyzes the rationale for the
development of such a Web-based teacher-upgrading program and discusses
the main issues of professional upgrading addressed in the development.
Issues of adult learning in a learner-controlled adaptive learning
environment and lifelong learning were addressed through an IT-infused
asynchronous mode, providing the much-needed freedom in time
management for the course participants. The development also includes
delivery of high definition graphics through a customized hybrid system of
CD-ROM and Web that addresses image-downloading bottleneck and
thereby overcomes a basic problem of distance learning in geospatial
education. Finally, reflections on the attending adult learners’ responses
to such an upgrading program are discussed.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

302 Chatterjea

INTRODUCTION

Learning through distance education using computer-mediated environment
has become not only an accepted norm, but also a necessity in the field of
retraining working professionals. As McIssac and Gunawardena (1996) put it,
it currently has become the fastest growing form of education. The fast flow of
information and the resultant rapid change in all disciplines have made it
necessary for continuous upgrading of all professionals. With the workforce
engaged in work-related commitments, distance learning using computers has
enhanced the learning environment whereby the learners have access to
upgrading programs without having to physically go for the courses and yet being
in constant contact with the facilitator as well as the other participants. In
response to such demands for training and available technological infrastructure,
there is a growing trend for blended-approach upgrading courses. One such
example of matching the upgrading requirement with the available technology is
the teacher training courses offered through the World Wide Web (WWW) to
the teachers in Singapore. Singapore’s IT thrust started with the effort to
integrate information technology in education under the first IT Master Plan in
1997; a corollary to the introduction of IT-infused learning is teacher education
and training, not just for the sake of learning IT skills, but also to incorporate IT
into their overall training programs. This is where the in-service upgrading
programs fit in. Each year, the teachers attend courses in content and pedagogy
to keep abreast of the latest developments while still continuing their school
commitments. The courses in discussion were a part of such upgrading of
teachers and were delivered for content upgrading between 1999 and 2001 via
the WWW. This chapter analyzes the rationale for the development of such a
Web-based distance teacher retraining program, discusses the strategies adopted
to align discipline and training requirements with the available technology, and
reflects some on how the Web-based retraining program was used by the
participating teachers. The latter reflects to a certain extent on how computermediated learning environment influences upgrading of working professionals.

SPECIFICS OF THE SINGAPORE
CASE AT HAND

The following excerpt from the Year 2000 Teacher Training Prospectus of
the Ministry of Education (MOE, 2000), Singapore, underlines the teacher
training initiatives that have been a part of the in-service teacher training scene
in Singapore.

TEAM LinG

A Blended Approach Learning Strategy for Teacher Development 303

From Year 2000, core upgrading for teachers will be implemented. Each
teacher will need to attend at least three core activities within each 5-year
cycle, starting from 2000. The aim is to enable teachers update themselves
on the latest developments in education and to upgrade their teaching skills
or content so as to stay relevant and competent in their profession. (Ministry
of Education, 2000, p. v)

PRESENT TREND IN
UPGRADING PROGRAMS

As a rule, most in-service courses for the teachers are requested by the
Ministry of Education, Singapore, and offered by the National Institute of
Education (NIE). Traditionally, these courses are conducted synchronously at
the NIE campus, although this is slowly changing. In 2002, NIE offered as many
as 150 in-service courses, 35 of which were Web-based. While physical distance
is not a real problem in Singapore, one reason for this move toward delivery via
the World Wide Web is to free the participating teachers from a fixed time
constraint. This series of three courses, named Geomorphology Online, was
delivered as stand-alone upgrading courses between 1999 and 2001. The main
objective was to support content upgrading of the participating teachers. The
courses were designed for a blended-approach, dual-mode delivery, incorporating asynchronous Web-based as well as synchronous face-to-face sessions.
Following Johanssen et al.’s (1991) 4-square map of distance education technology options, this present delivery could be described as combining the Different
Time/Different Place Instruction and the Same Time/Same Place Instruction to
provide not only the freedom of access but also the much-needed laboratory and
field experience for a course in Geomorphology. Through the provisions of online
content delivery and online communication coupled with the integrated face-toface sessions, the course aims to deliver what McIssac and Gunawardena (1996)
describe as “individualized and collaborative learning” in a combined setup of
both distance and traditional education (McIssac & Gunawardena, 1996, p. 403).

WHY USE THE INTERNET PLATFORM?

Freedom From Time and Space Constraints. Prior to 1999, in-service
courses were delivered through synchronous sessions where participating
teachers had to attend on-campus classes for three hours every Saturday for 10
consecutive weeks. But a survey done by Chatterjea and Goh (1999) revealed
that many of the participating teachers had too many concurrent work commitments, even though classes were held on Saturdays, a day of no regular lessons
in Singapore schools. This affected the attendance in the on-campus teacher

TEAM LinG

304 Chatterjea

upgrading courses and, in turn, affected the learning outcomes. As a countermeasure, the author offered to deliver the required upgrading courses using the
Internet platform to allow the flexibility of time, so that the teachers could access
the lessons anytime and anywhere, in spite of the many concurrently running
school activities. Each of the developed Web-based courses involved 18 hours
worth of online self-study and four three-hour sessions (total of 12 hours) of
integrated on-campus lab work. This reduced the number of visits to the campus
for the teachers and was clearly an advantage over the mandatory attendance
required in a synchronous course. As Porter (1997) points out, this facility of
learning anytime and anywhere, using the Web-based platform creates a
congenial environment for lifelong learning, and the concept of lifelong learning
becomes acceptable. Following Flowers and Reeve (2000), it may be emphasized here that the provision of such an online course offers an answer to the
constraints of physical distance and time. While considerations of space constraints, higher student population, and the like might be the driving forces for
opting for online courses in some universities (Pallof & Pratt, 1999), the
Singapore teacher education scene is filled with other factors. Here, providing
a continuum of knowledge upgrading with minimal infringement on the existing
teacher responsibility and commitment is the prime mover, and, from this
perspective, provision of online courses for teacher development seems to be one
of the best options the in-service service providers have.
IT Immersion: The second objective for introducing an asynchronous,
Web-based training package is to initiate teachers into an IT-immersed training
environment. Use of IT in education is one of the mainstays in the education
environment of Singapore. The first IT Masterplan of Singapore aims to provide
not only a pupil-computer ratio of 2:1 and a teacher-computer ratio of 2:1 by the
end of 2002, but it also aims to set aside 30% of the curriculum time for IT-based
curriculum (MOE, 2002). With this great emphasis on the use of information
technology in every aspect of education in Singapore, this initiative of Web-based
teacher upgrading seems most timely. Even though expressions such as “get
online or get out,” “virtualize or die” (Flowers & Reeve, 2000) are too drastic,
information technology pervades the educational scene in Singapore. Taking part
in an online course provides the much required IT proficiency and, therefore,
relevance to the participants. Moreover, since this upgrading tool is accessible
to all teachers, older teachers who have not grown up in the present IT-infused
environment can take this as an opportunity to narrow the gap between
themselves and their younger colleagues. In this respect, it may be apt to say that
while online delivery and its appropriateness is being debated in forums, in the
Singapore teacher training context, delivery of online upgrading programs seems
like a very appropriate learning technology therapy (Flowers & Reeve, 2000). In
response to the Ministry of Education’s thrust on introducing IT in all fields of

TEAM LinG

A Blended Approach Learning Strategy for Teacher Development 305

education in Singapore, NIE is presently mounting as many as 35 Web-based
courses under advanced diploma, advanced post-graduate diploma, and masters
degree programs, especially catering to the needs of the practicing teachers. In
addition, there is an increasing number of stand-alone Web-based in-service
courses (e.g., Geomorphology Online) that do not lead to formal qualifications
but are offered to teachers for their discipline area and pedagogic skills
upgrading. These initiatives come as a planned response to the needs of the
upgrading of teachers. The present provision of lifelong learning using the Internet
platform is a part of this ongoing strong thrust into technology-mediated learning
in Singapore, which particularly suits the needs of working professionals.

DEVELOPMENT CONSIDERATIONS

The Web-based courses on Geomorphology for the secondary school
teachers were developed with the following considerations:
1.

3.
4.

The courses need to be goal-oriented and adaptive in order to cater to
the varying requirements of the participating teachers. Participants in such
courses generally fall under two categories: (1) those with some years of
teaching experience but requiring upgrading of knowledge and (2) those
new to the profession that might lack the experience and sometimes the
specific knowledge base required to cope with the school syllabus (Chatterjea
& Yee, 1997). To cater to these disparate teacher-requirements, in-service
courses need to allow adaptive, non-linear access to information upgrading
rather than a set path for all. An online course can offer this provision
without much difficulty, which is a clear advantage over the traditional
classroom-based system of instruction.
The upgrading course should allow avenues of communication. Many
teacher participants, though experienced, may have reservations in a faceto-face exchange of ideas and, as pointed out by S. W. Schimdt (2000),
might find it daunting to speak up. Bulletin board discussions incorporated
in a distance learning setup removes this barrier, and academic discourse
is more possible than the face-to-face session.
A Web-based course with the option of infinite repetitions on demand
offers the right degree of freedom to teachers for effective learning without
making them feel left out in a fast-progressing class.
An asynchronous intervention provides freedom to the learner, not just in
terms of modes of learning, but also in temporal terms. The full-time
teachers, in spite of their other work-related responsibilities, can access the
course in their own spare time. This gives them better control over their
learning process.

TEAM LinG

306 Chatterjea

MAIN ISSUES ADDRESSED
IN THE DEVELOPMENT

To cater to the specific requirements of the working professionals, online
courses were offered to provide freedom from fixed-time, fixed-format, and
fixed-paced classes. While the course involved asynchronous online delivery and
guidance on attaining content upgrading and asynchronous online communication
for exchange of ideas, it also included integrated synchronous face-to-face
sessions to cover the lab-based segment of the training program and to keep all
participants on track. The five main issues addressed in the development are (1)
lifelong learning, (2) enhanced visual learning, (3) dynamic knowledge building,
(4) asynchronous knowledge sharing, and (5) personalized guidance during
integrated face-to-face sessions. Together, these features aim to provide
freedom of learning options through the use of online mode, while maintaining the
face-to-face sessions to provide subject-specific lab-based skills and knowledge.
Lifelong Learning Through Learner Control. The primary objective of
this course was to provide a continuum of content upgrading to geography
teachers in an environment of rapid changes and adjustments to the school
syllabus. This rapid and regular updating of school syllabi and the pedagogic
strategies through the use of information technology could be likened to the
“Permanent White Water” metaphor used by Vaill (1996), where the participants are constantly confronted with a continuously changing stream of developments. The players in this system constantly are required to adjust to these
changes through various programs, initiatives, or other learning strategies to feel
sufficiently updated both in content and in the technologies. In this respect, the
in-service course in question tends to support Vaill’s definition of learning, which
refers to “changes a person makes in himself or herself that increase the knowwhy and/ or the know-what and/ or the know-how the person possesses with
respect to a given subject” (Vaill, 1996, p. 21). This constant wave of changes
makes the issue of lifelong learning not just an awareness, but a reality and a
requirement to stay relevant. This course development focuses on this issue of
lifelong learning and offers an environment to engage in self-directed learning to
sustain lifelong learning. Teachers who participated in the in-service course
were exposed to a learner-controlled environment. Instead of providing all
required information and explanations on the Web page, two main textbooks
were adopted. For each point of discussion, a guided reading list from these two
books and from other published sources (i.e., journals, newspaper clips, relevant
Web sites) were provided at the bottom of each page (Figure 1). All this
information is given in a window set aside for this information on the desktop.
This provision is meant to initiate a “guided discovery” (Boyle, 1999, p. 49)

TEAM LinG

A Blended Approach Learning Strategy for Teacher Development 307

Figure 1. A sample page from the courseware
A sample page from the course

approach placing the primary emphasis on the learner. The role of the courseware
is to create an effective environment for learning that goes beyond the temporal
restrictions of the course. Once introduced to finding information from books and
other sources, it was hoped that the teachers would continue to do so and thus
extend the learning beyond the limits of the course. It is designed optimally to
support learner-oriented strategies, as the learning experience is defined by the
learner.
Another unique feature of this reading window is that the information is
delivered via the WWW. This gives it a dynamic nature allowing immediate
update in response to a relevant event. For example, if there is an event of a flood,
an earthquake, or a similar occurrence anywhere in the world that can be used
to illustrate the points on the Web page, the relevant readings can be added
immediately to this reading list. This adds relevance and dynamism to the course
materials and also shifts the onus of learning to the learner as he or she now has
to deal with the option of taking advantage of the available material. The
facilitator’s job here is only to make the source of information known. This shifts
radically from the traditional system of providing all information on the Web site,
which often leads to inactivity on the part of the course participant. This has an
added relevance in teacher education, as teachers, being facilitators themselves,
often need to be reminded of their role as independent learners.
Instantly Delivered Image-Rich Learning Environment. Branden
(1996), while reviewing the literature on visual illustrations, pointed out the
importance using visuals in conjunction with text materials. Peek (1974) emphaCopyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

308 Chatterjea

sized the same. When pictures and texts are used together, retention is
facilitated, and illustrations facilitate delayed recall, according to Peek (1974).
Pictures have been said to arouse interest, set mood, arouse curiosity, make
reading more enjoyable, and create positive attitudes toward subject content and
toward reading itself. Levie and Lentz (1982) reviewed results of 155 experiments and found on the average that group scores for the illustrated-text groups
were 36% better than for text-alone groups; they commented that learning is
better with pictures in most cases. Illustrations can help learners understand and
remember what they read, enhance learner enjoyment, and evoke affective
reactions. Race (1992) lists 10 reasons for making learning packages visual.
Visual images are particularly useful and, in some environments, indispensable
to explain spatial and process relationships, to synthesize, to develop contextual
backgrounds, and so forth. Geography, especially physical geography, is a
discipline that almost demands an image-enriched environment for effective
learning and knowledge assimilation. In a scenario of asynchronous learning,
where the learner is physically separated from the lecturer, visuals are even
more important to bring concepts to the learner. Textual delivery offers a
learning material that, at its best, has to fight for attention from the learner. As
Subbaro and Langa-Spencer (2000) exclaim, “Text loading cannot be turned
off!” (p113). In comparison, “pictures and illustrations catch the eye. They
enhance the attractiveness of the system and (more importantly) deepen the
information channel” (Boyle, 1997, p. 151). Visual images play a very important
role in establishing links in the concepts. In the absence of the lecturer, an
increased use of visual images helps to explain the points in question and captures
the attention of the learner. As Race (1992) points out, illustrations can set a
scene for the learner so that the idea unfolds and helps the learner to see the
broader scene. This is especially important in an asynchronous Web-based
environment, where the learner is left with the visual display to assimilate a
concept from the information depicted on the screen. Visual images also play a
great role in bringing the outside world into the classroom or, in this case, onto
the monitor. Many research papers have emphasized the importance of the use
of visual images (Broudy, 1987; Gardner, 1993; Hutton & Lescohier, 1983; Race,
1989; Salomon, 1997). Thus, visual images were considered as one of the key
requirements during the planning stage of this developed courseware in order to
inject the visual reinforcement while the learners dealt with the learning
themselves.
The decision of creating an image-intensive learning environment on the
Internet platform has its almost insurmountable problems. In spite of much
advancement in software technology in image compression, high-definition
image delivery still makes the downloading of visual information excruciatingly
slow, even via broadband delivery. The meager use of visuals on the majority of
Web pages merely points blatantly to this constraint. Subbaro and LangaSpencer (2000), while discussing the problems of slow downloading on the Web,
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

A Blended Approach Learning Strategy for Teacher Development 309

have painted quite a grim picture of the constraints that Web course designers
face and the constraints and the restrictions with which they have to operate.
Reiterating a study done by the Graphics, Visualization and Usability (GVU)
Centre at Georgia Tech (Subbaro & Langa-Spencer (2000), they quite rightly
almost warn developers that download times of Web pages needs to be strictly
monitored and, if necessary, “ruthlessly scrutinized” (p. 113). They also suggest
that images only should be used if they are absolutely indispensable, and, when
given, images preferably should be given with <ALT> coding. This would allow
the reader the option of bypassing the whole image to save time. We often see
thumbnail-sized images on Web pages with an option for clicking on them to see
the full image. As Race (1989) points out, such thumbnails are not effective as
illustrations. Therefore, such compromises do not serve the intended purpose.
The author is of the opinion that such a system fails to deliver pedagogically sound
courseware. After all, if something may or may not be used at all, the purpose
of it being there is not served. The fact that an image is necessary for the
illustration of the point makes it mandatory that the image is actually seen by the
learner. Otherwise, the image should not be included at all.
With this conviction, the present courseware planned to incorporate as
many visual images (i.e., maps, diagrams, photographs, graphs, toposheets, etc.)
as was deemed necessary to explain the concepts. The idea was that a distance
learner in no way should be disadvantaged by not coming to a face-to-face class
where the use of slides, maps, and diagrams are constrained only by time and not
by technology. The choice of Macromedia Authorware as the authoring tool for
this courseware was based on its ability to allow a hybrid development where the
textual material is delivered via the Web, while visual images are all delivered
from a CD-ROM. The CD-ROM works essentially as a storage space for all
image library files and is accessed by the program to extract the right images at
the right time. This effectively eliminates all waiting time for the numerous
graphic files that are used heavily in the explanations of the various geographic
Table 1. Comparison of downloading times for a 30K file using different
downloading devices
Image delivered
via the Web

Image delivered
from the CD-ROM

Device Used

14.4 kbs
modem
28.8 kbs
modem
56 kbs modem
ISDN
connection
T1 line
16X CD-ROM
Drive

Time taken
to
download
(sec)
20.9
15.54

Source of
Information
Subbaro and
Langa-Spencer,
2000

7.78
2.81
1.1
Instant

Author’s own
records

TEAM LinG

310 Chatterjea

concepts. Table 1 gives a comparison of downloading times of graphic files when
they are delivered via the Web and clearly shows the edge this present hybrid
courseware has over the existing system of delivery.
In comparison, a 2.4Mb mpeg format file delivered from the courseware
CD-ROM takes three seconds to load with a 16X CD-ROM drive, which is well
within the usual time any user is willing to wait for loading a page, as mentioned
by Subbaro and Langa-Spencer (2000). This capability enables the courseware
to deliver what Novitzki (2000) calls a key issue in any asynchronous learning
program. According to him, material presented in a wide variety of ways,
including text, graphics, video, and audio, gains attention and accommodates
students with differing learning styles. In-service courses for teacher development needs to operate for a wide spectrum of teacher profiles, and much of the
success of such a program running asynchronously depends on how well it
addresses the issues of learner engagement. Therefore, the issue of facilitating
the delivery of a visually enriched learning environment was taken as one major
Table 2. Organization of the delivery of teaching materials in the
Geomorphology online courseware
Courseware
Components
Main teaching
concepts (texts)
Explanations
(texts)
Readings
(texts)
Notices and
announcements for
the course (texts)
Discussion facilities on
Bulletin Board (texts)
Hyperlinks (texts)
Additional
explanations in
response to participant
requests (texts).
Navigation buttons
such as page sliders
(graphics).
All video clips
All graphic files
showing graphs,
maps, diagrams,
photographs,
cartoons, as well as
the courseware
templates

Mode of Delivery
On the
From the
CD-ROM
Web

Comments on the Advantages
These are delivered dynamically and can be
added/appended/updated anytime. There are
provisions to add new explanation pages in
response to participant requirements. The
diagrams from the existing pages can be used
repeatedly with these updated pages.
The reading list is updated on the Web. This
provides the possibility of dynamic delivery of
information from news items, TV programs,
journals, and so forth.
This is a mode of communication with the
participants and is updated on the Web.
This discussion forum is asynchronous Webbased in nature.
This is updated on the Web.
These pages can be added on the Web, using
images from the CD-ROM.

These are given in the CD-ROM for instant
loading.
These are given in the CD-ROM for instant
loading. There is virtually no restriction on the
number of images used. The availability of
storage allows pictures of high quality.
Therefore, there is no compromise on the
illustrations, especially in the use of
toposheets, which require high resolution
images for clarity.

TEAM LinG

A Blended Approach Learning Strategy for Teacher Development 311

requirement for the courseware. Table 2 gives details of the mode of delivery of
materials in the developed courseware. It shows how the hybrid concept using
Web and CD-ROM has been used in the development of the courseware. The
main delivery on the Internet platform is not burdened by the graphic usage, and the
user gets an instant loading of the graphic files whenever a new page is called up.
The facility of ultra-quick loading as provided in Geomorphology Online has
several advantages:

•

Not only is there freedom from the network speed or lack of it, but also the
downloading of the graphics does not depend on the user’s hardware
capabilities, as there is very little difference in time taken by CD-ROM
drives of different speeds to open the graphic files.
It frees the courseware developer from the severe constraint in the use of
visuals. By doing so, it allows flexibility in using illustrations, an essential
element of a course in geospatial education. No longer does the learner
have to accommodate the restricted access to images, and no longer does
the developer have to hold back on the use of images, whether to illustrate
points or even to make it more visually attractive and cognitively sound. The
Web-based part of the courseware can be enhanced visually through the
use of the library files that are stored in the CD-ROM. This can create quite
a transformation of the way the Web-based courseware looks without
sacrificing the sanity of mind. Particularly for geography, where maps are
an essential teaching/learning element, such a system has proven to be
extremely useful. This element has put the asynchronous course truly at par
with the classroom-based courses of the past, since the use of huge-sized
visuals is no longer a hindrance.

Dynamic Course Material and Explanations. Text-based concepts and
explanations in the courseware are delivered via the Web. This gives flexibility
like any other Web-based course, where the material can be uploaded/updated/
appended at any time during the course. A prominent feature of the way this
material is presented is the lack of too much detail in the information provided on
the screen. Morkes and Nielson (1998) mention that only 16% of readers read
word-for-word from the screen. Novitzki (2000) also observes that many
students prefer reading from textbooks than from online materials. “No students
spend as much time reading material off the screen as either the instructor or
course designer believes” (Novitzki, 2000, p. 72). Readers, therefore, should be
guided in noting the important points instead of giving a lot of printed information
on the screen. To supplement this concise text, two textbooks are adopted; for
each point of discussion, the learner is guided to the relevant section of the books
(See Reading Window in Figure 1). Inclusion of blank spaces, short paragraphs,
subheadings, bulleted lists, and concise texts are all part of an effort to offer
enhanced readability to the course material.
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

312 Chatterjea

Another feature of the textual content pages is the predictability of scheme.
Charney (1994) and Subbaro and Langa-Spencer (2000) emphasize that mental
representations of the structure of texts are essential to comprehension. Texts
in the courseware are delivered following a standardized format with fixed color
codes and font types for the main point, explanations, informal notes, and
humorous comments. This latter element is a deliberate addition to the pages to
simulate classroom conditions and attempts to lighten the solitary environment of
an online session. The informality and spontaneity of language is aimed at
overcoming the lack of face-to-face contact and “is working to change academic
discourse” (Moran & Hawisher, 1998, p. 94).
There are provisions to add new pages for explanation or for updating
information as events take place. These additional pages can use the graphics
that are already provided in the library files in the CD-ROM. The usefulness of
this provision is that the appended pages can be used to provide additional
explanation in response to online discussions or e-mail queries from participants.
This feature of help-on-demand is one of the key requirements in a JIT (just-intime) training situation. Thus, the system is equipped to handle a full learnercontrolled learning environment with a facilitator providing the right amount of
support as and when necessary.
Online Forum for Discussions. To compensate for the reduced synchronous sessions, provisions are made for an online forum for discussion and
exchange of ideas. This provision is kept asynchronous, as opposed to live chat
sessions, to provide teachers the freedom of time. Teachers who have their own
work commitments cannot be expected to logon simultaneously. An asynchronous forum still allows the participating teachers the scope of exchanging their
ideas and reflections, even when they have concurrently running work commitments. The asynchronous forum also allows time for reflection. Another
additional advantage is that such asynchronous communication does not alienate
the shy participant. Additional lesson/explanation pages can be mounted in
response to forum discussions. This integration with the bulletin board discourse
to develop the course material goes in line with the JIT concept, and it also
provides greater learner control of the material. In-service teachers, being goaloriented, can make greater use of this facility and have the freedom to have a
say in the development of the material to suit their purpose. This fits in Pallof and
Pratt’s (1999) framework for distance learning by contributing to “active
creation of knowledge” through interaction and feedback, mutually negotiated
guidelines, shared goals, and collaborative learning.
Integrated Synchronous Sessions. The asynchronous online part of the
course is blended with some on-campus sessions at designated intervals. These
are held at fixed intervals after the participants are scheduled to cover given

TEAM LinG

A Blended Approach Learning Strategy for Teacher Development 313

sections of the course. This is planned with several objectives in mind. First, a
course in geomorphology involves, among other things, interpretation of aerial
photographs (which needs stereoscopes), identification of rocks and minerals
(which needs handling of rock and mineral samples), neither of which can be
done on the WWW. In addition, it is felt that some face-to-face interaction with
the facilitator and other participants is necessary in order to build a lasting rapport
as well as to gain from each other’s experiences. This is particularly important
as the teachers form part of a well-knit society of geography teachers in
Singapore. Therefore, the synchronous sessions aim to provide the specific
requirements of the discipline as well as the social needs of a learning community. However, they are aimed also at ensuring the success of the asynchronous
sessions. In any learning situation, there can be some participants who are less
enthusiastic about independent learning. If left to handle themselves through an
asynchronous mode, these participants tend to lag behind, and along the way,
they might totally abandon the course. The synchronous sessions at regular
intervals, with prerequisites of a certain degree of coverage of the course, can
help to force this group to keep pace with the rest of the class. The chances of
some participants not benefiting from the course thus are reduced.

Reflections on the Asynchronous Mode of Delivery

Geomorphology Online was the first venture into providing in-service
course in geomorphology on the Internet. Triggered by the observed problems
of running synchronous iIn-service courses and the continuing emphasis on using
IT for education, this course was set to break the tradition in providing avenues
of lifelong learning. On reflection, some points need to be mentioned that are
course development-related, participant-related, and developer/ facilitator-related.

Reflections on Course Development

The two most important features of the course at hand were (1) provision
for an image-enriched learning environment without any bandwidth constraint
and (2) an integrated Web cum face-to-face delivery. While the first was related
to subject requirements and learning effectiveness, the latter was related to
ensuring effective participation in the upgrading program.
Image-Enriched Learning Environment. Participant response on the
image-intensive delivery in the course was overwhelmingly positive. Use of
graphic illustrations was unanimously voted the most interesting aspect of the
course development. Participants also felt that it allowed more in-depth learning,
helped in understanding of difficult concepts, and illustrated issues (more)
effectively. From this respect, the objective of creating a near-classroom
learning environment was achieved. No technical hurdles (i.e., slow downloadCopyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

314 Chatterjea

ing even with a modem connection or during peak hours) were encountered to
get the image-rich course materials as this issue was tackled in the development.
The learner, therefore, was freed of the operating hurdles—an issue of great
importance when technologically-mediated learning systems are used.
Integrated Web Cum Face-to-Face Delivery. When asked about
negative points of Web-based courses in general, 54% of the participants
commented that there are scopes to stray on a fully Web-based course.
However, 90% of the participants said partly synchronous Web-based courses
such as the present one helped them to stay on track without drifting. When
asked about their preferred choice for future in-service courses, an emphatic
92% opted for an integrated course like the present one with integrated
synchronous sessions, while 8% opted for a full face-to-face delivery. That
makes a 100% learner demand for an integrated Web cum synchronous delivery
for those learners who opt for some IT-infused learning delivery system.
Obviously, the scope for straying was minimized through this integrated synchronous session and allowed the learners the required freedom to learn in their own
time and yet provide the compulsion to push through the course, in spite of work
commitments and other distractions. This learner response and preference set
the goal for future course deliveries where the Internet is used not just to impose
a new learning style, but also to use the facilities offered by the environment to
enhance and facilitate the learning and training of busy professionals who would
otherwise find it difficult to attend courses but also might stray from the course
for lack of personal contact. Adult individuals are mostly intrinsically motivated
toward learning, be it for induced professional upgrading requirements or for
personal motivations. Yet, work commitments and hours of lonely learning
environment are not seen as conducive to learning. A learning package that
integrates Web-based delivery with synchronous sessions creates an opportunity
to maintain the personal contacts and keeps the learners on track. Tracking of
the learners through the learning management system showed that the most
favored hit periods always were a few days to a day before the impending
synchronous sessions, supporting the view of the participants that the synchronous sessions helped them to be on track. Participants even talked about being
encouraged by the interest shown by the facilitator and other participants during
the synchronous sessions, something only achieved through personal contacts.

Reflections on Participants and Their Participation
On Web-Based Delivery. Web-based delivery of courses definitely has
caught on, as 31% of the participants mentioned they prefer Web-based courses;
23% said they are adjusting and beginning to like it; 46% said they can live with
it; and none were uncomfortable with it. This means that in an IT-infused society,

TEAM LinG

A Blended Approach Learning Strategy for Teacher Development 315

Table 3. Response of teachers to the various aspects of the course
Aspect of the Course
Focused and customized
learning
Lifelong learning through
reading guidance
Freedom from travel
Opportunity to learn at own
pace

Teacher Response (Out of 89 Participants)
90% said this suited them more than a broad-based
general course.
67% said customized explanations helped them
understand better
95.5% said the Web-based strategies of self-determined
scope of learning helped them to carry on learning
beyond the course and later
31% showed this as an incentive for joining a Web-based
course
46% said they think this is a positive point of the Webbased course.

learning via the WWW is accepted as a norm, even in groups where 23% of the
learners have been teaching for more than 15 years and, therefore, are from a
pre-Internet era in Singapore. Learning via the WWW has become, therefore,
more of a willing compliance and an accepted norm than an imposed strategy in
the Singapore environment. Table 3 shows the responses of the participating
teachers to the various aspects of the course and sums up their views on Webbased delivery.
Perception of Online Delivery. While the figures in favor of online
course delivery say volumes about the acceptance of such a mode of delivery,
it is quite clear that online learning is still equated with online content delivery.
A review of the participants’ login details reveals that the content areas of the
course were used far more than any others, and the use of communication areas
was minimal (Figure 2).
There appears to be a large disparity in the way online courses are perceived
and followed. First, the built-in flexibility of time, in a way, was abused with quite
a few teachers logging on only before the synchronous sessions. I was happy to
have the synchronous component that actually pushed these not-so-initiated
Figure 2. Areas of the courseware most frequented by the participants
Percentage of access
100.00%
80.00%
Content areas

60.00%

Communication areas

40.00%

Group areas
Student areas

20.00%
0.00%

Group 1 Group 2 Group 3

TEAM LinG

316 Chatterjea

teachers into doing some work. But that also meant they failed to get the benefit
from a system that is designed to guide them gradually to relevant learning
resources along the way. The reason cited by the participants was lack of free
time. Login records revealed that most teachers accessed the course during
weekends and at nights. In other words, they were using their personal times to
access the course. Although access to the course is available from the schools,
lack of time during or even after regular school hours prevents teachers from
being regular in their accesses. Although irregular login may not provide
regularity to the training program, login from home during weekends needs not
be seen as a drawback, as one of the initial objectives of developing this course
was to make it accessible at all times. From this perspective, asynchronous
delivery did allow the teachers to access the course at their convenience. By
their positive responses regarding freedom to learn at their own paces (Table 4),
the teachers meant that they do not need to do the learning modules regularly.
From this response, it appears that the learners hold a very different view of the
freedom to learn on their own. Obviously, they are not thinking of the offered
freedom to learn according to learner abilities or aptitudes. They simply are
categorizing this as a freedom from restricted time for learning. Perhaps this
aspect is particularly applicable for working professionals who have workrelated commitments and are mostly interested in acquiring the knowledge when
they have the time.
Online Communication. While some teachers used the bulletin board for
discussions, a large group did not. Even though additional explanations were
provided to answer specific questions, more could be achieved to provide helpon-demand. There could be two reasons for this pattern:
1.

Use of the bulletin board as a communication media is not accepted yet by
the vast majority of the learners. Participants feel more comfortable
discussing things in person and are not keen on opening up to strangers and
taking them as partners in the learning process. In a face-to-face situation
they still can go to the lecturer in person at the end of the class, but on the
bulletin board, everything is so open—what if I make a mistake—a
response from one of the participants.
Although nearly all participants mentioned during the survey that they opted
for the Web-based course and that they prefer it this way since it saves a
lot of time, the participants do not see online communication as a part of the
learning package.

The perception about this learning platform is clear from the survey, where
46% viewed that, in spite of the online discussion board, they do not feel there
is actual interaction in a fully Web-based delivery system.

TEAM LinG

A Blended Approach Learning Strategy for Teacher Development 317

I have learnt much from you esp the clarifications of concepts made during
class interaction [excerpt from teacher response].
Actually what I think most teachers need is the exchange and sharing of
ideas face to face, be it in content or methodology. We need some professional
stimulations esp after so many years of teaching. People like you would help us
to keep abreast with times in the developing of the subject and also to clarify grey
areas. [excerpt from teacher response].
While there was fervent discussion during the face-to-face sessions, there
was, at best, only coerced postings on the bulletin board from teachers and, even
then, mostly on facilitator-initiated issues.
On reflection, it may be said that at least a part of the reason for this
apathetic response was due to the fact that, even though the teachers had signed
up for a Web-based course, they still were not completely at the comfort level
of using an asynchronous online mode for all aspects of learning. In this respect,
it must be mentioned that all the teachers involved were computer-savvy, having
been trained and immersed in computer-mediated environment both at work and
at home for more than a decade; therefore, they had no problem using the ITinitiated media. However, it is the author’s conclusion that using computermediated communication for negotiating learning as well as for self-learning
were both very different from their usually held concept of retraining courses.
Therefore, in spite of the technical know-how and apparent acceptance of the
medium, computer-mediated learning through online communication has not
caught on yet among practicing teachers. However, it was encouraging to note
that at the end of the course, more teachers were logging on to discuss. It is
hoped, therefore, that with more courses that are run in this manner, in time,
teachers will be more receptive to the idea of learning on their own. Asynchronous interventions then would lead to a continuum of teacher education.

Reflections on the Developer/Facilitator and the
Development Environment

As pointed out by Novitzki (2000), the developer not only needs to be an
expert on the discipline, but also needs to have a sound working knowledge of the
software. In fact, much of the time in the beginning is spent not on answering
questions on the content but on the basic use of the courseware (i.e., installation
of Web players, etc.), the learning management system, and the login details. The
time spent on answering e-mail inquiries begins to encroach upon other work
commitments, and one begins to see the merit of the warning given by Rea et al.
(2000) that “students become increasingly demanding of the instructor’s time
[and] care must be taken to avoid becoming a full time consultant, advisor, and
assistant for the students” (in this case, the participating teachers) (p. 142).
Another aspect to be pointed here is that of the long development time for
a courseware. Even though the idea of infusing IT into the various programs is
Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written
permission of Idea Group Inc. is prohibited.

TEAM LinG

318 Chatterjea

generally looked upon positively, there is as yet no set guideline about the
operations. Therefore, there is no fixed workload offset for the developer. Since
any online course development requires a lot of development time, this only can
add up to the developer’s workload without much respite from the other usual
workload. Similar concerns are voiced by Robinson and Borokowski (2000). This
can be a problem and can affect the quality of the courseware. However, it is
expected that with more and more such courses offered, some norms will be
implemented.

CONCLUSION

In conclusion, it can be said that asynchronous intervention using the
Internet can be seen as a major mode of providing teacher development in the
21st century. It can satisfy the usual requirements of any adult learning situation
(i.e., available anytime and anywhere; a resource for learning just in time; a
platform for interaction, even when participants are physically separated; a
platform for introducing the participants to IT-enriched learning environment,
etc.). However, in case of flagging interest in lonely online courses, provision for
integrated face-to-face sessions like the ones offered in the present geomorphology courses is an effective encouragement to carry on in this asynchronous
learning environment. But the success of such an asynchronous course depends
also on its ability to simulate the positive learning environment of a classroombased system. Each discipline has its own requirement for knowledge dissemination. For a course in geospatial education, the ability of the courseware to
provide ample and unrestricted visual illustrations becomes an added advantage
when presented with the other usual basic requirements. This issue has been
handled adequately in the present courseware. The lessons learned for the
developer/course facilitator could be summarized as follows:
i)

Teachers, like other adult learners, do have intrinsic motivation, but they
expect to get only what they need. So, the course for retraining purposes
ideally should have the flexibility of providing just-in-time learning components. In this respect, IT-mediated systems can offer non-linear, randomly
accessible learning experiences at different levels suitable for such learners with prior experiences in the discipline. The teachers, being already
exposed to the content area, have a fair understanding of the discipline. Any
retraining program, therefore, has to offer only specific scaffolding to the
learner to provide the required upgrading instead of an entire curriculum.
IT mediation is one way of providing this just-in-time and randomly
accessible content material where the learners are free to access as much
as they need.

TEAM LinG

A Blended Approach Learning Strategy for Teacher Development 319

ii)

iii)

iv)

Although learning from remote locations, learners expect to get information
in much the same way as they do in campus-based schools. In this respect,
if the discipline requires an image-enriched environment, the Web-based
development should endeavor to incorporate that element in the development to maintain learner interest.
Freedom from routine learning schedules is seen as a major pull factor by
all participants in these courses. Although the expected style of regular
login to get the benefit of evenly-spaced-out learning did not happen, it is
seen as a useful element, especially since the courses were meant for busy
professionals whose time was too divided among other work commitments.
The fact that they accessed the course from home and even during
weekends says that the flexibility of time did help.
Up until now, the teachers involved have not seen computer-mediated
learning as a negotiating platform. But with time, as they attend more and
more such courses, this might change. For now, pitching online communication with some compulsory assignment might help generate more incentive to participate in online communication.
If distances permit (like in Singapore), having some synchronous sessions
do help to keep learners in line with the flow of the course, ensuring that
learners do not lag behind and, more importantly, ensure that the human
touch of being in contact with other participants remains.

Distance education programs continue to grow due to the ever-growing
global need for an educated workforce and; in this case, an ever-growing need
for updating teachers. Upon considering the trends in distance education, it is
clear that lifelong learning potential for adult learners in computer-mediated
environments will play a vital role as it allows for independent learning by
working adults. Along with the growing preference for a distance learning
environment, there is a great need to align learning programs with the needs and
styles of the learners; and a successful educational package should pay adequate
attention to the specific requirements of learners, their learning styles, technology of the delivery system, and interaction with the instructor. So far, the already
IT-initiated workforce either has shown preference or willing compliance to
computer-mediated retraining programs. But much of the preference, it appears,
still is limited to the advantages of accessibility of information without the spatial
proximity demands, as provided by the asynchronous setup. Training via the
WWW still is seen as a convenient resource base rather than a dynamic platform
for knowledge negotiation. The concurrently running workload is cited as one of
the hindrances of using the online discourse platform. Some judicious persuasion
seems to work, at best, in a passive way. But the traffic usually improves with
some built-in compulsion. This might lead the way to developing into willing
compliance much like the way Web-based lesson delivery once did.

TEAM LinG

320 Chatterjea

ACKNOWLEDGMENT

Development of Geomorphology Online is funded by NIE Research Fund
(Reference no. RP 18/98KC). The author thanks Mr. Supriyo Chatterjea for his
technical assistance in the development and incorporation of the hybrid system.

REFERENCES

Boyle, T. (1997). Design for multimedia learning. London: Prentice Hall.
Braden, R.A. (1996). Visual literacy. In D.H. Jonassen (Ed.), Handbook of
research for educational communications and technology (pp. 491520). New York: Macmillan.
Broudy, H.S. (1987). The role of imagery in learning [occasional paper 1]. Los
Angeles: The Getty Centre for Education in the Arts.
Charney, D. (1994). The effect of hypertext on the processes of reading and
writing. In C. Selfe & S. Hilligloss (eds.), Literacy and computers: The
complications of teaching and learning with technology (pp. 238-263).
New York: Modern Language Association of America.
Chatterjea, K., & Goh, K.C. (1999). In-service teacher training As a strategy for
geography teacher development in Singapore. Proceedings of the International Conference on Teacher Education, Hong Kong.
Chatterjea, K., & Yee, S.O. (1997). Lifelong learning in ASEAN: Singapore’s
in-service programmes in geography. In T.C. Wong & M. Singh (Eds.),
Development and challenge: Southeast Asia in the new millennium (pp.
239-256). Singapore: Times Academic Press.
Flowers, S., & Reeve, S. (2000). Positioning Web-based learning in the higher
education portfolio: Too much, too soon? In L. Lloyd (Ed.), Teaching with
technology (pp. 133-151). Medford, NJ: Information Today, Inc.
Gardner, H. (1993). Frames of mind. London: Fontana Press.
Hutton, D.W., & Lescohier, J.A. (1983). Seeing to learn: Using mental imagery
in the classroom. In M.L. Fleming, & D.W. Hutton (Eds.), Mental imagery
and learning (p. 157). Englewood Cliffs, NJ: Educational Technology
Publications.
Johanssen, R., Martin, A., Mittman, R., & Saffo, P. (1991). Leading business
teams: How teams can use technology and group process tools to
enhance performance. Reading, MA: Addison-Wesley.
Levie, W.H., & Lentz, R. (1982). Effects of text illustrations: A review of
research. Educational Communication and Technology Journal, 30,
195-232.
McIssac, M.S., & Gunawardena, C.N. (1996). Distance education. In D.H.
Jonassen (Ed.), Handbook for research for educational communications and technology. New York: Simon and Schuster.

TEAM LinG

A Blended Approach Learning Strategy for Teacher Development 321

MOE (2000). TRAISI, training for the new millennium, Prospectus 2000.
Singapore: Ministry of Education.
MOE (2002). Under implementation: Phases. Retrieved February 22, 2005, from
http://www1.moe.edu.sg/iteducation/masterplan/brochure.htm
Moran, C., & Hawisher, G.E. (1998). The rhetorics and languages of electronic
mail. In I. Snyder (Ed.), Page to screen (pp. 80-101). London: Routledge.
Morkes, J., & Nielson, J. (1997). Concise, SCANNABLE, and objective: How
to write for the Web. Sun Microsystems. Retrieved June 10, 1998, from
http://www.useit.com/papers/Webwriting/writing.html
Novitzki, J.E. (2000). Asynchronous learning tools: What is really needed,
wanted and used? In A. Aggarwal (Ed.), Web-based learning and
teaching technologies (pp. 60-78). Hershey, PA: Idea Group Publishing.
Pallof, R., & Pratt, K. (1999). Building learning communities in cyberspace. San
Francisco: Jossey-Bass Publishers.
Peek, J. (1974). Retention of pictorial and verbal content of a text with
illustrations, Journal of Educational Psychology, 66, 880-888.
Porter, L.R. (1997). Creating the virtual classroom distance learning with
the Internet. New York: John Wiley & Sons, Inc.
Race, P. (1989). The open learning handbook. London: Kogan Page.
Race, P. (1992). 53 interesting ways to write open learning materials.
Bristol, UK: Technical and Educational Services Ltd.
Rea, A., White, D., McHaney, R., & Sanchez, C. (2000). Pedagogical methodology in virtual courses. In A. Aggarwal (Ed.), Web-based learning and
teaching technologies: Opportunities and challenges (pp. 135-154).
Hershey, PA: Idea Group Publishing.
Robinson, P., & Borokowski, E.Y. (2000). Faculty development for Web-based
teaching: Weaving pedagogy with skills training. In A. Aggarwal (Ed.),
Web-based learning and teaching technologies: Opportunities and
challenges (pp. 216-226). Hershey, PA: Idea Group Publishing.
Salomon, G. (1997). Of mind and media. Ann Arbor, MI: Phi Delta Kappan.
Schimdt, S.W. (2000). Distance education 2010: A virtual odyssey. In L. Lloyd
(Ed.), Teaching with technology (pp. 75-90). Medford, NJ: Information
Today, Inc.
Subbaro, S., & Langa-Spencer, L. (2000). When less is more: Some ergonomic
considerations in course page design. In L. Lloyd (Ed.), Teaching with
technology (pp. 109-126). Medford, NJ: Information Today, Inc.
Vaill, P.B. (1996). Learning as a way of being. San Francisco: Jossey-Bass
Publishers.

TEAM LinG

322 About the Editor

About the Editor

M. Adam Mahmood is a professor of computer information systems, Department of Information and Decision Sciences. He also holds the Ellis and Susan
Mayfield Professorship in the College of Business Administration. He is a
visiting faculty at the Helsinki School of Economics and Business Administration,
Finland, and the University of Canterbury, New Zealand. Dr. Mahmood’s
scholarly and service experience includes a number of responsibilities. He is
presently serving as editor-in-chief of the Journal of Organizational and End
User Computing. He has also recently served as guest editor of the International Journal of Electronic Commerce and the Journal of Management
Information Systems. Dr. Mahmood’s research interests center on the utilization of information technology including electronic commerce for managerial
decision making, strategic and competitive advantage, group decision support
systems, and information systems success as it relates to organizational and end
user computing. On this topic and others, he has published four edited books and
87 technical research papers in some of the leading journals and conference
proceedings in the IT field. These include Management Information Systems
Quarterly, Decision Sciences, Journal of Management Information Systems, International Journal of Electronic Commerce, European Journal of
Information Systems, INFOR — Canadian Journal of Operation Research
and Information Processing, Journal of Information Systems, Information
and Management, Journal of End User Computing, Information Resources
Management Journal, Journal of Computer-Based Instruction, Data Base,
and others. He has also presented papers in numerous regional, national, and
international conferences. In recognition of his research, he has received several
“outstanding research” awards from various professional organizations.

TEAM LinG

About the Authors 323

About the Authors

Barbara Adams, MSHI, is vice president of Cyrus Medical Systems and her
focus includes addressing user resistance to the implementation of new software
systems in health care organizations. She received her bachelor’s degree and her
Master of Science in health informatics from the University of Alabama at
Birmingham. She has published in various journals and presented at industry
conferences including the International Journal of Medical Informatics and
the American Association of Tissue Banks Annual Conference. She is the
recipient of the 2003 Samuel B. Barker Award for Excellence in Graduate
Studies at the Master’s Level and the 2003 Outstanding Graduate Student
Award in Health Informatics.
Kregg Aytes is a professor and chair of the Computer Information Systems
Department in the College of Business, Idaho State University. He holds a PhD
in business administration (MIS) from the University of Arizona. His research
interests include the use of collaborative technologies, information security, and
pedagogy.
Eta S. Berner, EdD, is professor in the Health Informatics Program, Department of Health Services Administration in the School of Health Related
Professions at the University of Alabama at Birmingham. She received her
bachelor’s degree from the University of Rochester and her doctorate from the
Harvard Graduate School of Education. She has published in a variety of journals
and is on the editorial boards of the Journal of the American Medical
Informatics Association (AMIA) and the Journal of Healthcare Information

TEAM LinG

324 About the Authors

Management. She was elected secretary of AMIA in 2004 and is a fellow in
AMIA’s College of Informatics.
Summer E. Bartczak is an assistant professor of information resource management at the Air Force Institute of Technology. As the IRM program director, she
is responsible for the graduate education of officer and enlisted candidates
selected from across the Department of Defense. Lieutenant Colonel Bartczak
is a U.S. Air Force Academy graduate (1986), a Squadron Officer’s School
distinguished graduate (1993), and an Air Command and Staff College (inresidence) graduate (1999). She completed her PhD program in management
information systems at Auburn University. Her research interests include
information and knowledge management, information and knowledge strategy,
and information and knowledge system implementation.
Ernst Bekkering is assistant professor in management information systems at
Northeastern State University in Tahlequah, OK. Dr. Bekkering received his MS
and PhD degrees in MIS from Mississippi State University. His current research
interests include adoption of new technologies, security, and telecommunications. He has published in the Communications of the ACM, Journal of
Organizational and End User Computing, the Journal for the Advancement
of Marketing Education, and several conference proceedings.
Kalyani Chatterjea is an associate professor at the National Institute of
Education, Nanyang Technological University, Singapore. Her main area of
specialization and research is geomorphology and more specifically, soil erosion
and slope stability, geographic education, and in-service education. She regularly
conducts courses, workshops, and field trips for teachers in schools and junior
colleges and for Geography Teachers’ Association, Singapore. She is involved
in in-service training for teachers, for many years. She has been developing
Web-based in-service training packages for secondary school and junior collage
teachers in Singapore for more than five years. She has also published in journals
and books on topics related to her areas of specialization, in geomorphology and
geography education.
Terry Connolly is the FINOVA professor and head of the Management and
Policy Department, Eller College, at the University of Arizona. His research and
publications are mainly in judgment and decision making. His most recent book
is titled Judgment and Decision Making: An Interdisciplinary Reader (2nd
ed., Arkes and Hammond; Cambridge, 2002). He is past president of the Society
for Judgment and Decision Making.

TEAM LinG

About the Authors 325

Kimberly Davis is assistant professor of information systems at Mississippi
State University, where her research interests include information assurance
and security, information privacy, and software outsourcing relationships. She
earned a BA in economics (magna cum laude and Phi Beta Kappa) and an
MBA from the University of Nebraska. She earned her doctorate in law (JD),
cum laude, from Harvard Law School. As an attorney, Professor Davis
practiced in the area of intellectual property, with a focus on software licensing.
Professor Davis has published in various outlets, including the European
Journal of Operational Research, Journal of Organizational and End User
Computing, and PC AI.
James P. Downey is an assistant professor of MIS at the University of Central
Arkansas. He was formerly an instructor in the Department of Computer
Science and Information Technology at the U.S. Naval Academy and is still an
active duty Naval officer with 24 years of service. He completed his PhD from
Auburn University. His research interests include IT in the workplace, humancomputer interaction, database technology, and technology trends.
Evan W. Duggan is assistant professor of MIS in the Culverhouse College of
Commerce & Business Administration, University of Alabama. He obtained a
PhD and MBA from Georgia State University and a BSc from the University of
the West Indies, Jamaica. He has more than 25 years of IT experience in
industry. His research interests involve the management of information systems
(IS) in corporations with reference to IS success factors and quality, sociotechnical
issues, and systems development and project management methodologies implementation. He has publications (or articles forthcoming) in the International
Journal of Industrial Engineering, Journal of International Technology
and Information Management, Information Technology & Management,
Journal of End User Computing, Information Resources Management
Journal, Human-Computer Interactions, Information & Management, Electronic Journal of Information Systems in Developing Countries, and Communications of the Association of Information Systems. Dr. Duggan has
taught MIS and decision sciences courses at the graduate and undergraduate
levels, including executive MBA programs, in several U.S. and international
institutions.
Manish Gupta is an executive at M&T Bank in the Information Security
Division. He recently graduated from SUNY Buffalo with an MBA.
Kun S. Im is an assistant professor of MIS at School of Business, Yonsei
University. He holds PhD in MIS from the University of South Carolina and a
PhD in accounting from Yonsei University. His current research interests

TEAM LinG

326 About the Authors

include IT adoption, impacts of IT on organizational structure, and valuation of
IT investments. He has published his papers in the areas of information systems
research and in the Journal of Information Technology Management.
Mary C. Jones is an associate professor of information systems at the
University of North Texas. She received her doctorate from the University of
Oklahoma in 1990. Dr. Jones has published articles in such journals as Information and Management, Information Resources Management Journal, European Journal of Information Systems, Journal of Computer Information
Systems, and Behavioral Science. Her research interests are in the management and integration of emerging electronic commerce technologies and in
organizational factors associated with enterprise-wide systems.
Nory B. Jones is an assistant professor of MIS at the Maine Business School,
University of Maine, in Orono. Her research interests are in the areas of
knowledge management and collaborative technologies as well as the adoption
and diffusion of technological innovations. She holds a PhD in information
systems from the University of Missouri in Columbia. She has published in
Performance Improvement Quarterly, Technology Horizons in Education,
and E-learning in Corporations (Prentice Hall).
Thomas R. Kochtanek is an associate professor of information science in the
School of Information Science and Learning Technologies at the University of
Missouri in Columbia. His research interests focus on information storage and
retrieval systems, digital libraries and asynchronous learning environments. He
holds the BS in management science and a PhD in information science, both from
Case Western Reserve University. He has more than 50 publications in
international and national journals including Information Processing and
Management, Journal of the American Society for Information Science,
Online Information Retrieval, and many others.
Liping Liu is an associate professor of management and information systems
at the University of Akron. He received a BS in applied mathematics from
Huazhong University of Science and Technology, China (1986), a BE in river
dynamics from Wuhan University, China (1987), an ME in systems engineering
from Huazhong University of Science and Technology, China (1991), and a PhD
in business from the University of Kansas (1995). His research interests have
been in the areas of uncertainty reasoning and decision making in artificial
intelligence, electronic business, systems analysis and design, technology adoption, and data quality. His articles have appeared in Decision Support Systems,
European Journal of Operational Research, International Journal of
Approximate Reasoning, Information Systems Frontier, Journal of Risk and

TEAM LinG

About the Authors 327

Uncertainty, and others. He has served as a guest editor for the International
Journal of Intelligent Systems and co-editor for Classic Works on the
Dempster-Shafer Theory of Belief Functions. He served on the program
committee or as a track chair for INFORMS, AMCIS, IRMA, IIT, etc. He has
strong practical and teaching interests in emerging e-business technologies and
systems design and development using advanced DBMS, CASE, and RAD tools.
He has won two teaching awards. His recent consulting experience includes
designing and developing a patient record management system, a payroll system,
a course management system, and an e-travel agent, and administering Oracle
databases for medium and large corporations.
Qingxiong Ma is an assistant professor of computer information systems at
Central Missouri State University. He received his MBA from Eastern Illinois
University, Charleston (1999), and his PhD in MIS at Southern Illinois University
at Carbondale (2004). His research interests include information technology
adoption/diffusion, electronic commerce, and information security management.
His articles were published in the International Journal of Healthcare
Technology and Management and the Journal of Organizational and End
User Computing. He presented numerous papers at the America’s Conference
on Information Systems and Decision Sciences Institute annual meetings.
Information systems classes he has taught include: systems analysis and design,
data communication and networks, management of information systems, and
database management systems.
Thomas E. Marshall is an associate professor of management information
systems at Auburn University. His previous research has been published in
journals including Information & Management, Information Resource Management Journal, and Journal of Database Management. His research
interests include CASE technologies, cognitive modeling, accounting information
systems, information security and database technologies.
Tanya McGill is a senior lecturer in the School of Information Technology at
Murdoch University, Western Australia. She has a PhD from Murdoch University. Her major research interests include end user computing and information
technology education. Her work has appeared in various journals including the
Information Resources Management Journal, Journal of Research on
Computing in Education, European Journal of Psychology of Education,
Journal of the American Society for Information Science, and Journal of
End User Computing.
Steven A. Morris is associate professor of computer information systems at
Middle Tennessee State University, Murfreesboro, Tennessee. Dr. Morris

TEAM LinG

328 About the Authors

received in PhD in MIS from Auburn University in 1999. His research interests
include cognitive factors in information systems, organizational design, and
various issues regarding system analysis and design. Dr. Morris has published in
numerous IS journals including Journal of Organizational and End User
Computing and Information Resources Management Journal.
R. Leon Price’s research interests include management information systems,
systems analysis and design and computer auditing. Professor Price’s articles
have been published in Academy of Management Review, Journal of Systems
Management, Journal of End User Computing, Journal of Microcomputer
System Management, DATA Management, Information Resource Management, Information Executive, Behavioral Science, Journal of Purchasing
and Materials Management and The American Oil and Gas Reporter. He has
also presented papers at numerous professional meetings and seminars such as
ICIS and DSI. He handles review assignments for several journals. Professor
Price is a member of the International Conference of Information Systems,
Decision Sciences Institute, Association of Information Technology Professionals and Association for Information Systems. He was selected as the industry
contact for the International Conference on Information Systems and has served
as MIS chairperson for the Decision Sciences Institute. Professor Price has won
more than 20 teaching awards during his tenure at the University of Oklahoma.
He has also served as faculty coordinator for the annual Oklahoma Business
Conference and on numerous university committees.
H. Raghav Rao is a professor of MIS and an adjunct professor of CSE at SUNY
Buffalo.
Peter B. Seddon, PhD, is an associate professor in the Department of
Information Systems at The University of Melbourne, Australia. His teaching
and research interests focus on helping people and organizations make more
effective use of IT. His particular research interests are (1) evaluation of
information systems success, (2) packaged enterprise application software, and
(3) IT outsourcing. Peter is on the editorial boards of a number of publications,
and has recently completed a term as an associate editor for Management
Information Systems Quarterly.
Bernd Carsten Stahl is a senior lecturer in the Faculty of Computer Sciences
and Engineering and a research associate at the Centre for Computing and Social
Responsibility of De Montfort University, Leicester, UK. His area of research
consists of philosophical, more specifically of normative, questions arising from
the use of information and communication technology. The emphasis in this area
is on the notion of responsibility. He researches the application of such normative

TEAM LinG

About the Authors 329

questions in economic organizations, but also educational and governmental
institutions. His second area of interest consists of epistemological questions in
Information Systems research. Dr. Stahl has published more than 40 papers in
refereed journals, books, and conferences. His first book Responsible Management of Information Systems has recently been published by Idea Group
Publishing. He is the editor-in-chief of the International Journal of Technology and Human Interaction.
D. Sandy Staples, PhD, is an associate professor in the School of Business at
Queen’s University, Kingston, Canada. His research interests include the
enabling role of information systems for working virtually and knowledge
management, and assessing the effectiveness of information systems and IS
practices. Sandy has published articles in various journals and magazines
including Organization Science, Information Systems Research, Information
& Management, Journal of Strategic Information Systems, Journal of
Management Information Systems, Journal of End-User Computing,
OMEGA, and KM Review. He is currently an associate editor of MIS Quarterly
and serves on the editorial board of other journals.
Cherian S. Thachenkary is associate professor of management in the J. Mack
Robinson College of Business at Georgia State University in Atlanta, Georgia,
USA. He obtained his PhD and MASc degrees in management sciences from the
Faculty of Engineering, University of Waterloo, Canada, and a BSc from the
University of Toronto. Dr. Thachenkary conducts teaching and research in the
area of management of technology. His current focus is on the digital economy
— the cost/benefits of high bandwidth networks and electronic services/
applications. Dr. Thachenkary’s research has been published in the Journal of
Telemedicine and e-Health, OR/MS Today, Computer Networks and ISDN
Systems, IEEE Transactions on Engineering Management, Office: Technology and People, and Computerworld. He has served as a senior editor of
Computer Networks and ISDN Systems, and as a departmental editor of
DATABASE. Dr. Thachenkary has also held faculty appointments at the
University of Waterloo in Canada, The University of Waikato in New Zealand,
and at Cairo University, Egypt.
Shambhu Upadhyaya is an associate professor of CSE and director of the
Center of Excellence in Information Systems Assurance Research and Education at SUNY Buffalo.
Merrill Warkentin is a professor of information systems at Mississippi State
University. His research, primarily in e-commerce, virtual teams, and computer
security, has appeared in such journals as MIS Quarterly, Decision Sciences,

TEAM LinG

330 About the Authors

Decision Support Systems, Communications of the AIS, Information Systems
Journal, Journal of End User Computing, Journal of Global Information
Management, Journal of Electronic Commerce Research, and Journal of
Computer Information Systems. Professor Warkentin’s newest book will
focus on Information System Assurance and Security. He is associate editor of
Information Resources Management Journal, Journal of Information Systems Security, and eGovernment Quarterly, and can be reached at
www.MISProfessor.com.
Joni Rousse Wyatt, MHA, MHS-HIA is presently the manager of operations
for Norwood Clinic, Inc., a 40-physician multispecialty clinic in Birmingham, AL.
She earned a master’s degree in health administration and a master’s degree in
health science with a focus in health information administration from the Medical
University of South Carolina in 2000. Her current focus is on process improvement as a precursor to technological advancement.
Mun Y. Yi is an assistant professor of MIS at the Moore School of Business,
University of South Carolina. He earned his PhD in information systems from the
University of Maryland, College Park. His current research focuses on computer skill acquisition and training, information technology adoption and diffusion,
electronic commerce, and IS project management. His work has been published
in journals such as Information Systems Research, Decision Sciences, Information and Management, International Journal of Human-Computer Studies, Journal of End User Computing, and Journal of Applied Psychology.

TEAM LinG

Index 331

Index

A
adoption of technology innovations 131
appraisal 92
assets management (AM) 212
asynchronous intervention 305
authentication 239
automated teller machine (ATM) 234
availability 242

B
banking service hosting 235
behavioral control 92
blended approach learning strategy
301
BSCW (basic support for cooperative
work) 137

C
centralized database 209
change management process 223
check-off password system (COPS)
281
chemicals 215

cognitive control 92
collaborative technologies 130
computer learning 66
computer security 257
computer self-efficacy 72
computer software training 68
computer task performance 65
computer viruses 258
computer-supported collaborative work
(CSCW) 131
computing behaviors 259
confidentiality 242, 280
consequences of use 51
control 91

D
decision support systems (DSS) 6
decisional control 92
digital 130
direct dial access (DDA) 234
distance education 302

E
E&P 214

TEAM LinG

332 Index

e-banking 234
e-mail 263
end user 3, 188
end user computing (EUC) 1, 22
end user development 21
end user dimension 5
end user support 7
end-user productivity 66
enterprise resource planning (ERP)
208
ethics 189
experimental social psychology 92
extreme programming (XP) 154

F
feedback condition 95
feedback content 95
feedback sequence 95
feedback timeframe 95
feedback-control paradigm 92
FI/CO (financial accounting and
controlling) 212
Frese model 94

G
goal content 95
goal realistic 95
goal sequence 95
goal stable 95
goal timeframe 95
group decision support systems
(GDSS) 6
group support systems (GSS) 6

H
health care 176
Herzberg’s two-factor model 145
human memory 285

immoral behavior 190
in-service upgrading 301
infoculture 132
information assurance 189, 233
information gathering 92
information system security 281
information systems 176
information systems implementation
177
information technology (IT) 66, 189,
258
infostructure 132
infrastructure 132
integration 226
integrity 241, 280
interactive voice response (IVRs) 236
Internet 233, 303
IT management 113

J
joint application development (JAD)
151

K
keystroke dynamics 283
knowledge 94

L
learner control 306

M
materials management (MM) 212
meta-analysis 116
motivation-maintenance technology
implementation 145

N
nominal group technique (NGT) 151

image-intensive learning environment
308

online course 304

TEAM LinG

Index 333

P
password 259, 282
performance improvements 133
personal finance managers (PFMs)
236
personal goal 65
personal identification number (PIN)
234
plant maintenance (PM) 212
PLS structural model 75
privacy 190, 242, 280
privacy-enhancing technologies (PETs)
196
production planning (PP) 212
project systems (PS) 212

R
rapid application development (RAD)
154
rational choice model 261
resistance 179

S
sales and distribution (SD) 212
security 186
share knowledge 130
short-term memory 282
Singapore 302
single system sign on (SSO) 283
social engineering 258
social psychology theory 114
software modules 209
stimulus–response 92
support staff 45
system usage 90
systems requirements determination
(SRD) 152

task organized 95
task performance 66
task realistic 95
task sequence 95
task timeframe 95
task-technology fit (TTF) 47
teacher development 301
team-oriented collaborative culture 134
technical customer support division
134
technology acceptance model (TAM)
113, 259
technology-to-performance chain
(TPC) 42
theory of reasoned action (TRA)
47, 114, 259
training 210
transition of IPS knowledge 218

U
user developed applications (UDAs)
21
user resistance 176
user satisfaction 23, 90
user-centered theories 176
USWhole 214

W
Web banking 234
Web-based course 305

T
task content 95
task flexible 95

TEAM LinG

Instant access to the latest offerings of Idea Group, Inc. in the fields of
I NFORMATION SCIENCE , T ECHNOLOGY AND MANAGEMENT!

InfoSci-Online
Database
BOOK CHAPTERS
JOURNAL AR TICLES
C ONFERENCE PROCEEDINGS
C ASE STUDIES

“

The Bottom Line: With easy
to use access to solid, current
and in-demand information,
InfoSci-Online, reasonably
priced, is recommended for
academic libraries.

The InfoSci-Online database is the
most comprehensive collection of
full-text literature published by
Idea Group, Inc. in:

”

- Excerpted with permission from
Library Journal, July 2003 Issue, Page 140

n
n
n
n
n
n
n
n
n

Distance Learning
Knowledge Management
Global Information Technology
Data Mining & Warehousing
E-Commerce & E-Government
IT Engineering & Modeling
Human Side of IT
Multimedia Networking
IT Virtual Organizations

BENEFITS
n Instant Access
n Full-Text
n Affordable
n Continuously Updated
n Advanced Searching Capabilities

Start exploring at
www.infosci-online.com

Recommend to your Library Today!
Complimentary 30-Day Trial Access Available!
A product of:

Information Science Publishing*
Enhancing knowledge through information science

*A company of Idea Group, Inc.
www.idea-group.com

TEAM LinG

New Releases from Idea Group Reference

Idea Group
REFERENCE

The Premier Reference Source for Information Science and Technology Research
ENCYCLOPEDIA OF

ENCYCLOPEDIA OF

DATA WAREHOUSING
AND MINING

INFORMATION SCIENCE
AND TECHNOLOGY
AVAILABLE NOW!

Edited by: John Wang,
Montclair State University, USA
Two-Volume Set • April 2005 • 1700 pp
ISBN: 1-59140-557-2; US $495.00 h/c
Pre-Publication Price: US $425.00*
*Pre-pub price is good through one month
after the publication date

Provides a comprehensive, critical and descriptive examination of concepts, issues, trends, and challenges in this
rapidly expanding field of data warehousing and mining
A single source of knowledge and latest discoveries in the
field, consisting of more than 350 contributors from 32
countries

Five-Volume Set • January 2005 • 3807 pp
ISBN: 1-59140-553-X; US $1125.00 h/c

ENCYCLOPEDIA OF

DATABASE TECHNOLOGIES
AND APPLICATIONS

Offers in-depth coverage of evolutions, theories, methodologies, functionalities, and applications of DWM in such
interdisciplinary industries as healthcare informatics, artificial intelligence, financial modeling, and applied statistics
Supplies over 1,300 terms and definitions, and more than
3,200 references

DISTANCE LEARNING

April 2005 • 650 pp
ISBN: 1-59140-560-2; US $275.00 h/c
Pre-Publication Price: US $235.00*
*Pre-publication price good through
one month after publication date

Four-Volume Set • April 2005 • 2500+ pp
ISBN: 1-59140-555-6; US $995.00 h/c
Pre-Pub Price: US $850.00*
*Pre-pub price is good through one
month after the publication date

MULTIMEDIA TECHNOLOGY
AND NETWORKING

ENCYCLOPEDIA OF

More than 450 international contributors provide extensive coverage of topics such as workforce training,
accessing education, digital divide, and the evolution of
distance and online education into a multibillion dollar
enterprise
Offers over 3,000 terms and definitions and more than
6,000 references in the field of distance learning
Excellent source of comprehensive knowledge and literature on the topic of distance learning programs
Provides the most comprehensive coverage of the issues,
concepts, trends, and technologies of distance learning

April 2005 • 650 pp
ISBN: 1-59140-561-0; US $275.00 h/c
Pre-Publication Price: US $235.00*
*Pre-pub price is good through
one month after publication date

www.idea-group-ref.com

Idea Group Reference is pleased to offer complimentary access to the electronic version
for the life of edition when your library purchases a print copy of an encyclopedia
For a complete catalog of our new & upcoming encyclopedias, please contact:
701 E. Chocolate Ave., Suite 200 • Hershey PA 17033, USA • 1-866-342-6657 (toll free) • cust@idea-group.com

TEAM LinG

IT Solutions Series
Humanizing Information Technology: Advice from Experts
Authored by:
Shannon Schelin, PhD, North Carolina State University, USA
G. David Garson, PhD, North Carolina State University, USA

With the alarming rate of information technology changes over the past two decades, it is not unexpected
that there is an evolution of the human side of IT that has forced many organizations to rethink their
strategies in dealing with the human side of IT. People, just like computers, are main components of any
information systems. And just as successful organizations must be willing to upgrade their equipment and
facilities, they must also be alert to changing their viewpoints on various aspects of human behavior. New
and emerging technologies result in human behavior responses, which must be addressed with a view
toward developing better theories about people and IT. This book brings out a variety of views expressed
by practitioners from corporate and public settings offer their experiences in dealing with the human
byproduct of IT.

Information Technology Security: Advice from Experts
Edited by:
Lawrence Oliva, PhD, Intelligent Decisions LLC, USA

As the value of the information portfolio has increased, IT security has changed from a product focus to
a business management process. Today, IT security is not just about controlling internal access to data
and systems but managing a portfolio of services including wireless networks, cyberterrorism protection
and business continuity planning in case of disaster. With this new perspective, the role of IT executives
has changed from protecting against external threats to building trusted security infrastructures linked to
business processes driving financial returns. As technology continues to expand in complexity, databases
increase in value, and as information privacy liability broadens exponentially, security processes
developed during the last century will not work. IT leaders must prepare their organizations for previously
unimagined situations. IT security has become both a necessary service and a business revenue
opportunity. Balancing both perspectives requires a business portfolio approach to managing investment
with income, user access with control, and trust with authentication. This book is a collection of
interviews of corporate IT security practitioners offering various viewpoint on successes and failures in
managing IT security in organizations.

Managing Data Mining: Advice from Experts
Edited by:
Stephan Kudyba, PhD, New Jersey Institute of Technology, USA
Foreword by Dr. Jim Goodnight, SAS Inc, USA

Managing Data Mining: Advice from Experts is a collection of leading business applications in the data mining
and multivariate modeling spectrum provided by experts in the field at leading US corporations. Each contributor
provides valued insights as to the importance quantitative modeling provides in helping their corresponding organizations
manage risk, increase productivity and drive profits in the market in which they operate. Additionally, the expert
contributors address other important areas which are involved in the utilization of data mining and multivariate modeling
that include various aspects in the data management spectrum (e.g. data collection, cleansing and general
organization).

E-Commerce Security: Advice from Experts
Edited by:
Mehdi Khosrow-Pour, D.B.A.,
Information Resources Management Association, USA

The e-commerce revolution has allowed many organizations around the world to become more effective and efficient
in managing their resources. Through the use of e-commerce many businesses can now cut the cost of doing
business with their customers in a speed that could only be imagined a decade ago. However, doing business on
the Internet has opened up business to additional vulnerabilities and misuse. It has been estimated that the cost
of misuse and criminal activities related to e-commerce now exceeds 10 billion dollars per year, and many experts
predict that this number will increase in the future. This book provides insight and practical knowledge obtained from
industry leaders regarding the overall successful management of e-commerce practices and solutions.

Its Easy to Order! Order online at www.cybertech-pub.com,
www.idea-group.com or call 717/533-8845 x10
Mon-Fri 8:30 am-5:00 pm (est) or fax 24 hours a day 717/533-8661

CyberTech Publishing
Hershey • London • Melbourne • Singapore

Excellent additions to your library!

TEAM LinG