1. Information & Management 49 (2012) 36–46
Contents lists available at SciVerse ScienceDirect
Information & Management
journal homepage: www.elsevier.com/locate/im
A comprehensive study of the relationship between enterprise resource planning
selection criteria and enterprise resource planning system success
Wen-Hsien Tsai *, Pei-Ling Lee, Yu-Shan Shen, Hsiu-Ling Lin
Department of Business Administration, National Central University, Taoyuan 32001, Jhongli, Taiwan
A R T I C L E I N F O
A B S T R A C T
Article history:
Received 23 February 2010
Received in revised form 3 June 2011
Accepted 30 September 2011
Available online 21 October 2011
We developed a conceptual framework for investigating how ERP selection criteria are linked to system
quality and the service provided by suppliers and consultants, and thus how these influenced ERP
implementation success. Through a cross-sectional survey of the top 5000 largest companies in Taiwan,
using a balanced scorecard concept and path analysis, we showed that four system selection criteria
(consultant’s suggestion, a certified high-stability system, compatibility between the system and the
business process, and the provision of best practices) were positively related to system quality. Three
supplier selection criteria (international market position, training support by the supplier and supplier
technical support and experience) had a significant influence on supplier service quality, and two
consultant selection criteria (consultant’s ERP implementation experience in a similar industry and
consultant’s support after going live) were related to consultant service quality. However, we found that
most organizations did not consider all these criteria when implementing ERP systems. Our study also
suggested that enhanced system quality and service quality could increase user perspective and ERP
success.
ß 2011 Published by Elsevier B.V.
Keywords:
ERP systems
ERP suppliers
ERP consultants
System quality
SERVQUAL
Balanced scorecard
Path analysis
Selection criteria
1. Introduction
An ERP system is an integrated information technology (IT) that
uses common databases and consistent cross-functional information flow to allow organizations to integrate information from
different departments and locations. Their availability has
prompted many small and medium sized organizations to shift
their IT strategies from in-house development to purchasing
application software [10]. Clearly, ERP systems can enhance
business operations. However, there are several difficulties that
must be overcome for the successful implementation of an ERP
system [5]. The company must be aware of the factors that
influence the success of its implementation to prevent failures.
The perceived characteristics of the product play a major role in
the final decision of companies considering buying an ERP system
[1]. In choosing a specific system, organizations use a variety of
criteria to determine the most suitable one for them. Several
previous papers on ERP system selection have discussed the major
criteria for the evaluation of the best implementation processes
[3,22]. These have shown the differences in the characteristics of
the available ERP systems and suppliers but the work has mainly
been confined to the ERP selection criteria. Few studies have
* Corresponding author. Tel.: +886 3 4267247; fax: +886 3 4222891.
E-mail address: whtsai@mgt.ncu.edu.tw (W.-H. Tsai).
0378-7206/$ – see front matter ß 2011 Published by Elsevier B.V.
doi:10.1016/j.im.2011.09.007
examined the relationship between ERP selection criteria and ERP
system success among Taiwanese companies.
The selection of an ERP system involves more than interviewing
a few suppliers. After deciding that an ERP is right for the company,
choosing the most suitable supplier becomes important. The
correct supplier will become a long-term partner. Furthermore,
ERP suppliers must incorporate the latest IT trends in their systems
to remain competitive [20]. During the ERP implementation
process, most organizations collaborate with their ERP suppliers
and/or consultants. These suppliers and consultants are external
facilitators that affect ERP success [25]. ERP suppliers and
consultants help not only in improving the quality of the ERP
products, but also in ensuring user knowledge and involvement.
Most research to date has considered only ERP suppliers, ERP
systems, or ERP consultants, or two of these; seldom have all three
been considered simultaneously in their effect on the success of ERP
implementation. Furthermore, the research has mainly discussed
the pre-implementation stage; few researchers have done further
investigation on subsequent effects, such as the influence that the
selection criteria have on the success of the organization.
We therefore decided to develop a conceptual framework for
investigating how ERP selection criteria are linked to system
quality and service quality and thus influence ERP system
implementation success. Our study adopted a cross-sectional
survey of the top 5000 largest companies in Taiwan to examine the
influence of various ERP selection criteria.

2. W.-H. Tsai et al. / Information & Management 49 (2012) 36–46
2. Literature review
2.1. Pre-implementation consideration factors
The factors that underlie ERP system success are not the same as
those that determine implementation success. For the former,
selecting the right solution is critical, whereas the latter depends
on software and hardware characteristics. Organizations usually
treat system suppliers and implementation consultants as major
external support. It is therefore important that consultants and
suppliers understand the business and translate the ERP requirements to the organizational and process levels [17]. In this stage
the determination of pre-implementation factors, such as ERP
system selection criteria, ERP supplier selection criteria, and ERP
consultant selection criteria, are critical.
2.1.1. ERP system selection criteria
The characteristics of the ERP system should match the criteria
used by the company to select an IS. Youakim and Jean [26] pointed
out that risk management is an important part of every successful
business model when dealing with socio-economic changes. Also,
security concerns play a major role in minimizing risk by protecting
a business’ intangible resources and knowledge. Therefore, risk
management and security control might be considered major factors
that should be used to evaluate system quality. Functionality,
system reliability, and fit with the systems of parent and/or other
allied organizations were found to be the three most important
criteria by Kumar et al. [14]. In an overlapping study, Birdogan and
Kemal used 17 selection criteria as the determinants of ERP package
selection: fit with parent organization systems, cross-module
integration, compatibility with other systems, references of the
vendor, vision, functionally, system reliability, consultancy, technical aspects, implementation time, methodology of the software,
market position of the vendor, ease of customization, best fit with
organizational structure, service and support, and cost and domain
knowledge of the vendor. Based on a review of the criteria used in
prior studies, we chose eleven ERP system selection criteria as the
most important for our study:
a) consultant’s suggestions,
b) flexibility in adjusting demands according to business requirements,
c) a complete mechanism for risk management and security
control,
d) the ability to integrate different platforms and data,
e) ERP systems that are used by customers and suppliers,
f) ease of integration with other systems (e.g., CRM and SCM),
g) a certified high-stability system,
h) ease of use and maintenance,
i) compatibility between the system and the business process,
j) the provision of best practices, and
k) implementation time.
37
Table 1
Results of the literature review of ERP system selection criteria.
Selection criteria
References
a. Consultant’s suggestions
b. Flexibility in adjusting demands according to business
requirements
c. A complete mechanism for risk management and security
control
d. The ability to integrate different platforms and data
e. ERP systems that are used by customers and suppliers
f. Ease of integration with other systems (e.g., CRM and SCM)
g. A certified high-stability system
h. Ease of use and maintenance
i. Compatibility between the system and the business process
j. The provision of best practices
k. Implementation time
[3]
[3,14]
[26]
[3,14]
[3]
[3,14,21]
[3,14]
[3]
[3,5]
[14]
[3]
assistance, emergency maintenance, updates, service responsiveness, solutions provision, design, customization support, and user
training [27]. Some companies also place emphasis on learning from
past experience and service infrastructure when selecting their ERP
system. Thus, among the supplier evaluation process criteria,
consideration should be given to supplier reputation, financial
stability, and supplier vision. Moreover, sales references and the
internationality of the supplier may be important in the selection
process. As Somers and Nelson [19] mention, many projects have
failed due to a lack of proper training support during the ERP
implementation process. Finally, to cope with technical or other
problems during installation, implementation, or go-live periods,
companies need support from suppliers in terms of IT expertise and
domain knowledge. Verville and Halingten proposed three supplier
evaluation criteria: size, financial stability, and reputation. From
these suggestions, we selected six criteria as for our study:
a)
b)
c)
d)
e)
f)
international market position,
a good reputation in the field,
training support by the supplier,
financial position,
technical support and experience, and
support for maintaining and updating the system after going
live.
Table 2 summarizes the results of the literature review on ERP
supplier selection criteria.
H3a–H3f. ERP supplier selection criteria (a–f) is positively correlated with ERP supplier service quality.
H4a–H4f. ERP supplier selection criteria (a–f) is positively correlated with net benefits.
Table 1 summarizes the literature on the ERP system selection
criteria.
2.1.3. ERP implementation consultant selection criteria
Many organizations use external consulting organizations to
help them in the ERP selection and implementation process and in
the technical evaluation of the chosen solution. Consultants are not
only well trained in ERP implementation methodologies but also
H1a–H1k. ERP system selection criteria (a–k) is positively correlated with ERP system quality
Table 2
Results of the literature review of ERP supplier selection criteria.
H2a–H2k. ERP system selection criteria (a–k) is positively correlated with net benefits.
2.1.2. ERP supplier selection criteria
In addition to the ERP system selection criteria, the selection of
the system supplier also affects ERP system success. Here, primary
considerations generally includes the need for extended technical
Selection criteria
References
a. International market position
b. A good reputation in the field
c. Training support by the supplier
d. Financial position
e. Technical support and experience
f. Support for maintaining and updating
the system after going live
[3,14]
[3,14,22]
[14,19,27]
[22]
[3,14,21,23,27]
[3,14,27]

3. 38
W.-H. Tsai et al. / Information & Management 49 (2012) 36–46
have real system deployment experience. However, consultant
selection demands a complex decision making process. Although
consultants transfer the knowledge and business practices to the
client, they must also have as system knowledge and industryspecific expertise and this complicates their function and role [23].
The consultant’s domain knowledge and fee also affect the
implementation [15,16]. Cheung et al. [6] structured their
consultant selection criteria: on past performance, capacity to
accomplish the work, project approach, and fees. Furthermore, the
transfer of consultants’ implementation knowledge was deemed
critical to meeting the perceived needs of the client, responding to
changing business processes, conducting ongoing administration
and maintenance, minimizing training, gaining new in-house
capabilities, and deploying a high-quality system [13]. After a
literature review and discussions with experts, we selected seven
consultant selection criteria as those to use in our study:
a)
b)
c)
d)
e)
consulting fee,
consultant’s project management abilities,
consultant’s domain knowledge,
consultant’s ERP implementation experience,
consultant’s ERP implementation experience in a similar
industry,
f) consultant’s ERP implementation approaches and tools, and
g) consultant’s support after going live.
Table 3 summarizes the results of the literature review on the
ERP consultant selection criteria.
H5a–H5g. ERP consultant selection criteria (a–g) is positively
correlated to ERP consultant service quality.
H6a–H6g. ERP consultant selection criteria (a–g) is positively
correlated to net benefits.
2.2. IS success model
IS success is a difficult notion to define. Many prior studies have
attempted to explain and justify their assessment of IS success [9].
The original and updated DeLone and McLean IS success model
provided perhaps the most comprehensive system for success
measurement to date, and it has been widely used by others. It has
been validated and applied to measure the success of several ecommerce ventures [2]. The purpose of the original model was to
synthesize work involving individual assessments into a single
coherent model. The authors posited that system quality and
information quality were the two main determinants of IS use and
user satisfaction, and they directly influenced individual impact,
and that this, in turn, had a positive effect on net benefits. The
updated model speculated that information quality, system
quality, and service quality affected subsequent use and user
satisfaction, and that net benefits occurred as a result of this use
and resulting user satisfaction. Subsequently, Ifinedo [11]
Table 3
Results of the literature review of ERP consultant selection criteria.
Selection criteria
References
a. Consulting fee
b. Consultant’s project management abilities
c. Consultant’s domain knowledge
d. Consultant’s ERP implementation experience
e. Consultant’s ERP implementation experience
in a similar industry
f. Consultant’s ERP implementation approaches and tools
g. Consultant’s support after going live
[6,15]
[6,23]
[15,23]
[6,13,16]
[6,23]
[16]
[13]
extended the model, incorporating two new factors: the vendor/
consultant quality (VQ) and workgroup impact (WI). He identified
the relevance measures relating to the cooperative role and system
suppliers as success measures, grouping them together and
incorporating them into the original model.
A series of concrete measurement constructs for the updated
success model was applied to evaluate performance during
implementation. The model included six IS success factors:
(1) system quality ! the performance characteristics of the ERP
systems, including reliability, flexibility, ease of use, and
response time [18],
(2) information quality ! such as credibility, timeliness, usefulness, understandability, and relevance of output,
(3) service quality ! the overall support delivered by the service
supplier: users are now customers and poor user support can
translate into lost customers and lost sales,
(4) the use of an ERP system (the frequency with which an IS is
used, including its use in decision making, charges for ERP
system use, and connection time),
(5) user satisfaction ! successful interaction between the IS and
its users, covering the entire customer experience cycle from
information retrieval through purchase, payment, receipt, etc.,
and
(6) net benefits ! capturing the balance of positive and negative
impacts of the system on the organization.
In addition to the updated IS success model of DeLone and
MacLean, we utilized the balanced scorecard (BSC) technique as a
financial measure to evaluate the ERP performance of the net
benefits dimension. The four constructs of the BSC are; financial,
customer, internal business process, and learning and growth.
We arrived at two hypotheses:
H7. ERP system quality is positively related to the user’s perspective of an ERP system.
H8. The user’s perspective of an ERP system is positively related to
net benefits.
2.3. Measurement of service quality – SERVQUAL
Service quality was explored via an extensive series of focus
group interviews. These centered on the idea that it involved
customer evaluations made on the basis of their expectations of
service and their perception of the way the service had been
performed. SERVQUAL, an instrument developed in the marketing
arena, has been used as a measure of IS service quality. The
majority of such research has attempted to use SERVQUAL to
measure service quality [21]. The SERVQUAL instrument has been a
major technique used to evaluate the degree of satisfaction with a
service, as indicated by consumer perceptions of quality. In our
study, we examined the applicability of SERVQUAL in the ERP
environment. The SERVQUAL instrument uses the dimensions of
tangibles, reliability, responsiveness, assurance, and empathy to
measure service quality. The instrument items include:
(1) Tangibles: Suppliers and/or Consultants provide up-to-date
hardware and software.
(2) Reliability: Suppliers and/or Consultants are reliable.
(3) Responsiveness: Suppliers and/or Consultants provide prompt
service to users.
(4) Assurance: Suppliers and/or Consultants have the knowledge to
do their job well.
(5) Empathy: Suppliers and/or Consultants have users’ best
interests at heart.

4. W.-H. Tsai et al. / Information & Management 49 (2012) 36–46
39
SERVQUAL &
IS Success Model
Beforehand
Consideration Factor
H2a+~H2k+
System
Selections
H1a+~H1k+
Net Benefits
System
Quality
H7+
Service Quality
Supplier
Selections
Consultant
Selections
H3a+~H3f+
H5a+~H5g+
Suppliers
Quality
Direct Impact
H4a+~H4f+
H9+
Consultants
Quality
Financial
Perspectives
User Perspective
User
Satisfaction
Individual
Impact
H8+
Customer
Perspectives
Indirect Impact
Internal Business
Perspectives
Innovation &
Learning
H6a+~H6g+
Fig. 1. Research model and the hypotheses.
Here, we hypothesized a positive relationship between the
degree of satisfaction with the service quality of an ERP supplier
and/or consultant and user satisfaction with an ERP system.
H9. The degree of satisfaction with the service quality of an ERP
supplier is positively related to user perspective of an ERP system.
2.4. Balanced scorecard
Kaplan and Norton developed the balanced scorecard concept,
which includes financial and non-financial measures for the
estimation of the state of an organization. It incorporates four
measurement dimensions: financial, customer, internal business,
and innovation and learning. Its measure of success is the financial
perspective. Financial measures calculate the fiscal influences of
activities, in this case specifying whether the implementation of
an ERP system contributes to the company’s financial position.
Customer perspective is used to determine the degree of customer
satisfaction in areas such as on-time delivery ratio, on-time
invoice ratio, and response time to customer problems/complaints. The internal business perspective involves the efficiency
of the production processes. Thus a company should recognize
activities and major processes involved in achieving the aims with
respect to the financial and customer perspectives. The innovation
and learning perspectives include the aims and measures
determining the progress of the organization and employee
growth.
Our research model is presented in Fig. 1.
3. Methodology
The survey examined the ERP implementation experiences of
the top 5000 largest corporations in Taiwan to explore the status of
their ERP implementation. We used questionnaires that focused on
five areas: the characteristics of ERP implementation, the ERP
implementation status, the motivation and evaluation of the preimplementation process, the implementation experience and ERP
system configuration, and the benefits of the ERP system and
future directions. In our study, we used some of our survey data to
examine whether there were any significant effects on the net
benefits stemming from the choice of selection criteria.
3.1. Survey procedures and sample characteristics
Our research instrument was developed by adapting existing
scales to enhance validity; the items were also slightly altered to fit
our research context. Based on an extensive literature review of
ERP systems, eleven, six, and seven selection criteria were
identified for the ERP system, supplier, and consultant, respectively. To ensure the face validity and clarify the wording of the
questionnaire, we conducted an iterative process of personal
interviews with eight knowledgeable people (two IS faculty, two
ERP suppliers, two ERP consultants, and two manager-users) and
the questionnaire was suitably modified before sending it out.
Empirical data for testing the hypothesized relationships were
obtained using a mail survey of Taiwanese manufacturing firms
that had implemented ERP systems. Data collected from a pilot test
were used for instrument validation and refinement and factorial
validity, whereas data from the major survey were used for
confirmatory analysis of the measurement properties and hypothesis testing of the model, as shown in Fig. 1. Usually, multiple
surveys should not be aggregated unless the population surveyed,
questionnaire, and sampling methodology are identical [12]. In our
study, therefore, the two samples could not be combined because
they were not homogeneous: the pilot test involved the top 5000
largest corporations in Taiwan, whereas the second sample
included companies of different sizes (via stratified sampling)
but only those in the manufacturing sector. In both cases, the
survey questionnaires were mailed to the ERP project managers
and senior project team members of the selected companies along
with a letter outlining the purpose of the research and soliciting
participation in the survey; a postage-paid return envelope was
included for the completed response forms. No incentive was
provided to participants for completing the survey other than the
promise of a copy of the aggregated results.

5. 40
W.-H. Tsai et al. / Information & Management 49 (2012) 36–46
The pilot test survey was directed to the 5000 largest
corporations in Taiwan in 2008; a total of 620 responses were
obtained after three rounds of follow-up requests to nonrespondents. After deleting missing responses and unusable ones,
this resulted in a data response rate of 14.4%. The second survey
was directed at a stratified sample of Taiwan manufacturing firms.
We decided to focus our analysis on manufacturing firms for two
reasons: first, ERP implementation seemed to be particularly
widespread among these firms, and second, we wanted to decrease
potential confounding effects due to industry variation. To
alleviate concerns about the sample distribution, we stratified
the total employment numbers of approximately 4300 firms in the
top 5000 largest corporations in 2009 into four categories based on
their employee counts: 1000 and over, 500–999, 100–499, and less
than 99. The survey was mailed to a random sample of firms in
each of the four strata. In our study, only organizations with prior
experience in implementing ERP systems were used in our
samples. A total of 603 firms were contacted; after three followup mailings, a total of 278 usable responses were obtained, for a
respectable response rate of 46%. A time-trend extrapolation test
was used to examine non-response bias. We assumed that the
responses of the non-respondents were more similar to late than
early respondents. Taking the first 25% as early and the last 25% as
non-respondents, a multivariate analysis of variance of all 37
variances indicated no significant difference (Wilk’s Lambda = 0.972; p = 0.264). This finding is consistent with the absence
of non-response bias.
In the questionnaires, YES/NO selections were used to determine if the selection criteria had been taken into account by the
firms when selecting their ERP packages, ERP suppliers, and
implementation consultants. In the second stage, the respondents
were asked to evaluate the ERP performance improvement level
and its importance using 7-point Likert-type scale ranging from 1
(substantial deterioration) to 7 (substantial improvement) and
from 1 (extremely unimportant) to 7 (extremely important),
respectively. The importance measures were based on SERVQUAL
and focused on the respondents’ degree of satisfaction with system
quality, suppliers quality, and consultants quality, and their
evaluation of the individual impact and the net benefits of the
system. The data on importance levels were applied to determine
the relative weights of the measures. Although the original
SERVQUAL scale is gap-based, performance-only service quality
measures have been demonstrated to be superior to gap-based
measures [4]; therefore, we adopted a performance-only approach.
Table 4 shows the descriptive statistics and the response rate by
firm strata.
3.2. Data operation
Although the DeLone and McLean IS success model depicts a bidirectional effect between system use and user satisfaction, we
considered, only user satisfaction because the sample data
included only those organizations that had implemented an ERP
software package. Since all organizations had used ERP systems,
determinants of system use were dropped from the model.
To explore important factors influencing ERP performance
improvement and develop appropriate performance measures, we
combined the IS success measurement categories in the DeLone
and McLean original and updated IS success models. The scheme is
presented in Fig. 1.
In order to evaluate the importance level of multiple measures
and their degree of improvement in a reliable way, we used
weighted averages of the measures: a two-stage approach in the
design of the performance evaluation questionnaires was adopted.
Stage 1: To find the relative weight of the kth measure of the jth
dimension relative to the measures within the jth dimension, we
Table 4
Characteristics of the samples (N = 278).
Firms in database
(frequency)
Survey returned (%)
Employment numbers
<100
100–499
500–999
1000 and over
65
139
32
42
23
50
12
15
Company age
<10 years
10–<20 years
20–<30 years
!30 years
37
72
61
108
13
26
22
39
Annual revenue (NT$ billion)
<$20
7
$10–<$50
47
$50–<$100
61
$100–<$250
62
$250–<$500
43
!$500
58
3
17
22
22
15
21
Capital amount (NT$ billion)
17
<$8
$8–<$20
45
$20–<$50
79
$50–<$100
50
$100–<$250
39
$250–<$500
19
!$500
29
6
16
28
18
14
7
11
calculated the importance level and the degree of performance
improvement:
¯
W jk ¼
PN
W i jk
;
N
i¼1
i ¼ 1 to N
(1)
In this, Wijk is the importance level score (1–7) of the kth
measure of the jth dimension as perceived by the ith respondent’s
¯
company. W jk is the average importance level score of the kth
measure of the jth dimension as perceived by N respondents.
Stage 2: Since average importance score rankings were acquired
at Stage 1, the level of performance improvement of the jth
dimension for the ith respondent’s company can be calculated as:
0
1
lj
X P i jk à W jk
¯
@
A;
Pi j ¼
Pl j
¯
k¼1
k¼1 W jk
i¼1
to
N
and
j¼1À3
(2)
Pj
¯
¯
Here, ðW jk = lk¼1 W jk Þ, is the relative weight of the kth measure
of the jth dimension relative to the measures within the jth
dimension. Meanwhile, Pijk is the performance improvement level
score of the kth measure of the jth dimension for the ith
respondent’s company, and lj is the number of chosen measures
of the jth dimension and P i j is the performance improvement level
score of the jth dimension for the ith respondent’s company. Note
j
that l is the number of chosen measures for the jth dimension.
4. Data analysis and findings
4.1. Data analysis and model testing
Initially, we tested for common method bias using a post hoc
procedure. Under principal components factor analysis, evidence
for common method bias exists when a single factor emerges from
the analysis, or one general factor accounts for the majority of the
covariance in the interdependent and dependent variables.
Because the analysis of our pilot data (the first survey), and data
from the second survey resulted in multiple factors, we concluded
that the data do not indicate substantial common method bias.

6. W.-H. Tsai et al. / Information & Management 49 (2012) 36–46
Table 5
Exploratory factor analysis of Pilot test data.
41
Table 6
Factor loadings, reliability and validity.
Items
1
2
3
4
5
SEVQUAL1-CON3
SEVQUAL2-CON2
SEVQUAL3-CON4
SEVQUAL4-SUP3
SEVQUAL5-CON5
SEVQUAL6-SUP4
SEVQUAL7-SUP2
SEVQUAL8-SUP5
SEVQUAL9-SUP1
SEVQUAL10-CON1
0.853
0.819
0.809
0.807
0.801
0.796
0.779
0.754
0.691
0.678
0.144
0.195
0.014
0.204
À0.009
0.073
0.247
0.072
0.129
0.095
0.158
0.051
0.286
À0.007
0.218
0.130
À0.043
0.113
À0.102
0.035
À0.004
0.014
À0.012
0.055
À0.002
0.143
0.132
0.143
0.093
À0.041
0.083
0.119
0.175
0.041
0.135
0.123
0.063
À0.022
0.095
0.048
UserPec1-USER4
UserPec2-USER3
UserPec3-USER2
UserPec4-USER5
UserPec5-USER1
UserPec6-IND3
UserPec7-IND2
UserPec8-IND4
UserPec9-IND5
UserPec10-IND1
0.197
0.166
0.230
0.200
0.218
0.127
0.170
0.151
0.141
0.241
0.823
0.817
0.808
0.792
0.749
0.628
0.579
0.558
0.546
0.531
0.234
0.259
0.200
0.213
0.193
0.500
0.490
0.515
0.504
0.524
0.218
0.191
0.283
0.225
0.303
0.129
0.207
0.102
0.167
0.206
0.216
0.056
0.149
0.201
0.268
0.237
0.259
0.307
0.288
0.261
NetBenefitII1-IBP2
NetBenefitII2-IBP 3
NetBenefitII3-IBP1
NetBenefitII4-IL1
NetBenefitII5-IL3
NetBenefitII6-IL2
0.118
0.091
0.144
0.146
0.006
0.094
0.207
0.383
0.204
0.363
0.243
0.503
0.790
0.669
0.666
0.596
0.591
0.543
0.179
0.250
0.230
0.333
0.414
0.399
0.245
0.257
0.352
0.137
À0.129
0.122
NetBenefitDI1-FP1
NetBenefitDI2-FP 3
NetBenefitDI3-FP 2
NetBenefitDI4-CP1
NetBenefitDI5-CP2
NetBenefitDI6-CP3
System quality2 – SQ2
System quality4 – SQ4
System quality3 – SQ3
System quality1 – SQ1
System quality5 – SQ5
0.076
0.055
0.092
0.090
0.053
0.075
0.187
0.180
0.155
0.143
0.189
0.245
0.267
0.229
0.285
0.265
0.151
0.115
0.401
0.205
0.393
0.334
0.076
0.184
0.250
0.396
0.486
0.507
0.292
0.146
0.401
0.033
0.305
0.778
0.775
0.757
0.705
0.676
0.592
0.187
0.320
0.155
0.347
0.267
0.352
0.267
0.285
0.110
0.023
0.248
0.779
0.713
0.697
0.674
0.571
Inventory
Notes. Principal axis factoring: Promax with Kaiser normalization.
Data from the first survey were subjected to exploratory factor
analysis (EFA) to gain insights as to the multidimensionality of the
items (see Table 5). The results of this data analysis revealed five
factors with an eigenvalue greater than one and no single factor
was consistent with the absence of a significant variance common
to the measures.
Data from the second survey (manufacturing companies) were
used for a confirmatory factor analysis (CFA) on the measurement
scales and to test the model hypotheses, because the construct
validity was preliminarily verified with an EFA from the first
survey. A CFA using a complete standardized solution in LISREL
8.52 showed that all 37 items loaded on their corresponding
factors, which provided strong empirical evidence of construct
independence and the validity of the construct. Moreover, most
reliability measures were well above the recommended level of
0.7, indicating adequate internal consistency. SPSS 17 was used for
the analysis of construct item reliability. As indicated by
Cronbach’s a coefficients, all constructs exhibited high composite
reliability. Table 6 shows that most of the measures had significant
loadings that were much higher than the suggested threshold. The
two exceptions were the first two items in the system supplier and
consultant scales of SERVQUAL, where loadings were slightly
below 0.7. Content validity defines how representative and
comprehensive the items were in testing the hypotheses. Our
definitions of systems quality, service quality, user perspective,
and net benefits were based on our literature review. For
unidimensionality testing, a measurement model was constructed
using the five latent factors. Each item was restricted to load on
Service quality
SERVQUAL of system suppliers
Tangibles-SUP1
Reliability-SUP2
Responsiveness-SUP3
Assurance-SUP4
Empathy-SUP5
SERVQUAL of consultants
Tangibles-CON1
Reliability-CON2
Responsiveness-CON3
Assurance-CON4
Empathy-CON5
System quality
Reliability-SQ1
Flexibility-SQ2
Ease of use-SQ3
Accuracy-SQ4
Response time-SQ5
User perspective
User satisfaction
Information satisfaction-USER1
Software satisfaction-USER2
System Interface
satisfaction-USER3
System satisfaction-USER4
ERP Project satisfaction-USER5
Individual impact
Improved job performance-IND1
Improved productivity-IND2
Improved decision making
ability-IND3
Improved identification problem
ability-IND3
Improved trouble-shooting
ability-IND4
Net benefits – direct impact
Financial perspective
Inventory levels-FP1
Purchasing costs-FP2
Inventory turnover-FP3
Customer perspective
On time delivery ratio-CP1
Response time customer
complaint-CP2
On time invoice ratio-CP3
Net benefits – indirect impact
Internal business perspective
Internal data transmission time-IBP1
Internal interaction frequency-IBP2
Response time to environmental
volatility-IBP3
Innovation and learning
Understanding of business process-IL1
Job achievement of employees-IL2
Product development to market-IL3
Standard
loading
CR
AVE
Cronbach’s
0.95
0.64
0.917
0.94
0.75
0.919
0.97
0.76
0.948
0.95
0.75
0.928
0.93
0.68
0.914
a
0.65
0.78
0.80
0.81
0.73
0.67
0.87
0.92
0.87
0.85
0.87
0.88
0.80
0.97
0.83
0.89
0.89
0.86
0.92
0.91
0.86
0.87
0.87
0.83
0.83
0.86
0.90
0.90
0.86
0.81
0.84
0.85
0.90
0.90
0.80
0.83
0.62
All ratio of improvement were measured by using seven-point Likert-type scales
ranging from 1 (substantial deteriorate) to 7 (substantial improvement).
only one specific latent variable, and one latent variable per
indicator was allowed. Convergent validity is demonstrated when
items load highly (loading >0.7) on their associated factors and
constructs have an average variance extracted (AVE) of at least 0.5
and composite reliability (CR) of at least 0.7. After the assessment
of reliability and validity, the overall fit and the explanatory power
of the research model were examined together with the relative
strengths of the individual causal path. Seven common modelfitting measures were used to assess the model’s overall goodnessof-fit: a normalized x2 test (x2 divided by degrees of freedom; x2/
df), a goodness-of-fit index (GFI), an adjusted goodness-of-fit index

7. W.-H. Tsai et al. / Information & Management 49 (2012) 36–46
42
Table 7
Goodness of fit statistics for the overall structure model.
Fit Indices
Threshold
x /DF
GFI
AGFI
NFI
NNFI
CFI
RMSEA
RMR
Output
<2
>0.80
>0.80
>0.80
>0.80
>0.8
<0.05
<0.1
2
1.26
0.96
0.88
0.95
0.86
0.95
0.00
0.05
(AGFI), a normalized fit index (NFI), a non-normalized fit index
(NNFI), a comparative fit index (CFI), a root mean square error of
approximation (RMSEA), and a root mean square residual (RMR).
All reflective indicators were standardized to be consistent with
recommendations for dealing with interactions [7].
Table 6 summarizes the overall fit statistics of the measurement
model. All model indices fit well with the recommendations
suggested by earlier studies. Therefore, the research model
provided a good fit to the data (Table 7).
We further examined the significance of individual paths, and
the results are shown summarized in Fig. 2. All paths displayed a
significant effect, with p-values less than 0.05. As hypothesized,
system quality and service quality had positive effects on user
perspectives, with path coefficients of 0.61 and 0.17, respectively.
Thus, Hypotheses 7 and 9 were supported. Consistent with
Hypothesis 8, user perspective had a positive effect on net
benefits, with a path coefficient of 0.74. Table 8 presents the four
system selection criteria, consultant’s suggestion, a certified highstability system, compatibility between the system and the business
process, and the provision of best practices, had significant influences
on system quality. Thus, Hypotheses 1a, 1g, 1i, and 1j were
supported. Table 9 shows that three selection criteria, international
market position, suppliers’ technical support and experience, and
support for maintaining and updating the system after going live, had
significant influences on supplier service quality. Therefore,
Hypotheses 2a, 2e, and 2f were supported. In Table 10, the
Beforehand
Consideration Factor
System
Selections
Supplier
Selections
H1a+~H1k+
SERVQUAL &
IS Success Model
System Quality
0.61***
R2 =0.68
User
Perspective
H2a+~H2f+
R2 =0.65
0.74***
Net Benefits
0.17***
Service Quality
Consultant
Selections
H3a+~H3g+
Note: *p-value<0.05, **p-value<0.01, ***p-value<0.001
Fig. 2. Result of the path analysis.
Table 8
Testing results concerning ERP system selection criteria.
ERP system selection criteria
Consider or not
Freq.
p-Value
ERP system selection
criteria and system quality
a. Consultant’s suggestions
b. Flexibility in adjusting demands according to business requirements
c. A complete mechanism for risk management and security control
d. The ability to integrate different platforms and data
e. ERP systems that are used by customers and suppliers
f. Ease of integration with other systems (CRM, SCM)
g. A certified high-stability system
h. Ease of use and maintain
i. Compatibility between the system and the business process
j. The provision of best practices
k. Implementation time
*
p < 10%;
p < 5%;
p < 1%; number of respondents: 278.
**
***
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
245
33
151
127
183
95
226
52
259
19
235
43
188
90
158
120
130
148
229
49
200
78
ERP system selection
criteria and net benefits
0.042**
0.463
0.329
0.403
0.720
0.179
0.998
0.134
0.954
0.267
0.121
0.011
0.008
***
0119
0.173
0.700
0.042**
0.279
**
0.033
0.356
0.597
0.416

8. W.-H. Tsai et al. / Information & Management 49 (2012) 36–46
43
Table 9
Testing results concerning ERP supplier selection criteria.
ERP supplier selection criteria
Consider or not
Freq.
p-Value
ERP supplier selection criteria
and ERP supplier service quality
a. International market position
b. A good reputation in the field
c. Training support by the supplier
d. Financial position
e. Technical support and experience
f. Support for maintaining and updating after going live
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
0.003***
0.137
0.261
0.364
0.553
0.623
0.904
0.476
0.015**
0.338
0.061*
215
63
163
115
182
96
249
29
132
146
138
140
ERP supplier selection
criteria and net benefits
0.664
*
p < 10%;
p < 5%;
***
p < 1%; number of respondents: 278.
**
Table 10
Testing results concerning ERP consultant selection criteria.
ERP consultant selection criteria
Consider or not
Freq.
p-Value
ERP consultant selection criteria
and ERP consultant service quality
a. Consulting fee
b. Consultant’s project management abilities
c. Consultant’s domain knowledge
d. Consultant’s ERP implementation experience
e. Consultant’s ERP implementation experience in a similar industry
f. Consultant’s ERP implementation approaches and tools
g. Consultant’s support after going live
**
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
180
98
143
135
113
165
76
202
124
154
210
68
198
80
p < 5%, ***p < 1%; number of respondents: 278.
*
p < 10%
Fig. 3. ERP system selection criteria.
ERP consultant selection
criteria and net benefits
0.278
0.609
0.262
0.378
0.211
0.209
0.207
0.261
*
0.062
0.770
0.256
0.601
0.088*
0.595

9. 44
W.-H. Tsai et al. / Information & Management 49 (2012) 36–46
Fig. 4. ERP supplier selection criteria.
Fig. 5. ERP consultant selection criteria.
selection criteria, consultant’s ERP implementation experience in a
similar industry, and consultant’s support after going live, exhibited
strong correlations with consultant service quality. Hypotheses 3e
and 3g were thus supported.
Fig. 2 also shows the explanatory power of the research model:
the R2 values show that system quality and service quality
accounted for 68% of the variance in user perspective, which
accounted for 65% of the variance in net benefits. Given the high
explanatory power of the resulting model, it is likely that the
model can predict the results of a quality process from a user
perspective and enhance ERP systems understanding. The results
also implied that the selection criteria had positive influences on
net benefits through service quality process and user perspective.
4.2. Results and management implications
The survey evidence for the percentages and the ranking of
selection criteria are listed in Figs. 3–5.
Our first findings were that: the results were consistent with
previous IS success model research [8] and a balanced scorecard
was able to translate ERP performance more effectively across the
entire enterprise and manage it effectively throughout its
execution.
Second, we found that system quality and service quality in
terms of pre-implementation consideration factors may have
major and moderating effects on user perspectives. These results
may imply that the consistency of perceived system quality and
service quality between ERP systems, ERP system suppliers, and
implementation consultants is not only important in increasing
the degree of user satisfaction but also critical in accommodating
firms’ operations needs and determining the value of an ERP [24].
Consequently, we emphasize the importance of a good fit between
these two roles (i.e., one with the smallest gap) when selecting an
ERP package, and hiring an implementation consultant with the
highest degree of fit competence.
Last, but not least, we found that among the system selection
criteria, consultant’s suggestion presented a significant influence
on the service quality provided by the implementation consultants, and, among the consultant selection criteria, consultant’s
ERP implementation experience in a similar industry and
consultant’s support after going live presented significant influences. However, most organizations only considered compatibility
between the system and the business process among the selection
criterion; the three remaining significant software selection
criteria were not the main considerations when organizations
selected ERP software.

10. W.-H. Tsai et al. / Information & Management 49 (2012) 36–46
5. Conclusions and limitations
ERP systems are, in principle, applicable to all industries and
although it is costly, in general, it may bring major benefits. In our
study, we evaluated the various selection criteria that may directly
influence the service quality of an ERP and its further success. By
confirming our success model, we found that enhanced system
quality and service quality can increase user satisfaction. Although
system quality and service quality were as important in ERP
systems as in the traditional IS environment, a balanced scorecard
concept played an important role in the evaluation of ERP
performance.
Moreover, our findings revealed that the selection criteria
which most users considered to be important for ERP implementation actually had no obvious influence on the success of an ERP
system. This finding has not been reported to date! We conclude
that for an ERP implementation process to be reliable, corporations
should place more emphasis on new selection criteria. Specifically,
they should focus on the criteria shown to facilitate successful ERP
implementation and system success.
Some limitations of our study need mentioning. First, our
results reflected only Taiwanese perspectives; different industries,
cultural contexts and environmental settings may generate
different results. Second, there is an obvious survey bias in the
sample, because those who felt dissatisfied with the ERP systems
were inclined to refuse to participate in the survey. Third, although
there are numerous factors affecting the success of ERP systems,
we only focused on the IS success constructs in our research model.
Finally, the performance evaluation could be a limitation. In a
questionnaire investigation, it is difficult to estimate the various
variables by using the practical data obtained from real financial
statements, management reports, and so on.
f) support for maintaining and updating the system after going
live.
3. Selection criteria for ERP consultant
a. consulting fee,
b. consultant’s project management abilities,
c. consultant’s domain knowledge,
d. consultant’s ERP implementation experience,
e. consultant’s ERP implementation experience in a similar
industry,
f. consultant’s ERP implementation approaches and tools, and
g. consultant’s support after going live.
Appendix B. Benefits of an ERP system
Please mark the appropriate number to indicate the extent to
which the important factors influencing the ERP performance
improvement. Using 7-point Likert-type scales ranging from 1
(substantial deterioration) to 7 (substantial improvement) and from
1 (extremely unimportant) to 7 (extremely important) to evaluate the
ERP performance improvement levels and its importance for their
degree, respectively.
Measurement categories
Please mark the selection criteria which you considered before
you implemented an ERP system (multiple selection).
Service quality
SERVQUAL of system suppliers
Tangibles
Reliability
Responsiveness
Assurance
Empathy
SERVQUAL of consultants
Tangibles
Reliability
Responsiveness
Assurance
Empathy
System quality
Reliability
Flexibility
Ease of use
Accuracy
Response Time
1. Selection criteria for an ERP system
a) consultant’s suggestions,
b) flexibility in adjusting demands according to business requirements,
c) a complete mechanism for risk management and security
control,
d) the ability to integrate different platforms and data,
e) ERP systems that are used by customers and suppliers,
f) ease of integration with other systems (e.g., CRM and SCM),
g) a certified high-stability system,
h) ease of use and maintenance,
i) compatibility between the system and the business process,
j) the provision of best practices, and
k) implementation time.
2. Selection criteria for ERP supplier
a) international market position,
b) a good reputation in the field,
c) training support by the supplier,
d) financial position,
e) technical support and experience, and
User perspective
User satisfaction
Information satisfaction
Software satisfaction
System interface satisfaction
System satisfaction
ERP project satisfaction
Individual impact
Improved job performance
Improved productivity
Improved decision-making ability
Improved identification
problem ability
Improved trouble-shooting ability
Net benefits-direct impact
Financial perspective
Inventory levels
Purchasing costs
Inventory turnover
Customer perspective
On time delivery ratio
Customer complaint response time
On time invoice ratio
Net benefits-indirect impact
Internal business perspective
Acknowledgment
The authors would like to thank the National Science Council of
Taiwan for financially supporting this research under Grant
NSC96-2416-H-008-015.
Appendix A. Motivation and evaluation of pre-implementation
process
45
The performance
improvement
levels after ERP
implementation
Important
factors of
the categories
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567

11. W.-H. Tsai et al. / Information & Management 49 (2012) 36–46
46
Appendix B (Continued )
Measurement categories
Internal data transmission time
Internal interaction frequency
Response time to environmental
volatility
Innovation and learning
Understanding of business process
Job achievement of employees
Product development to market
The performance
improvement
levels after ERP
implementation
Important
factors of
the categories
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
1234567
References
[1] M. Al-Mashari, A. Al-Mudimigh, M. Zairi, Enterprise resource planning: a taxonomy of critical factors, European Journal of Operational Research 146 (2), 2003, pp.
352–364.
[2] E.W.N. Bernroider, IT governance for enterprise resource planning supported by
the DeLone-McLean model of information systems success, Information & Management 45 (5), 2008, pp. 257–269.
[3] B. Birdogan, C. Kemal, Determining the ERP package-selecting criteria – the case of
Turkish manufacturing companies, Business Process Management Journal 11 (1),
2005, pp. 75–86.
[4] M.K. Brady, J.J. Cronin, R.R. Brand, Performance-only measurement of service
quality: a replication and extension, Journal of Business Research 55 (1), 2002, pp.
17–31.
[5] C. Brown, I. Vesey, Managing the next wave of enterprise systems: leveraging
lessons from ERP, MIS Quarterly 2 (1), 2003, pp. 65–77.
[6] F.K.T. Cheung, J.L.F. Kuen, M. Skitmore, Multi-criteria evaluation model for the
selection of architectural consultants, Construction Management and Economics
20, 2002, pp. 569–580.
[7] W.W. Chin, B. Marcolin, P.A. Newsted, partial least squares latent variable
modeling approach for measuring interaction effects: Results from a Monte Carlo
simulation study and an electronic-mail emotion/adoption study, Information
Systems Research 14 (2), 2003, pp. 189–217.
[8] W.H. DeLone, E.R. McLean, Measuring e-commerce success: applying the DeLone
and McLean information systems success model, International Journal of Electronic Commerce 9 (1), 2004, pp. p.31–p.47.
[9] W.H. DeLone, E.R. McLean, The DeLone and McLean model of information systems
success: a ten-year update, Journal of Management Information System 19 (4),
2003, pp. 9–30.
[10] K.K. Hong, Y.G. Kim, The critical success factors for ERP implication: an organizational fit perspective, Information & Management 40 (1), 2002, pp. 25–40.
[11] P. Ifinedo, Extending the Gable et al. enterprise systems success measurement
model: a preliminary study, Journal of Information Technology Management 17
(1), 2006, pp. 14–33.
[12] J. Karimi, T. Somers, A. Bhattacherjee, The impact of ERP implementation on
business process outcomes: a factor-based study, Journal of Management Information Systems 24 (1), 2007, pp. 101–134.
[13] D.G. Ko, Consultant competence trust doesn’t pay off, but benevolent trust does!
Managing knowledge with care Journal of Knowledge Management 14 (2), 2010,
pp. 202–213.
[14] V. Kumar, B. Maheshwari, U. Kumar, An investigation of critical management
issues in ERP implementation: empirical evidence from Canadian organizations,
Technovation 23 (10), 2003, pp. 793–807.
[15] W. Luo, M.J. Liberatore, Achieving it consultant objectives through client project
success, Information & Management 46 (5), 2009, pp. 259–266.
[16] D.G. Ko, L.J. Kirsch, W.R. King, Antecedents of knowledge transfer from consultants to clients in enterprise system implementations, MIS Quarterly 29 (1), 2005,
pp. 59–85.
[17] C. Rettig, The trouble with enterprise software, MIT Sloan Management Review 49
(1), 2007, pp. 21–27.
[18] A. Rai, S.S. Lang, R.B. Welker, Assessing the validity of IS success models: an
empirical test and theoretical analysis, Information System Research 13 (1), 2002,
pp. 50–69.
[19] T.M. Somers, K.G. Nelson, A taxonomy of players and activities across the ERP
project life cycle, Information & Management 41 (3), 2004, pp. 257–278.
[20] F.K. Stage, H.C. Carter, A. Nora, Path analysis: an introduction and analysis of a
decade of research, Journal of Educational Research 98 (1), 2004, pp. 5–12.
[21] W.H. Tsai, M.J. Shaw, Y.W. Fan, J.Y. Liu, K.C. Lee, H.C. Cheng, An empirical
investigation of the impacts of internal/external facilitators on the project success
of ERP: A structural equation model, Decision Support Systems 50 (2), 2011, pp.
480–490.
[22] J. Verville, A. Halingten, An investigation of the decision process for selecting ERP
software: the case of ESC, Management Decision 40 (3), 2002, pp. 206–216.
[23] E.T.G. Wang, K. Gary, J.J. James, ERP Misfit: country of origin and organizational
factors, Journal of Management Information Systems 23 (1), 2006, pp. 263–292.
[24] E.T.G. Wang, H.F. Chen Jessica, Effects of internal support and consultant quality
on the consulting process and ERP system quality, Decision Support Systems 42,
2006, pp. 1029–1041.
[25] J. Wu, Y. Wang, Measuring ERP success: the key-users’ viewpoint of the ERP to
produce a viable IS in the organization, Computer in Human Behavior 23 (3), 2007,
pp. 1582–1596.
[26] B. Youakim, S. Jean, Security and risk management in supply chins, Journal of
Information Assurance and Security (2), 2007, pp. 288–296.
[27] Z. Zhang, M.K.O. Lee, P. Huang, L. Zhang, X. Huang, A framework of ERP systems
implementation success in China: an empirical study, International Journal of
Production Economics 98 (1), 2005, pp. 56–80.
Wen-Hsien Tsai is a professor of accounting and
information systems in the Department of Business
Administration, National Central University, Taiwan.
He is also a certified consultant of SAP financial module.
He received his PhD degree in industrial management
from the National Taiwan Science and Technology
University. He received his MBA degree and his MS
degree in Industrial Engineering from the National
Taiwan University and National Tsing-Hwa University,
respectively. His research interests include ERP implementation and auditing, activity-based costing (ABC),
green production and optimization decision, and
International Financial Reporting Standards (IFRS). He
has published several papers in Decision Support Systems, Omega – The
International Journal of Management Science, Transportation Science, Industrial
Marketing Management, Journal of the Operational Research Society, Tourism
Management, Family Business Review, Enterprise Information Systems, Int. J.
Production Economics, Computers and Operations Research, Computers and
Industrial Engineering, Int. J. Production Research, etc.
Pei-Ling Lee is a PhD candidate in financial management at the Department of Business Administration,
National Central University, Taiwan. She has done her
MSc in finance and investment management at the
University of Aberdeen in Scotland, UK and her BA in
industrial engineering at Chun-Yuan Christian University in Taiwan. Her current research interests are focus
on ERP implementation and management, ERP performance measurement, multi-criteria decision making,
entrepreneurial policy in SME, management accounting
and financial management.
Yu-Shan Shen is a PhD student in financial management at the Department of Business Administration,
National Central University, Taiwan. She has completed
her MSc in international business administration at
Chinese Culture University in Taiwan. Her BA in the
department of International Trade at Chinese Culture
University in Taiwan. Her current research interests
include ERP implementation and management, ERP
performance measurement, activity-based costing and
entrepreneurial policy in SME.
Hsiu-Ling Lin is a PhD student in financial management
in the Department of Business Administration at
National Central University, Taiwan. She is also secretary general, Taiwan Corporate Governance Association. Her research interests include ERP auditing,
corporate governance, ERP performance measurement
and activity-based costing.