Howdifferentsurveytechniquescanimpactconsultantrecommendations.pdf

How Different Survey Techniques Can Impact
Consultant Recommendations

A Scientist-Practitioner Study Comparing Two
Popular Methods

James D. Westaby Teachers College, Columbia University

Very little research has compared how consul-
tant survey methods may result in different rec-
ommendations for change. This study compared
the job satisfaction method to the reason method
to understand employee turnover motives. To
control for content, measures from both meth-
ods were applied to the same attribute catego-
ries, such as job pay, the work itself, and job
security. Results indicated that the reason ap-
proach demonstrated considerably more predic-
tive validity than the job satisfaction approach
and provided unique insight into specific turn-
over motives (e.g., salary and supervisor rela-
tionships). These results directly impacted sub-
sequent intervention choices of the client. Re-
sults suggest that consultants can generate even
better recommendations for clients if they in-
clude reason scales in their survey-based
projects.

Keywords: job satisfaction, reasons, intention,
turnover, survey research

Consultants frequently collect survey in-
formation to understand important organi-
zational behaviors (Shahani-Denning, Bar-
riere, Shapiro, Kaplan, & Metlay, 1999).
Such information is important because it
can help consultants inform clients about
potential targets for change (Sala &
Dwight, 2002). For example, if consultants
find that turnover is significantly correlated
with poor supervisory relationships, con-
sultants can recommend to clients that in-
terventions need to address supervisory is-
sues, such as through leadership develop-
ment training. Fortunately, there are a
number of survey techniques that consult-
ants can use to identify important behav-
ioral determinants, such as scales measur-

ing satisfaction facets, reasons, and beliefs.
However, despite the wide variety of sur-
vey tools used in consulting, surprisingly
little systematic research has compared the
validity of different approaches in actual
field projects (Fuqua, Newman, & Dick-
man, 1999). Hence, it is unclear whether
different survey methods make different
inferences about what drives employee be-
havior. In other words, do the survey meth-
ods that consultants implement create dif-
ferent conclusions that impact recommen-
dations for change? If so, consultants may
be making less than ideal recommendations
in some situations. Thus, the purpose of
this study is to systematically compare two
popular survey methods for identifying the
underlying determinants of behavior. The
first method uses satisfaction scales and the
second uses reason scales.

Before detailing the methods, it is im-
portant to explain how this line of research
helps advance the consulting literature.
First, by using scientist-practitioner ap-
proaches that are “grounded in theory and
research” (Burke, 1993, p. 14), this line of
work scientifically evaluates the effective-
ness of survey methods commonly used in
practice. Pragmatically, identifying scien-
tific methods that demonstrate validity can

Correspondence concerning this article
should be addressed to James D. Westaby, De-
partment of Organization and Leadership,
Box 6, Teachers College, Columbia University,
525 W. 120th Street, New York, NY 10027-
6696. E-mail: [email protected]

Copyright 2006 by the American Psychological Association and the Society of Consulting , 1065-9293/06/$12.00
DOI: 10.1037/1065-9293.58.4.195
Consulting Journal: Practice and Research, Vol. 58, No. 4, 195–205

195

enhance consultants’ recommendations for
change. Second, unlike many studies which
use only one survey method, this study
compares two methods using parallel mea-
sures. Such comparisons not only advance
the psychological literature (Westaby,
2002), but also inform consultants about
effective methods for applied research.
Third, this study compares the methods in a
real consulting project examining em-
ployee turnover (Wilpert, 1995). Examin-
ing turnover is an important topic in orga-
nizational consulting because high levels of
turnover can increase costs in recruitment,
selection, and training (Huselid, 1995).
Hence, many consultants are asked to pro-
vide insight into turnover and retention is-
sues in a wide variety of organizations.
Practically, our examination of survey
methods in this study should provide the
client with helpful information about im-
portant motives underlying employee turn-
over.

In this article, we first define each sur-
vey method used in this study along with a
description of the supporting theory (Blan-
ton, 2000; Burke, 1976). Second, we
present an empirical test of the methods
along with comparative results. Finally, we
articulate how the different methods im-
pacted our recommendations to the client
organization. The general approach taken
in this study is also responsive to Alderfer’s
(1998) recommendation that professional
practice show an “interdependence among
theory, data, method, and values” (p .69).

Survey Techniques

Job Satisfaction Method

Consultants often use job satisfaction
scales in their survey-based projects to under-
stand employee turnover. This is also theo-
retically justified in the psychology literature.
For example, turnover linkage theory (Mob-
ley, 1977) suggests that job satisfaction lies at
the root of the turnover linkage process.
When employees are dissatisfied with various

aspects of their job, they start thinking about
quitting, which drives eventual turnover de-
cisions. Satisfaction variables should thus be
related to turnover criteria because they the-
oretically serve as primary antecedents of the
turnover process. Moreover, because job sat-
isfaction represents an attitude toward work
and because attitudes have been strong pre-
dictors of intentions and behavior (Fishbein
& Ajzen, 1975), job satisfaction should be
related to turnover intentions and behavior. In
support of these expectations, considerable
research has shown that job satisfaction is a
robust predictor of employee turnover (Cot-
ton & Tuttle, 1986).

In the context of this study, the satisfac-
tion method is defined as a measurement
system that assesses the degree to which
employees are satisfied or dissatisfied with
various aspects of the job. For example,
consultants can use this system to assess
employees’ satisfaction with their cowork-
ers, their opportunities for advancement,
and their job security. In practical consult-
ing projects, those specific factors that are
found to be significantly correlated with
intentions or behavior are considered prime
targets for change (Shahani-Denning et al.,
1999). In summary, consistent with past
theory, research, and consulting practice,
the specific factors measured by the job
satisfaction method are expected to be re-
lated to turnover intentions and behavior
(Hypothesis 1). However, despite the pop-
ularity of the satisfaction method, little re-
search has compared how well satisfaction
measures perform in comparison to other
techniques. Thus, we now turn to the rea-
son method, which has also been shown to
be a powerful tool in applied consulting
projects (Westaby & Braithwaite, 2003).

Reason Method

To identify important targets for behav-
ioral change, consultants frequently assess
employees’ reasons for their behavior in
survey instruments (Westaby, Fishbein, &
Aherin, 1997; Westaby, Versenyi, & Haus-

196 Consulting Journal: Practice and Research
Fall 2006

mann, 2005). Reasons theory also provides
a theoretical justification for this in the psy-
chological literature (Westaby, 2005a).
Grounded in contemporary attitude and de-
cision-making models (Clary et al., 1998;
Pennington & Hastie, 1988), reasons theory
states that “reasons” serve as critical psy-
chological determinants of intentions and
behavior (Westaby & Fishbein, 1996). Ac-
cording to the theory, “reasons” are defined
as the specific subjective factors individu-
als use to explain their behavior (Westaby,
2005a). Reasons are also subdivided into
“reasons for” and “reasons against” the be-
havior. This subdivision allows the con-
struct to subsume well established pro/con,
benefit/cost, and facilitator/constraint con-
cepts (Ajzen, 1991; Lewin, 1951; Velicer,
DiClemente, Prochaska, & Brandenburg,
1985).1 Theoretically, reasons theory as-
sumes that reasons are key behavioral deter-
minants because they help individuals justify
and defend their actions, which helps individ-
uals maintain their self-worth (Hsee, 1995;
Kunda, 1990; Steel & Mento, 1986; Steele,
Spencer, & Lynch, 1993). Thus, in the con-
text of employee turnover, employees would
be presumed to feel better about themselves
when they have justifiable reasons to support
their future retention or turnover behavior
(Pieters & Zeelenberg, 2005). Finally, rea-
sons also help people make sense of their
world by providing them with causal expla-
nations for their behavior, the behavior of
others, and causal relationships in their envi-
ronment.

Reasons theory also describes the role
that reasons play in the behavioral deci-
sion-making and intention formation pro-
cess. Specifically, in the context of em-
ployee turnover, the theory states that the
turnover decision-making process begins
with a state of uncertainty and that employ-
ees start resolving the uncertainty by eval-
uating their beliefs associated with the de-
cision options presented (e.g., beliefs about
their supervisory interactions). During the
processing of their beliefs, individuals are

presumed to search for the option that has
the most justifiable and defensible set of
reasons. When that alternative is identified,
intentions are solidified and behaviors are
executed with reasons continuing to sup-
port and defend the behavior over time
(Gollwitzer & Brandstaetter, 1997; Krug-
lanski & Webster, 1996; Phillips, 2002).
For example, if an employee discerns a
much stronger set of reasons for quitting
than for staying, the employee is pre-
dicted to activate an intention to quit.
Hence, because of their motivational po-
tential, reasons should be related to turn-
over intentions.

From a consulting perspective, reasons
are expected to provide pragmatic insight
into behavioral change processes. Accord-
ing to the theory, individuals are expected
to interrupt their behavior when new infor-
mation is presented that causes them to
question their reasons. At this time, their
current reasons may not be sufficient to
counterargue the new information, thus
making them uncertain about their behav-
ior. Consequently, the decision-making
process would be reengaged to identify a
new set of reasons, a new intention, and a
new course of action (Antaki, 1994). For
example, although an employee enjoys his
or her job, the employee may consider quit-
ting because of a potential reconfiguration
of the compensation system. Hence, target-
ing reasons is presumed to be a critical
consultant strategy for developing effective
change interventions (Westaby & Braith-
waite, 2003). Because of their multifaceted
influence on behavioral processes, reason
variables are expected to be associated with
turnover intentions and behavior (Hypoth-
esis 2).

1Reasons are different from Herzberg’s
(1966) two-factor model, which does not posit
explanation processes. Also, research has had
difficulty supporting hygiene and motivator dis-
tinctions in the framework (Gordon, Pryor, &
Harris, 1974).

197Consulting Journal: Practice and Research
Fall 2006

Differences Between Methods

It is important to note key differences
between the reason and job satisfaction
methods. Reasons are conceptually differ-
entiated from other belief-based constructs,
such as expectancy, value, and job satisfac-
tion facets because they explicitly assess
the specific factors people use to explain
their anticipated behavior. Although rea-
sons are connected to explanations, belief-
based concepts can be broad and not con-
nected to explanations. For instance, a per-
son can be satisfied with his or her
coworkers and pay, but these are not the
reasons the person uses to explain his or her
anticipated turnover behavior. Instead, the
person states that he or she will likely leave
the company because of health reasons.
Thus, the positive beliefs about the com-
pany do not become reasons in the person’s
anticipated explanation.

The reason method is also expected to
provide unique insight into employees’ de-
cision-making that is not accounted for by
the traditional job satisfaction approach.
First, reasons capture powerful justification
and defense mechanisms that are not theo-
retically accounted for by job satisfaction
theory (Steele et al., 1993; Wicklund &
Brehm, 1998). Second, consistent with
Kunda’s (1990) theory, reasons may have a
direct effect on intentions because of peo-
ple’s strong motivation to use reasons that
support their overall positions (Boiney,
Kennedy, & Nye, 1997). Finally, reasons
explicitly differentiate between “reasons
for” and “reasons against” performing a
behavior, which may allow the concept to
provide unique insight into behavior (Veli-
cer et al., 1985). The job satisfaction
method, in contrast, does not formally posit
such a distinction.

As empirical support for the above ar-
guments, a recent study demonstrated that
the specific factors measured via reasons
theory explained an additional 13% of the

variance in attitudes toward an organiza-
tional change initiative over and above that
explained by the expectancy-value method
(Westaby, 2002). Furthermore, when re-
sults from both approaches were manip-
ulated in an intervention, manipulations
derived from the reason approach had a
stronger impact on attitude change than
manipulations derived from the expec-
tancy-value approach. Moreover, Westaby
and Braithwaite (2003) found that reasons
provided unique insight into the determi-
nants of unemployed individuals’ reem-
ployment self-efficacy. Results also in-
formed executive management at the De-
partment of Labor about several areas to
target in their change interventions.
These results had a direct impact on the
Commissioner’s subsequent policy deci-
sions. Thus, we expect that the reason
method will provide additional insight
into turnover motives in this consulting
project beyond that explained by the job
satisfaction method (Hypothesis 3).

Method

This consulting project took the follow-
ing form, consistent with recommendations
in the consulting literature (Fuqua et al.,
1999; Nadler, 1977; Shahani-Denning et
al., 1999): (1) executed a planned data col-
lection, (2) assessed relevant data from or-
ganizational members, (3) used appropriate
statistical analyses, (4) synthesized data
from various vantage points, (5) fed back
results to the client organization, (6) fol-
lowed-up with communication to ensure
appropriate interpretation, and (7) gener-
ated action plans based upon survey results.

Participants and Procedure

Surveys were administered to 472 em-
ployees from several units of an organiza-
tion experiencing high levels of turnover.
The client organization was primarily inter-
ested in collecting information to identify

198 Consulting Journal: Practice and Research
Fall 2006

specific turnover motives. Two hundred
and ten individuals agreed to participate
(response rate � .44). Given the impor-
tance of confidentiality and ethics in orga-
nizational consulting (Newman & Robin-
son, 1991; Westaby & Lowe, 2005), at the
beginning of the project, participants were
informed that (1) their responses were com-
pletely confidential, (2) independent re-
search consultants had exclusive access to
their data and that only aggregate data
would be reported to employers, and (3)
they had the right to refuse to participate at
any time. Data were collected over multiple
phases. At Phase 1, a qualitative elicitation
study was conducted to develop items for
the main study. At Phase 2, a closed-ended
survey assessed job satisfaction facets, rea-
sons, and intentions to quit. Finally, at
Phase 3, turnover was assessed approxi-
mately one year after Phase 2.

Survey Development

Phase 1 Qualitative Elicitation Survey

To ensure that our survey was “tailored
to the specific needs of the target organiza-
tion” (Fuqua, Newman, & Dickman, 1999,
p. 18) and was “relevant” to the organiza-
tion’s concerns (Shahani-Denning et al.,
1999), a qualitative elicitation study was
conducted at Phase 1 on 74 employees.
Employees were asked open-ended ques-
tions in which they listed important factors
impacting their job satisfaction and inten-
tions to quit or stay at their job. Data was
content analyzed in accord with procedures
set forth by Fishbein and Ajzen (1975;
Ajzen & Fishbein, 1980). The analysis re-
sulted in seven substantive categories: job
security, coworker relationships, supervisor
relationship, perceived employment oppor-
tunity, pay and benefits, opportunities for
advancement, and the work itself. These
categories were used to develop closed-
ended items for both the satisfaction and
reason method at Phase 2. The same cate-
gories were applied to both methods, which

provided methodological control for our
planned comparisons.2

Phase 2 Survey

To assess job satisfaction factors, partic-
ipants were asked to rate their job satisfac-
tion/dissatisfaction on each of the follow-
ing dimensions discerned from the elicita-
tion study: coworker relationships,
supervisor relationships, pay and benefits,
opportunities for advancement, the work
itself, job security, and perceived employ-
ment opportunity. At least two items rep-
resented each dimension and all alphas
were greater than .83. Consistent with typ-
ical scaling parameters (Weiss, Dawis, En-
gland, & Lofquist, 1967), participants re-
sponded to each item on a 5-point scale
ranging from 1 (very dissatisfied) to 5 (very
satisfied). Item placement was randomly
ordered and a composite was created for
each satisfaction factor.

To properly compare the job satisfaction
method to the reason method, reasons for
staying and reasons against staying (a.k.a.,
reasons for leaving) were assessed on the
same dimensions derived from the elicita-
tion study. To assess reasons for staying,
participants were first presented with the
following statement: “Please rate the rea-
sons why you will stay at this organiza-
tion.” Participants then rated the extent to
which various items corresponding to each
of the following categories represented
their reasons for staying on 3-point scales
with ratings of 1 (not a reason), 2 (a rea-
son), and 3 (a strong reason): good job

2Although the elicitation study indicated that
very few employees stated that “job insecurity”
was a reason for leaving and “not having other
employment opportunities” was a reason for
staying, measures for both dimensions were in-
cluded in the reason sets in order to fully com-
pare the reason factors to the job satisfaction
factors. Manifest items were consistently paral-
lel across the methods for all dimensions to
ensure a fair test of the approaches.

199Consulting Journal: Practice and Research
Fall 2006

security, enjoyable work, lack of other em-
ployment opportunities, good coworker re-
lationships, good pay and benefits, good
opportunities for advancement, and good
supervisor relationships. At least two items
represented each category with all alphas
greater than .82.

To measure reasons for leaving, par-
ticipants were first presented with the fol-
lowing statement: “Please rate the rea-
sons why you will leave this organiza-
tion.” Participants then rated the extent to
which various items corresponding to
each of the following categories repre-
sented their reasons for leaving on
3-point scales with 1 (not a reason), 2 (a
reason), and 3 (a strong reason): poor
coworker relationships, poor supervisor
relationships, inadequate pay and bene-
fits, job insecurity, lack of opportunities
for advancement, unenjoyable work , and
other employment opportunities. Consis-
tent with our previous assessments, at
least two items represented each cate-
gory. All alphas were greater than .80.
Composite scores were created for each
dimension.

Turnover intention was assessed with
four items: There is a strong chance that I
will quit this job within the next year, I will
stay at this organization (reversed), I will
not quit this job within the next year (re-
versed), and I intend to quit this job no
matter what (� � .86). Items were scored
on a 5-point scale ranging from 1 (strongly
disagree) to 5 (strongly agree).

Phase 3 Behavioral Assessment

Because both objective and longitudinal
data bolsters the validity of consulting
projects (Fuqua et al., 1999; Sala &
Dwight, 2002), objective retention data was
collected directly from employers approxi-
mately one year after Time 1. Individuals
working in the organization were scored 1
while individuals not working in the orga-
nization were scored 0.

Systems Approach

In line with systems theory, because it is
critical in consulting work to not focus on a
single aspect of an organization without
considering its relationship to the whole of
the organization (Williams, 1978), we had
extensive discussions with various groups
and representatives in the organization
about our survey findings and their rele-
vance to other aspects of the organization
(Fuqua et al., 1999). Thus, our recommen-
dations for change were sensitized to im-
portant issues occurring throughout the or-
ganization at various levels (Van DeVen &
Ferry, 1980). This should also help mitigate
concerns about relying only upon survey
data when making recommendations to
management (Beer & Spector, 1993).

Results

Regression analyses were used to exam-
ine the predictive validity of the two meth-
ods on turnover criteria. To examine the job
satisfaction method, intention was re-
gressed on the seven satisfaction factors.
Results indicate a significant association
with intention, as expected (R2 � .22; Adj.
R2 � .18). The work itself factor was inde-
pendently associated with intention. Table
1 presents regression coefficients and fit
statistics for both models tested. To test the
method on behavior, a logistic regression
was conducted with longitudinal retention
behavior used as the criterion and the seven
job satisfaction factors entered as a block.
A significant omnibus effect was observed
(Cox & Snell, R2 � .14), with the work
itself variable demonstrating the only sig-
nificant effect.

To examine the validity of the reason
method, the set of reason factors were used
as independent variables in a hierarchical
regression with intention serving as the cri-
terion. As expected, a significant associa-
tion was observed (R2 � .46; Adj. R2 �
.40). Four reasons for staying and two rea-
sons for leaving were independently related

200 Consulting Journal: Practice and Research
Fall 2006

to intention. Unlike the satisfaction
method, the reason method was thus able to
discern additional predictors of intention
beyond the work itself factor. As a supple-
mental test, a logistic regression was con-
ducted with retention behavior used as the
criterion. Results indicated that several rea-
sons were directly related to longitudinal
retention behavior (Cox & Snell, R2 � .36),
such as salary, the work itself, and lack of
opportunities for advancement. Table 2
presents coefficients and fit statistics for
both logistic regressions.

To examine the potential incremental
validity of reasons over job satisfaction,
two additional hierarchical regressions

were conducted. For the first regression,
intention served as the criteria and job sat-
isfaction factors were entered on Step 1.
Reasons were entered on Step 2. In support
of expectations, results indicate a 23% in-
crease in variance explained on Step 2. For
the second regression, behavior served as
the criterion using logistic regression. Job
satisfaction was again entered on Step 1
and reasons on Step 2. As expected, a sig-
nificant increment in fit was observed on
Step 2 (Cox & Snell, R2 � .36)

Discussion

This study examined the extent to which
two psychologically based survey methods
resulted in different conclusions about the
underlying determinants of employee turn-
over. The first method used job satisfaction
scales while the second used reason scales.
In line with expectations, results indicated
that job satisfaction was related to turnover
criteria, with the work itself variable dem-
onstrating significance. This finding is con-
sistent with the organizational psychology
literature, which shows that the work itself
facet captures the majority of variance in
job satisfaction (Roznowski, 1989). Thus,
our first formal recommendation to the cli-
ent was to focus on improving various as-
pects of the work of employees. In response
to this, management committed to examin-

Table 1
Summary of Hierarchical Regressions Explaining
Variance in Turnover Intention

Variable

Job
satisfaction

facets

Reasons
for

leaving

Reasons
for

staying
� � �

Coworker .07 �.03 .02
Pay .11 .00 �.24**

Work �.24** .18** �.22**

Supervisor �.08 .10 �.18**

Promotions �.13 .11 �.14*

Security .01 .01 �.03
Opportunities .02 .21** .00
R2 .22** .46**

Adjusted R2 .18 .40
F (df) 6.55 (7) 12.11 (14)

* p � .05. ** p � .01.

Table 2
Summary of Logistic Regressions Predicting Time 2 Retention Behavior

Variable

Job satisfaction facets Reasons for leaving Reasons for staying

Wald R Odds ratio Wald R Odds ratio Wald R Odds ratio

Coworker 0.15 .00 1.11 0.91 .00 1.71 0.79 .00 1.32
Pay 2.20 .05 1.34 0.35 .00 1.14 6.32** .17 2.08
Work itself 4.23** .11 1.70 6.21** �.17 0.47 9.13** .20 2.21
Supervisor 0.15 .00 0.91 0.55 .00 0.66 0.89 .00 0.70
Promotions 3.11 .09 1.41 3.88** �.11 0.47 2.92 .06 1.90
Security 0.01 .00 1.01 1.80 .01 0.56 0.00 .00 0.96
Opportunities 0.68 .00 0.81 0.02 .00 0.95 0.95 .00 1.74
�2 (df) 22.08** (7) 59.06** (14)
�2 log likelihood 167.72 105.22
Cox & Snell R2 .14 .36

** p � .01.

201Consulting Journal: Practice and Research
Fall 2006

ing several relevant issues, such as job de-
sign, job enrichment, workplace ergonom-
ics, and work flow processes. However,
there was a concern about the job satisfac-
tion method because it only identified the
work itself variable as significant. Thus, the
method provided relatively little diagnostic
insight into the underlying determinants of
turnover.

In contrast, the reason method provided
a more diverse understanding of turnover
intentions and behavior. Beyond the work
itself factor, the reason method also found
that compensation, supervision, opportuni-
ties for advancement, and employment op-
portunity reasons were also independently
related to turnover. These findings had a
direct impact on our recommendations for
improving retention in the units, which our
client found highly relevant and valuable.
For example, unlike the job satisfaction
method, which found no impact for salary
satisfaction on turnover criteria, the reason
method found that salary was an important
predictor of both intentions and behavior,
which is consistent with other findings in
the human resource management literature
(Huselid, 1995). Our client concurred with
this finding because it corresponded with
the client’s anecdotal experiences as to why
some employees were leaving the units.
Thus, our second formal recommendation
to management was to engage in a formal
evaluation of the current compensation and
benefits packages, including focus group
discussions with relevant parties and a po-
tential search for new “benefits vendors.”

The next issue in our recommendations
dealt with opportunities for advancement.
Because the reason method found that the
opportunity for advancement reason was
significantly related to retention criteria,
our third formal recommendation to the
client was to improve employee develop-
ment and succession planning programs. In
response to this recommendation, the client
commissioned a task force to investigate

specific processes to enact this recommen-
dation over the next calendar year.

The reason method also indicated that
supervisor relationships were a concern.
That is, employees were forming intentions
to quit when they expressed reasons that
dealt with poor supervisory relationships.
Thus, our fourth recommendation for
change suggested leadership development
training for supervisors and team building
activities for select workgroups. Team
building activities may improve the climate
between supervisors and their employees,
which has been shown to be a potent vari-
able in effective organizational functioning
(Barriere, Anson, Ording, & Rogers, 2002;
Sheridan, 1992). The client used this rec-
ommendation as a motive to schedule a
variety of such activities among relevant
units.

Finally, given the finding that employ-
ment opportunity reasons were a critical
correlate of turnover intention (Hulin,
Roznowski, & Hachiya, 1985), we recom-
mended that management attempt to collect
ongoing information about why their de-
parting employees are attracted to other
employers, such as through formal exit in-
terview techniques (Campion, 1991). Fur-
thermore, we recommended that the client
more seriously compare their human re-
source practices with other organizations in
their industry, such …