Faculty
Richard Ilkka - Research
Improving Faculty Selection: The Critical Indices Approach
Published--The Department Chair, 6, Summer, 1995 (Anker Publishing, Bolton, MA)
Hiring a new faculty member may be among the most important decisions a department ever makes. The new hire represents the department’s future and along with other new hires will help shape image, culture, and students for years to come. The hiring decision is also an expensive one, possibly accounting for around a million dollars or more of the university’s money over the course of a career. Even more will be spent if poor choices are made and new searches are required. In such cases, the department chair is at least somewhat accountable if for no other reason than at most colleges and universities, it is the department chair who is primarily responsible for creating and overseeing the selection process.
That responsibility begins with effective selection of both a search committee chair and committee membership (e.g., Ilkka, 1995). It continues with the department chair facilitating the committee’s efforts in any number of unobstrusive ways ranging from making sure permission to hire has been accomplished to providing secretarial and budget support. But perhaps the most important facilitation may involve helping the search committee develop and ask job relevant questions as well as engage in meaningful answer assessment. Encouraging the search committee to attend to effective questions development is crucial for a couple of reasons. First of all, the low validity and reliability of the employment interview as an assessment measure across various professions is fairly well established in the interviewing literature (e.g., Dipboye, 1992). The reasons for such problems of validity and reliability are related to such concerns as lack of position clarity, problems with interview structure, and of particular interest here, skill in developing and asking position relevant questions. Secondly, university faculty search committees are generally not composed of individuals with extensive prior training in personnel assessment and thus, they are neither any more or less skilled as employment interviewers than those who conduct interviews in other professions. In turn, while it may be presumed that faculty avoid judgments based on race, sex, age, disabilities, or attractiveness, it is not unreasonable to assume that some future faculty search committees at otherwise fine institutions will still (1) ask far too many job irrelevant questions (versus job relevant ones); (2) not ask these questions in a consistent manner, e.g., different questions and different sequencing of questions; and (3) not meaningfully compare candidate answers against previously established and agreed upon benchmark responses. Given these assertions, the remainder of this article offers an approach to question development which is easily implemented and decidedly more job relevant.
Question Development
Critical Indices questions are questions which emerge from a thorough job analysis and which focus upon specific knowledge, skills and abilities (KSAs) the candidate may need to exhibit in order to meet certain job tasks. While the committee may develop a large number of useful questions, the department chair should encourage the committee to develop at least five or six Critical Indices questions which speak to the KSAs needed to accomplish various job requirements. The committee might actually wish to develop two sets of such questions, one set for a screening interview (e.g., teleconference) which focuses upon establishing comparative professional quality, and another set for the on-site interview which further examines professional quality but also addresses personal fit.
As developed here, the Critical Indices approach draws from and combines ideas developed by Feild and Gatewood (1989), Janz (1989), and Janz, Hellervik, and Gilmore (1986). While others (e.g., Coady, 1990; Watts, 1993) have emphasized the value of descriptive interviewing, this approach incorporates both descriptive and situational interviewing, agreeing that one of the best indicators of future behavior is past behavior (descriptive focus--"what did you do when..."), but unwilling to discard the value of well developed, hypothetically construed contexts (situational focus--"what would you do if....") for assessing some aspects of a candidate's potential (Janz, 1989). The Critical Indices approach also recognizes the value of what is termed interrogative questions, follow up questions for both descriptive and situationally based interview questions (see Appendix A). Below, each of the three question types is further discussed and illustrated.
Descriptive Behavior Questions -- Janz (1989) notes that there are four types of interview information: credentials, experience descriptions (e.g., surface discussion on duties, responsibilities), opinions (includes self perceptions and commentary on other contexts, plans, goals, etc.), and behavior description (detailed accounts of actual events from the applicant's work and life situations). Janz believe the latter category is most useful in that it reveals specific choices made as well as indications of the circumstances encompassing such choices. Descriptive behavioral patterned interviewing involves questions which seek to contextualize and specify the more general and often philosophical questions asked in interviews. Instead of a question which asks, "what kind of relationship between teacher and student best fosters learning?" the descriptive approach asks, "tell me about a teaching situation from your past which best illustrates the kind of relationship that should exist between teachers and students?" In any event, the descriptive question forces the candidate away from abstract and perhaps even "canned" answers to instances which reveal choices made and values actualized. In a sense, the shift is simply from conjecture to specific instance, but in another and more important sense, it is a shift from detached reflection to more personal revelation (see Appendix B).
Situational Questions -- To ask candidates to answer all questions based on specific, past experiences could easily become counter-productive if for no other reason than candidates may not have the requisite experience from which to draw. Thus, the use of hypothetically construed or situational questions are appropriate, especially for candidates with limited job experience. For example, suppose the job analysis underscored the importance of development and delivery on an effective introductory course in the discipline. The experienced teacher might simply be asked to discuss her most recent rendition of the introductory course. The novice, however, might be asked to construct a hypothetical course syllabus with such attendant questions as, how would you structure the course? Explain what materials you would select and within that corpus, what might you emphasize? What key readings? Assignment? Evaluation system? --and of course, the lingering "why" is also available. Admittedly, the hypothetical situation reduces expectations for a richer, more personal and contextualized answer, however it may still provide committee members with the opportunity to assess the candidate's ability along a variety of relevant dimensions, for example, problem solving, organizational skills, subject matter competency, and resource awareness (see Appendix B).
Interrogative Questions -- These are the "why" and "how come" probes of the candidate's initial answers, and can serve one or more functions, (a) promoting understanding, (b) providing modest confrontation in order to test commitment to views, (c) illustrating the candidates’s ability to re-examine a response based upon new information and/or perspective offered by a committee member. Moreover, such answers might provide some initial guidance for the kind and amount of mentoring likely to be needed should the candidate subsequently be selected.
Sequencing
While Critical Indices questions may constitute only a small portion of the total questions asked by a committee, it is important that all the Critical Indices questions asked of one candidate be asked of all candidates, and to the extent possible in the same order and with the same amount of time available for an answer. Without scripting the interview too tightly, the selection committee chair might actually organize and in order, cue members assigned to ask predetermined, critical incident questions. While it is impossible to control every intervening variable in every interview, it is important to provide each candidate with as even an opportunity to respond to Critical Indices questions as is possible. And, while other information and answers to non-Critical Indices questions may significantly impact on candidate assessment, evaluation of candidate answers will be significantly enhanced if at least some answers to identical questions can be meaningfully compared across all candidates.
Benchmarking
Once the primary candidates have been screened and after the top few have made an on-site visit and been interviewed by the search committee, it is imperative for both legal and ethical concerns that the decision to offer the position to a given candidate be based upon objective, interview-based information. For example, while decisions based on age or sex are illegal, the search committee could also not invite and then later reject a candidate based on a "disqualifier" previously evident in the written materials (e.g., lack of a terminal degree). Instead, it is to everyone’s advantage to be able to make a clear and thorough comparison of each candidate based upon answers given to undeniably job relevant questions which were asked of every candidate in approximately the same way and sequence. And, while such answers as noted on paper (or with permission, as recorded on tape), might be compared among candidates, the committee should also have developed its versions of high quality answers, acceptable answers, and an unacceptable answers for each Critical Indices question. Following each interview, individual members would evaluate answers to each of the Critical Indices questions. Such evaluations could then be collated and discussed with reference to the committee developed benchmarks. Obviously, unanticipated and yet excellent variations on the benchmark answers might emerge, and as a consequence, the committee may have to re-consider its benchmark and/or reconsider the viability of the question itself.
Conclusion
While there are many ways in which the department chair can facilitate a more effective search process, the chair should make every effort to assure that the search committee will ask questions which allow for a useful, comparative assessment of the candidates who have made it to the short list. Questions which address specific behaviors, which are asked in the same way and the same sequence for every candidate, and which are evaluated against established benchmarks, should improve the value of the screening and selection interviews in particular as well as the overall search process. To the extent that the department chair is able to foster the use of a question development approach as outlined above, the interests of the department, the institution, the students, and the eventual hire, are more likely to be served.
References
Coady, S. (1990). “Hiring faculty: A system for making good decisions.” CUPA Journal, 41, 5-8.
Dipboye, R.L. (1992). Selection interviews: Process perspectives. Cincinnati, OH: South-Western Publishing Co.
Feild, H.S., & Gatewood, R.D. (1989). “Development of a selection interview: A job content strategy.” Eds. R.W. Eder & G.R. Ferris, The employment interview: Theory, research, and practice. Newbury Park, CA: Sage.
Ilkka, R.J. (1995). “Selecting Members of The Faculty Search Committee.” The Department Chair, 5, 11-13.
Janz, J.T. (1989). “The patterned behavior description interview: The best prophet of the future is the past.” Eds. R.W. Eder & G.R. Ferris, The employment interview: Theory, research, and practice. Newbury Park, CA: Sage.
Janz, J.T., Hellervik, L., & Gilmore, D.C. (1986). Behavior description interviewing: New, accurate, cost effective. Newton, MA: Allyn & Bacon.
Watts, G.E. (1993). “How to hire good faculty.” AACC Journal, Jun/Jul, 29-33.
Appendix B
--Sample Questions Sheet--
TRADITIONAL QUESTIONS
- How do you define good teaching?
- How have the works of leading scholars in our field influenced you?
- How would you describe your classroom relationship with students?
- What are your greatest strengths & weaknesses as a teacher (scholar)?
- What is your philosophy regarding the evaluation of students?
- In terms of your professional goals, where would you like to be in the next five years?
BEHAVIORAL QUESTIONS
- Tell me about a time where you excelled as a teacher?
- Given who you consider to be among the most influential scholars in our field, how have they impacted on your teaching (or your scholarship)?
- If a student challenged your views in class in a fairly convincing manner, how would you respond to the student and class?
- What would you say were the strongest and weakest aspects of your teaching the last time you taught the introductory course? (or in the last article you published?)
- What evaluation system did you use in the last graduate seminar you taught?
- What two or three accomplishments from the past five years might best indicate where your professional career will take you in the next five years?