Advertisement

Pilot Situational Judgment Test

Description:

This measure utilizes a situational judgment test (SJT) format to assess the judgment skills of general aviation (GA) pilots. An initial 51-item test was constructed in which each item described an in-flight situation that required a decision on the part of the pilot and 4 alternative solutions to the situation. Subject matter experts were used to generate a scoring key from the alternatives that the experts recommended for a typical GA pilot.

Two development studies were conducted. In Study 1, the test was administered in paper-and-pencil format to 246 participants. Results from that first study showed that the test had acceptable psychometric properties in terms of internal consistency and distribution. In Study 2, the test was administered to 467 participants over the Internet. Analyses resulted in the reduction of the test to 39 items.

Evaluation of mode of administration (paper-and-pencil vs. Internet) showed that the 2 forms were equivalent. The results from the two studies suggest that the SJT has potential for use in the assessment of judgment or aeronautical decision making by GA pilots, and might be useful in the evaluation of training.

Number of Items: 51 in full version. 39 in reduced version.

Format:

Multiple-choice format. The stem consists of a one-paragraph description of an aviation situation. Four alternatives are presented, from which the subject must select the alternative that best solves the problem.

Scoring:

A key is provided, based on recommendations from subject matter experts. This key also matches the modal response of a large sample of subjects. Each item is scored as right (1) or wrong (0). Maximum score (for version 2) is therefore 39. Alternately, percent correct (a linear transform of the sum) may be used.

The 39-item version consists of the following items:

N39_rite = SUM(q1a, q2a, q3a,q5a,q7a,q8a,q9a,q10a,q11a,q12a
,q13a,q15a,q16a,q17a,q18a,q19a,q21a,q22a,q23a,q25a,q27a
,q28a,q30a,q31a,q32a,q34a,q35a,q36a,q37a,q38a,q39a,q41a
,q43a,q44a,q45a,q46a,q47a,q50a,q51a).

Factor Composition:

No factor analysis studies reported.

Reliability:

Internal consistency (Alpha) for the 39-item version is .747, for a sample of N = 467.

Construct Validity:

Construct validation was accomplished through correlation with a measure of the number of times the participant had been involved in an accident or other hazardous aviation event. Pilots who had higher (better) scores on the SJT were found to have experienced fewer hazardous events (HES), which was taken as supporting the construct validity of the test.

For N = 115, the correlation between the PJT and the HES was –. 215 (p = .021).

Normative Information:

From Study 2, (N = 467) the mean number-right score for the 39-item scale was 23.5 (60.2% correct); the standard deviation was 5.32. Scores were approximately normally distributed.

References:

Hunter, D.R. (2003). Measuring general aviation pilot judgment using a situational judgment technique. International Journal of Aviation Psychology, 13, 373-386.

Download the scale (Word format)