KEYS 2.0 Survey
Development of the KEYS 2.0 Survey
The NEA KEYS 2.0 Research The revision of the KEYS instrument was accomplished through a two-phased process. Phase 1 included the revision of existing items and the development of new items specifically related to teaching and learning concepts. Phase 2 involved the pilot testing of the newly revised instrument in schools across the country and the development of preliminary national norms and best practices benchmarks.
Phase 1: The Revision of Survey Items
The Advisory Group
An advisory committee was formed to provide guidance and direction. Committee members included Jomills Braddock, University of Miami; Helen Mark, Ohio State University; Sylvia Rosenfield, University of Maryland; Mark Smylie, University of Illinois at Chicago; and Floraline Stevens, a private consultant from Los Angeles, California. Since the original KEYS had been used successfully and our purposes for the revisions were to refine and improve the instrument, consensus was reached to retain from the original survey a core set of questions that was most predictive of student achievement within each of the five major KEYS. Thus, the KEYS 2.0 instrument retains the “best” of the original KEYS including the five original KEYS (shared understanding and commitment to high goals, open communication and collaborative problem solving, continuous assessment for teaching and learning, personal and professional learning, and resources to support teaching and learning). As part of the revision, all of the questions retained were reviewed and clarified, where necessary, to eliminate ambiguity that could be a source of error.
Expanding the Scope of the Instrument
In addition to the revisions of questionnaire items, KEYS 2.0 provided a major opportunity to expand the scope of the instrument. Whereas the original instrument was limited to an assessment of a school’s organizational conditions that support teaching and learning, KEYS 2.0 includes a large set of questions that focus most directly on issues of teaching and learning such as the school’s quality of its content standards, curriculum, assessments, and instruction. Theoretical Underpinning of Curriculum and Instruction Items The curriculum and instruction concepts and items included in the revised questionnaire focused on general, research-based principles rather than specific pedagogical styles or curricular programs. The items were drawn from several sources including Fred Newman and Gary Wehlage’s work on authentic instruction and assessment at the Center on Organization and Restructuring of Schools, Charlotte Danielson’s framework for teaching, the American Psychological Association’s Learner Centered Psychological Principles, the five core propositions of the National Board for Professional Teaching Standards, and the indicators of school quality identified by the National Study of School Evaluation. Feedback from Focus Groups As part of NEA’s efforts to improve the instrument, the revised and newly developed draft questions (curriculum and instruction) were reviewed by groups of NEA teachers and education support professionals as part of four focus group discussions. As a result of these group discussions, the items were further revised and assembled into a draft questionnaire ready for pilot testing.
Phase 2: The Pilot Test of the Revised KEYS 2.0 Instrument
The School as the Unit of Analysis
Unlike the original KEYS research where the questionnaire was administered to a random sample of NEA teacher members, the KEYS 2.0 instrument was administered to all education employees in several schools. Thus, for the KEYS 2.0 research, the data points used to identify the indicators of school quality represent very stable and valid measures. Thirty-eight schools, representing 84 percent of the 45 schools designated, participated in the KEYS 2.0 pilot test, including 28 schools affiliated with NEA and 10 with AFT. The sample was a national random sample of schools drawn from the U.S. Department of Education, National Center for Education Statistics database. The randomness of the sample was constrained, however, to ensure that all six NEA regions were represented and that the sample included elementary, middle, and high schools in urban, suburban, and rural settings. The sample included 13 elementary schools (five urban, six suburban, two rural); 13 middle schools (four urban, three suburban, six rural); and 12 high schools (four urban, five suburban, three rural). The total number of individuals responding to the questionnaire was 1,491 representing 52 percent of the total population. Seventy-one percent (1,061) were teachers; the other 29 percent included 378 education support professionals and 52 administrators.
Analysis of the KEYS 2.0 Data
In addition to the questionnaire data from the staff in each school, several measures of student achievement data were collected. The measures included all available school test data as well as respondents’ perceptions of how well students were achieving. The school questionnaire data were aggregated and examined using a series of factor analyses to determine if and how items clustered together; that is the extent to which certain items statistically tended to measure the same underlying concepts. This procedure allows for the identification of the conditions or indicators of quality that could be measured. The second part of the analyses involved a series of linear regressions to determine the extent to which the indicators of quality identified were positively related to student achievement.
Results of KEYS 2.0 Research
Forty-two indicators of quality clustered into six main “KEYS” were identified. The first five keys are the same as the ones identified in the original KEYS. The sixth key, curriculum and instruction, is new. Based on the NEA research, as well as reports in the education research literature, all 42 quality indicators are considered important elements of a high-performing school. High scores on all of the 42 indicators, clustered around the six keys, represent an integrated view of a well-functioning, high-achieving school. In addition to identifying the six major keys and the 42 indicators of a high performing school, the NEA KEYS 2.0 research allows each school the opportunity to compare its scores on each indicator with the average scores of all the schools in the pilot as well as with the school scoring in the 90th percentile of the distribution of all the schools in the sample.