NCATE and the Accreditation Process

The purpose of the unit's assessment system is to ensure that all candidates exit the programs with the knowledge, skills, and dispositions necessary to be scholarly, creative, reflective, and community-minded professionals. Able to work in diverse settings.  Data from the system is used to strengthen the programs as well as to improve program operations and the overall management of the unit.

Element 2A:  Assessment Plan

Assessment of Teacher Candidates:
The faculty and administration of the unit have approved overall guidelines for the assessment of candidates' knowledge, skills, and dispositions.  These guidelines include:

• The use of intended learning outcomes that are subsumed as proficiencies under the goals and values of the unit's Conceptual Framework.
• The recognition of key transition points, allowing for progressive assessment of outcomes as candidates move from applicant status, through candidate performance, and ultimately to professional practice as graduates of the programs.
• The utilization of multiple assessment tools at multiple levels of analysis.  These tools, many of which have been in place for years, allow for the tracking of important variables related to both candidate performance (e.g., entering GPAs for program admission, student teaching evaluations, state examinations, and graduating GPAs) and program and unit quality (e.g., course evaluations and alumni and employer surveys).
• The development and incorporation of program-based portfolios as a new formative and summative assessment tool.  Portfolio-based assessments, both during specific courses and at transition points, are unique to individual programs, although certain "benchmarks" are designed to provide evaluative information to the entire unit. 

Quantitative and qualitative data resulting from myriad assessment tools are aggregated at the unit level, allowing for triangulated measures of success.  Some of these tools are specific to candidate achievement, including test scores and cooperating teacher feedback, while others, such as course evaluations, candidate satisfaction measures, and faculty surveys, focus on instruction.  Still other measures, including results of employer surveys, provide insight into unit quality as seen by members of the professional community.

The unit has created an assessment database in a centralized NCATE office.  This office serves as a unified location for the compilation of both program-specific and unit assessment results, including existing data that are stored at the university level.  More specifically, the database provides the following:

• support for NCATE and SPA reporting needs
• a system to track candidates' progress
• a platform to support data-based decision making

By locating this function in a single office, the unit has enabled more timely collection, analysis, and distribution of performance data. 

Table 2-1 lists candidate and program performance and unit operations on a range of assessments organized in categories of "internal" and "external" indicators.

Several of the assessment indicators for candidate performance and program performance are described in the paragraphs below.  Those involved in assessment of unit operations are described in Standard 6 of this report.


Candidate Performance
In Spring 2002, the initial components of an assessment system were in place.  All teacher preparation programs were required to submit explicit criteria for admission, progress and exit standards to the New York State Education Department.  These requirements, along with the INTASC and NCATE standards, formed the basis of the assessment system that was submitted as the pre-conditions Standard 4.4 document (Exhibit S2-1).  To establish an organizational structure to formally coordinate the development and implementation of a unit assessment plan, an Assessment Committee was created as a standing committee of the Ruth S. Ammon School of Education.  The committee included representatives from each of the certification programs, the office of School/Community Partnership, the office of the Dean, and the university's Faculty Center for Professional Excellence.  At strategic points, other members of the professional education community were asked to review and provide input into the development of the assessment system. 

A Portfolio Subcommittee was created from the Assessment Committee to evaluate software to implement a unit-wide electronic portfolio that would collect, store, and calculate program data.  A recommendation to use College LiveText was accepted by the faculty, and faculty and candidates in the unit are currently employing it for portfolio and other data management and review.  Agendas and minutes of the Assessment Committee and Portfolio Subcommittee are available as Exhibit S2-2; sample pages from the LiveText-based portfolio work by candidates are available as Exhibit S2-3.  A developing part in the implementation of the assessment system has been selecting a computer information management system for data collection, storage, and aggregation. The evolution to the digital portfolio is addressed in greater detail at the following website: http://fcpe.adelphi.edu/News/Issue8/n8_steve2.htm.

Each of the initial and advanced programs offered by the unit has developed their own candidate outcomes assessment plan.  Each plan must conform to the overall unit guidelines for candidate assessment, in addition to reflecting the standards of its specialized professional association (SPA).  Although SPA requirements are customized to conform to the standards and assessments unique to each discipline, there is substantial overlap. Programs that admit candidates for initial and advanced certification have distinct requirements as evidenced by distinguishable syllabi and program templates for the professional portfolio.  The assessments administered in Educational Leadership and Literacy Education are tailored to candidates who have been admitted with an advanced certificate in their fields. Psychology and Communication Sciences and Disorders are post-baccalaureate programs preparing candidates to work in roles other than teaching. Differences in assessment with initial programs are related to the practicum requirements.  Comparing the initial program and advanced program requirements to comply with the elements of NCATE Standard 1 over the various transition stages demonstrates the assessment differences in the capstone experience and field preparation. The practicum and clinical experiences in Literacy, and Educational Leadership for example, include earlier implementation of fieldwork methodology and action research, as well as a focused set of in-service criteria (e.g., Literacy Center Observations) for certification.

New York State Certification Examinations (NYSTCE):  Each of the initial and advanced programs has New York State certification examination requirements that are part of their assessment plans.  Effective February 2, 2004, candidates to receive an initial New York State teaching certificate in most fields are required to achieve passing scores on the Liberal Arts and Science Test (LAST), the elementary or the secondary version of the Assessment of Teaching Skills-Written (ATS-W), and on a Content Specialty Test (CST) in the content area of the certification. Most initial programs have established Exploration as the program point where the Liberal Arts and Sciences Test (LAST), consisting of multiple-choice test questions and a written assignment, is taken. Candidates who take the test are asked to demonstrate conceptual and analytical skills, critical thinking and communication skills, and multicultural awareness. The test covers scientific, mathematical, and technological processes; historical and social awareness; artistic expression and the humanities; communication and research skills, and written analysis and expression.  Before candidates move to the "Reflection" point of their program they have to take the CST examination, consisting of multiple choice test questions and a written assignment.

Candidates in the advanced programs must pass tests reflecting their disciplines, as well.  Literacy candidates take the New York State teaching certification examinations, those in Communications Sciences and Disorders programs take the PRAXIS II.  The majority of candidates in the Communications Sciences and Disorders also seek certification as speech teachers and also take the State certification exams (LAST and ATS-W).  New York State has postponed a proposed certification test for those seeking educational leadership (building-level administration) certification until 2007.  School psychology uses the PRAXIS II voluntarily in order to evaluate candidate progress.

Benchmark Assignments and Portfolio Assessment: All programs were asked to provide evidence that they had defined specific candidate and measurement processes consistent with the Conceptual Framework, state licensure requirements, and professional standards.  The faculty created templates (Exhibits S2-4) for each course in their program, identifying proficiencies based on the Conceptual Framework's six core values, as well as benchmark assignments (Exhibit S2-5).  These benchmark assignments and their accompanying rubrics were developed in accordance with the Specialized Professional Associations standards.  The templates also identify the state standards, and any 'field work' associated with the course.  The course templates were prepared using a consistent format.  While the scope and sequence of the initial and advanced programs vary, no difference in the format used for the courses exists among the programs.  The expectation is that there will be evidence of growth as candidates progress through their coursework.

Benchmark assignments are an important measure used to monitor candidates' growth.  They are considered by the programs' faculty to be key assignments within these courses, reflecting knowledge, skills, and dispositions that are explicit performance outcomes for the courses which are in alignment with the professional and state standards, on the one hand, and candidate proficiencies from the Conceptual Framework, on the other.   They vary in scope and complexity, depending on the nature of the course and its "placement" within the overall program.  A foundations course, for example, might require a benchmark assignment of a professional autobiography drawing on the seminal readings for the class.  A methods course might require a content-based unit plan that would be implemented in a classroom.  Taken together, they are representative of the array of knowledge and skills that candidates should have mastered as they exit the program and begin work as a professional.  These benchmark assignments are evaluated by the use of rubrics that explicitly establish levels of knowledge and skill along a continuum used for all of the benchmark assignments across programs:  "Distinguished," "Proficient," "Basic," and "Unsatisfactory."   The dimensions of the rubric evaluated in these four categories include the knowledge/content, conventions, and organization/development and, where possible, the elements of the Conceptual Framework and related proficiencies.

Early in their programs, candidates are expected to build a "developmental" portfolio containing these assignments, as well as artifacts and personal reflections from other courses and learning experiences. These developmental portfolios are reviewed at transition points identified by each program.  After admission to the program (entitled "Induction" below), the candidate is introduced to the portfolio process ("Exploration" below) and the portfolio work is reviewed during "Synthesis" and "Reflection" transition points of the candidate's program.  The STEP program has an additional evaluation point as candidates are evaluated after they express an interest in becoming a teacher to determine their "candidacy" status. At these points, the candidate reflects back on growth and accomplishments with a faculty member serving as mentor and guide, not just evaluator. Most initial programs house these evaluation points in specific courses, while the advanced programs evaluate candidates at certain points of their program via assigned advisors. Sample portfolios of candidates' work are offered as Exhibit S2-6.

Table 2-2 identifies the transition assessment points in relation to initial and advanced programs of preparation for our candidates.  The assessment points for the initial programs are the same.  However, there are some differences among the advanced programs based on significant differences in the professional training models used by each.  As indicated in the table, the advanced programs do not distinguish between the Exploration and Synthesis transition points.  The program advisement sheets for the initial and advanced programs are provided as Exhibit S2-7.  They identify for the candidates the courses that contain the benchmark assignments and the points at which the developmental portfolio is to be submitted.

Table 2-2: Transition Points

As shown in Table 2-2 the assessment system is divided into phases:

Induction:  Admission to program.  Candidates applying for admission to initial and advanced professional education programs must adhere to the specific criteria identified by the "unit" and by specific program requirements.  The assessment plan (available in the data books, explained below, and collectively as Exhibit S2-8) of each program identifies these requirements under "Induction". Applicants either seeking to begin an initial certification at the undergraduate level or contemplating a graduate program to change a career are assessed to ascertain their academic ability and their dispositions to become teachers at this first transition point. Applicants for advanced programs, adding to their initial certification, are assessed to ascertain their professional experience, academic ability and appropriateness for the new certification license sought.  Several programs use a rubric to assess potential candidates to their program.  The measures of compliance with the rubric vary from program to program. The rubric is in alignment with those used for the benchmark assessments within the programs' courses and emphasizes two additional elements of the Framework:  reflective activity and creativity (of expression).  Additional points are given for the essay based on evidence of the applicant's experience and/or emphasis on the other three elements of the Framework, that being social justice, inclusive environments, and wellness.  Letters of recommendation are also scored.  Sample admissions rubrics by program are presented collectively as an exhibit and in the program data books (Exhibit S1-3).   The unit requires a baccalaureate degree from an accredited educational institution for all candidates admitted to the graduate initial programs. At the point of admission graduate candidates must have a liberal arts or science background and demonstrated knowledge (30 credit hours) in a specific content area (in the University graduate bulletin and advisement sheets for initial programs, Exhibits S2-7).  In relation to the Conceptual Framework, the academic record of the applicant is viewed as a reflection of their scholarship potential (the first element of the Framework).

Exploration:  Foundation coursework. By the end of this transition point, candidates have explored and analyzed the foundations of their disciplines while taking the first steps towards developing their professional identities as educators. Depending upon program of study, the portfolio will be introduced during or shortly after the completion of the first 12 credits of study.  Candidates demonstrate their knowledge, skills, and dispositions though a variety of course-related assignments. This transition point is used to monitor candidate preparedness for meeting professional standards regarding their written expression and academic research skills, in alignment with the scholarship, reflection and creativity elements of the Conceptual Framework.  This aim is to identify candidates—both in initial and in advanced programs--who may need to further develop their writing or learn new research skills to fully participate in academic and professional work.  Candidates' growth in the other three elements of the Framework—social justice, inclusive environments, and wellness—is evaluated in some programs by the rubric scores on the benchmarks.  Each program requires the completion of at least two benchmark assignments during the exploration stage as well as having passed the NYSTCE LAST.  GPA, portfolio assessment, and test scores will be used to evaluate whether a candidate can move into the "synthesis" point of the program.

Synthesis: Pedagogical coursework and Field experiences. This transition point prepares the candidates for their clinical or practicum experience.  This is true for both initial and advanced programs because the latter involves learning new knowledge and skills that are specific to a different professional role (for example, going from a background and degree in classroom teaching to a specialist in reading or to a building administrator).  The courses during this stage allow candidates to integrate their general education and content specialization with pedagogical/professional knowledge. Candidates must demonstrate aptitudes and abilities enabling the synthesis of foundational knowledge in the field of education, disciplinary knowledge, and curriculum theory. Many of the professional education courses during this phase of the programs have field experiences/internships. Both through the benchmark assignments (which typically involve unit and/or lesson plans) and the fieldwork (with related evaluation of the candidates' knowledge and skill), the candidates' growth in the elements of the Conceptual Framework is highlighted:  evidence of scholarship and reflective activity, social justice, inclusive environments, and wellness (working with the whole child).  Portfolios at this transition point should reflect each candidate's ability to integrate theory with practice as well as their creativity in designing curriculum, the sixth element of the Framework, which is included in the rubric used to evaluate the portfolio. At the end of this transition point, the candidate's eligibility for student teaching or internship must be approved by the advisor or program director.  The candidate's developmental portfolio, submitted at this transition point, is crucial to this decision.  Candidates are required to file a student teaching application with the Office of School/Community Partnerships.  Candidates in advanced programs follow similar processes but submit their applications for their professional field experiences directly to the programs in which they are enrolled. Again, each program requires the completion of at least two benchmark assignments during the synthesis stage as well as having taken the CST and the ATS-W NYSTCE.

Reflective Practice:  Clinical practice/Internship.  In order to complete the clinical practice/internship successfully, candidates are required to complete the established hours of clinical practice in appropriate placements and submit their developmental portfolios with adequate evidence of basic competency in the areas defined by the unit, professional, and state standards. The portfolio at this point must demonstrate each candidate's aptitudes and abilities to analyze complex texts and situations, to synthesize theory into her/his planning and instruction, as well as to reflect on their work in the classroom. As such, the portfolios illuminate the candidates' developing scholarship and reflective activity of the Conceptual Framework.  As well, candidates must demonstrate professional knowledge while clearly establishing her/his roles and identities as professional educators engaged in their school communities (representative of inclusive environments) and committed to all of their students (representative of social justice and wellness of the Conceptual Framework). Again, this is true for both initial and advanced programs that require a supervised field experience in the area of specialization that candidates seek.  Exhibit S2-8a provides examples of student teaching and internship evaluation forms specific to each program.

Professional Practice:  Program Completion.  To graduate from the master's level professional education programs, all candidates in the initial programs are required to successfully complete the requisite coursework and clinical/internship experiences, as well as present a performance portfolio for assessment.  In addition, they must complete a research project in the capstone course (Master's Seminar).  Examples of candidate's performance portfolios and Master's Seminar research projects can be found in the Exhibit Room (Exhibit S2-9). The internships in advanced level programs follow very similar processes.  In at least some of the programs, the Master's Seminar research projects and capstone assignments occur before Reflective Practice.

Dispositions Assessment:  The unit's faculty is committed to evaluating candidates' appropriateness for entry into their chosen professional field beyond their academic achievement. Each program faculty established its own dispositional attributes by aligning them to these six broadly defined dispositions to assess their candidates. Each program's instrument for assessing candidate's dispositions can be found in the program data books and as Exhibit S1-21. Each program's assessment plan identifies at which stage candidate's dispositions are assessed.  The dispositions instrument and its use are discussed in greater detail in Standard 1F of this report.

Portfolio Artifacts/Capstone Projects/Comprehensive Examinations:  Along with the above assessment instruments related to the portfolio and transition points, some of the programs' faculty members have developed additional components in their plans to evaluate candidates' progress.  These have included the candidates' selection of artifacts for their portfolios that they view as portraying their knowledge, skills and dispositions as well as capstone projects in the Masters' Seminars and, in two of the advanced programs, comprehensive examinations that must be passed before candidates graduate.  The range of artifacts, capstone projects, and comprehensive examinations are offered as Exhibits S2-10, S2-11, S2-12, respectively.

Pathwise System:  The unit has implemented an assessment instrument based on the work of Charlotte Danielson (Using Enhancing Professional Practice: A Framework for Teaching, 1996) that defines professional competencies and parallels the Interstate New Teacher Assessment and Support Consortium (INTASC) standards, known as "Pathwise."  This instrument and its use is fully described in Standard 1D of this report and is available as Exhibit S1-19.

These assessment activities, along with candidates' grade point averages, candidate concern forms and evaluations by cooperating teachers, allow the faculty to carefully document and monitor candidate performance through the programs on the various elements of Standard 1.  Candidates are informed about the above assessments through the institution's undergraduate and graduate bulletins, and program handbooks for candidates. Syllabus templates indicate where and how outcomes are assessed.  Similarly, field manuals have been or will be revised to incorporate this information. Instructors introduce these concepts to their candidates at the first meeting of their courses and in seminars prior to student teaching. 

The instruments and measures used in each program and their frequency are offered as an overall summary in a matrix that is available in the Exhibit Room (Exhibit S2-13).  The document addresses the assessments for each transition point, including the required GPA, number of courses, benchmarks, portfolio reviews, State tests, and Pathwise evaluations.  As of this writing, several of the programs have or are actively development databases of these assessments to evaluate the candidates' progress for review by the faculty on an ongoing basis.  A unit-wide database of these assessments as well as those described for program performance has also been developed, available as Exhibit S2-14.


Program Performance
The individual programs use the aggregated candidate assessment results as evidence for meeting standards and improving candidate performance related to content knowledge, student learning, and pedagogical and professional knowledge, skills, and dispositions.

The assessment system calls for cyclical (semester by semester and annual) evaluation.  Oversight is the responsibility of the unit's Assessment Committee.  Implementation is carried out within each program, where faculty members are responsible for collecting data on candidate performance and progress.  The NCATE office places candidate performance and progress data, including such internal and external measurements as GPAs, state test scores, and portfolio scores, in program data books for directors to use with their program faculty.  These data books are updated each semester.  At the end of each year, benchmark data are aggregated across each transition point of each program.  The program data books are also used to track candidate progress.  For individual candidates, each of the portfolio transition points provides summative judgment about qualifications relative to recommending movement to the next transition point of the program.

Exhibit S2-15 identifies the "content" components of the data books.  The program data books contain data from the benchmark assessments, assessments for field experiences, and clinical practice from the "Pathwise Rubric", the NYSTCE tests, candidate and program dispositional and portfolio assessment, the course evaluation's "five questions", and exit, alumni, and employer surveys.  The data books also report the "aggregated" results of the assessment data.  The data books present and aggregated program-based assessment measurements.  They also include data for which the unit as a whole is the sole level of analysis, including results of alumni surveys.  Using these comparative and holistic data sources, the unit's faculty can come to summative judgments regarding the quality of the programs.  The data books for each of the programs are available in the Exhibit Room.

Program directors reviewed the data books with their members in Spring 2006, for data through Fall 2005, and again in Summer 2006, with Spring 2006 semester data.  These reviews—along with the review of other data each program had collected through contacts with employers, alumni, candidates, and faculty—resulted in changes to both the programs and the assessment plans, themselves.  The related reports for each of the programs are offered as Exhibit S2-16 and are covered in 2C of this report.

In addition, a number of instruments were designed and used for the purposes of looking at the programs and how we can best support our candidates to reach professional and state standards.  These instruments have included: 

Extended Course Evaluation Forms: The University issues a course evaluation form on which the unit faculty has been able to include additional items pertaining to areas of candidate growth.   Candidates are asked to respond on a four-point scale ("1" being "strongly agree") to five items.

• As specified in the course outline, this course contributed to my understanding of the School of Education's Conceptual Framework.
• This course helped me develop the dispositions required for becoming a professional.
• This course incorporated the use of meaningful technology.
• This course addressed diversity issues (gender, culture, special populations, etc.).
• Field experiences (in this course) contributed to my professional growth. 

Their responses are then collected for analyses in relation to the Conceptual Framework.

Graduate Exit Survey: A follow-up survey was conducted graduates from 2004 through 2006, parallel to NCATE standards and the Conceptual Framework competencies.  Graduates were to rate items in terms of their importance for their professional careers using a five-point scale; one being "not important" and five being "highly important."  A copy of the survey is provided as Exhibit S2-17.  The data from the exit survey for 2004-2005 was not disaggregated to provide evidence of candidate knowledge, which was rectified for 2005-06.  Over 650 have been received since 2003-2004 and are covered in Standard 1.

Evaluation of the Program: Student teaching candidates are provided an opportunity to evaluate their experiences in the program, particularly the field components.  A copy of the evaluation appears as Exhibit S2-18, which was revised in 2005-2006.

Alumni and Employers Surveys: The alumni survey is issued annually by the University and the most recent version includes questions pertaining the unit.  A copy of the survey is available as Exhibit S2-19.  This past year, 131 alumni responded to and returned the survey, covered in Standard 1.  The employers' survey is administered to, or on behalf of, candidates working professionally in the field.  One form was distributed to over 100 school districts in October 2005.  As a validation procedure, this form and a newly-designed form more explicitly related to the Conceptual Framework were sent out to over 100 school districts in June 2006 with a request for employers to fill out the form they perceived as more in line with what the important knowledge and skills were for a position in their school(s).  A total of thirty-six responses were received during 2005-2006.  Copies of the two forms and the cover letters sent to employers are available as Exhibit S2-20.