|
Element 2B: Data Collection, Analysis, and Evaluation
The unit assessment system incorporates the collection and analyses of data and the use of this information for candidate, program, and unit evaluation for both initial and advanced programs. The responsibility for collecting and analyzing data on candidates' performance while they are in the program lies with program faculty with the support of the Dean's Office. The NCATE office and the Assessment Committee coordinate and provide support for program level data collection as well as post-graduate surveys and focus groups.
Table 2-3 offers the assessment indicators, timelines, and functions of assessment across the candidates' programs. The columns list how candidates are assessed, when they are assessed, and how the data is used. Rows distinguish between types of assessments.
|
Table 2-3: Assessment Indicators, Timelines, and Function of Assessment: Initial and Advanced
|
|
|
|
How Assessed
|
When Assessed
|
Use of Assessment Data
|
|
|
Candidates
|
|
|
Internal Assessment
|
GPA: Major
|
End of each transition phase
|
Candidate progression through program
Candidate review for possible remedial support (strengths and weaknesses) and candidate admission and dismissal
|
|
|
GPA: Education
|
|
|
GPA; Cumulative
|
|
|
Benchmark Assignments
|
End of each semester
|
|
|
Candidate Interviews
|
Beginning of Induction phase
|
|
|
Comprehensive Exams
|
End of synthesis phase
|
|
|
Portfolio Artifacts
|
End of each transition phase
|
|
|
Dispositions Assessment
|
|
|
Capstone Projects
|
|
|
|
Candidate Concern Forms
|
|
|
|
Letters of Recommendation for Student Teaching
|
Beginning of Reflective Practice phase
|
|
|
Pathwise System
|
During each semester in the Reflective Practice phase
|
|
|
Clinical/Field Evaluations
|
End of each semester
|
Candidates can ask themselves "Do I want to teach?" Self improvement
|
|
External Assessments
|
Letters of Recommendation
|
Beginning of Induction phase
|
Candidate Admission
|
|
|
SAT/GRE Scores
|
Beginning of Induction phase
|
|
|
NYSTCE: LAST, CST,
ATS-W
|
Eight times a year
|
Candidate progression through program
|
|
|
Clinical/Field Evaluations
|
End of each semester
|
Candidates can ask themselves "Do I want to teach?" Self improvement
|
|
|
Program
|
|
|
Internal Assessment
|
Aggregated Benchmark
|
End of academic year
|
Program Improvement
|
|
|
Course Evals. (5 Questions)
|
End of each semester
|
|
|
Graduate Exit Survey
|
|
|
Candidate Focus Groups
|
Each academic year
|
|
|
SOE Standing Committees
|
During each semester
|
|
External Assessment
|
SOE Alumni Survey
|
Every 2 years
|
Program Improvement
State Approval
Continued National Recognition
|
|
|
Employer Survey
|
Once a year (October)
|
|
|
Aggregated NYSTCE Scores
|
End of each academic year
|
|
|
NYS Dept. of Education
|
Every five years
|
By the beginning of each semester, the program directors and the Dean's Office update the data detailing the number of candidates, grades, benchmark/portfolio scores, dispositions and the number of candidates who were successful/unsuccessful or left the program, at each transition point. As described before, several of the programs are now creating program databases of these data on EXCEL spreadsheets, available in the program data books in the Exhibit Room.
The unit has adopted a model of self-monitoring for each of the programs. Program faculty has the responsibility for ensuring fair, accurate, and reliable assessments. These are demonstrated through the use of multiple assessments and multiple independent raters. Within programs, faculty have collectively designed, administered, and analyzed candidate assessments. One of the advantages of the introduction of Livetext to "capture" program-based data is that there is the opportunity to look at interrater reliability with regard to benchmark assignments and portfolios for some programs in order to adjust scoring as needed. Exhibit S2-21 offer examples of portfolios scores for the Standards across faculty members in an individual program. Program faculty appraises assessment tools for validity in the review of candidate growth, as well as, for overall review of the program. The faculty continuously monitor procedures and assessment activities and will often conduct a review to assure accuracy in the collection, reporting, and analyses of the data. Faculty members, as a group, make changes to procedures or assessments as deemed appropriate.
Candidates in all initial and advanced programs must maintain a "B" average. As indicated in the Adelphi Graduate Bulletin ( Exhibit G-3), any candidate who receives a second C+, or lower, will be put on academic probation and blocked from further registration until receiving academic clearance from an advisor. Any program judging a candidate to be performing below expectations can require additional coursework as a means to evaluate the candidate's continuation within the certification program or at the institution. Because of the assessment plan that was adopted by the unit, the monitoring of candidate progress has been strengthened. The Communication Sciences and Disorders Department has begun use of the Knowledge and Skills Assessment issued from ASHA, monitoring candidates of concern and holding faculty meetings on candidate issues every two weeks during the year (available as Exhibit S2-22). And in Spring 2006, the faculty in several of the initial and advanced programs adopted an identification and intervention system of candidate progress involving two levels: 1) identification within and during individual classes; 2) identification during formal review of all candidates' progress, whether this occurs at the end of every semester or academic year.
Level #1: Identification of candidates for intervention during classes: candidates are identified as being of concern to faculty and a form ( Exhibit S2-23) is filled out within the first ten hours of class meeting time and forwarded to the program director or department chair. A meeting of program faculty (either the entire program or a subcommittee) occurs before mid-semester, devoted to the review of these candidates, the discussion of the criteria leading to the candidates' identification, and establishment (with the candidates' faculty members) of intervention (e.g. referral for literacy support, personal counseling, and accommodation of candidates' needs through instructional strategies). The candidates identified on these forms are then reviewed again in terms of their progress at the end of the semester by a second meeting of faculty for possible action (see #2 below). A key activity for the faculty is to discuss the behavioral indicators that they believe would warrant concern about candidates' progress, related to the checklist on the attached form. This is important for greater congruence within the program, through the establishment of expectations for candidates that can be explicitly communicated to them, and also allows for potential professional development topics to emerge for the program faculty with regard to candidate needs and pedagogy.|
Level #2: Identification of candidates for intervention during semester/annual review: faculty meet about candidate progress at the end of every semester. While each program may have its own structure for review of data, the previously-described template is intended to integrate information from admissions (a profile of candidates' achievement upon entry), the registrar's office (candidates' grades), and program data (benchmarks, dispositions, and letters of concern) on individual candidates for review of their progress. Criteria for determining intervention were developed by each program's faculty and disseminated to all of those teaching in the program, as well as the candidates. When candidates have been identified as in need of intervention, the program/department faculty decides the course of action (based on previously-established policy that has been formally communicated to candidates). The following are recommendations for possible program options to support the candidates with progress of concern, assuming the available resources:
- re-enrollment and/or auditing of courses (depending on whether candidate received failing or passing grade)
- enrollment in a non-credit or limited-credit skills course (depending on area of need, for example, speech, social/communication, time management/study strategies) prior to culminating field experiences
- completion of degree without field experience/certification by program
Candidates are advised of the recommendations of the program faculty with regard to these options in person and in writing prior to the beginning of the following academic semester (including summer).
The unit maintains a record of formal candidate complaints and documentation of their resolution. The protocol for candidate complaints concerning the actions of members of faculty is outlined in the Guide to Student Life: pp.118-119 (Exhibit S2-24). Candidates may petition the School of Education Review Board to hear grievances that have arisen in an academic context and could not be settled at the department level. Typical complaints include, but are not limited to: problems with coursework, dismissal from a course, removal from the student teaching/practicum, academic standing in a program, and inappropriate conduct.
The Review Board is composed of seven members (five faculty members and two candidates). The Board's recommendations are forwarded to the Dean of the SOE for review and final disposition. Since 2001, twelve candidates have filed academic grievances and four have made their way to the unit's Review Board. The final resolutions are housed in the Dean's office. (Exhibit S2-25 offers these resolutions.)
|