Measures of Program Impact and Program Outcomes and Consumer Information: Fall 2018-Spring 2019
Measure 1: Impact on P-12 learning and development (Component 4.1)
IUP's program completers are making a positive impact on P-12 student learning and development. The Pennsylvania Department of Education (PDE) does not provide institutions of higher education with completers' year-end evaluation documents or their students' assessment data. Thus, a case study approach using intentional sampling was piloted to demonstrate the impact of IUP's EPP completers on P-12 student learning and development, classroom instruction, and schools and to assess the satisfaction of its completers and their employers with the relevance and effectiveness of their preparation. Upon obtaining approval from IUP's Institutional Review Board and site approval from building administrators, the following process was implemented:
-
The Dean's Associate for Educator Preparation and initial program coordinators identified three groups to target. The three groups included completers currently employed with our local and best-established PDS partner, Indiana Area School District (IASD) (See IASD Case Studies), completers who were graduates of our Urban Track program and employed by Pittsburgh Public School District (PPSD - see PPSD Case Studies), and recent completers representing a range of programs hired by other school districts within and outside the Commonwealth of Pennsylvania (see Recent Graduates' Case Studies).
Potential participants from those groups received letters of invitation and consent forms to sign if they were willing to participate. Consent to participate included giving permission to survey their direct employers. -
Volunteer completers were sent a validated survey that sought educational background information about the teacher, Likert scale items based on the 10 InTASC standards and ten critical dispositional standards within the InTASC standards, and five open-ended questions (4.4) (see CAEP 4 Completer Survey and Validity Results).
-
Interview questions based on the completers' responses were composed and three additional questions were posed to gain further (confidential) insight into participants' perceptions of preparedness and their classroom demographic information (4.4).
-
Completers were asked to provide IUP with copies of their principals' or supervisors' classroom observations (4.2), principals' or supervisors' end-of-year evaluations of the completers (4.1, 4.2, 4.3), and/or student assessment data that demonstrate student learning (4.1).
-
Consenters' principals were sent surveys to evaluate IUP Educator Preparation Program Completers' preparedness for the classroom (4.3). The employers' validated survey was the same as the completers' survey and was based on the InTASC Standards (see CAEP 4 Employer Survey and Validity Results).
A total of 38 teachers participated in the study: 13 completers are in their first 0-5 years of teaching; four completers have between 6-10 years of experience; 10 completers have between 11-20 years of experience, and 11 completers have 21 or more years in the teaching profession. Thirty-six IASD teachers identified themselves as being IUP EPP completers who agreed to participate; 28 (78%) actually participated. Teaching experience ranged from first-year teachers to teachers with 34 years of experience. Grade spans included kindergarten through grade 12. The following subjects are represented: Art, Early Childhood grades K-4, English, English as a Second Language, Mathematics, Reading, and Social Studies. Five IUP completers who were candidates in the Urban Track Professional Development School (PDS) program and are now teaching in the PPSD were invited to participate in the study. Four of the PPSD completers are in regular grades K-4 classrooms, and one teacher is teaching fourth- and fifth-grade children who are identified as needing learning support (see PPSD Case Studies).
All but two (36/38) of the case study participants are from Pennsylvania. PDE's goal for educator effectiveness is “To develop educator effectiveness models that will reform the way we evaluate school professionals as well as the critical components of training and professional growth.” PDE adopted Act 82 of 2012 that describes the Educator Effectiveness Model. Classroom teaching professionals are evaluated using the Model. Observational Practices (Danielson Domains) are 50 percent of the evaluation. Elective Data (District Designed Measures and Examinations, Nationally Recognized Standardized Tests, Industry Certification Examinations, and Student Projects Pursuant to Local Requirements) are 20 percent of the evaluation. Fifteen percent is based on Building Level Data (School Performance Profiles), and 15 percent is related to Teacher Specific Data (Student Performance on Assessments, PVAAS three-year average, IEP Progress, LEA Developed Rubrics). Teaching professionals are ranked overall as Satisfactory or Unsatisfactory. The EPP discusses findings as they relate to each Standard 4 component. More information may be found in the attached narrative.
Additionally, aggregate data supplied by Pittsburgh Public Schools demonstrates impact on P-12 learning and development and is provided as an attachment. Analysis of this data is located in the attached narrative.
SDUC Report, Indiana University of Pennsylvania
Measure 2: Indicators of teaching effectiveness (Component 4.2)
Observation instruments based on Charlotte Danielson's Framework for Professional Practice demonstrate that IUP completers effectively apply the professional knowledge, skills, and dispositions that preparation experiences were designed to achieve. According to the Danielson Group website, the Framework for Teaching (FFT) is a “valid instrument for defining effective teaching. Several large research studies (the MET project, a study in Chicago) demonstrated its predictive validity; that is, when teachers demonstrate high levels of proficiency on the FFT, their students show greater learning gains than do the students of teachers who perform less well.” Twenty-eight completers presented final observation data from their building supervisors. In 2014-15, five completers shared their observation data; all were rated as Distinguished or Proficient in all domains. In 2015-16, nine completers shared their observation data; all of them were rated as Distinguished or Proficient in the four domains. In 2016-17, nine completers shared their observation data; data illustrated that 100 percent were rated as Distinguished or Proficient in all domains.
In addition, a summary of completers' effective application of knowledge, skills, and dispositions, completer observation data, and candidates' Teacher Work Sample and Student Teaching Part I data are provided.
The EPP also conducts a yearly Data Retreat to review data from candidates and completers to reflect on programming and candidate and completer teacher effectiveness. School personnel, including teachers, administrators, and program faculty from IUP, attend the event. Insights from all educators are used to continuously develop and revise programming. Professional development is also provided. This day represents an opportunity to partner with rural and urban districts and better meet the needs of potential employers. Sample documents from the Fall Data Retreat are attached. The data from 2018-19 were utilized at this retreat. Approximately 30 individuals participated in at least some portion of the day-long event based on work commitments.
Completers Observation Data Table
IUP Educator Preparation Data Retreat Evaluation September 23, 2019, Comments Only
Data Retreat Evaluation Fall 2019
SDUC Report, Indiana University of Pennsylvania
Measure 3: Satisfaction of employers and employment milestones (Components 4.3 and A.4.1)
Employer surveys demonstrate that employers are satisfied with IUP completers' preparation for their assigned responsibilities working with P-12 students. Results of the completers' surveys and interviews demonstrate that IUP completers are reaching employment milestones such as promotion and retention, and they are continuing their professional growth. As per the information in the introduction to this Standard, many completers remain in the profession.
The EPP Dean's Associates and Clinical Services Specialist actively participate in the Superintendent's Advisory Council (SAC) and are a paid member of the School District University Collaborative (SDUC) of Pittsburgh Public Schools. The local intermediate unit, ARIN IU28, hosts and participates in SAC. This forum of superintendents, leaders of the intermediate unit, and IUP's dean of Education and Communications and the dean's associates of Education, provides a partnership of two-way networking, exchange of current and relevant information, and a direct link to central administration of regional PK-12 school districts. The SDUC consists of central administration from PPS and several regional universities. The SDUC meets several times per year to encourage preservice teacher placement, professional development, networking, and partnership within urban school settings. The SDUC provides comprehensive data regarding completers performance, PK-12 impact, and retention in the attached SDUC performance report of aggregate data from July 2010 to May 2017. Data is currently being collected for the next report.
A summary of employer surveys and teacher demographic information from case studies are provided as well as performance data from IUP completers employed at Pittsburgh Public Schools.
CAEP 4 Employer Survey Results
District Demographics and School Profiles 0-5-Year Teachers
SDUC Report Indiana University of Pennsylvania
Measure 4: Satisfaction of Completers (Components 4.4 and A.4.2)
Surveys and cooperating mentor surveys are attached. A narrative is included to summarize the findings.
Cooperating Mentor Teacher Survey 2018-19 nc
Measure 5: Graduation Rates (Initial and Advanced Levels)
Completer graduation rates have been added to the attached documents. Completer rates for initial and advanced programming are listed on separate documents. Candidate enrollment information is also included in two additional attachments.
IUP UG Completers for Academic Years 2012-18
IUP UG Enrolls Candidates for Academic Years 2012-18
2018-19 IUP Graduate Completers for Academic Years 2012-18 Initial and Advanced vMarch 2020
2018-19 IUP Graduate Enrolls Candidates for Academic Years 2012-18 Initial and Advanced vMarch 2020
Measure 6: Ability of completers to meet licensing (certification) and any additional state requirements; Title II (Initial and Advanced Levels)
The provisions of the 1998 Amendments to Title II of the Higher Education Act of 1965 require Indiana University of Pennsylvania to provide information to the public about students who complete our teacher education programs. It is important to recognize that Pennsylvania Department of Education uses a sliding scale to determine pass rates that is not captured in the data within Title II reporting. Specific information regarding pass rates and the sliding scale may be found on PDE's website. All IUP teacher candidates must pass testing requirements for certification, as per the Teacher Education Coordinator Council (TECC), before their student teaching. Therefore, IUP's teacher candidate pass rate is 100%.
Certification Testing Requirements
For further information on Title II and state reports, visit the Title II homepage.
Measure 7: Ability of completers to be hired in education positions for which they have prepared (Initial and Advanced Levels)
The majority of completers of both Initial and Advanced Level programs are hired in education positions or continue their education. The Career and Professional Development Center of IUP provides completer data from the 2018-19 academic school year. These attached reports have had identifiable data redacted to maintain confidentiality of the respondents. The Handshaker Reports reveal that a majority of responding completers acquire a position, if not already held, within their field and/or continue their education. Fifty-seven percent of initial program completers and 80 percent of advanced programming completers satisfy this criteria.
Additional employment rate information is provided in the IUP Student Survey, CAEP 4 Employer Results, and the Graduate First Destination Report.
Please note that the Graduate Reports for 2019 will be uploaded when they are published and available.
CAEP 4 Employer Survey Results
Graduate Survey Report Graduate First Destination Report for Class of 2018
Graduate Survey Report Class of 2018 HRFP FINAL 2
COEC 2019 Handshake Report Initial Programs
COEC 2019 Handshake Report Advanced Programs
Measure 8: Student loan default rates and other consumer information (Initial and Advanced Levels)
IUP student loan default rates, see below:
IUP Default Rates for 2018-19 report
2015 Fiscal Year Official Cohort Default Rate