Introduction & program demographics
Felician College , an independent co-educational Catholic/Franciscan College sponsored by the Felician Sisters, was founded in 1923 as Immaculate Conception Normal School. It has campuses in Lodi and Rutherford, New Jersey, and is dedicated to developing students academically and spiritually.

Felician College offers programs in four areas: Arts and Sciences, Business and Management Sciences, Nursing and Health Management, and Teacher Education. It is primarily a commuter college enrolling 1654 undergraduate and 152 graduate students from 23 states and 21 foreign countries. Approximately 34% of the 1806 students are minority students. The student/faculty ratio is fifteen to one, with 82% of the classes enrolling fewer than 20 students. Primarily a computer campus, Felician College permits students who do not have necessary entrance credentials to enroll on a conditional basis both at the graduate and undergraduate levels.

The Division of Teacher Education offers a Masters of Arts in Education entitled “Teacher as Leader” that is designed to prepare leaders in curriculum and instruction supervision who meet the New Jersey Professional Leadership Standards. Students may receive a Master of Arts in Education, Supervisor Endorsement or, if they enter Felician College with an existing Masters Degree, they may receive a Supervisor Endorsement. The program is currently undergoing revision in response to changes in the New Jersey Administrative Code limiting the size of district administrative offices. Currently, there are 13 students enrolled in the graduate program and no students seeking the supervisor endorsement. In the 2005-06 academic year, there were two full-time education faculty and 5 adjunct faculty in the Supervision Program.

The supervisory track of the graduate program has three components: (1) core courses, (2) professional studies, and (3) Masters Thesis (Capstone Action Research Project). All components are aligned with the New Jersey Professional Teaching Standards, the Interstate New Teacher Assessment and Support Consortium Standards (INTASC), the New Jersey Professional Standards for School Leaders (NJPSSL), and the appropriate Interstate School Leaders Licensure Consortium Standards (ISLLC). The supervisory strand shares the philosophy and mission of the Division of Teacher Education and therefore is influenced by a similar body of educational research.

Program’s claim(s)
Within the broad goal to produce supervisors who know sound educational principles and can help teachers implement them, the Division of Teacher Education at Felician College makes three claims about the graduates of the supervisor strand that are aligned with TEAC’s Quality Principle I.

TEAC 1.1 Professional knowledge

1. They understand curriculum and instruction and are able to work collaboratively to promote curriculum development that supports the New Jersey Core Curriculum Content Standards.

TEAC 1.2 Strategic decision-making

1. They engage in reflective practice and strategic decision-making to support the development of collaborative learning communities.

TEAC 1.3 Caring leadership

1. They understand the adult learner and are able to use caring leadership skills to improve and enhance instruction through knowledge of the New Jersey Professional Teaching Standards and the New Jersey Professional Standards for School Leaders.

In other words, the program’s graduates understand the subject matter of supervision, can use it to make effective decisions that create and facilitate learning communities, and they can implement their decisions in caring manner that improves and enhances learning in the community.

Method and categories of evidence supporting the claims
The evidence for meeting their claims will come from all graduates in the supervisory strand of the teacher education program, and the faculty proposes to investigate these claims over the next five years. The claims for this strand are based on the following sources:

1. Overall GPA in the graduate program
2. GPA in four required courses for the supervisory strand
3. GPA in three required courses leading to the masters thesis
4. Ratings of the Professional portfolio artifacts
5. Ratings of the capstone action research project or masters thesis
6. Student self-ratings at the time of exiting the program
7. Alumni survey of graduates’ leadership roles and professional service activities

Because this is an Inquiry Brief Proposal, the college describes what it will assess in greater detail than what it has discovered by examining existing evidence. The faculty has plans to determine the reliability and validity of their measures through correlations of GPA scores and analysis of how the ratings compare to licensure test scores and ratings of the senior research project.

This program is undergoing revision and it will become a principal training program in all probability. The faculty is unclear whether the claims and sources of evidence will be appropriate for the revised program, or whether the 5-year plan to collect evidence applies to the current program or the revised program.

Results
Preliminary results are presented for six program students, but these were restricted to course grades. In addition the faculty will rate the professional portfolio and the Capstone Action Research Project (Masters Thesis).

The faculty is aware that it has little reliable evidence with which to support its claims for students in the supervisory strand of the graduate program. As a result, the faculty intends to design and implement a Professional Portfolio that will include artifacts organized by claim to gather appropriate evidence.

Plan for program improvement
The faculty has identified a number of areas that require attention before it will be able to produce an Inquiry Brief. The program plans to develop essential artifacts and rubrics for the Professional Portfolio and field test the process in spring 2006, a series of surveys is being launched, an improved data collection system is being considered, and a new mechanism to probe the quality of student support services is being designed.

Internal audit results
The Internal Audit of the Master of Arts in Education Supervision was conducted in two parts after completion of realignment with New Jersey Licensure Code. The first audit, in February 2006, was conducted by the Associate Dean of Graduate Studies and two professors from the Division of Teacher Education to determine whether the quality control system was operating effectively. The second audit, April 2006, was conducted by the Associate Dean of Graduate Studies and one of the two professors who participated in the February audit to determine whether the status quo represented satisfactory conditions. In both cases, the audit team reviewed the files of all of the graduates from the program. The findings of the teams were combined and the final report was written by Dr. Donna Barron-Baker and Ms. Bernadette Casey. Faculty discussed the probes in a designated meeting and reviewed and added to the depth of the audit findings. At this stage the Vice President of Academic Affairs was interviewed and provided additional information to the brief.

The audit team examined course syllabi to see if they were on file and current [they were]; determined that there was documentation of State approved programs; investigated evidence of course descriptions being on file with the Registrar [they were]; reviewed classroom environment for equipment and accessibility to faculty [yes to both]; investigated data collection procedures and found them to be satisfactory;; in addition to numerous other nuances identifiable with quality programs.

Evidence of commitment and capacity
The Internal Audit team examined 26 indicators of commitment/capacity with respect to the program and the institution overall. The probes and the results are virtually the same as those of the Felician College Teacher Education Program. Overall, the audit revealed in very general terms the processes and procedures of the Division of Teacher Education and its graduate program in supervision. The authors reported that they felt there was evidence to support their claim that the college has the capacity to offer a quality program because:

• their program met state standards,
• their integral audit validated the quality control mechanisms,
• curriculum requirements in the graduate program are consistent with those of other graduate programs in the college,
• program completers possess the professional knowledge, caring leadership skills, strategic decision making abilities, and cross-cutting themes outlined in TEAC Quality Principle I.

Areas of concern included a need to monitor curriculum more closely, inadequate facilities and resources, need for additional faculty, need for alumni surveys and other evidence in terms of student work.

Audit opinion and commitment finding
The original Inquiry Brief Proposal was not fully auditable during the audit visit and the program re-submitted its case for preaccreditation as an addendum to the Inquiry Brief Proposal (IBP) for its teacher education program. That IBP, of which this addendum is now a part, fully satisfied TEAC’s capacity and commitment requirements.

Of the 17 audit task targets devoted to the addendum, 16 were verified, but nine had errors of some kind, which meant that the addendum earn a qualified audit opinion owing to the relatively high number of errors (53%).

The IBP is primarily focused on the rationale for the assessments and 11 of the 17 tasks were devoted to targets in this area. Ninety-one percent were verified, but 55% had errors associated with the target. All the targets associated with Quality Principle III were verified, but again a relatively large number (50%) had errors of some sort.

Were it not for the errors, only one of which was serious, the IBP would have earned a clean audit opinion. On the other hand, the sheer number of errors may indicate that the faculty’s proposal for its Inquiry Brief needs further clarification and precision.

In sum, the IBP is acceptably accurate and the faculty members by all accounts truly mean what they have said.

The auditors are guided in their award of clean or qualified audit opinions by the following considerations: an element (1.0 –4.0), or component of 4.0, receives a clean opinion if at least 90% of its targets are confirmed. An element, etc., is given a qualified opinion when at least 75%, but less than 90%, of its targets are confirmed. An element that would otherwise receive a clean opinion is also given a qualified opinion if more than 25% of the targets reveal misstatements of any kind (that is, trivial or consequential). If more than 75% of the targets cannot be verified, the element or component receives an adverse opinion (or a disclaimer if more than 75% of the audit tasks cannot be performed or completed).

These guidelines are not strict rules, because a simple counting of outcomes of probes may be misleading with regard to the trustworthiness of the Brief. Some audit tasks may be more revealing than others. For example, some may have targeted only minor points, and some may be merely following up on other audit tasks on a single point. Others may probe significant and central targets in the case for accreditation. The guidelines may prove unreliable in cases where the number of audit tasks is small.

The audit team knows that they are not to treat the guidelines or heuristics as rule that can be mechanically applied. If the findings suggest anomalies that make the heuristic unworkable, the auditors rely on their good judgments, explaining in their audit report the difficulties they experienced and the reasons for their opinions.

The auditors are also instructed to be alert to any event that is at variance with how the program is represented in the Brief. The auditors’ report events and experiences during the audit that were not fully consistent with the manner in which the program is portrayed in the Brief.

Finally, it must be emphasized again that the audit opinion is not an opinion about the quality of the program or the degree to which the evidence in the Brief satisfies TEAC’s quality principles and capacity standards. It is solely an opinion about whether the Brief is accurate as written. The issue in the audit is only whether what was in the Brief was accurate, not whether it could be made, or was made, more accurate by additional work on the part of the program faculty or the TEAC auditors during or after the audit.