Possible Audit Activities

1. Checking records, such as minutes of meetings or memos on file of faculty actions in making program decisions in approving the Inquiry Brief, and similar group decisions.

2. Reviewing notes taken of interviews with focus groups and with students at their exit from the program from which summaries are prepared or generalizations included in the Brief are induced.

3. Inspecting the responses received from stakeholders who are surveyed in the self-study process when the responses are summarized in tables or in narrative and included in the Brief.

4. Re-computing percentages, means, and standard deviations reported in the Brief from the original raw data taken from institutional files, State reports, or other sources.

5. Using institutional records to re-compute the means and standard deviations of grade point averages reported in the Inquiry Brief.

6. Tracing the sources of claims having to do with budget allocations, space allocations, and similar matters with institutional officials (e.g., provosts, bursars, or vice presidents).

7. Re-applying the coding schemes used to draw inferences from qualitative data to see if the results can be reproduced.

8. Interviewing faculty and/or staff who applied the coding schemes to qualitative data to learn how well they were trained, how their reliability was checked, and the depth of their understanding of the process.

9. Re-computing correlations and other statistics that were used to support claims of reliability and validity using, when appropriate, the faculty’s statistical packages.

10. Checking out reports concerning the reliability of multiple observers by inspecting the raw reports and re-computing the appropriate coefficients.

11. Touring the campus to check out claims concerning services available to program candidates, the availability of computers, faculty office space, and other capacity related issues.

12. Checking brochures, catalogs, and local websites to make sure the information found in the Inquiry Brief is consistent with the information found in these sources.

13. Examining both the data (video tapes, transcripts, field notes) and the procedures for coding the data if the faculty’s claims are supported by qualitative analyses of interviews with informants.

14. Examining data sets (also institutional reports where those same data are reported) to verify evidence present in comparisons of funds, space, full-time faculty equivalent per student enrollment of the target program with other programs on campus.

15. Interviewing faculty who participated in the deliberations leading to program change, examining minutes of meetings, and inspecting the copies of proposals that were taken to the faculty or administration for action to determine if the Inquiry Brief claims that changes were made in the program after considering data generated by the quality control system.

16. Targeting text to see if there is in fact a referent for the language, particularly if the language is educational jargon or is unqualified.

17. Interviewing faculty who conducted the internal audit probes and asking that their efforts be described to discern how familiar faculty members were with the internal audit and its purposes, findings, and recommendations.

18. Interviewing students who were the focus of the internal audit probes to ascertain that the characterizations found in the Internal Audit Report in Appendix A about these students are accurate.

19. Interviewing faculty who were the focus of an internal audit probe to ascertain that the characterizations of those faculty members found in the internal audit are accurate.

20. Examining files and archives describing actions taken by the faculty to improve the program to document the accuracy of the characterizations of these actions in the Inquiry Brief.

Return to Verifying the evidence related to specific claims