Download
entire document [Word
file or PDF]
May 2003
A committee composed of the following members of our Faculty
developed the process outlined in this document.
Undergraduate
Program Evaluation Sub-committee
Mr. Simon Albon, Faculty
Dr. Gail Bellward, Faculty
Dr. David Fielding, Faculty (Chair)
Dr. Rosemin Kassam, Faculty
Amanda Lai, Student
Dr. Kath MacLeod, Faculty
Abbas Merali, Student
Dr. James McCormack, Faculty
Dr. Keith McErlane, Faculty
Dr. John McNeill, Faculty
Ms. Marion Pearson, Faculty
Dr. Ingrid Price, Faculty
Bryce Wong, Student
Ms. Marg. Yee, Faculty
Caveat:
“Not everything that can be counted counts and not
everything that counts can be counted.”
Albert Einstein
Comprehensive
Process for Program Evaluation
Faculty’s
Mandate: The Faculty’s primary mandate is “to
maximize the health and well being of the citizens of British
Columbia and beyond” through service, education and research.
To measure success at achieving this mandate, our Faculty is
implementing a comprehensive program evaluation process. This
proposed approach is a modification of Robert Stake’s
“Countenance Model” of program evaluation (Stake,
1967).
Creating
a Culture of Evaluation: It is our hope that the implementation
of this process will create a “culture of evaluation”
in which faculty, staff and students routinely gather evaluation
data, as part of their day-to-day activities. Both program-specific
and general evaluation policies and procedures are essential
to collect data in a focused and efficient manner. The application
of these policies and procedures to each of the Faculty’s
three responsibility areas (education, research and service)
will result in discrete data sets summarizing inputs, processes
and outcomes. These data will provide the evidence to guide
judgments of quality for each responsibility area and, when
taken collectively, will provide a means to direct continuous
overall Faculty improvement.
The
Process: The program evaluation process is summarized
in Appendix B. At present, the educational
program of the Faculty can be sub-divided into entry-to-practice
(B.Sc. (Pharm.)), advanced-clinical practitioner (Pharm. D.),
continuing professional (licensed practitioners) and graduate
education (M.Sc., & Ph.D.). For each of these, an evaluation
sub-committee has or is being established.
The
undergraduate education sub-committee (with representatives
from all Divisions, relevant committees and the undergraduate
student society) has been meeting regularly since December 2002,
and the following outlines the evaluation process under development.
It is envisaged that this model will be adapted to the three
other areas of educational responsibility (Pharm.D., CPE, and
Graduate).
Step
1: Program Elements
Program elements (important program activities/ functions that
have an impact on program quality and program outcomes, e.g.,
student recruitment, student selection, instruction, or, would
indicate program quality, e.g., program outputs and outcomes)
are identified.
Step 2: Program Element’s Objective(s)
For each program element, specific objectives (desired outcomes)
are identified. Why is this element important to the undergraduate
program? How would it influence/ indicate quality?
Step 3: What Evidence?
What data should be collected to determine if this program’s
element objective has been achieved and how would these data
indicate quality?
Step
4: When, How, How Often and by Whom?
When, how and how often should this evidence be gathered and
by whom? This information will be used to provide guidance and
to serve as the basis for development of necessary policies,
procedures and data collection templates. To the greatest extent
possible, the program evaluation data gathering would be an
on-going process and integrated into daily routines/practices
and use existing sources.
Step
5: Gather The Evidence
In fulfilling every program activity/function, evaluation data
are collected and recorded.
Step
6: Compare Results and Make Judgment
Apply specific standards, criteria or expectations set for each
program element to make a judgment as to whether the intent/objective
of this program element was achieved. The standards, criteria
or expectations could be: national standards set by AFPC or
CCAPP; informal but generally accepted targets used by all Faculties
of Pharmacy in Canada; or internal expectations of an individual
Faculty.
Step
7: Unintended Consequences
Record and consider the implications for program quality of
all unintended consequences (good and bad) resulting from the
implementation of specific objectives set for a program element.
For the most part, these will only become evident when comparing
what has occurred with what was intended.
Step
8: Recommend Program Refinements
The goal of program evaluation is program improvement. As a
result of the program evaluation process, what changes are necessary
to enhance the competence (quality) of our B.Sc. (Pharm.) graduates?
Appendices
C to N outline the suggested program elements for the
undergraduate program. (Although every attempt has been made
to make this as comprehensive as possible, other important elements
may need to be added.) For each, specific objectives are set
and the “what, how, when, where and by whom” of
data collection are suggested. When reviewed collectively, such
data will permit an evaluation of the overall undergraduate
program and provide direction for continuous program refinement
and growth.
Appendix
O is a representation of the Committee’s view
that, to the greatest extent possible, program evaluation items
should be “embedded” into routine Faculty activities.
