Curriculum assessment and evaluation projects may not always spark immediate enthusiasm, but a recent well-organized team project in our higher education program demonstrated the power of collaboration. Thanks to the incredible participation of our faculty, improvements in student learning were not just planned, but successfully implemented. The changes made as part of this academic assessment/evaluation project to enhance student success yielded significant improvements. The positive impact on student learning results is rewarding when the impact on student learning results is positive!
The measures are crucial to our curriculum assessment/evaluation, program accreditation, and appraising evidence of student success. A key aspect of our approach is the use of both indirect and direct measures to gauge student learning within program and course assessments. This method allows us to redesign course assessments, incorporating authentic direct and indirect measures to foster student success.
What is a Direct Measure Assessment?
Our program faculty evaluated and updated our program curriculum, including assessment measures, as an aspect of continuous quality improvement within our educational assessment program. A direct measure objectively quantifies student learning (Bugeja & Garrett, 2019). It is crucial that these direct measures, which include course assignments (for example, papers, demonstrations, worksheets, presentations, virtual simulations, live simulations), examinations, capstone projects, student portfolios, and internship evaluations (Bugeja & Garrett, 2019), align with learning outcomes. This alignment ensures that the assessment accurately reflects the intended learning goals.
Well-constructed rubrics further enhance the quality of the assessment measure by communicating the criteria to the student and increasing the objectivity of the measure (Postmes et al., 2023). Linking examinations to learning outcomes and blueprints (List of topics the examination is assessing) increases authenticity (Abdellatif & Al-Shahrani, 2019). See the sample alignment template included at the end of this article.
What is an Indirect Measure Assessment?
An indirect measure is a subjective perception of the experience (Bugeja & Garrett, 2019). Examples of indirect measures include surveys (student, employer), focus groups, and exit interviews (Bugeja & Garrett, 2019). Indirect measures should not solely measure a course outcome or competency but should support a direct assessment measure. For example, in a major course assignment or project such as a Change Theory Application Project paper or poster presentation (direct measure), the student also completes a reflective self-evaluation survey of the project specific to learning outcomes or competencies (indirect measure). The indirect measure is valuable because it provides a different perspective but supports the direct measure.
Implementing Direct and Indirect Assessments
The program-level outcomes measure student success at program graduation. They are also known as end-of-program outcomes or terminal program outcomes. Course-level outcomes should match the attainment of the program-level outcomes.
After reviewing program-level assessment plans, the focus becomes the course level. We recommend reviewing and updating an existing alignment table or creating one that includes the course outcomes, module/unit objectives, competencies, learning activities, and assessments (modify the items per your school). We have included a sample alignment table at the end of this article. The alignment table map helps check the Revised Bloom’s Taxonomy Leveling and ensures all course outcomes and competencies link to course assessments (direct and indirect measures).
Next, examine the current course assessments. Identify course assessments worthy of updating and revision or develop a new assessment measure. Use mostly direct assessments with a few supporting indirect assessments, such as exit evaluations, focus groups, alums, and employer surveys. Remember, an indirect assessment should only add another viewpoint that supports the direct assessment measure.
Map the course outcomes, module learning objectives, course-assigned knowledge, skills, attitudes/values, competencies, and learning activities (for example, readings, videos, posters) in an alignment table. Add the final assessments to the alignment table after updating course assessment measures. Check that most measures are direct and that all course outcomes, objectives, and competencies link to a direct measure. Use only a few supporting indirect measures to safeguard genuine assessment of learning.
Summary
Educational assessment is valuable for informing curricula, demonstrating student learning, and determining program effectiveness for academic accreditation. Using direct and indirect measures in education provides data about student learning and program evaluation. Alignment mapping can help plan and check for consistency in the course. Multiple data measures, such as direct measures and a few supporting indirect measures, increase confidence in the findings and analysis, known as triangulation (Demeter et al., 2019). After completing course revisions with a particular emphasis on using direct and indirect assessment measures, we have noted improvements in student learning assessment at the course level.
Course Level Alignment Table Map
| Course Outcomes | Module Objectives | Course Competencies | Learning Activities | Course Learning Assessments (Identify Direct and Indirect Measures) |
Course Level Alignment Table Map Tips:
- Add and delete rows and columns as needed.
- The verb levels in the outcomes, objectives, and competencies should align with the learning activities and assessment measures.
- Not all courses will have competencies; this may vary depending on the discipline.
- Colleges and universities vary in the word use of outcomes and objectives, so modify the headings in the alignment table to match your school.
Nancyruth Leibold, EdD, RN, MSN, PHN, CNE, AHN-BC is an Associate Professor of Nursing at Minnesota State University, Mankato. Nancyruth is the Health Sciences Editor at Multimedia Educational Resource for Learning and Online Teaching (MERLOT).
Laura Schwarz, DNP, RN, PHN, CNE, AHN-BC is a Professor at Minnesota State University, Mankato. Laura is a Certified Peer Reviewer for Quality Matters.
References
Abdellatif, H., & Al-Shahrani, A. M. (2019). Effect of blueprinting methods on test difficulty, discrimination, and reliability indices: cross-sectional study in an integrated learning program. Advances in Medical Education and Practice, 10, 23–30. https://doi.org/10.2147/AMEP.S190827
Bugeja, M., & Garrett, M. (2019). “Making the connection”: Aggregate internship data as direct and indirect measure informing curricula and assessment. Journalism & Mass Communication Educator, 74(1), 17–30. https://doi.org/10.1177/1077695817749077
Demeter, E., Robinson, C., & Frederick, J. G. (2019). Holistically assessing critical thinking and written communication learning outcomes with direct and indirect measures. Research & Practice in Assessment, 14(1), 41-51.
Postmes, L., Bouwmeester, R., de Kleijn, R., & van der Schaaf, M. (2023). Supervisors’ untrained postgraduate rubric use for formative and summative purposes. Assessment and Evaluation in Higher Education, 48(1), 41–55. https://doi.org/10.1080/02602938.2021.2021390