Leadership in Assessment

Throughout my time in PQP, it has become clear that leadership plays a crucial role in creating assessment policies in schools as it sets the tone for the entire school community and ensures that the policies are aligned with the school's vision and goals. Effective leadership in this area can lead to better assessment practices, which can have a positive impact on student learning outcomes.

Effective leaders involve all stakeholders in the assessment policy-making process, including teachers, parents, and students, to ensure that everyone understands and shares the same vision for the school's assessment practices.  These practices align with board mission and vision, and SIPSAW and BIPSAW goals.   Using these guides, leaders set clear expectations for assessment policies and practices, and communicate them effectively to all stakeholders. This helps to ensure that assessments are administered consistently and fairly across the school.  A school that has consistent practices create an environment where staff are more likely to collaborate with shared goals.

When consistent practices are prioritized, leaders provide the necessary resources, such as professional development opportunities and assessment tools, to support teachers in implementing the assessment policies effectively.  They encourage a culture of continuous improvement, where teachers regularly reflect on their assessment practices and make adjustments as needed to improve student learning outcomes.  Leaders ensure that resources and time are allotted to allowing staff to work to improve practices regularly, ensuring that assessment policies comply with equity and Ministry standards.

As outlined in the Catholic leadership framework, Catholic leaders must set directions, build relationships, develop people and the organization, improve the instructional program and secure accountability.  Leadership is crucial in creating assessment policies in schools as it sets the direction and expectations for the entire school community and ensures that assessment practices align with the school's vision and goals.  Allowing time to develop these practices creates a better school environment where students can trust in their teachers to be partners in their success.  When a team is behind a student, they feel safe to make mistakes, knowing that the purpose of assessment is to improve.

You're Too Late? Lateness and Assessment

 

The issue of deducting marks for lateness is a highly debated issue in evaluation and assessment study and among teachers.  Teachers reference “fairness” and “encouragement” and “long term consequences” as the reasons why lateness should both be penalized and why it should not (Sun et al 2014).  There are 17 suggestions in Growing Success (pg. 43) to help prevent and/or address late assignments.  Only one of these suggestions involves deducting marks for late assignments.  That provision allows for “deducting marks for late assignments, up to and including the full value of the assignment” (pg. 43).  Some school policies add that a teacher may deduct up to one achievement level for late assignments and further adds “Once a class set of assignments has been returned to students, a teacher is under no obligation to accept late assignments”.  This arbitrary addition eliminates the fairness of the policy since it is open to interpretation, marking speed and bias of the teacher and is inequitable on a number of levels.  Further, this added statement violates the classroom assessment standards in the area of use, both in effective feedback and grades and summary comments and quality in unbiased and fair assessment.

Effective feedback is considered fair when it is transparent, consistent and justified (Rasooli et al, 2018).  The standards of practice build on this and define effective feedback as “timely and useful in improving student learning” (Klinger et al, 2015).  Klinger further explains that students will be able to use the feedback to “revise their work to meet the learning expectations” and provide “opportunities for self-reflection" (pg. 28).  This standard is supported by the work of Tierney (2004) and Williams (2011), who argue that assessment as learning can effectively support learning, encourage student engagement in productive learning and add value to an assessment.  Valuing a task is critical not only for having continuing interest in a topic, but also for voluntarily electing to do a task which leads to the pursuit of mastery (Brookhart et al, 2003). William states that for assessment to encourage learning, it "must be provided in a way that encourages the learner to direct energy towards growth, rather than well-being", where attention, instead, is focused on preventing threat, harm or loss (2011).  By denying students the opportunity to submit work after an inexact timeline, a student’s focus shifts to wellbeing and they are denied the opportunity to grow and learn through the assessment process.  Effective feedback can allow a student to improve by developing effective error detection skills and will lead to self-feedback which will allow them to strategize and regulate to solve problems or use their self-regulatory proficiencies (Hattie et al, 2007). The students who submitted on time will be advantaged in this (if given the opportunity to resubmit) or the next assessment by critically examining their work through the lens of previously received feedback.

The assessment standard of use in “grades and summary comments” states that “grades and comments should reflect student achievement of the learning expectations” (Klinger et al, 2015).  This standard insists that class assessment grades and comments reflect a students’ level of mastery of the learning expectations.  Lowering a student’s mark or giving a mark of zero based on lateness clearly does not reflect a student’s achievement since the student has not yet demonstrated their learning.  Learning cannot be evaluated appropriately with insufficient opportunity for its demonstration (Tierney 2004). Rasooli et al (2018) noted that “students perceive unfairness when teachers base their grade decisions on insufficient, unsuitable, and insignificant data”. Klinger suggests using tardiness of an assignment in the comments but states that “summary comments provide additional information, but they should not be used to adjust the grades” (Klinger et al, 2015).  Feedback that does include these construct irrelevant factors and strays from intended goals results in lower engagement, motivation and effort and is the least effective way to create self-regulation (Hattie et al, 2007, Brookhart et al, 2003).  When a mark does not reflect a student’s achievement, it renders the mark meaningless.  “Including both achievement and non-achievement factors in grading threatens the interpretability of the grades assigned by teachers” (Sun et al 2014). This is particularly true if assessment is being used a tool to gauge a student against the standard or against others in the class (Baharloo, 2013). 

The third, and perhaps most important, assessment standard that some school lateness policies violates is the quality standard in the area of “unbiased and fair assessment”.  This standard states that “classroom assessment practices and subsequent decisions should be free from all factors unrelated to the intended purposes of the assessment (Klinger et al, 2015).  Certainly, the policy of refusing to accept late work ignores the variety of reasons that a student may have to submit work late and relies on the generosity of the teacher.  Effective teaching, learning and assessment practices are dependent upon the knowledge and value systems of educators and many teachers struggle with the idea that good work habits can be encouraged without grading penalties, despite evidence to the contrary (Tierney et al, 2011). Sun et al (2014) noted that “even when teachers use the same grading scale and the same grading guidelines, there is little consistency in teachers’ grading across schools”.  Rasooli et al (2018) also noted this saying “teachers show high levels of disagreements and numerous conflicts in the scenarios associated with avoiding score pollution”. Although teachers agree that grade decisions based on students’ misbehavior and gender are unfair, teachers may consider factors such as “effort, progress, compassion for students, desire to teach life lesson, and impact of grades on students’ opportunities in their future lives as fair when deciding or adjusting grades (Rasooli et al, 2018).  The variability and ambiguity of some school late policies is too reliant on these values of the teacher and is, therefore, vulnerable to unconscious or conscious bias.  Judging lateness may also unfairly reward some students since higher income parents, who understand the qualities needed to achieve higher education, can use their financial resources to improve the performance of their children through buying private lessons and learning materials (Tieben et al, 2010).  Students from low-income households may lack support at home for a number of reasons, including the need to work, and therefore may not obtain work habits associated with higher achievement (Tieben et al, 2010).  Those students may need more time to fine tune assignments and may lack the time to complete assignments in a timely manner if they, themselves are working to maintain the household.  Penalizing these irrelevant factors directly violates the assessment standards (Klinger et al, 2015) and creates an unfair advantage for some over others, most often those who are marginalized.

Although Growing Success (Pg. 42) acknowledges a student’s responsibility to submit work on time and accept the consequences of lateness, it offers a variety of methods before marks should be deducted.  Teachers want an assessment that is both fair and beneficial to students (Sun et al 2014). The purpose of assessment, itself, is at question here.  If assessment is used as a measure of achievement, it is clear that using study skills or other personality factors to determine marks is an unfair practice (unless a teacher has come to believe that equality, not equity, is a measure of fairness).  If, however, assessment is being used to encourage good qualities like study habits and esteem or to reduce the long-term consequences of not developing effort or responsibility, deducting marks for lateness seems reasonable.  Unfortunately, teachers often blend these two, depending on previous knowledge of the student and their own lived experience (Sun et al 2014), allowing for bias when grading.  It is this possibility for bias that makes many school policies on lateness unfair.

 

 

 

 

 

References

Baharloo, A. (2013). Test fairness in traditional and dynamic assessment. Theory and Practice in Language Studies, 3(10), 1930+. https://link.gale.com/apps/doc/A351082074/GPS?u=ko_k12hs_d2&sid=GPS&xid=e027de3d

Brookhart, S. M., & Durkin, D. T. (2003). Classroom Assessment, Student Motivation, and Achievement in High School Social Studies Classes. Applied Measurement in Education, 16(1), 27–54. https://doi-org.proxy.queensu.ca/10.1207/S15324818AME1601_2

Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102_00346543077001081

Klinger, D.A., McDivitt, P.R.,Howard, B.B., Munoz, M.A., Rogers, W.T., & Wylie, E.C. (2015). The Classroom Assessment Standards for PreK-12 Teachers. Kindle Direct Press.

McMillan, J. H., Hellsten, L., & Klinger, D. (2011). Classroom assessment: principles and practice for effective standards-based instruction. Pearson Canada.

Rasooli, A., Zandi, H., & DeLuca, C. (2018). Re-conceptualizing classroom assessment fairness: A systematic meta-ethnography of assessment literature and beyond. Studies in Educational Evaluation, 56, 164–181. https://doi-org.proxy.queensu.ca/10.1016/j.stueduc.2017.12.008

Sun, Y., & Cheng, L. (2014). Teachers’ Grading Practices: Meaning and Values Assigned. Assessment in Education: Principles, Policy & Practice, 21(3), 326–343.

Tieben, N., & Wolbers, M. (2010). Success and failure in secondary education: socio-economic background effects on secondary school outcome in the Netherlands, 1927-1998. British Journal of Sociology of Education, 31(3), 277–290. https://doi-org.proxy.queensu.ca/10.1080/01425691003700516

Tierney, R. D. (2014). Fairness as a multifaceted quality in classroom assessment. Studies in Educational Evaluation, 43, 55–69. https://doi-org.proxy.queensu.ca/10.1016/j.stueduc.2013.12.003

Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3–14.

Ontario Ministry of Education, Growing success: assessment, evaluation and reporting: improving student learning (2010). Toronto.  http://www.edu.gov.on.ca/eng/policyfunding/growSuccess.pdf.

Student Voice and Assessment

  

"In education, student voice refers to the values, opinions, beliefs, perspectives, and cultural backgrounds of individual students and groups of students in a school, and to instructional approaches and techniques that are based on student choices, interests, passions, and ambitions." Link to definition

The idea of student voice is not a new one. In it's most basic form, student councils have served for years as the voice of the student body within a school. In recent years, the idea of student voice has expanded to include not just school based decision making but also classroom and community based decisions. The actual power of these student-led groups, however, was often limited in actual power to make changes. Historically, schools and classrooms have been run with adults, not students, making decisions that directly affect them.


For my own inquiry question, I wondered how student voice could be used more effectively to guide instruction and assessment to create engaged, knowledgeable and thoughtful learners. The hope is that students will be more engaged and more willing to learn if they are involved in the process of selecting activities and assessment methods. Through invested engagement, students will become more knowledgeable, thoughtful learners.


Melissa Schlinger, in her article "Elevating Voice Through Social & Emotional Learning: Finding new ways for students to practice skills as decision makers and problem solvers.", suggests that learners need social-emotional skills before they can actively engage in using their voice.


"Social and emotional learning provides opportunities for all students to develop and practice the skills they need to become leaders, decision makers and problem solvers. It also helps adults and students to jointly create learning places that elevate everyone's voice and strengthens the community." (Schlinger 2020)

She further defines the areas that will be necessary for development. They include: gaining awareness of personal strengths and self-efficacy, practicing self-motivation and organizational skills, recognizing others' perspectives, communicating and collaborating with others, and identifying solutions to issues.


I had not considered the extensive teaching that would be necessary before undertaking student voice. In fact, this is not a task that can be undertaken in a 75 minute lesson or even over the course of a semester. Development of the social-emotional skills necessary for student voice to be fully impactful would take years to achieve. An entire system shift would have to take place. Boards could take this on as a priority in board and school improvement planning at all levels and introduced at an early age.


So what can be done? At the most basic level, student voice is about including choice - in lesson, assessment and school planning. The organized forms of student voice certainly have a place but informal or formal conversation and relationship building to understand the pulse of students seems to be how student voice can be best understood to start.

 
Domenech, Daniel A. "One Pandemic Plus: Greater Voice for Students." School Administrator, vol. 77, no. 6, June 2020, p. 54. Gale General OneFile, https://link.gale.com/apps/doc/A635067994/GPS?u=ko_k12hs_d2&sid=GPS&xid=3e5b19dc. Accessed 6 Oct. 2020.
 
Schlinger, Melissa. "Elevating Voice Through Social & Emotional Learning: Finding new ways for students to practice skills as decision makers and problem solvers." School Administrator, vol. 77, no. 6, June 2020, p. 44+. Gale General OneFile, https://link.gale.com/apps/doc/A635067971/GPS?u=ko_k12hs_d2&sid=GPS&xid=6544f64e. Accessed 6 Oct. 2020.
 
“Student Voice Definition.” The Glossary of Education Reform, 20 Dec. 2013, www.edglossary.org/student-voice/. 

Why Effort Should Not Be Evaluated

 Why Effort Should Not be Included in Grading

When assessing the fairness of an assessment task, three areas can be examined; bias, equity in opportunity to learn the construct of interest and equitable treatment of the subject of the evaluation (Baharloo, 2013).  When creating assessments and accompanying grading rubrics, educators need to consider the issues of equity, fairness and validity.  When these factors are not considered, assessment and evaluative tools can be used as a means to compare marginalized students (those living in poverty, with learning difficulties or other types of discrimination) to more advantaged students and thus push them even further to the margins, when it should be employed to support their development as diverse human beings (Milner, 2018).

Construct Irrelevant Factors

Construct irrelevance (or construct irrelevant variance) occurs when an assessment is measuring more variables than just the construct of interest, many of which are irrelevant to the content standards (Haladyna & Downing, 2004). The underlying assumption of all assessments is that the score that is produced reflects a student’s ability on the construct of interest. The validity of the assessment is measured by considering if the task adequately reflects the domain of knowledge and skills of interest and if it can be used as the basis for the inference of proficiency (Ferrier et al, 2011, Growing Success pg. 156). The meaningfulness and accuracy of assessment results, therefore, is adversely affected by focus on irrelevant factors (Brookhart, 2019).  When an assessment is valid, the construct (ability, skill, or knowledge) that it is designed to measure is the source of students’ score on the assessment. An assessment is invalid, then, when factors that have nothing to do with the target construct affect a student’s score (Ferrier et al, 2011, Baharloo, 2013).  By assessing these construct irrelevant factors, such as effort, misinterpretations could be made about a student’s proficiency and undermine the accuracy of scoring data (Allen 2005).  In fact, “teachers' practice of including a variety of criteria in the assignment of grades distorts grades, rendering them invalid and meaningless” (Olson, 1989).

Why Teachers Believe That Effort Should Be Graded

Teachers believe that students who try harder should be rewarded and that more effort results in more knowledge acquisition (Howley, 1999, McMillan, 2011), some, in fact, believe that effort is more important than amount of academic content learned (Allen 2005).  They may even believe that they are helping low-achieving students to stay engaged with the material and that grading effort could be used effectively as a motivational tool (McMillan, 2011).

A study at Benedict College seemed to confirm this, concluding “strong evidence that more student effort does lead to increase learning” (Swinton, 2010).  The study also found, however, that “better measure for effort than previous research and a more appropriate estimation procedure” were needed for this correlation to be noted, that regularly high-achieving students did not report this same result because they can choose when to implement effort and that the impact on graduation rates could not be noted (Swinton, 2010).

Another reason teachers reward perceived effort may be that in some troubled schools, where compliance and cooperation are lacking, good behavior may replace achievement as the desired response of students (Howley 1999).  However, when marginalized students receive grades that fail to provide an accurate representation of their achievement (either inflation or deflation), they can be seriously harmed (Allen, 2005, Howley, 1999).  When marks are inflated, students have an unrealistic view of their ability, setting them up for failure later in life and when they are deflated, opportunities for employment and higher education are significantly reduced (Lipman, 1997, Howley, 1999).

Evaluator Bias in Judging Effort

Assessment bias occurs whenever assessment items “unfairly penalize students for reasons related to students’ personal characteristics, such as their race, gender, ethnicity, religion, or socioeconomic status” (Popham, 2011). In essence, biased assessment favors one group of students over another or negatively impacts one group of students over another.  Effort can be difficult to measure and define and as such be inconsistently evaluated (Allen, 2005).  Teachers tend to have a bias towards learners who exhibit qualities of a “good student” regardless of actual performance competency (Allen, 2005).  This includes things like "being on task”, “class participation”, “homework completion” and including additions to assignments like fancy paper or use of stencils on posters, costumes for oral presentations, high tech elements, and decorations that match themes, to name a few (McMillan, 2011).  Measuring effort can, therefore, create an inequitable standard for students on a number of levels, including gender differences, socio-economic factors and parental support. “Regardless of the strength of their rationale for doing so, teachers' practice of confounding effort and compliance with achievement in their construction of students' grades makes grading vulnerable to race and class bias” (Howley, 1999).

Gender Bias

One possible explanation for multiple findings that teachers treat and evaluate boys and girls differently is that “girls tend to have more positive attitudes toward school and put more effort into their schoolwork, and teachers not only evaluate students’ competence, but also reward effort” (Krkovic, 2014).  A study at the University of New Brunswick found that stereotypical patterns of boys scoring higher in math and science only occurred on standardized achievement tests even though girls scored higher across every subject in school tasks (Voyer, 2014).  This shows a clear difference in actual knowledge acquisition and in school performance.  This could be due to the perception by teachers that girls are “less disruptive than boys and this could also result in teachers overestimating girls’ performance” (Krkovic, 2014).

Socio-economic Bias and Parental Support

There is significant evidence to show that working-class pupils are implicitly seen as less able than their more affluent counterparts (Creasey, 2016).  Evidence seems to support that fewer marginalized children will develop to the full measure of their potential, achieve at a high level or acquire advanced intellectual competencies and academic skills (Kitano, 2003, Edgerton, 2008).  This may be because higher income parents can use their financial resources to improve the performance of their children through buying private lessons and learning materials (Tieben et al, 2010) and are better able to address specific learning strategies like monitoring homework, reducing television time, supporting development of good study habits, and high expectations that appear most likely to have positive effects on children's academic performance (Kitano, 2003). Lack of support at home for a number of reasons, including the need to work, makes students less able to obtain these skills and parents may lack the technical know how or time to provide new technology and apps (Tieben et al, 2010, Edgerton, 2008).  Students who arrive in class without the appropriate materials and those who do not complete homework may be accused of being “irresponsible” or “unprepared” and may be seen by their teachers as lacking in effort.  Further, parents who have obtained higher education themselves are not only able to help and support their children in education but are familiar with requirements of higher educational tracks. (Tieben et al,  2010, Edgerton, 2008) Where teachers do recognize that children from different backgrounds have different needs, there seems to be a double standard of success. One study indicated that for students whom teachers assumed lacked a positive home environment “success was feeling good about school, adjusting to rules and expectations, having positive interactions with adults, and attaining a sense of belonging” and lacked any correlation between success and academic achievement (Lipman, 1997, Howley, 1999).

Moving Passed Effort as a Measure of Evaluation

For assessment tasks to be regarded as fair and valid, the results must reliably indicate a student’s academic achievement and ability on the construct of interest and that construct needs to be clear to both the educator and the student.  When effort is graded as part of an assessment, the data can no longer accurately communicate a student’s ability and can have long term negative effects on the student being unfairly graded.  Evaluation standards of effort currently lack consistency and are subject to bias and unfairly disadvantage those who are marginalized.  Effective teaching, learning and assessment practices are dependent upon the knowledge and value systems of educators (Tierney et al, 2011) Inadequate education in valid assessment and grading principles and practices is a reason many teachers continue to perpetuate invalid grading practices with students (Allen, 2005).  Many teachers struggle with the idea that good work habits can be encouraged without grading penalties, despite evidence to the contrary (Tierney et al, 2011). Teachers may also have ingrained experience, beliefs and long-standing habits about assessment that must be overcome (Brookhart, 2019). Campbell argues that equity, diversity, and inclusion (EDI) educational programs are central to raising participants’ awareness of behaviours that might have discriminatory effects, even when those effects are not intended (Campbell, 2021). If teacher education adopted both fair assessment and EDI training, old patterns of evaluation using construct irrelevant factors like effort could be eliminated, resulting in more accurate results for all stakeholders.

 

 

Resources

Allen, James. (2005). Grades as Valid Measures of Academic Achievement of Classroom Learning. The Clearing House.

Baharloo, A. (2013). Test fairness in traditional and dynamic assessment. Theory and Practice in Language Studies, 3(10), 1930+. https://link.gale.com/apps/doc/A351082074/GPS?u=ko_k12hs_d2&sid=GPS&xid=e027de3d

Brookhart, S. M., & Nitko, A. J. (2019). Educational assessment of students. Pearson.

Budge, K. M., & Parrett, W. (2018). Disrupting poverty: 5 powerful classroom practices. ASCD.

Campbell, A. (2021). Equity education initiatives within Canadian universities: promise and limits. Perspectives: Policy and Practice in Higher Education, 25(2), 51–61. https://doi.org/10.1080/13603108.2019.1631226

Creasey, S. (2016). Is unconscious bias holding your pupils back? TES: Times Educational Supplement, 5185, 26–32.

Edgerton, J. D., Peter, T., & Roberts, L. W. (2008). Back to the Basics: Socio-Economic, Gender, and Regional Disparities in Canada’s Educational System. Canadian Journal of Education, 31(4), 861–888.

Ferrier, David & Lovett, Benjamin & Jordan, Alexander. (2011). Construct-Irrelevant Variance in Achievement Test Scores: A Social Cognitive Perspective.

Haladyna, T. M., & Downing, S. M. (2004). Construct-Irrelevant Variance in High-Stakes Testing. Educational Measurement: Issues & Practice, 23(1), 17–27. https://doi-org.proxy.queensu.ca/10.1111/j.1745-3992.2004.tb00149.x

Howley, A., Kusimo, P. S., & Parrott, L. (1999). Grading and the Ethos of Effort.

Jensen, E. (2010). Teaching with poverty in mind: what being poor does to kids' brains and what schools can do about it. ASCD.

Kitano, M. K. (2003). Gifted Potential and Poverty: A Call for Extraordinary Action. Journal for the Education of the Gifted, 26(4), 292–303.

Krkovic, K. (2014). Teacher evaluation of student ability: what roles do teacher gender, student gender, and their interaction play? Educational Research., 56(2), 244–257.

Lipman, P. (1997). Restructuring in context: a case study of teacher participation and the dynamics of ideology, race, and power. American Educational Research Journal, 34, 3–37.

McMillan, J. H., Hellsten, L., & Klinger, D. (2011). Classroom assessment: principles and practice for effective standards-based instruction. Pearson Canada.

Milner, H. R. (2018). Confronting Inequity / Assessment for Equity. Educational Leadership, 75(5), 88–89. https://doi-org/http://www.ascd.org/publications/educational-leadership/feb18/vol75/num05/Assessment-for-Equity.aspx

Olson, G.H. (1989, March). On the validity of performance grades: The relationship between teacher-assigned grades and standard measures of subject matter acquisition. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco, CA.

Ontario Ministry of Education, Growing success: assessment, evaluation and reporting: improving student learning (2010). Toronto.  http://www.edu.gov.on.ca/eng/policyfunding/growSuccess.pdf.

Payne, R. K. (2013). Framework for understanding poverty. Aha! Process.

Popham, James W. (2011). Popham - Assessment bias.mp4. YouTube. https://www.youtube.com/watch?v=HjID39cqmzw.

Schleicher, A. (2009). Securing quality and equity in education: Lessons from PISA. Prospects (00331538), 39(3), 251–263. https://doi-org.proxy.queensu.ca/10.1007/s11125-009-9126-x

Swinton, O. H. (2010). The Effect of Effort Grading on Learning. Economics of Education Review, 29(6), 1176–1182

Tieben, N., & Wolbers, M. (2010). Success and failure in secondary education: socio-economic background effects on secondary school outcome in the Netherlands, 1927-1998. British Journal of Sociology of Education, 31(3), 277–290. https://doi-org.proxy.queensu.ca/10.1080/01425691003700516

Tierney, R., Simon, M. & Charland, J. (2011), Being Fair: Teachers’ Interpretations of Principles for Standards based Grading, The Educational Forum 75(3): 210-227.

Voyer, D., & Voyer, S. D. (2014, April 28). Girls make higher grades than boys in all school subjects, analysis finds. American Psychological Association.

 

 

 

Using Achievement Charts to Ensure Validity

 A rubric can help an evaluator eliminate their evaluator bias but eliminating all construct irrelevant factors.  

Reminder: When an assessment is valid, the construct (ability, skill, or knowledge) that it is designed to measure is the source of students’ score on the assessment.

This can be done using the Achievement Chart (available in all curriculum documents)

The benefits of using achievement charts:

• It is a standard province-wide guide and is to be used by all teachers as a framework within which to assess and evaluate student achievement of the expectations in the particular subject or discipline.
• It enables teachers to make consistent judgements about the quality of student learning based on clear performance standards and on a body of evidence collected over time.
• It provides teachers with a foundation for developing clear and specific feedback for students and parents

2 Ways to Adapt Achievement Charts to Ensure Validity and UDL Adaptability

#1 - Modify the descriptors

#2 - Add descriptions under the categories

What to Include/Not Include in Rubrics

Validity in Assessment

 

Validity

The validity of the assessment is measured by considering if the task

•       adequately reflects the domain of knowledge and skills of interest and

•       if it can be used as the basis for the inference of proficiency

(Growing Success pg. 156) 

***When an assessment is valid, the construct (ability, skill, or knowledge) that it is designed to measure is the source of students’ score on the assessment. In other words, it measures what it is designed to measure.

Construct irrelevant variance (CIV) is the introduction of extraneous, variables that affect assessment outcomes. The meaningfulness and accuracy of examination results is adversely affected, the legitimacy of decisions made upon results is affected, and the validity is reduced. 

Construct irrelevant factors can be:​

​

·           Physical (hunger)​

·           Environmental (noise in the hallway)​

·           Social (trouble with friends)​

·           Emotional (anxiety)​

·           Bias (neater writing is marked higher)​

·           Systemic or Random​

·           Can occur before, during or after an assessment.

 There are variables that we can control and ones that we can’t.  For variables we can control (classroom set up, font choices), we sometimes know that they are concerns and can fix them.  For others, we may not know that they are a factors.  Building trusting relationships with students and getting to know your learner is the only way to find out that one of these factors is in play.  

For the factors that we cannot control, we sometimes have to make decisions based on what is best for students.  Maybe the assessment needs to be pushed off a day or a few hours until issues have resolved.  Sometimes we need to rework the assignment so that these factors are diminished for students.  Referring to a question as a "written response" and not an "essay", for example, could be enough to diminish a student's anxiety.  Providing the materials, in class time and a variety of methods (as outlined in the UDL post) can eliminate some of these factors for a student.

Making sure the rubric is measuring what it is meant to measure will be the focus of the next blog post.

A bit on UDL

  

Universal Design for Learning, or UDL, is a theoretical framework of learning, developed by the Center for Applied Special Technologies (CAST) in the 1980s, that draws on research in neuroscience, education, and technology.  Originally as an attempt to incorporate students with learning difficulties into mainstream classes and later expanded into a more universal vision, it values the individual needs and interests of the learner in an understanding of how real learning occurs.  Lesson, assessment and school design begin with those individual needs, offering choice for everything from seating, approach and output.  Student voice and needs are considered the primary guiding factor in all educational decisions.  Like universal design in other areas, universal design for learning is about providing access to the course material for all learners, regardless of barriers that may be present.

 “Students have varying abilities, preferences, cultures, languages, and experiences, all of which affect how they learn” (Meyer, Rose, & Gordon, 2014)  It is this understanding that guides practice.  It is a changing of the way that “fairness” is seen within an educational setting.  UDL moves from an idea of equality to equity. According to the UDL principles, each area of the curriculum should provide multiple, varied, and flexible options for representation, expression, and engagement.

 

 

So, how does UDL create engaged, knowledgeable and thoughtful learners?

The goal of learning is make experts out of learners.  As defined by Meyer et al, “Expert learners are defined as being: (a) purposeful and motivated; (b) resourceful and knowledgeable; and (c) strategic and goal directed” (Meyer, Rose, & Gordon, 2014).  Using the guidelines of UDL, students use choice to engage with the material and extend learning using their own lives and experience and express that learning in a meaningful, deeper way.  With this understanding of mastery, all learners can become experts, despite the complexity of student needs and individual barriers.  CAST defines the guidelines as universal – applicable to all students in all subject areas.

Using effective UDL strategies, students are engaged with their own education, they learn in more in depth way, they achieve at higher levels and they are motivated to continue to learn.

 I’ll examine more specific ways that UDL can be used to create engaged, knowledgeable and thoughtful learners in future blog posts and unpack research from educators who have already begun seeing the results of these principals in action.

 

 CAST (2011). Universal design for learning guidelines version 2.0. Wakefield, MA: Author

 Meyer, A., Rose, D.H., & Gordon, D. (2014) Universal design for learning: Theory and practice, Wakefield MA: CAST.

 Hartmann, Elizabeth. "Universal Design for Learning (UDL) and learners with severe support needs." International Journal of Whole Schooling, vol. 11, no. 1, 2015, p. 54+. Gale In Context: Canada, https://link.gale.com/apps/doc/A420051484/GPS?u=ko_k12hs_d2&sid=GPS&xid=a7b757b0. Accessed 15 Oct. 2020.