Why Effort Should Not be Included in Grading
When assessing the fairness of an
assessment task, three areas can be examined; bias, equity in opportunity to
learn the construct of interest and equitable treatment of the subject of the
evaluation (Baharloo, 2013). When
creating assessments and accompanying grading rubrics, educators need to
consider the issues of equity, fairness and validity. When these factors are not considered, assessment
and evaluative tools can be used as a means to compare marginalized students (those
living in poverty, with learning difficulties or other types of discrimination)
to more advantaged students and thus push them even further to the margins,
when it should be employed to support their development as diverse human beings
(Milner, 2018).
Construct Irrelevant Factors
Construct irrelevance (or construct irrelevant variance)
occurs when an assessment is measuring more variables than just the construct
of interest, many of which are irrelevant to the content standards (Haladyna &
Downing, 2004). The underlying assumption of all assessments is that the score
that is produced reflects a student’s ability on the construct of interest. The
validity of the assessment is measured by considering if the task adequately
reflects the domain of knowledge and skills of interest and if it can be used
as the basis for the inference of proficiency (Ferrier et al, 2011, Growing
Success pg. 156). The meaningfulness and accuracy of assessment results,
therefore, is adversely affected by focus on irrelevant factors (Brookhart,
2019). When an assessment is valid, the
construct (ability, skill, or knowledge) that it is designed to measure is the
source of students’ score on the assessment. An assessment is invalid, then, when
factors that have nothing to do with the target construct affect a student’s
score (Ferrier et al, 2011, Baharloo, 2013).
By assessing these construct irrelevant factors, such as effort, misinterpretations
could be made about a student’s proficiency and undermine the accuracy of
scoring data (Allen 2005). In fact, “teachers'
practice of including a variety of criteria in the assignment of grades
distorts grades, rendering them invalid and meaningless” (Olson, 1989).
Why Teachers Believe That Effort Should Be Graded
Teachers believe that students who
try harder should be rewarded and that more effort results in more knowledge
acquisition (Howley, 1999, McMillan, 2011), some, in fact, believe that effort
is more important than amount of academic content learned (Allen 2005). They may even believe that they are helping low-achieving
students to stay engaged with the material and that grading effort could be
used effectively as a motivational tool (McMillan, 2011).
A study at Benedict College seemed to confirm this, concluding
“strong evidence that more student effort does lead to increase learning”
(Swinton, 2010). The study also found,
however, that “better measure for effort than previous research and a more
appropriate estimation procedure” were needed for this correlation to be noted,
that regularly high-achieving students did not report this same result because
they can choose when to implement effort and that the impact on graduation
rates could not be noted (Swinton, 2010).
Another reason teachers reward perceived effort may be that in
some troubled schools, where compliance and cooperation are lacking, good
behavior may replace achievement as the desired response of students (Howley
1999). However, when marginalized students
receive grades that fail to provide an accurate representation of their
achievement (either inflation or deflation), they can be seriously harmed (Allen,
2005, Howley, 1999). When marks are
inflated, students have an unrealistic view of their ability, setting them up
for failure later in life and when they are deflated, opportunities for
employment and higher education are significantly reduced (Lipman, 1997, Howley,
1999).
Evaluator Bias in Judging Effort
Assessment bias occurs whenever assessment
items “unfairly penalize students for reasons related to students’ personal
characteristics, such as their race, gender, ethnicity, religion, or
socioeconomic status” (Popham, 2011). In essence, biased assessment favors one
group of students over another or negatively impacts one group of students over
another. Effort can be difficult to
measure and define and as such be inconsistently evaluated (Allen, 2005). Teachers tend to have a bias towards learners
who exhibit qualities of a “good student” regardless of actual performance
competency (Allen, 2005). This includes
things like "being on task”, “class participation”, “homework completion” and
including additions to assignments like fancy paper or use of stencils on
posters, costumes for oral presentations, high tech elements, and decorations
that match themes, to name a few (McMillan, 2011). Measuring effort can, therefore, create an
inequitable standard for students on a number of levels, including gender
differences, socio-economic factors and parental support. “Regardless of the
strength of their rationale for doing so, teachers' practice of confounding
effort and compliance with achievement in their construction of students'
grades makes grading vulnerable to race and class bias” (Howley, 1999).
Gender Bias
One possible explanation for multiple findings that teachers
treat and evaluate boys and girls differently is that “girls tend to have more
positive attitudes toward school and put more effort into their schoolwork, and
teachers not only evaluate students’ competence, but also reward effort” (Krkovic,
2014). A study at the University of New
Brunswick found that stereotypical patterns of boys scoring higher in math and
science only occurred on standardized achievement tests even though girls
scored higher across every subject in school tasks (Voyer, 2014). This shows a clear difference in actual
knowledge acquisition and in school performance. This could be due to the perception by
teachers that girls are “less disruptive than boys and this could also result
in teachers overestimating girls’ performance” (Krkovic, 2014).
Socio-economic Bias and Parental Support
There is significant evidence to show that working-class
pupils are implicitly seen as less able than their more affluent counterparts (Creasey,
2016). Evidence seems to support that
fewer marginalized children will develop to the full measure of their potential,
achieve at a high level or acquire advanced intellectual competencies and
academic skills (Kitano, 2003, Edgerton, 2008).
This may be because higher income parents can use their financial
resources to improve the performance of their children through buying private lessons
and learning materials (Tieben et al, 2010) and are better able to address
specific learning strategies like monitoring homework, reducing television
time, supporting development of good study habits, and high expectations that
appear most likely to have positive effects on children's academic performance (Kitano,
2003). Lack of support at home for a number of reasons, including the need to
work, makes students less able to obtain these skills and parents may lack the
technical know how or time to provide new technology and apps (Tieben et al,
2010, Edgerton, 2008). Students who
arrive in class without the appropriate materials and those who do not complete
homework may be accused of being “irresponsible” or “unprepared” and may be seen
by their teachers as lacking in effort. Further,
parents who have obtained higher education themselves are not only able to help
and support their children in education but are familiar with requirements of
higher educational tracks. (Tieben et al,
2010, Edgerton, 2008) Where teachers do recognize that children from
different backgrounds have different needs, there seems to be a double standard
of success. One study indicated that for students whom teachers assumed lacked
a positive home environment “success was feeling good about school, adjusting
to rules and expectations, having positive interactions with adults, and
attaining a sense of belonging” and lacked any correlation between success and
academic achievement (Lipman, 1997, Howley, 1999).
Moving Passed Effort as a Measure of Evaluation
For assessment tasks to be regarded
as fair and valid, the results must reliably indicate a student’s academic
achievement and ability on the construct of interest and that construct needs
to be clear to both the educator and the student. When effort is graded as part of an
assessment, the data can no longer accurately communicate a student’s ability
and can have long term negative effects on the student being unfairly graded. Evaluation standards of effort currently lack
consistency and are subject to bias and unfairly disadvantage those who are
marginalized. Effective teaching,
learning and assessment practices are dependent upon the knowledge and value
systems of educators (Tierney et al, 2011) Inadequate education in valid
assessment and grading principles and practices is a reason many teachers
continue to perpetuate invalid grading practices with students (Allen, 2005). Many teachers struggle with the idea that
good work habits can be encouraged without grading penalties, despite evidence
to the contrary (Tierney et al, 2011). Teachers may also have ingrained
experience, beliefs and long-standing habits about assessment that must be
overcome (Brookhart, 2019). Campbell argues that equity, diversity, and
inclusion (EDI) educational programs are central to raising participants’
awareness of behaviours that might have discriminatory effects, even when those
effects are not intended (Campbell, 2021). If teacher education adopted both
fair assessment and EDI training, old patterns of evaluation using construct
irrelevant factors like effort could be eliminated, resulting in more accurate
results for all stakeholders.
Resources
Allen,
James. (2005). Grades as Valid Measures of Academic Achievement of Classroom
Learning. The Clearing House.
Baharloo, A. (2013). Test fairness
in traditional and dynamic assessment. Theory and Practice in Language
Studies, 3(10), 1930+.
https://link.gale.com/apps/doc/A351082074/GPS?u=ko_k12hs_d2&sid=GPS&xid=e027de3d
Brookhart, S. M., & Nitko, A. J.
(2019). Educational assessment of students. Pearson.
Budge, K. M., & Parrett, W.
(2018). Disrupting poverty: 5 powerful classroom practices. ASCD.
Campbell, A.
(2021). Equity education initiatives within Canadian universities: promise and
limits. Perspectives: Policy and Practice in Higher Education, 25(2),
51–61. https://doi.org/10.1080/13603108.2019.1631226
Creasey, S.
(2016). Is unconscious bias holding your pupils back? TES: Times Educational
Supplement, 5185, 26–32.
Edgerton, J.
D., Peter, T., & Roberts, L. W. (2008). Back to the Basics: Socio-Economic,
Gender, and Regional Disparities in Canada’s Educational System. Canadian
Journal of Education, 31(4), 861–888.
Ferrier,
David & Lovett, Benjamin & Jordan, Alexander. (2011).
Construct-Irrelevant Variance in Achievement Test Scores: A Social Cognitive
Perspective.
Haladyna, T.
M., & Downing, S. M. (2004). Construct-Irrelevant Variance in High-Stakes
Testing. Educational Measurement: Issues & Practice, 23(1),
17–27. https://doi-org.proxy.queensu.ca/10.1111/j.1745-3992.2004.tb00149.x
Howley, A.,
Kusimo, P. S., & Parrott, L. (1999). Grading and the Ethos of Effort.
Jensen, E. (2010). Teaching with
poverty in mind: what being poor does to kids' brains and what schools can do
about it. ASCD.
Kitano, M.
K. (2003). Gifted Potential and Poverty: A Call for Extraordinary Action. Journal
for the Education of the Gifted, 26(4), 292–303.
Krkovic, K.
(2014). Teacher evaluation of student ability: what roles do teacher gender,
student gender, and their interaction play? Educational Research., 56(2),
244–257.
Lipman, P.
(1997). Restructuring in context: a case study of teacher participation and the
dynamics of ideology, race, and power. American Educational Research Journal,
34, 3–37.
McMillan, J. H., Hellsten, L., &
Klinger, D. (2011). Classroom assessment: principles and practice for
effective standards-based instruction. Pearson Canada.
Milner, H. R. (2018). Confronting
Inequity / Assessment for Equity. Educational Leadership, 75(5),
88–89. https://doi-org/http://www.ascd.org/publications/educational-leadership/feb18/vol75/num05/Assessment-for-Equity.aspx
Olson, G.H. (1989, March). On the
validity of performance grades: The relationship between teacher-assigned
grades and standard measures of subject matter acquisition. Paper presented at
the annual meeting of the National Council on Measurement in Education, San
Francisco, CA.
Ontario
Ministry of Education, Growing success: assessment, evaluation and reporting:
improving student learning (2010). Toronto.
http://www.edu.gov.on.ca/eng/policyfunding/growSuccess.pdf.
Payne, R. K. (2013). Framework
for understanding poverty. Aha! Process.
Popham, James W. (2011). Popham -
Assessment bias.mp4. YouTube. https://www.youtube.com/watch?v=HjID39cqmzw.
Schleicher,
A. (2009). Securing quality and equity in education: Lessons from PISA. Prospects
(00331538), 39(3), 251–263. https://doi-org.proxy.queensu.ca/10.1007/s11125-009-9126-x
Swinton, O.
H. (2010). The Effect of Effort Grading on Learning. Economics of Education
Review, 29(6), 1176–1182
Tieben, N.,
& Wolbers, M. (2010). Success and failure in secondary education:
socio-economic background effects on secondary school outcome in the
Netherlands, 1927-1998. British Journal of Sociology of Education, 31(3),
277–290. https://doi-org.proxy.queensu.ca/10.1080/01425691003700516
Tierney, R.,
Simon, M. & Charland, J. (2011), Being Fair: Teachers’ Interpretations of
Principles for Standards based Grading, The Educational Forum 75(3): 210-227.
Voyer, D., & Voyer, S. D. (2014,
April 28). Girls make higher grades than boys in all school subjects,
analysis finds. American Psychological Association.
