Why Effort Should Not Be Evaluated

 Why Effort Should Not be Included in Grading

When assessing the fairness of an assessment task, three areas can be examined; bias, equity in opportunity to learn the construct of interest and equitable treatment of the subject of the evaluation (Baharloo, 2013).  When creating assessments and accompanying grading rubrics, educators need to consider the issues of equity, fairness and validity.  When these factors are not considered, assessment and evaluative tools can be used as a means to compare marginalized students (those living in poverty, with learning difficulties or other types of discrimination) to more advantaged students and thus push them even further to the margins, when it should be employed to support their development as diverse human beings (Milner, 2018).

Construct Irrelevant Factors

Construct irrelevance (or construct irrelevant variance) occurs when an assessment is measuring more variables than just the construct of interest, many of which are irrelevant to the content standards (Haladyna & Downing, 2004). The underlying assumption of all assessments is that the score that is produced reflects a student’s ability on the construct of interest. The validity of the assessment is measured by considering if the task adequately reflects the domain of knowledge and skills of interest and if it can be used as the basis for the inference of proficiency (Ferrier et al, 2011, Growing Success pg. 156). The meaningfulness and accuracy of assessment results, therefore, is adversely affected by focus on irrelevant factors (Brookhart, 2019).  When an assessment is valid, the construct (ability, skill, or knowledge) that it is designed to measure is the source of students’ score on the assessment. An assessment is invalid, then, when factors that have nothing to do with the target construct affect a student’s score (Ferrier et al, 2011, Baharloo, 2013).  By assessing these construct irrelevant factors, such as effort, misinterpretations could be made about a student’s proficiency and undermine the accuracy of scoring data (Allen 2005).  In fact, “teachers' practice of including a variety of criteria in the assignment of grades distorts grades, rendering them invalid and meaningless” (Olson, 1989).

Why Teachers Believe That Effort Should Be Graded

Teachers believe that students who try harder should be rewarded and that more effort results in more knowledge acquisition (Howley, 1999, McMillan, 2011), some, in fact, believe that effort is more important than amount of academic content learned (Allen 2005).  They may even believe that they are helping low-achieving students to stay engaged with the material and that grading effort could be used effectively as a motivational tool (McMillan, 2011).

A study at Benedict College seemed to confirm this, concluding “strong evidence that more student effort does lead to increase learning” (Swinton, 2010).  The study also found, however, that “better measure for effort than previous research and a more appropriate estimation procedure” were needed for this correlation to be noted, that regularly high-achieving students did not report this same result because they can choose when to implement effort and that the impact on graduation rates could not be noted (Swinton, 2010).

Another reason teachers reward perceived effort may be that in some troubled schools, where compliance and cooperation are lacking, good behavior may replace achievement as the desired response of students (Howley 1999).  However, when marginalized students receive grades that fail to provide an accurate representation of their achievement (either inflation or deflation), they can be seriously harmed (Allen, 2005, Howley, 1999).  When marks are inflated, students have an unrealistic view of their ability, setting them up for failure later in life and when they are deflated, opportunities for employment and higher education are significantly reduced (Lipman, 1997, Howley, 1999).

Evaluator Bias in Judging Effort

Assessment bias occurs whenever assessment items “unfairly penalize students for reasons related to students’ personal characteristics, such as their race, gender, ethnicity, religion, or socioeconomic status” (Popham, 2011). In essence, biased assessment favors one group of students over another or negatively impacts one group of students over another.  Effort can be difficult to measure and define and as such be inconsistently evaluated (Allen, 2005).  Teachers tend to have a bias towards learners who exhibit qualities of a “good student” regardless of actual performance competency (Allen, 2005).  This includes things like "being on task”, “class participation”, “homework completion” and including additions to assignments like fancy paper or use of stencils on posters, costumes for oral presentations, high tech elements, and decorations that match themes, to name a few (McMillan, 2011).  Measuring effort can, therefore, create an inequitable standard for students on a number of levels, including gender differences, socio-economic factors and parental support. “Regardless of the strength of their rationale for doing so, teachers' practice of confounding effort and compliance with achievement in their construction of students' grades makes grading vulnerable to race and class bias” (Howley, 1999).

Gender Bias

One possible explanation for multiple findings that teachers treat and evaluate boys and girls differently is that “girls tend to have more positive attitudes toward school and put more effort into their schoolwork, and teachers not only evaluate students’ competence, but also reward effort” (Krkovic, 2014).  A study at the University of New Brunswick found that stereotypical patterns of boys scoring higher in math and science only occurred on standardized achievement tests even though girls scored higher across every subject in school tasks (Voyer, 2014).  This shows a clear difference in actual knowledge acquisition and in school performance.  This could be due to the perception by teachers that girls are “less disruptive than boys and this could also result in teachers overestimating girls’ performance” (Krkovic, 2014).

Socio-economic Bias and Parental Support

There is significant evidence to show that working-class pupils are implicitly seen as less able than their more affluent counterparts (Creasey, 2016).  Evidence seems to support that fewer marginalized children will develop to the full measure of their potential, achieve at a high level or acquire advanced intellectual competencies and academic skills (Kitano, 2003, Edgerton, 2008).  This may be because higher income parents can use their financial resources to improve the performance of their children through buying private lessons and learning materials (Tieben et al, 2010) and are better able to address specific learning strategies like monitoring homework, reducing television time, supporting development of good study habits, and high expectations that appear most likely to have positive effects on children's academic performance (Kitano, 2003). Lack of support at home for a number of reasons, including the need to work, makes students less able to obtain these skills and parents may lack the technical know how or time to provide new technology and apps (Tieben et al, 2010, Edgerton, 2008).  Students who arrive in class without the appropriate materials and those who do not complete homework may be accused of being “irresponsible” or “unprepared” and may be seen by their teachers as lacking in effort.  Further, parents who have obtained higher education themselves are not only able to help and support their children in education but are familiar with requirements of higher educational tracks. (Tieben et al,  2010, Edgerton, 2008) Where teachers do recognize that children from different backgrounds have different needs, there seems to be a double standard of success. One study indicated that for students whom teachers assumed lacked a positive home environment “success was feeling good about school, adjusting to rules and expectations, having positive interactions with adults, and attaining a sense of belonging” and lacked any correlation between success and academic achievement (Lipman, 1997, Howley, 1999).

Moving Passed Effort as a Measure of Evaluation

For assessment tasks to be regarded as fair and valid, the results must reliably indicate a student’s academic achievement and ability on the construct of interest and that construct needs to be clear to both the educator and the student.  When effort is graded as part of an assessment, the data can no longer accurately communicate a student’s ability and can have long term negative effects on the student being unfairly graded.  Evaluation standards of effort currently lack consistency and are subject to bias and unfairly disadvantage those who are marginalized.  Effective teaching, learning and assessment practices are dependent upon the knowledge and value systems of educators (Tierney et al, 2011) Inadequate education in valid assessment and grading principles and practices is a reason many teachers continue to perpetuate invalid grading practices with students (Allen, 2005).  Many teachers struggle with the idea that good work habits can be encouraged without grading penalties, despite evidence to the contrary (Tierney et al, 2011). Teachers may also have ingrained experience, beliefs and long-standing habits about assessment that must be overcome (Brookhart, 2019). Campbell argues that equity, diversity, and inclusion (EDI) educational programs are central to raising participants’ awareness of behaviours that might have discriminatory effects, even when those effects are not intended (Campbell, 2021). If teacher education adopted both fair assessment and EDI training, old patterns of evaluation using construct irrelevant factors like effort could be eliminated, resulting in more accurate results for all stakeholders.

 

 

Resources

Allen, James. (2005). Grades as Valid Measures of Academic Achievement of Classroom Learning. The Clearing House.

Baharloo, A. (2013). Test fairness in traditional and dynamic assessment. Theory and Practice in Language Studies, 3(10), 1930+. https://link.gale.com/apps/doc/A351082074/GPS?u=ko_k12hs_d2&sid=GPS&xid=e027de3d

Brookhart, S. M., & Nitko, A. J. (2019). Educational assessment of students. Pearson.

Budge, K. M., & Parrett, W. (2018). Disrupting poverty: 5 powerful classroom practices. ASCD.

Campbell, A. (2021). Equity education initiatives within Canadian universities: promise and limits. Perspectives: Policy and Practice in Higher Education, 25(2), 51–61. https://doi.org/10.1080/13603108.2019.1631226

Creasey, S. (2016). Is unconscious bias holding your pupils back? TES: Times Educational Supplement, 5185, 26–32.

Edgerton, J. D., Peter, T., & Roberts, L. W. (2008). Back to the Basics: Socio-Economic, Gender, and Regional Disparities in Canada’s Educational System. Canadian Journal of Education, 31(4), 861–888.

Ferrier, David & Lovett, Benjamin & Jordan, Alexander. (2011). Construct-Irrelevant Variance in Achievement Test Scores: A Social Cognitive Perspective.

Haladyna, T. M., & Downing, S. M. (2004). Construct-Irrelevant Variance in High-Stakes Testing. Educational Measurement: Issues & Practice, 23(1), 17–27. https://doi-org.proxy.queensu.ca/10.1111/j.1745-3992.2004.tb00149.x

Howley, A., Kusimo, P. S., & Parrott, L. (1999). Grading and the Ethos of Effort.

Jensen, E. (2010). Teaching with poverty in mind: what being poor does to kids' brains and what schools can do about it. ASCD.

Kitano, M. K. (2003). Gifted Potential and Poverty: A Call for Extraordinary Action. Journal for the Education of the Gifted, 26(4), 292–303.

Krkovic, K. (2014). Teacher evaluation of student ability: what roles do teacher gender, student gender, and their interaction play? Educational Research., 56(2), 244–257.

Lipman, P. (1997). Restructuring in context: a case study of teacher participation and the dynamics of ideology, race, and power. American Educational Research Journal, 34, 3–37.

McMillan, J. H., Hellsten, L., & Klinger, D. (2011). Classroom assessment: principles and practice for effective standards-based instruction. Pearson Canada.

Milner, H. R. (2018). Confronting Inequity / Assessment for Equity. Educational Leadership, 75(5), 88–89. https://doi-org/http://www.ascd.org/publications/educational-leadership/feb18/vol75/num05/Assessment-for-Equity.aspx

Olson, G.H. (1989, March). On the validity of performance grades: The relationship between teacher-assigned grades and standard measures of subject matter acquisition. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco, CA.

Ontario Ministry of Education, Growing success: assessment, evaluation and reporting: improving student learning (2010). Toronto.  http://www.edu.gov.on.ca/eng/policyfunding/growSuccess.pdf.

Payne, R. K. (2013). Framework for understanding poverty. Aha! Process.

Popham, James W. (2011). Popham - Assessment bias.mp4. YouTube. https://www.youtube.com/watch?v=HjID39cqmzw.

Schleicher, A. (2009). Securing quality and equity in education: Lessons from PISA. Prospects (00331538), 39(3), 251–263. https://doi-org.proxy.queensu.ca/10.1007/s11125-009-9126-x

Swinton, O. H. (2010). The Effect of Effort Grading on Learning. Economics of Education Review, 29(6), 1176–1182

Tieben, N., & Wolbers, M. (2010). Success and failure in secondary education: socio-economic background effects on secondary school outcome in the Netherlands, 1927-1998. British Journal of Sociology of Education, 31(3), 277–290. https://doi-org.proxy.queensu.ca/10.1080/01425691003700516

Tierney, R., Simon, M. & Charland, J. (2011), Being Fair: Teachers’ Interpretations of Principles for Standards based Grading, The Educational Forum 75(3): 210-227.

Voyer, D., & Voyer, S. D. (2014, April 28). Girls make higher grades than boys in all school subjects, analysis finds. American Psychological Association.