Data gurus: I've been analyzing student success data in my courses. What do you make of this?

[Coventina]

So, I've been analyzing performance (passing) rates in my classes over the past 5 spring semesters: 18, 19, 20, 21, 22

I've looked at it from every demographic measurement the school provides me: age, gender, ethnicity

The only significant performance difference I can find is by gender. (Some of the populations are too small for me to really measure).

Females have always consistently outperformed males in my classes:

It was at a high in the spring of 18: 17% points! Women were ahead of the males.

By spring of 22, it was down to 8% points. Both genders have declined, but females more dramatically.

I'm guessing that the biggest factor might be the pandemic, it was extremely disruptive to education, although I have done my best to make all accommodations for students that I can. I accept late work, I accept many more absences, etc. etc.

While it makes sense that both genders would be affected, why do you think females have been hit harder? And why do both genders continue to decline? Is it too early to look for recovery from the pandemic?

Just wondering what those of you who like to crunch numbers might think.

[WhiskeyGrinder]

1. Female students may have been saddled with more pandemic responsibilities, such as caregiving, or

may have been harder on themselves mentally as far as caregiving goes, resulting in greater stress and lower performance.

[Gidney N Cloyd]

2. I'd expect women to be more affected by the pandemic since they tend to be the caretakers.

[yardwork]

3. There's a mental health crisis among young people, and it's not improving.

Apparently it's affecting young women even more than young men. I wonder if it's because women count on their support networks more than men. Social isolation is having a bad effect on all young people, but more on women than men. I can think of several hypotheses for why.

Also, I don't know the ages of your students, but women are more likely to have caregiver roles than men, and all those stresses have been exacerbated by the pandemic. Loss of child care, worry about aging parents who are affected by the pandemic's isolation, etc.

I've always thought that you are a very dedicated and caring professor. Now more than ever, I'm sure that means the world to all your students.

[Coventina]

4. Awe, thanks yardwork! I do really care about my students.

The vast majority of my students are typical "college age" students.
The older ones are usually there because of personal interest, so they tend to do really well.
Plus, they are so small a population, I don't really count them in my data sets.

Every year or so, I try to look at the population of my classes, who is doing well, who is doing not so well....

This year I was really hoping for an uptick in performances in both genders, but it hasn't happened yet.

I guess this belief that "life is returning to normal" is a bit premature.

[MichMan]

5. Have you considered that your tests and material may be biased for one gender vs another ?

[Coventina]

10. I would certainly be willing to consider that, but I don't know how.

I don't give tests, actually.

I have a variety of types of assignments that make up grades.

It's a mix of written, oral, group, partner, assignments.

Students write papers, prepare presentations, do (small) research assignments, give presentations, work in groups to answer questions or prepare projects, etc. I try to make my assignments as varied as possible to engage as many different styles of learners as possible.

[Ms. Toad]

16. Most of those tools are biased toward women.

As a general rule anything which is an open ended question favors women; anything which presents choices favors men. Things with greater time pressure are more varied, but slightly favor men.

I was on the state committee evaluating changing bar exam tests. Using the scores earned on the individual components (multiple choice, short essay, long essay)

The weighting on the first exam was 33.3% multiple choice, 13.3% long essay, 53.4% short essay
The new exam was weighted 50% multiple choice, 20% long essay, 30% short essay

On the first exam (in place for years), men passed at a rate 3% higher than women.

When we reweighted the exams (exact same scores - only the weighting changed) to estimate the impact of moving to the new exam, the men passed at a significantly higher rate than women.

When we mushed all of the written scores together (67% written, rather than 50% written, based solely on the short essays), but shifted to 50% weighting for multiple choice men passed at a 6% greater rate than women.

When we weighted the short essays and long essays separately (increasing the weight for long essays, decreasing the weight for short essays) men passed at a 7% greater rate than women.

Again - the exact same work was used. About 1200 individual scores x 3 years of exams. The only difference was the weighting.

The groups studying the exam in other states found similar outcomes: In every jurisdiction in which the weight of multiple choice was increased, more men passed relative to women. As far as I know we were the only state which individually weighted short v. long essays - I had to push pretty hard based on my experience preparing folks for the test. I know, from personal experience, that the long essay scores are the least predictable - and as I expected, more individual students would have shifted from pass to fail, or fail to pass than changing the weight of multiple choice alone. What I didn't expect was that it bumped the advantage to men up by a full percent.

The difference as to the long v. short essays seems to be that it is pretty easy to manage the short essays in the time allotted, but the long ones are a mad rush against time. Men seem to perform better under time pressure, on average.

[Coventina]

18. My discipline is not well-measured by multiple choice exams, though.

Giving a multiple choice exam basically asks students to memorize discrete facts, which is a very low level of learning, and, those facts are quickly forgotten after the exam.

I'm looking for students' abilities to analyze, synthesize, and build arguments to support a thesis.

[Ms. Toad]

19. I wasn't suggesting you switch

Just demonstrating how much difference the measuring tool influences who passes.

But there are lots of ways to use multiple choice questions, which have little to nothing to do with memorization of facts.

[Yavin4]

6. What do you teach?

Is it freshmen level? Sophomore? Higher?

[Coventina]

7. Mostly freshmen, a little sophomore. n/t

[Yavin4]

9. What's the subject?

Apologies if I missed it in your post.

[Coventina]

11. Whoops, no, I forgot to answer your question: Art History. n/t

[Yavin4]

13. Students today are under pressure to perform in STEM classes.

Time lost during the pandemic is contributing to the pressure to keep up. They're probably de-prioritizing your class to catch up with other subjects.

[Coventina]

14. Thats....disheartening but logical.

[Yavin4]

17. Indeed.

There's intense pressure to chase high paying tech jobs after college.

[diane in sf]

8. I haven't been as quantitative with my classes. But performance has decayed since COVID. I

believe students got ‘unschooled’ during that time. I’ve been very accommodating about absence and late work since the pandemic. This fall semester it was common to have only 25-50% of the class attend in person. And it didn’t help the missing students performances. I asked my attending students if the low attendance situation was true of other classes they were taking. They said it was.

Even though I posted recordings of all the lectures and demos, and written descriptions of what was expected for their projects, I was amazed at how badly a number of students did. I have never handed out so many Fs and D-s.

This spring semester I’m going to require attendance unless the student has a doctor, disability, or athletic travel letter. And all late homework will lose half a grade in value. Where I’m accommodating is that it can be as late as they need without losing any increasing points. Recorded lectures will still be available and of course written assignment descriptions and PDFs of PowerPoints.

I don’t think there is a good substitute for in class time with peers. And students are losing one of the most valuable parts of their education by not participating in class.

Since some lack the self-discipline to get there on their own and get their assignments in, I’m going to apply discipline externally.

[Coventina]

12. Like you, I believe there is not substitute for the classroom experience.

I've always made attendance / participation part of their grade.

But, aside from that, there are a LOT of students who are just not bothering to turn in assignments.

[diane in sf]

15. I had a group who only turned in the first assignment. Their parents are not going to be

happy paying all that money for them to get Ds and Fs.

[Sympthsical]

20. Boys have been neglected by the education system for years

Plenty of research bearing this out (check the Department of Education for numerous stats and studies), but they're boys, so no one really cares. Say boys need help, and the argle bargle comes fast and furious and everyone gives up because no one wants to deal with the bullshit.

However, I think the pandemic became something of an equalizer - everyone was getting crappy education and lack of attention.

As someone currently in college for a career change and doing most of my classes remotely (outside of sciences and labs), I'll just say this - it can be very hard to focus when going to school remotely. I sit at my PC and have all my course modules and assignments and things. And there are times, even as an adult, where I'm working on something and suddenly it's, "Hrm, think I'll just watch this quick YouTube video . . ." and suddenly a half an hour has gone by and I've done nothing. Then an hour. "Hrm, I'll start this project on Friday and get a little bit done so it's finished by Monday. Out, look at this Reddit thread . . ." *twenty Reddit threads later* "Oh, my friend is messaging me over Facebook." And on. And on. Now imagine that as a teenager or young adult.

Classrooms help because you have no choice but to pay attention (depending on how strict teachers and profs are about laptops and smart phones). At home, you need a lot of self-discipline to get things done. Not just to get the assignments done, but to get the studying done. Once the assignments and studying are all mish mashed with everything else on your computer, it can sometimes take epic focus to really knuckle down and get done what needs to get done. To actually set aside an hour or two and only study something.

Fortunately, I'm in my early 40s, so I've learned how to get things done and schedule and bear down. However, some people need outside influence, pressure, and structure to focus and succeed.

The pandemic and remote learning obliterated that for a lot of students. I was on my nephews during that like a domineering helicopter uncle, because the teachers had absolutely checked out. Their assignments were so simplistic and filled with, "Whatever, remote, who gives a fuck," that I would often stare at their assignment sheets with total incredulity.

Pandemic learning was a shitshow. And here's the thing. In college? Remote learning is still a shitshow. Write some papers, read some things, make some posts to a bulletin board, maybe toss out a few hundred words about whatever. I have had multiple professors who just blatantly check out all semester. Some I never once e-mailed or spoke to. I've encountered kids who were in high school during the pandemic with whom I was quite genuinely wondering if they are capable of writing a sentence in grammatically correct English.

In person classes are nothing like this. They're challenging. There's written work. There are tests and quizzes you can't google your way through (this varies, based on if anyone bothers to monitor). The professors are generally guiding and keeping everything on track.

It all made a difference.