The Secret Problem With That Testing Column

[n2doc]

By Scott Huler | December 12, 2011



I hope you won’t object to a post not specifically on point about energy or transmission or connectivity. But I think it gets directly at how we understand those topics, so I think it’s worth mentioning.

Everybody has been reading and posting and cross-posting and commenting on this post, in the blog of Valerie Strauss, who writes The Answer Sheet blog for the Washington Post. In it Strauss allows Marion Brady, an educator living in Florida, to guest post, and Brady talks about a friend of his, Rick Roach, on the board of education in Orange County, Florida. He has, as Strauss says, “two masters degrees: in education and educational psychology. He has trained over 18,000 educators in classroom management and course delivery skills in six eastern states over the last 25 years.”

Brady’s post describes how Roach, to better understand the tests his students were constantly having to face, took a “version of the state standardized test and was horrified at what he found,” as Strauss says. Roach says he couldn’t answer a single one of the math questions but was able to guess right on one-sixth of them. He also says on the reading portion of the test he got a score of 62 percent.

The conclusion is simple: the tests make no sense; they test the wrong things; they don’t accurately reflect a student’s capacities, achievements, or aptitude for study. It’s a listing of problems expressing a distrust of the current educational focus on testing that’s familiar — Brady himself wrote a long guest post for Strauss on November 1 2011 on that exact topic, listing the many reasons educators resist test-based programs like No Child Left Behind.


more

http://blogs.scientificamerican.com/plugged-in/2011/12/12/the-secret-problem-with-that-testing-column/
Interesting column. Sounds like Roach is an idiot. Maybe a well meaning idiot, but still...

[FBaggins]

1. That was my conclusion. It's embarrassing that someone without an education...

...can end up on a school board. Let alone get graduate degrees in education and education psychology. Isn't that later degree an MS with a math requirement? Certainly statistics, no? I'm guessing that there should be at least three courses measurement/statistics.

He reportedly guessed on all 61 math questions and confesses that he didn't know how to do any of them (a fact which is confirmed by the fact that he did worse than would be expected from just random selection).

There may be a problem with this test, but all we know from this exercise is that this guy doesn't belong anywhere near the education system. His nonsense about how nobody uses that math in the real world is excusable ignorance in the 4th grade student who doesn't want to learn fractions... it's inexcusable here.

He has trained over 18,000 educators in classroom management and course delivery skills in six eastern states over the last 25 years.”

The good news is that it's possible that these statistics are self-reported. It may have only been 12 educators in three states last summer and he just multiplied it wrong.

[proud2BlibKansan]

8. Are you really wanting to restrict school board membership?

I have a problem with that. In many communities there are not a large number of college graduates to run for school board.

[FBaggins]

9. What's the difference between "restrict" and "high standards" in your mind?

I'm not saying that there should be a law requiring that prospective SB members pass a test. I'm saying that the school board should be made up of people who can actually pass the tests that the students have to take. Knowing now his... shall we say "academic weakness"... I can't imagine considering him the next time his term is up.

For heaven's sake. Some school boards have to deal with budgets. Wouldn't you like their votes to be at least moderately informed?

there are not a large number of college graduates to run for school board.

I didn't insist on college graduates. I merely expressed shock that someone could get multiple degrees without a HS education. I'm fine with some stay-at-home parent without any college background running for school board if they feel strongly about the issues. But I wouldn't consider them qualified if they couldn't come anywhere close to passing a 10th grade math test.

Just as importantly... I presume that you would insist on at least a college degree for teachers. IIRC you didn't even like the high-performing college graduates if they didn't then complete a post-graduate teacher certification program. School board authority varies from area to area, but these are often the people who set such standards, hire/fire teachers, select curricula, etc. How low shall we set the bar for them?

[proud2BlibKansan]

12. And I think school board members should be the ones the community chooses to serve

"We" have no right to set any bar for them.

[LWolf]

2. Back in November of '08,

Marion Brady was my 2nd choice for Sec. of Ed; I mentioned him in a letter I sent to Obama that was returned to me, unopened, because I sent a book about education with it.

http://www.democraticunderground.com/discuss/duboard.php?az=show_topic&forum=219&topic_id=11231#11239

My first pick, Gerry Bracey, has since died.

If the nation really wanted to heal and support public education, they'd listen to voices like Bracey, and like Brady, instead of Duncan, Gates, Rhee, etc..

Roach? An idiot? He had to pass college math classes to get those masters. I'd suggest that those who would brand him an "idiot" for having trouble with standardized tests take one themselves before they pass judgement.

I WISH school board members around the nation would start rejecting the assumptions about high-stakes tests and fight back.

[n2doc]

3. Did you bother to read the whole article?

If Roach can't answer the questions as described he is an idiot. Lots of these with college degrees. Especially in education admin.

[LWolf]

4. Yes.

I read it several days ago, before it was posted on DU.

You calling someone an idiot doesn't make it so.

[FBaggins]

5. "calling" doesn't make it so...

... but being unable to answer a single question on a standardized math test at that level is a pretty good indicator.

[LWolf]

6. Please take the test he referred to.

We're all waiting to see how you do.

[FBaggins]

7. I went through the online FCAT practice test - it isn't hard.

They let you use a calculator and give you the basic formulas that might be needed (so you don't even need to remember how to figure the midpoint on a line between two points for instance).

There were a number of "everyday" math problems that any adult should be able to do ("a contractor charges a flat fee for starting a job plus an hourly wage. If the flat fee is $45 and he works for three hours and charges a total of $145.50 - what is his hourly wage?" )

There were no questions that I considered difficult for any adult with multiple degrees (and therefore presumably acceptable SAT and GRE scores).

On edit (I love that DU3 lets you edit things so much later!) -
The test isn't multiple choice. So my earlier comment on his performance is a bit off base. He said that he didn't know how to do any of the problems and only got a few of them right because he guessed correctly... but this had to be at least a somewhat "educated" guess. The score is still clearly failing (and IIRC you can't graduate from HS in FL without passing) and there's no exuse in my mind for being accepted into a postgraduate program with this level of ignorance (let alone award multiple graduate degrees and a slot on a school board)... but it's not the total disaster that was implied by his comments and an assumption that it was a multiple-choice exam. Frankly I can't see how he could earn even an undergraduate degree in education.

[LWolf]

10. I spent this week looking at the approved practice tests

for the tests that my students take. Not math, since I don't teach math any more; reading. The practice tests, at least in my state, don't align well with the tests my students actually take. The practice tests are simpler. For the real thing, students all get a different test, based on their early answers. If they do well early, they get fewer, but more complex questions. If they don't, they get more, but simpler questions. Many of those questions are "choose the best answer" out of more than one reasonable choice. Being multiple choice, though, they don't get to defend why one choice is "best."

The practice test is a piece of cake in comparison.

[FBaggins]

11. Then let me correct/clarify

The "practice tests" that I found on their website are the actual tests from prior years.

For the real thing, students all get a different test, based on their early answers. If they do well early, they get fewer, but more complex questions. If they don't, they get more, but simpler questions.

That's more like the current GMAT and (I think) GRE tests. I happen to like that plan, but have no opinion on whether that's ideal for this level.

That last part, however, wouldn't be good news for this individual were it true - since he confesses to not knowing how to do any of the questions. Presumably he would be getting easier and easier questions as he went along.

On edit - a verifiable record of practice tests being insufficiently like the actual test would be a FAR better criticism of the standardized tests than this guy's ignorance... since presumably a high percentage of FL students significantly outperform him.

[LWolf]

13. That's a statistic that would be good to look at.

The states that I've worked in don't make previous years' tests public, because they don't publish new tests every year; that's cost prohibitive.

It's been about 8 years since I proctored a math test. The test I remember most distinctly was the math test for 2nd graders in CA. I don't know if they are still taking that test.

THAT math test was a better measure of auditory memory than math skills. You see, the test developers wanted to insure that the test results would not be affected by 7 yos who couldn't read the test fluently. So...most of the test was read aloud to the students. There was a small section with calculations, but the majority were "story problems." So the proctor would read the problem to the students.

The problem with that? The problem wasn't actually printed in the students' test booklets. They couldn't read along, or refer back. The proctor was allowed to read the problem ONCE. Students were allowed to use paper and pencil for whatever they needed. 7 yos, though, are not that efficient at note-taking, and that's not the end of it. Once read, proctors were to give them 10 seconds to solve, and then read the next problem. That's not enough time to do much of anything constructive with that paper and pencil.

Those students with good auditory memory, who could remember the whole problem and sort out the relevant information without seeing it or writing it down...those students did much better on that math test than the rest.

[FBaggins]

14. Can do.

http://fcat.fldoe.org/fcatrelease.asp

I have no beef with your explanation re: why you dislike a particular test. My point was never to defend a particular test, but instead to point out that the validity of a test is not refuted by poor performance by a particular adult when that performance is so bad that it's clearly an indictment of his own abilities and/or the education that he received.

Put more succinctly, we all tire of the student who whines "I don't need to learn this because I'll never use it in real life". It doesn't help to have a school board member show that he agrees.

[LWolf]

15. I had a student, just last week,

tell me that she agrees with my teaching partner, her math teacher, that she would never need history in real life, and shouldn't have to learn it.

[FBaggins]

16. I suppose you could fail the kid.

Bringing new meaning to "Those who won't learn History are doomed to repeat it"



Seriously... I think you see my point. You probably don't want your partner to win a seat on the school board where that opinion could damage more than just this one kid's future.

[LWolf]

17. No, I don't want him on the school board, lol.

He IS an effective math teacher, if otherwise clueless.

When the student said that, the whole room fell silent; they knew what was coming. I spoke pretty mildly, and briefly, about the importance of history, and ended by pointing out that my teaching partner's favorite book, the one he dips into most often, the one he thinks is the most important book in the world, IS a history.

She looked confused, until someone near her leaned forward and whispered, "The Bible." Then she blushed, and I moved us on to other things.