Monday, September 24, 2012

Before Tony Bennett's 2012 "State of Education" Speech, A Reminder of the Data Exaggerations from His 2010 Speech

Tony Bennett is scheduled to give his third “State of Education” tomorrow night (Tuesday) in Indianapolis.

After he gave his first such speech in 2010, I wrote an article about data exaggerations he used in his speech.  I have reviewed the article from two years ago, and I believe it is appropriate to reprint it for you here as preparation for the speech tomorrow night and as part of the “hold Tony Bennett accountable plan” which culminates in the election on November 6th

This explains one more reason why I am supporting Glenda Ritz for state superintendent in the election.  I lost confidence in Tony Bennett’s accuracy early in his term, and this article provides a clear illustration of this problem.

As you read, keep in mind that this article (in bold print below) was written in October, 2010.   I have not changed or updated it:

              “Waiting for Fact Checking:  How Many Drop Outs and Failing Third Graders Do We Really Have?” 

In his major State of Education address on August 23rd, Dr. Bennett cited the following statistics about deficiencies in Indiana schools: 

“We must close the door on a system that allows 23,000 Hoosier students to drop out of high school each year. We must close the door on a system in which 25,000 third graders cannot read at grade level.”

The editors of the Indianapolis Star were impressed with those numbers and featured them in their lead sentences of the Sunday Star editorial (8/29/10): “Each year in Indiana, about 25,000 children fail to acquire basic reading skills by the time they complete third grade.  Each year another 23,000 Indiana students also drop out of high school.”

I’ve been waiting for fact checks on these two attention-grabbing numbers.  But who would ever check?

Not teachers or administrators. They are far too busy.  Not the press, although one might argue that fact-checking is their role.  The Star editors were all too eager to repeat the numbers as stated.  Not university professors.  They are doing scientifically-based research.  I decided to look into the numbers myself.

IDOE data for the Class of 2009, the most recent available, show that 8.7% dropped out in the 4 years from 9thgrade to graduation.  With 75,905 in that cohort, 8.7% is 6604 dropouts, which of course includes not only declared dropouts but also students who disappear and can’t be documented as transfers, those known as “Undetermined.”  Now if one 4-year cohort had 6600 dropouts, what could Dr. Bennett have meant by saying our system “allows 23,000 Hoosier students to drop out each year”? 

I checked other cohorts. The Class of 2008, with 10.3% dropouts, had a total of 8000 (actually 7997) in four years.  The Class of 2007 had 9100 dropouts (actually 12.0%, 9107).  The Class of 2006 had 8600 (actually, 11.4%, 8596).  Add them up (2006-2009) and we get:  6600 + 8000 + 9100 + 8600 = 32,300 drop outs in four cohorts. 

That would be an average of 8000 per cohort.  The cohort count includes four years.  That’s still too many drop outs, but it’s a long way from 23,000 per year.  Not even close.

Would the 3rd grade statistic be easier to document?

I remembered that State Board member David Shane used numbers in testifying last January on the 3rd grade reading bill, saying that “18,929 3rd graders failed English/language arts on ISTEP.”
How did the third grade reading problem grow by 6000 students since January?   

 Checking Spring 2010 ISTEP+ actually revealed fewer failures in Grade 3, with 15,130 not passing the test. 15,000?  That’s a long way from 25,000.  Further, the IDOE data show that 4,997 of the 15,130 failures were special education students.  Only 10,133 were general education students, the population that would be considered for retention in the reading plan. 

That is a long way from 25,000.

Perhaps failing 3rd grade ISTEP+ was not what Dr. Bennett meant by “cannot read at grade level.”  ISTEP+ is a language arts test which includes reading but does not produce a reading score.  In fact, to implement the retention plan, a new reading test needs to be added to ISTEP+, to be piloted in Spring, 2011 and used to retain 3rd graders in Spring, 2012.

Given that we have no state reading test, citing a statistic on how many couldn’t read at grade level is a tall order. 

I started thinking about other data sources:  Acuity?  No state level data is reported.  National Assessment?  No, that’s 4th and 8th grade.  Wait, maybe Dr. Bennett made a mistake and meant 4th grade instead of 3rd grade.

National Assessment is a random sample test given every two years; a small sample of Hoosier students take it. In 2009, 70% of Indiana’s fourth graders reached the nationally defined basic level in reading, better than the national average of 66%.  The IDOE 4th grade count in 2008-09 was 78,805.  If 30% were below basic on NAEP, that would be 23,641, assuming the random sample truly reflected the entire group.  Still, that is 1400 below Dr. Bennett’s statistic and addresses 4th grade, not 3rd.      

I had to give up my quest and conclude that Dr. Bennett’s 3rd grade number could not be documented. Perhaps a new data source will someday enlighten me.

What’s going on here?  In the major speech of the year, two leading statistics can’t be documented.

Do officials ever footnote their statistics?  Not very often.  Do statistics ever get stretched for political purposes?  Yes.  Does anyone ever notice?  Not often.

Everyone wants to improve the status quo, but differences often erupt on which new change will bring the biggest return, e.g. whether to invest in early childhood initiatives or merit pay incentives.  Dr. Bennett, as he pushes changes for Indiana, should bring unassailable, thoroughly documented data to the debate. 

Accuracy matters.  Credibility matters.  I have lost confidence in Dr. Bennett’s numbers.  I hope he will start documenting his statistics with footnotes.

End of article.

In hindsight, Tony Bennett’s “enhanced” reading statistics used in the speech helped push through the controversial and, to many, unpopular IREAD program.  As far as drop outs, statistics have improved to 6.4% for the Class of 2010 and 6.1% for the Class of 2011.

We have gone long enough with a state superintendent who stretches the numbers for political purposes.   Another key example was the use of cherry-picked Florida-Indiana comparisons as the basis for the September 2009 Roundtable meeting when the claim was made that since Florida was ahead of Indiana in 4th grade reading on the National Assessment, we should change our policies to mirror Florida.  No one mentioned the fact that Indiana has consistently over many years outscored Florida on 8th grade reading as well as on 4th and 8th grade math.

I support Glenda Ritz to restore credibility and accuracy to the office of State Superintendent of Public Instruction.

I urge you to donate to the Ritz campaign, but more importantly, to talk with non-educators who trust you about the issues and enlist their support for Glenda Ritz.  It will take a lot of that if Tony Bennett is to be upset in this election. The urgency is clear.  I hope you will help.

One last question:  Will anyone be fact-checking tomorrow night?

Best wishes,
Vic Smith

 “Vic’s Election Notes on Education” is not linked to any organization.

No comments:

Post a Comment