.comment-link {margin-left:.6em;}

Ed Knows Policy

EKP -- a local (Washington, DC) and national blog about education policy, politics, and research.

Who is Ed Researcher?

Monday, July 31, 2006

DC elem. schools all failing?

It seems like when DCPS switched from the Stanford 9 to the new assessments that most elementary schools failed to make AYP . What I'd like to know is where is the data that school officials "released" according to this WaPo article>?

Click here for continuation of post (if any)

Wednesday, July 26, 2006

Petrilli commentary in Ed Week

This piece by Mike Petrilli is mandatory reading if you care about the U.S. Department of Education's Institute of Education Sciences (IES) noble but hopeless quest to learn "what works" in education. No time to blog it now, but since Ed Week charges, I'll put the full text in my extended section for you...

What Works vs. Whatever Works
Inside the No Child Left Behind law’s internal contradictions.
By Michael J. Petrilli

Are you still struggling to make heads or tails of the federal No Child Left Behind Act? Does the law remind you of Dr. Dolittle’s two-headed llama, the PushMePullyu? It should, because when it comes to the law’s “theory of action,” it’s heading in opposite directions at the same time. Specifically, the No Child Left Behind Act is the result of an uncomfortable truce between two groups of school reformers: the “what works” camp and the “whatever works” camp. The law is an amalgam of their ideas, and their ongoing competition will shape the contours of No Child Left Behind version 2.0.

First, let’s examine the what-works crowd. These reformers look across the education system and see its failings in terms of ignorance and ideology. They decry the pedagogical fads that sweep through our schools, bemoan educators’ resistance to scientifically proven reading-instruction methods, and abhor the quasi-religious nature of disliked educational “philosophies” such as constructivism and “multiple intelligences.” They seek to bring order to this chaos through the dispassionate eye of science. Using medicine as their model, they aim to employ rigorous research methods to determine what works, and then to use the force of law and regulation to ensure adoption of these methods throughout the land. After all, they say, we don’t allow doctors to wing it when they are practicing brain surgery; we expect them to use best practices in order to save lives.

The stamp of what-works advocates is clear throughout the No Child Left Behind legislation, but especially in two of its most controversial provisions: the Reading First program, and the “highly qualified teachers” mandate. The former requires schools to use funds for a narrowly defined type of reading instruction—namely, that which has been found by rigorous research to be effective. The U.S. Department of Education has dutifully implemented the program in this narrow, prescriptive way, leading to much gnashing of teeth and complaints of bullying. But for leaders of the what-works coalition, such as the former federal reading czar, G. Reid Lyon, anything else would amount to malpractice.

Then there’s the “highly qualified teachers” provision of the law, which demands that all teachers be able to demonstrate their subject-matter knowledge. This, too, is said to be based upon rigorous research, though even admirers of the mandate must admit that the evidence is somewhat flimsy. (Most studies linking subject-matter knowledge to teacher effectiveness have examined math or science at the secondary level; their applicability to elementary school, much less to subjects such as art, geography, or economics, is unknown.) But again, the what-works coalition borrows from the language of medicine, insisting that we wouldn’t allow doctors to practice if they didn’t have the relevant credentials. Surely we need teachers who are similarly well-qualified.


The whatever-works camp holds a very different worldview. These reformers look out across the education system and see its failings in terms of incentives, power, and politics. They decry the daily decisions made by school boards and district leaders that benefit adults instead of children (especially poor children). They abhor the red tape and bureaucratic inertia that keep educators from innovating. They don’t particularly care what happens inside the “black box” of classroom instruction; they just want children to be well-educated at the end of the day. They seek to right the system through the classic management model of “tight-loose”: Be tight about the results you expect, but loose as to the means. Put differently, the whatever-works camp combines accountability for student learning with flexibility around everything else. Using the entrepreneurial sector as its model, this camp aims to create a marketplace of schools, free to experiment, compete, and improve. After all, there’s a reason that America has the strongest economy in the world, they assert, and if we can empower educators with significant freedom (in return for getting results), they, too, will rise to the occasion.

The stamp of whatever-works advocates is also clear in the NCLB legislation. The very heart of the law is its accountability system, built into the Title I program, which was designed to create incentives for schools to boost student learning and close achievement gaps. Most important, the design of “adequate yearly progress,” with its disaggregation of test scores by racial groups, was meant to help local communities overcome the political barriers that keep resources and attention from flowing to needy children. In the spirit of “whatever works,” and in return for this increased accountability, the rules around the use of Title I funds were relaxed dramatically, and many more schools were given permission to use their federal dollars for “schoolwide” reforms. New “transferability” provisions were included in the law, too, allowing states and districts to move federal funds from one program to another. Just show us the results, Congress seemed to say, and we’ll leave you alone.

Is it any surprise, then, that educators feel whipsawed between competing demands? On the one hand, the federal government is saying to do whatever works to boost student learning, and on the other hand it’s saying to do things in a certain prescribed, preapproved way.

The result is frustration and anger. Imagine a poor, rural Title I school that is doing whatever works to get great results. In this case, it hires a former engineer from the local coal mine to teach 8th grade mathematics. She’s a natural, and her students’ test scores go through the roof. But because she didn’t major in math, she’s not considered “highly qualified.” How is that school’s principal going to feel when the feds come knocking, telling him to replace the teacher or risk being “out of compliance”? Or consider an elementary school whose reading results are soaring but which dares to use a reading program not on the state-approved Reading First list. Should the school’s principal be punished for insubordination, or celebrated for innovation and inventiveness?


What does all of this mean for the next version of the No Child Left Behind Act, due to be reauthorized in 2007? Surely both camps will try to consolidate their gains, further push their agenda, and avoid surrendering ground. The what-works camp will seek to expand its influence beyond reading to other areas, such as math. (This is especially true if the newly named National Mathematics Advisory Panel can identify rigorous evidence to support certain approaches to teaching math.) They will seek to close the loopholes in the law’s highly-qualified-teachers provision that allow veteran teachers to show their stuff without having a major or passing a test. And, of course, they will defend Reading First to the bitter end.

The whatever-works camp will attempt to expand the zone of flexibility. Beyond fighting to remove the highly-qualified-teachers mandate, and perhaps even Reading First, it will push for additional flexibility for local schools. Look for renewed calls for the “Straight A’s” plan, a 1990s creation that would have given states, districts, and schools near-complete discretion over how they spent their federal funds, so long as student learning rose. The whatever-works camp will try to expand on the transferability provision, perhaps allowing all of a state’s federal funds to flow through a simplified Title I formula. It will try to make it easier for schools to earn waivers from federal laws and regulations. And, perhaps most importantly, expect this camp to push for a larger role for charter schools, whose “accountability in return for flexibility” design epitomizes the whatever-works approach.

Sure enough, the big issues of accountability, adequate yearly progress, and funding will grab most of the headlines during reauthorization. But keep your eye on this more subtle battle between what works and whatever works. While the odds are that the basic outlines of the law won’t change much, the competition between these two camps is wide open. The direction in which this two-headed llama moves will have a big impact on the day-to-day operations of the nation’s schools, and the amount of freedom and flexibility educators enjoy.

Michael J. Petrilli was the U.S. Department of Education’s associate assistant deputy secretary for innovation and improvement during President Bush’s first term. He is the author, with Frederick M. Hess, of No Child Left Behind: A Primer (Peter Lang, 2006) and the vice president for national programs and policy at the Thomas B. Fordham Foundation, in Washington.

Click here for continuation of post (if any)

Friday, July 21, 2006

State Education Agencies: What do they do?

I am trying to figure out which candidate I will vote for for Mayor of DC. There is a strong feeling that the outgoing Mayor Anthony Williams has made a lot of progress on economic development, but his successor will have as Job One improving the city's education system. Given that this race is pretty much all about education, or should be, it's interesting that only the two frontrunners Fenty and Cropp have articulated concrete edupolicies.

What does this have to do with the title of my post? Click continue...

The next Mayor, if he/she does not take over the public schools outright (a risky proposition), he/she should strongthen the state education office to serve as counterweight to the DC Public Schools (DCPS). Clearly, there is momentum to move in this direction.

Either Mayor Fenty or Mayor Cropp, whichever it is, as a soon-to-be former member of the Council, can use that experience to get full legislative support behind a strong city government role in education that transcends the traditional public sector. We have a vibrant charter school movement that is educating a quarter of the District's children. We have a voucher program (foisted on us by out-of-state Republicans in collusion with Mayor Williams) that brings a host of private schools into the regulatory fold by putting them on the public payroll.

So what should the state role be?
1. Distribute federal funds. Mundane, but important.
2. Regulate traditional public and charter schools. NCLB compliance, etc. The State Education Office (SEO), can work with other sister city agencies like DCRA to more efficiently coordinate oversight.
3. Research and evaluation.
4. Create a centralized parent information center and centralize the school admissions process. This means have a single application and lottery, perhaps in two rounds, but coordinated with one application and a single notification process.

What else? Do any readers of this blog want to lend their ideas?

Click here for continuation of post (if any)

Separating the LEA from the SEA

Somebody is finally coming to their senses and trying to take state education functions away from the local education authority, the District of Columbia Public Schools. (WaPo: Board Might Be Stripped of Some Roles). The article mentions charges of incompetence on the part of DCPS, but that's not even the best reason. The issue is conflict of interest. Why should the DCPS Board be entrusted with managing federal programs like charter school assistance? It should be vested with the "state" government, which has jurisdiction over traditional public as well as charter and private school regulation.

Unfortunately, it had to be a Senate committee where someone has taken some action. What's wrong with this picture? Congressional overlords are doing the work that DC's legislature (the City Council) should have done long ago.

Having a stronger SEO is not a silver bullet. We still have to hold that office accountable. In a few months we will elect a new Mayor, and that is the person who must keep a close eye on the SEO with its new expanded powers and be accountable to the people who elect him or her.

Click here for continuation of post (if any)

Friday, July 14, 2006

Comparisons of public, private schools on NAEP test

I'm sure there will be a lot of interest in the latest comparison by NCES (researchers Henry Braun, Frank Jenkins, and Wendy Grigg) of public and private schools, including breakdowns by types of religious schools.

But the problem is that such comparisons are descriptive, and confound sorting effects with school quality. Also, it's NAEP, so you have test score levels in grades 4 and 8 only, not growth over a specified period using year-to-year performance changes. The test takers could be recently enrolled in public or private sector with scores that reflect where they've been for their whole school careers.

Continued on the flip...

But this is all I had to read:

Second, the data are obtained from an observational study rather than a randomized experiment, so the estimated effects should not be interpreted in terms of causal relationships. In particular, private schools are "schools of choice." Without further information, such as measures of prior achievement, there is no way to determine how patterns of self-selection may have affected the estimates presented. That is, the estimates of the average difference in school mean scores are confounded with average differences in the student populations, which are not fully captured by the selected student characteristics employed in this analysis.

Of course, "schools of choice" means that you are comparing kids as much as you are comparing schools. If you notice, the private school advantage shrinks when you start to control for student background characteristics, in some cases flipping the result to favor public schools. That should not be too suprising.

Click here for continuation of post (if any)

Thursday, July 13, 2006

Insight from rank-and-file teachers' union member

Nice post here from a teacher on union membership. "Know your contract" is good advice in many contexts, not just teaching.

Click here for continuation of post (if any)

Wednesday, July 12, 2006

Your edublog fix

A lot of teachers and students are slowing down their blogging over the summer, and I'm behind myself on reading and stuff, so it's nice to see these guys still rolling. Go read other good blogs like This Week In Education or VARC if you need a fix. Of course, Nathan at the very fine DC Education Blog is keeping us posted on the Master Facilities Plan (trust me, non-DC residents, it's a big deal -- DC Public Schools needs to unload millions of square footage of space because they lost 25% of their enrollment to charters) and the amendment to our "constitution", the Home Rule Fool Act to guarantee a free, high-quality education. I'm with Nathan and Rick Hess on this one. It's a foolish election year stunt by DC politicians.

Click here for continuation of post (if any)

Saturday, July 08, 2006

Teacher turnover in charter schools?

One of the big problems that plagues education is high turnover of teachers early in their career. The problem is especially acute in disadvantaged communities, which disproportionately hire and lose teachers. School administrators spend a lot of resources recruiting, screening, hiring, and training new teachers, so when a teacher finishes their stint and heads off for a wealthier district, more lucrative career, or stays home to raise kids, all that investment drains away also. And unlike the military, schools cannot just issue stop-loss orders to keep teachers in their ranks.

So is this problem more or less serious in charter schools? I'd like to see some research on the topic. So far, all we have is anecdotes, like this one about a DC charter school teacher whose school didn't make AYP and she decided to skip to South America. Bye, bye, Jenna. We'll miss you.

Hat tip to Education in Texas.

Click here for continuation of post (if any)

Friday, July 07, 2006

College choice of Texas's top high school grads

Marta Tienda and her colleagues at Princeton University have a paper (pdf), which just came out in a prestigious academic journal, describing the college choices of the top 10% of each high school in Texas. The topic is interesting, but I'm surprised they were allowed to publish such a mundane result, which is that high school grads are eager to get into selective colleges, although black and Hispanic students less so.

I guess this is important because of Texas' top 10% law, which tries to get around race-sensitive admission to state universities by capitalizing on the racial segregation by school and therefore giving school-based preferences.

Abstract of the paper below the fold...

Here's the abstract, with some edits for clarity. I put in some paragraph breaks, emphasis (all mine), and numbered the conclusions.

This paper addresses how institutional selectivity influences college preferences and enrollment decisions of Texas seniors in the presence of a putatively race-neutral admissions policy, the top 10% law.

We analyze a representative survey of Texas high school seniors as of spring, 2002, who were re-interviewed 1 year later to evaluate differences in selectivity of college preferences and enrollment decisions according to three criteria targeted by the new admissions law: high school type, class rank and minority group status.

[We reached] three major conclusions.

1. Texas seniors, and top decile graduates in particular, are highly responsive to institutional selectivity.

2. Graduates from feeder and resource-affluent high schools are more likely, whereas their counterparts who graduated from resource-poor, Longhorn or Century scholarship high schools are less likely, to choose selective institutions as their first preference. Both for first college preference and enrollment decisions, blacks and Hispanics are less likely than whites to opt for selective colleges.

3. Although disparities in selectivity of college preferences by high school type and minority group status persist among top decile graduates, these do not carry into actual matriculation, a result we attribute to the selection regime governing application and enrollment decisions.

Click here for continuation of post (if any)

Wednesday, July 05, 2006

More bad news for National Board

[Update: The National Council on Teacher Quality says in their latest newsletter that: "Except for some alert reporter at the Hampton Roads Daily Press, nary a mention of this study can be found anywhere." Gee, thanks, NCTQ, for plagiarizing my title and missing the fact that I scooped the HRDP by 5 days. I guess blogs don't count.]

The National Board for Professional Teaching Standards (NBPTS) can't catch a break. They put out an RFP in 2002 to let a thousand dozen (research) flowers bloom and they got a pile of, um, a pile of studies from North Carolina, mostly, that were riddled with methodological weaknesses, but consistent in their story: National Board certified teachers are probably a hair better than non-certified teachers, but otherwise indistinguishable.

Now another study (pdf) by researchers at one of the U.S. Department of Education's regional labs (SERVE) delivers a disappointing message: National Board certification had "no clear pattern of effects on student achievement based on whether the teacher was Board certified."
(continued below)

As with the previous studies, I'm not sure that this one is on such solid ground. Like the ones before it, the SERVE study compares test scores for Board-certified with non-Board certified teachers and relies on a set of easily obtained control variables to remove selection bias. (Actually, the report is candid about how even basic control variables are not so easily obtained).

Studies like this have to worry about selection bias because: (a) those who seek National Board certification are not a random sample of the population, so we're not sure if we're getting the selection/certification effect or the training/value added effect of the certification process; and (b) teachers are not randomly sorted into their districts, schools, and communities, so we don't know the direction of causality. Principals may explicitly assign different types of students to NBCTs. Their Board certification may change where they teach, to be in harder or easier situations. Controlling for prior test scores (or end of grade exam scores), race, and free lunch participation seems like a a small bandaid on a gushing, oozing sore of a methodological problem.

The analysis actually has two parts even though I focused on just the first one. The first part is the comparison of test scores between NBCTs and non-NBCTs. The second part tries to pick off the most and least effective comparison teachers and do more in-depth study of how and why they differ from NBCTs. But the authors admit that they got a small, self-selected sample of volunteers for the second part, so it's hard to take much away from it. In fact, the recommendations at the end of the report boil down to: do more research and be more careful about interpreting existing research.


Click here for continuation of post (if any)