.comment-link {margin-left:.6em;}

Ed Knows Policy

EKP -- a local (Washington, DC) and national blog about education policy, politics, and research.

Who is Ed Researcher?

Thursday, June 29, 2006

Bill Gates Gets Schooled

Jay Greene has what I would call a very useful article in Business Week about the Bill and Melinda Gates Foundation. It traces the experience of the Gates Foundation's cannonball into the edupool with its somewhat ill-conceived "smaller schools" initiative (Greene is suprisingly candid for a Gates grantee) [Update: It's not the same Jay Greene; this Business Week contributor is different from the Gates-funded researcher with the same name -- thanks to "rlf" for pointing this out] and walks us through how Gates squandered their first billion in education philanthropy. It makes me sick to think of how those resources could have been spent if they had had better experts and had not misread the social science. I have seen other corporate foundations and even some government agencies (but less so) move large piles of money in suprisingly naive and foolish ways because they become enamored by an "expert" who puts on a good show but is not really the authority that they seem to be. I won't mention names.

Business leaders have been trying to play the hero and solve education problems for a long time. Andrew Carnegie, John D. Rockefeller, H. Ross Perot, Louis Gerstner, Walter Annenberg, Bill Gates. Sometimes they see it as their extracurricular activity after their pile of wealth is getting too tall to see over and they want to leave their mark on society. Their choice of eduphilanthropy is a wonderful one, but they tend to range between two extreme categories. (1) hopelessly naive, charging like a bull in a china shop (2) careful, thoughtful, and successful in bringing about change.

So far, Bill Gates is the former category. (Click on continue for why).


The biggest problem with the BMGF (why personalize this?) is that they don't appreciate the value of real research. They pay lip service to research, but look at who and what they fund. I don't want to name names, but most of these are not serious social scientists seeking causal explanations, but "policy" researchers who are more advocates and aggregators of others' research than producers of knowledge themselves. Fine people, but not the ones you want to render judgment on the effectiveness and impact of multi-million dollar interventions.

The "evaluation" research (e.g. of the Gates Millenium Scholarship program), seems to be more outcomes tracking than any serious attempt to compare those outcomes to what would have happened to the awardees had the program not been available.

I have a great deal of hope that BMGF will evolve into a more questioning, research-savvy operation, but for the sake of the children who could benefit from their philanthropy, I hope it happens sooner rather than later.


Click here for continuation of post (if any)

Wednesday, June 28, 2006

Who signed the 100% document

I like Alexander Russo's breakdown of Who Signed, Who Didn't sign the 100% solution document fronted by Rod Paige, the one that everyone's talking about.


Click here for continuation of post (if any)

Extra, extra, get yer DC Edunews

Nathan at DC Education Blog is back from his week off, with local stories on DC education including principal firings reassignments and Council members who oppose mayoral control of the schools, unless of course they become Mayor themselves. Phew. We missed you.


Click here for continuation of post (if any)

Tuesday, June 27, 2006

Could raising standards hurt teacher quality?

This is like the paradox of class size reduction. When you reduce class size, you increase the demand for teachers, which can have the effect in the short term of forcing you to hire worse teachers out of necessity. The Dallas Morning News ponders the effect that increased high school course requirements in Texas might have on the demand for teachers who specialize in those course subjects.
Starting in 2007-08, all Texas students must take four years of high school math and science to graduate under the law, passed by the Legislature last month.

School districts have more than a year to get staff and equipment in place, but whether they'll be able to find qualified math and science teachers is a legitimate question when there's already a shortage in those areas.

"I think it'll be a real issue for all of us around the state," said Cathy Bryce, superintendent of Highland Park schools. "I really believe in requiring four years of science and math. But setting the goal and not paralleling that with some type of program to broaden the pipeline of certified teachers probably is a recipe for a problem."

The reporter, Kristen Holland, does a nice job digging facts on the hiring crunch and concludes thusly:
Where additional math and science teachers will come from is anyone's guess. Data compiled by several agencies, including the State Board for Educator Certification, indicate those teachers are hard to find.


Hard, indeed. I suspect the rural areas will be hit the hardest. I bet you $5 that Texas will be a proving ground for differential teacher pay aiming to get more math and science experts to enter and stay in the profession.

P.S. Go check out the latest carnival of education blogs hosted by the "why homeschool" blog or submit something to the next one. They asked me to plug them and provide a link. Since they linked here, I owe them one.
--Ed.


Click here for continuation of post (if any)

Monday, June 26, 2006

Psychoneuroeconometrics

This NBER working paper is a must-read, not so much for its new research evidence as for its novelty as an attempt by Nobel-prize winning economist James Heckman to show how well rounded and inquisitive he is. (In fairness, he IS well rounded and inquisitive and deserved a solo prize instead of sharing it with Daniel McFadden in 2000).



The thesis: The best way to improve the economy is to invest in very early intervention for disadvantaged children.

The evidence: Crude under-powered social experiments (Perry preschool and Abecedarian), animal studies (monkeys, rodents, and owls), arguments about how plastic the brain is at young ages, and fancy figures.

The conclusion: Early intervention is better than later intervention.

What's missing: Policy recommendations or even hints. "The issue of what the optimal strategies might be for maximizing the productivity of the future U.S. workforce is beyond the scope of the current discussion." Gee, thanks for nothing.

The funders: MacArther, Johnson & Johnson Pediatric Institute, Buffet Early Childhood Fund

Abstract after the jump...



ABSTRACT

A growing proportion of the U.S. workforce will have been raised in disadvantaged environments that are associated with relatively high proportions of individuals with diminished cognitive and social skills. A cross-disciplinary examination of research in economics, developmental psychology, and neurobiology reveals a striking convergence on a set of common principles that account for the potent effects of early environment on the capacity for human skill development. Central to these principles are the findings that early experiences have a uniquely powerful influence on the development of cognitive and social skills, as well as on brain architecture and neurochemistry; that both skill development and brain maturation are hierarchical processes in which higher level functions depend on, and build on, lower level functions; and that the capacity for change in the foundations of human skill development and neural circuitry is highest earlier in life and decreases over time. These findings lead to the conclusion that the most efficient strategy for strengthening the future workforce, both economically and neurobiologically, and for improving its quality of life is to invest in the environments of disadvantaged children during the early childhood years.


Click here for continuation of post (if any)

Sunday, June 25, 2006

NCLB-Inspired Tough Love in DC

A quarter of DC's teachers are not certified and the superintendant is laying down the law. 370 teachers are going to get the axe with another 450 at risk unless they get ther certificate soon. I hope they can find good, certified, replacements. HR director in a large city school district must be a thankless job.


Click here for continuation of post (if any)

Friday, June 23, 2006

Sherman Dorn makes you think about ed schools

Read his latest post called "Decline of a Profession." It's clever.


Click here for continuation of post (if any)

Wednesday, June 21, 2006

Do schools play the exemption game?

Do school administrators monkey with testing exemptions to make sure the dumb kids don't take the test? I'm not sure, but Cullen and Reback looked at some Texas data and seemed to have found some evidence that they do. I'm still reading their working paper (pdf), but you can read the abstract on the continuation page...




Abstract. We explore the extent to which schools manipulate the composition of students in the test-taking pool in order to maximize ratings under Texas’ accountability system in the 1990s. We first derive predictions from a static model of administrators’ incentives given the structure of the ratings criteria, and then test these predictions by comparing differential changes in exemption rates across student subgroups within campuses and across campuses and regimes. Our analyses uncover evidence of a moderate degree of strategic behavior, so that there is some tension between designing systems that account for heterogeneity in student populations and that are manipulation-free.


Click here for continuation of post (if any)

Tuesday, June 20, 2006

Evidence against small high schools

Here's the Ed Week story ($) covering Barbara Schneider, Adam Wysse, and Vanessa Keesler's analysis of the ELS-2002 data where they matched small high schools to bigger ones and found that size doesn't matter.

The paper was one of several presented at the May Brookings conference on the topic of wasting money on the effects of small schools and small classes.

The papers are not yet available, so we have to rely on the Ed Weak coverage -- (eyeroll). Debra Viadero is Ed Week's best research reporter, but it's not like being there. More on the flip...



There is plenty there to discuss and criticize, so why does Ed Weak not see that some "criticisms" are silly nitpicks? Are they just so desperate to include a quote from Valerie Lee?


But other experts attending the May 22-23 conference questioned the study methods. They noted, for instance, that the survey began in 10th grade, a year after students had entered high school and after many students had already dropped out.

“It’s very hard to talk about school effects when kids have already experienced half of high school at the beginning of the study,” said Valerie E. Lee, an education professor at the University of Michigan in Ann Arbor.


BFD. Hello, please, talk about the size and quality of the comparison group and the role of unobservable confounding variables.

The article mentions that they use "a technique pioneered in the 1970s by Harvard University statistician Donald Rubin" as a way say that the authors used propensity score matching. Oh, if a Harvard statistician in the 1970s pioneered it, then I as an Ed Weak reader will be impressed. If you take away the words propensity score, then you have the essence of what they did, which is just matching. ("Propensity score means they used a bunch of variables and combined them in an efficient way to create a singe matching index). And matching is only as good as the variables you have to match with.

To Ed Week's credit, at least they mention the sample size: 660 schools.

And they mention the control variables: "98 different characteristics. Those characteristics included the kinds of courses the students had taken and the extracurricular activities in which they participated, as well as traditional socioeconomic traits such as race and family-income levels."

Given the diversity of the schools and the confounding of urbanicity with size, I'm not convinced from the article that the paper isolated the true school size effect. Ed Week has a different idea of vetting research than most researchers do.

At least they got a quote from the Gates Foundation, who was in full spin mode.


Marie Groark, a spokeswoman for the Gates Foundation, which is underwriting improvement efforts in more than 1,600 high schools, did not disagree.

“We see really great results with our new small schools, but our existing schools have been slower to improve,” she said, referring to large existing high schools that have been restructured into small schools or learning communities.

“We do know size is not a panacea for improving high school outcomes,” Ms. Groark said. “It requires a focus on personalization, on ensuring kids won’t fall through the cracks, increasing expectations for all, and on improving instruction and curriculum so that the classroom is a lively, engaging, thought-provoking experience for kids.”


In other words, "forget resarch, we know that if it's done right that our half-baked policy idea is wonderful."

As most people now know, Gates splashed into the edupool cannonball style by putting a pile of money out there for an untested policy intervention and now they probably look pretty foolish. I hope they learn a lesson and try to sponsor demonstration programs with rigorous evaluation before they go to scale with their change-the-world ideas. Their money could be doing a lot of good in education, if only they invested it wisely.


Click here for continuation of post (if any)

Monday, June 19, 2006

DCPS may get out of chartering business

The District of Columbia Public Schools cannot and should not serve as a chartering agency. Having traditional public school systems approve and oversee their competitors (charter schools) was always a bad idea and hopefully it will end soon in DC.

As the WaPo article states, there is an issue about what to do with the existing 17 charter schools authorized by the Board of Education.

Update: Sara Mead at the Quick and the Ed defends DCPS's role as a charter authorizer. It's worth reading. She argues (admirably but not convincingly) that DCPS can use chartering to fill gaps in its own system. Her strongest argument would have been to go after the DC Public Charter School Board, which I think is no great shakes.


Click here for continuation of post (if any)

Thursday, June 15, 2006

Evidence that Teacher Incentives Work

North Carolina continues to be the nation's education research laboratory because the state (along with Florida) maintains such a comprehensive statewide database of test scores and student and teacher background. It's not as good as a randomized experiment, but this nice working paper (pdf) by Duke researchers provides interesting evidence on the possible effects of bonuses for teaching in shortage subject areas (math, science, spec ed) in high need schools.

The findings? The small, under-publicized bonus ($1,800) reduced attrition (my words).

Click "continue" for the full abstract in the authors' own words, or read it here.



The paper itself is worth a read. I've scanned it, but not studied it.

The real thorn is always whether the relationship is causal. In other words, were the attrition rates in these fields and schools going down over time anyway, for example, as a result of the attention that the problem was receiving and of economy-wide changes in the labor market during the study period?

Here's the abstract:

For a three-year time period beginning in 2001, North Carolina awarded an annual bonus of $1,800 to certified math, science and special education teachers working in high poverty or academically failing public secondary schools. Using longitudinal data on teachers, we estimate hazard models that identify the impact of this differential pay by comparing turnover patterns before and after the program’s implementation, across eligible and ineligible categories of teachers, and across eligible and barely-ineligible schools.

Results suggest that this bonus payment was sufficient to reduce mean turnover rates of the targeted teachers by 12%. Experienced teachers exhibited the strongest response to the program.

Finally, the effect of the program may have been at least partly undermined by the state’s failure to fully educate teachers regarding the eligibility criteria. Our estimates most likely underpredict the potential outcome of a program of permanent salary differentials operating under complete information.


Click here for continuation of post (if any)

DC Charter (KIPP) school + Public School Co-Locate

This is probably a good thing.

There are some implications for research on school choice. The unit of analysis is the educational market, not the school. One can no longer claim that the presence of charter schools does not affect public schools.


Click here for continuation of post (if any)

Wednesday, June 14, 2006

Exit, Voice, and Faith: Calling Horace Mann

Some Southern Baptists wanted the whole church and its members to band together and pull out of public schools, but the members said no.

I'm not well versed in the history of American education but this sounds similar to debates that went on in the 19th century over the Common School and immigrants.


Click here for continuation of post (if any)

National Board Certification: Educators agree to ignore research

I think the point of this Ed Week article ($) entitled, "NBPTS Upgrades Profession, Most Agree, Despite Test-Score Letdown" is that everyone loves National Board certification even if the research says that it has little power to predict student achievement gains (and even that NBC teachers have little influence outside the classroom).

Why do we even bother doing research when the education world just chooses to believe what they believe as an article of faith? Or maybe Ed Week got it all wrong.

Perhaps a better-written article would have said that many policy makers and educators like the idea of Board certification, even though National Board certification may not be such an informative credential...

A question that has dogged the NBPTS for years is whether the certification process merely identifies accomplished teachers or helps create them.


Well, gee, I guess it does neither.

Bess Keller got so many quotes from so many of the wrong people, wasting column space and her time and ours. Why didn't she get the state legislators and school board members who pony up extra money for Board certification and those who hire teachers to comment on whether the research affects their decision to use National Board certification for differentiating among teachers for promotion, pay, and other considerations?


Click here for continuation of post (if any)

Tuesday, June 13, 2006

Check out our OUR bald eagles, Andy

-- non Education Day continued --

Eduwonk doesn't accept comments, but if it did, I'd respond to this post on Andy's inspirational sighting of bald eagles in Virginia with an invitation. If you like seeing bald eagles nesting by a river, we got 'em for you right here, on the Anacostia River in Washington, DC.


Click here for continuation of post (if any)

DC Cabs: Not just meters vs. zones

Taking a quick break from Education to post this opinion I have. Taxicabs in DC use zones, not meters. It's a big issue here for those who ride or drive taxis and it's seemingly always on the verge of changing. (Click below to continue)


As a local resident who knows the zones, I benefit from them and game them. I walk across zone boundaries before catching a cab and get dropped off just shy of the zone boundaries. Still, meters seem fair, so I'm not strongly opposed.

The real debate over cab fares should be about rules for "extras." It currently costs $1.50 per extra passenger. This is ridiculous. It should not cost anything per passenger. It should only cost more for multiple dropoffs or extra bag handling. The cabdriver doesn't incur any additional cost per passenger, so why should the price differ? Why discourage cab sharing, which is efficient?

The rush hour surcharge makes somewhat more sense because traveling during rush hour imposes costs on everyone who uses the road. I wish they would narrow the window of what is considered rush hour, because you can't expect to affect behavior as much if you change the rates at 6:30 PM instead of 6:00, by which time the heavy traffic is already dying down in the city (it's all out there on the bridges and beltway).

Finally, the rule that allows a cabdriver to pick up additional passengers needs to be modified. It's just not fair to the paying passenger if the cab even stops or deviates half a block, let alone three blocks. I almost missed a flight because the driver (whom I arranged through a dispatcher and paid the dispatch fee) wanted to earn some extra money at 5 AM. And even if they don't actually pick up passengers, they drive differently when they're trolling for additional fares. These are just the wrong incentives for drivers. Give riders the incentives to double up, not drivers (see above).

So I hope the city looks beyond just meters vs. zones.


Click here for continuation of post (if any)

Monday, June 12, 2006

Jay Greene: Mythbuster?

Jay Greene has a piece in the conservative American Enterprise Institute's newsletter that purports to dispel myths about education. There isn't anything new in it. It looks designed just to get a rise out of people, by inspiring the right and inflaming the left in education debates. These are the main points:

* Schools are not underfunded
* Teachers are not underpaid
* Class size reduction is... maybe not cost effective
* Schools use kids' social problems as an excuse for their own poor performance
* Teacher qualifications like certification and experience are irrelevant
* Private schools are better than public schools because they are private, not because of resources
* School vouchers are effective

I'm not sure Greene is an expert on any of those except for the last two. What do you think?


Click here for continuation of post (if any)

Wednesday, June 07, 2006

Stinky Research or Stinky Programs?

Instead of blogging I've been ranting on other people's blogs, like D-ED Reckoning's posts on education research.

What set me off? The ridiculous over-use and misuse o rules of thumb about what effect size (percentage of a standard deviation) is meaningful.


Click here for continuation of post (if any)

Monday, June 05, 2006

Goldhaber breaks it down

Dueling studies on the impact of NBPTS? Not really.

The public always wants to see a fight, but Dan Goldhaber's guest-blogging appearance on Eduwonk where he compares his study with Emily Anthony of the impacts of National Board Certification (G&A) with a study by Sanders, Ashton, and Wright (SAW) on the same topic is sure to disappoint.

Dan's discourse is cautious and a bit wordy, so you may need my Cliffs Notes version in bullets along with some pictures of bikini-clad elementary teachers to stay awake:

1. There were some differences in data and methods
- While both studies used NC data, they had different samples (years, districts, and grades)
- G&A used fixed effects, SAW used random effects (HLM); neither is "wrong"
- G&A had more control variables (but did they use two years of pre-test?)

2. They reached similar conclusions, but put different spin on it
- Both studies found tiny differences in mean outcomes
- Goldhaber admits that "statistical significance" is only part of the story
- Only SAW stated the obvious, which is that NBPTS essentially fails if you think of their goal as being to identify the very best teachers

You know my take, which is that the claims made by NBPTS are probably hooey, but that their certificate may still be worth quite a bit if it improves even slightly over what we know now about predicting teacher quality, which is essentially nothing.


Click here for continuation of post (if any)

Friday, June 02, 2006

Evidence for Preschool Impacts

Here is a new working paper (pdf) by Berlinski, Galiani, and Gertler using Argentina data to estimate the impact of preschool (vs. no preschool) on performance in regular school.

The analysis is nonexperimental, but interesting.


Click here for continuation of post (if any)

The Mexicans are coming! The Mexicans are coming!

John Derbyshire = pendejo.


Click here for continuation of post (if any)

Thursday, June 01, 2006

More DC Local Stuff: School Board race heats up

This race should be interesting to watch.


WASHINGTON - A former D.C. Public Schools teacher-turned-advocate who is credited with leading the effort behind the recently approved multibillion-dollar schools modernization bill now is running for the D.C. Board of Education.


Marc Borbely, 33, filed his statement of candidacy with the D.C. Office of Campaign Finance on Wednesday to run for the District 3 seat being vacated by D.C. Council candidate Tommy Wells. Borbely, who is on the Advisory Neighborhood Commission, is the fourth person to announce their candidacy for the seat which represents Wards 5 and 6.

Academic researcher Stephane Baldi, charter school leader Lisa Raymond and parent Terrance McMichael have already filed, according to campaign documents.


Click here for continuation of post (if any)

I agree with JRB

You haven't lived in DC unless you've sat through the rants of local commentator Jonetta Rose Barras, frequent guest on the Kojo Naamdi Show on local politics. I actually agree with JRB on a lot of local issues, even though I can't stand actually reading or listening to her commentary (low signal-to-noise ratio, smart, but needs a good editor).

Her opinion piece in the Washington Examiner touches on a vital issue: the DC State Board of Education. Read and heed her wise words, edited and quoted on the continuation page. I'm doing you a huge favor by trimming the fat from her piece...


Will tens of thousands of young people ever be saved?
Maybe.

...if the District’s State Education Office is empowered.

The office, when originally conceived, was to be the big stick in the city’s so-called education reform movement. Legislation passed by the D.C. Council made it a twig: It counts students, creates rules to verify students’ residency, serves as a licensing agency for institutions of higher education and manages a feeding program.

Its counterparts in Maryland and Virginia are muscular operations — receiving and managing federal grants, defining academic standards and assuming control of poorly performing districts and schools.

In 2001, the SEO recommended assuming additional responsibilities comparable to its counterparts in other states: establishing licensing standards for instructional staff; establishing teacher certification requirements for all schools; and establishing rules for governing academic credits at all schools, among other things.

Years later, none of the city’s leaders has taken action.
The public has to start demanding action. The SEO, independent from DCPS, needs to control federal grant money. Right now the DCPS Office of Federal Grants oversees startup funding given to charter schools, the very organizations that compete with DCPS for enrollment. It's crazy.


Click here for continuation of post (if any)

Higher/Hire Quality Teachers

Kevin Carey has a long-winded but worthwhile post at the Quick and the Ed cogitating on the Deep Meaning of the Sanders et al. findings (pdf) on National Board certification(my very different cogitation here). I really like his writing and the post starts out very interesting, but then loses me as he sinks into despair that there is no way to predict teacher quality.


Click here for continuation of post (if any)