How to Arrive at the Best Health Policies

When the 48-year-old man from Oregon didn't have insurance, he felt he had no place to go but the emergency room. The man, who has diabetes, went to the emergency room often when he suffered from kidney stones. 'Emergency rooms, from what I understand, they can never turn you away,' he said. 'I mean, you don't have much options when you don't have insurance.'

Then, when he enrolled in the state of Oregon's Medicaid plan, that all changed. He started seeing doctors in their offices, and stayed away from the emergency room: 'I have had five appointments with my primary, one with the diabetic because they had me go to a diabetic educator, and then an appointment with my pharmacist, and then he does a phone-in thing with me every two weeks.'

His experience confirms common assumptions about how health care works. If we can just invest in preventive care, we can reduce the use of the emergency room and lower health care costs, the thinking goes. But it turns out that his experience wasn't typical. He was part of a giant social policy experiment that randomly assigned some eligible people to get Medicaid and others to remain uninsured. Over all, the study found that people who got insurance actually used the emergency room more than their uninsured peers.

Collecting data that can trump a powerful anecdote is the value of the randomized controlled trial, says Amy Finkelstein, an M.I.T. professor and a leader of the Oregon study, which has published a series of papers, most recently on emergency room use.

That's why this type of study - which randomly assigns some people to a new treatment and others to a placebo or an old approach - is the gold standard in evaluating the effectiveness of drugs: It can provide results that are both surprising and persuasive. But despite medical science's long history with such studies, when it comes to the best way to design health care delivery, the randomized evaluation is still an incredibly rare approach.

That may be starting to change.

Ms. Finkelstein is trying to help researchers find rigorous ways to evaluate new approaches to the health delivery system - questions about what sorts of services to offer different patients, financing methods for care, or other such questions. In a recent review of major medical journals, she found that more than 80 percent of published papers on medical interventions, like drugs and operations, involved randomized trials. Fewer than 20 percent of studies about how to actually practice medicine used randomized groups of patients or doctors.

Dr. Jeffrey Brenner runs a medical practice in Camden, N.J., that serves the extremely sick patients who tend to cost the health care system a lot of money. His approach, which focuses on extensive social services and one-on-one medical care for these costly patients, has received accolades and glowing press coverage. But Dr. Brenner said he can't really be sure his approach works without testing it, since he knows that many of the sickest patients in a given year would probably get better even without all that extra help. He has recently teamed with Ms. Finkelstein and her colleagues at M.I.T.'s J-PAL North America group to set up a system for randomizing possible patients and tracking their outcomes over time.

'We have these one-off stories of the miracle cure, and now we have to test it,' he said. 'The only way to know if our intervention is truly effective is to do a randomized trial.'

Randomized controlled trials are increasingly popular in designing international development programs overseas, and trials have been used in the United States to see if policies increase voter turnout or improve homelessness assistance. They have a long history in fields like employment training.

But there has been less appetite for using them in health care settings. Sometimes, the hesitation is because of ethical considerations: If we think we know a certain approach will be better, should we really withhold an intervention from some people?

Sometimes it's just practically and politically difficult to design trials, measure the results and wait all that time to know whether a given strategy achieves its goals. Dr. Brenner's trial, for example, will take more than two years to complete. If policy makers or hospital administrators can get approval for a sensible sounding change, they often prefer to adopt it right away.

'Doctors who insist on randomized controlled trials for whether they'll use drug X are a little lackadaisical on the more important sort of questions,' said Peter Orszag, a former budget director for President Obama who is now a Citigroup vice president. He has been advocating the use of more random evaluations in health delivery. Possibilities include studying how long women should stay in the hospital after cesarean sections, or exploring whether health insurers should charge higher co-payments for doctors who provide low-quality care.

The Oregon study shows the dangers of policy making by gut instinct. For years, advocates of expanded health insurance coverage have been fond of saying that insurance coverage is a surefire way to reduce health care spending, figuring that prevention is cheaper than treating medical emergencies. President Obama leaned heavily on such rhetoric in selling the Affordable Care Act.

The study also countered some conventional wisdom among Medicaid's critics. It turned out that the people who got coverage did have much better access to the care they needed than peers who relied on free clinics and other charity care. (The study had some limitations, too. It turned out to have too small a sample to answer questions about whether Medicaid led to improvements in key measures of health.)

Another randomized study surprised experts: one that looked at offering treatment to relieve suffering for people with advanced lung cancer. It turned out, as expected, that people who chose so-called palliative care had a better quality of life than those who got the usual care. But it also turned out that the patients who got palliative care lived longer.

Some people in health care are starting to become interested in the approach. The wide use of electronic health records is making it easier to track patients without a lot of additional record-keeping. New payment strategies from Medicare and private insurance companies are increasing pressure on hospitals to lower costs, and that may create incentives for them to study the most efficient ways to care for their patients.

Dr. Jeremy Boal, the medical director at the Mount Sinai Medical Center, said his organization was about to start a series of trials to see if its ideas for delivering better and cheaper care would really work.

'Everyone is going to be realizing that the tools we've used in the past on the health care delivery side aren't really up to the task,' he said. 'We're going to have to find new approaches.'