I recently wrote a Twitter thread (5/12/20) about another New York Times graphic feature that was a good idea, strangely executed.
The good idea, which appears on the Times‘ “Coronavirus in the US” page, is to sort states in terms of how well they’re controlling the coronavirus outbreak, using graphs of the daily count of new cases in each state. This seems like a good choice of metric and a useful thing to keep track of, especially given how fragmented the US response has been.
As for the strange execution—well, in the category of “where new cases are increasing,” the Times (as of May 16) includes both Virginia…
…and Louisiana:
Under “where new cases are mostly the same,” the Times includes both Arizona…
…and Montana:
And labeled as “where new cases are decreasing” are both New Hampshire…
…and Vermont:
So in every category, the Times lumps together states that are doing an outstanding job controlling new Covid-19 cases, and other states that show little or no sign of bringing the outbreak under control. Often it’s hard to discern why the Times puts a state in one category rather than another, but the bigger problem is conceptual: If you place a state that is down just slightly from a peak in a more favorable category than one that has brought new cases down to zero, because in the former cases are “decreasing” while in the latter they’re “mostly the same,” then you aren’t helping to hold state governments responsible; you’re actually obscuring which officials have implemented an effective anti-coronavirus strategy.
To demonstrate what seems to me a more useful approach to sorting states’ coronavirus records, I’ve made charts of each state (and selected territories) and graded them according to where they are between a peak of infection and bringing new cases down to zero, and whether the direction of the number of daily new cases is up, down or more or less plateaued.
I’ve used the site 91-DIVOC to make the charts; I used daily new cases averaged over seven days to reduce noise. The charts are laid out in a linear rather than a logarithmic scale, to make changes more apparent; they are scaled to the state’s own peak.
A couple of caveats: These grades do not take into account how high each state’s peak was, either in absolute or per capita terms; factoring that in would certainly change the rankings of some states. And there is some subjectivity involved in sorting this way; any two people might come up with slightly different classifications. I’m confident, though, that this ordering gives a more coherent picture than the New York Times does of which states are having more or less success at stopping the spread of the novel coronavirus.
A
These states and territories have been largely successful at controlling the coronavirus—bringing new cases from their peak down to zero, or close to it. They each have natural advantages: Either relatively low population density or some degree of geographic isolation. But other places with similar advantages did not do nearly so well.
Alaska
Hawaii
Guam
Montana
Vermont
Virgin Islands
B
The states in this group have brought new cases most of the way down from their peak, but have not yet brought them to zero. Both these states had particularly bad outbreaks.
New Jersey
New York
B-
These states have similarly brought new cases well down from their peak, but seem to have plateaued short of bringing them down to zero.
Idaho
Louisiana
Michigan
C
These states are about halfway between their peaks and zero, and their numbers are headed downwards.
Colorado
Kansas
Massachusetts
Missouri
Nebraska
Pennsylvania
Rhode Island
C-
These states are also about midway between peak and elimination, but have stopped making downward progress.
Connecticut
Florida
Nevada
Ohio
West Virginia
Washington State
Wyoming
D+
Close to their peak, cases in these states (and one territory) are just beginning to head down.
District of Columbia
Iowa
Indiana
Maryland
Maine
Minnesota
Mississippi
New Mexico
New Hampshire
Oregon
Puerto Rico
Wisconsin
D
This group of states has seen their new cases decline considerably from their peak, but they are now heading back up.
Arkansas
Kentucky
Tennessee
D-
The number of new cases in these states has plateaued near their peak.
Alabama
Delaware
Georgia
Illinois
North Dakota
Oklahoma
South Carolina
Utah
F
These states have shown little sign of controlling their coronavirus outbreaks, with new cases continuing to rise. Unfortunately, they include the most populous and second-most populous states in the union.
Arizona
California
North Carolina
Texas
Virginia
I want to reiterate that these grades reflect only the trajectory of each state’s outbreak, and not the absolute magnitude of their peaks. It does matter that on California’s worst day so far, it had 70 new cases per million, and on New York’s worst day it had 588 per million. But in terms of guiding the national coronavirus outbreak to a successful conclusion—one that does not involve the virus spreading disastrously through the entire population—the direction of each state’s infection rate is critical. We need to learn from the states that have managed to control the coronavirus, and see what lessons can be applied where it is still out of control.
I appreciate this analysis, but this methodology seems flawed as well. For example, Illinois has increased testing significantly in the past few weeks so it looks like their graph is has increased late and plateaued near their peak, but in reality it is a reflection of many more tests.
Yes! I came down here to suggest this. For example, NC has expanded tests, which no doubt contributes to its greatest single day increase of new cases, 853, on Saturday May 16. The approach on rt.live, by contrast, estimates the actual spread.
South Dakota is missing from this listing.
This still doesn’t make sense. The reason some states didn’t peak is those states had more controls to start with and are behind the others in terms of peaking. They had a slower climb and there was little or no testing in the beginning and that’s why they didn’t peak earlier. The increased testing adds more cases ( understandably). None of these graphs are accurate in comparison because testing was not the same in each state in terms of availability. Also, the graphs should compare hospitalized patients + deaths, and not compare mild cases and asymptomatic as the same as the worst cases. Without a vaccine, the hope is that people do get it as a mild flu and create their own antibodies. Why some get it bad and others don’t is a real question to ponder. We can’t continue to stay in our homes in perpetuity.
I would love to see these reformated for hospitalizations and deaths per million population
Your charges would be better if you used a standard metric like cases per 1000 residents, and if you were consistent in the scale for the y axis.
Dividing the case counts by a constant number such as the population would not change the shape of the graphs at all. That’s not really a problem with this methodology. A concern that’s not raised is that testing increases appear on the graph as disease incidence when that of course isn’t accurate.
Yes, but if you combine it with being consistent in the scale of the Y axis it would be clearer who has a bad outbreak and who doesn’t.
might well be that many places list deaths on the days they occur, so the five most recent days, even the ten most recent days don’t show all the deaths, and yesterday or the day before show almost none. As death notices come in they are added, so prior days grow. A better metric is deaths on days they are reported. It skews heavier mid week, as many people are off on the weekend but if you compare similar days of the week you get a feel for what’s up. Also hospital admissions is a good place to look. For my state I look at positive tests, percent of positive tests, hospital admissions, and deaths, and I look for trends. It’s not exact, and there is no exact number. Good luck.
I don’t think it’s as cut and dried as which states have implemented the best plan. There are, as you said, variations in population. Also, in many states, like my state of Pennsylvania, governors have taken swift action only to be thwarted by their legislatures egging on angry mobs. When you have a trifecta, like Cuomo does in NY, it’s much easier to get things done.
This is pointless. These curves show nothing – you can’t tell anything about per capita numbers. To get a lousy grade because you successfully flattened the curve, and are therefore still going up, but with a low per capita number (that’s what flattening the curve means, right? You’ll go up longer), and a good grade because you brought a horrific outbreak down just to pretty bad, is insane. The NYT charts aren’t great, but neither are these.
My bigger complaint about the NYT is that they used to have info available in the form of “deaths per 100,000 population”, and they have changed their format, so you can’t compare last week’s numbers to this week’s.
It’s a fairer assessment than the Times, but neglects a couple of key points: 1) The primary aim in most places was not elimination or even containment, but control and “flattening the peak.” Making sure resources weren’t outstripped and holding down growth of cases; 2) almost no state is uniform. Different demographic & urbanization conditions almost guaranteed a heterogeneous response. In CA, for example, Bay Area counties had early peaks but also early control. Rural areas, like Kings an Imperial counties were hit much later & control has been harder.
Exactly what Stuart said, 1. why did the goalpost get moved from ‘flattening the curve’ i.e. not overwhelming the hospital systems to somehow preventing the spread.
No epidemiologist has ever said we could totally eliminate the virus or prevent the spread.
2. Each county in a state should have been treated differently. Start with total lockdown. Two weeks in evaluate by geographic area – any area with less than 20% covid capacity use should have one low risk business type opened up.
Instead, we had huge swaths of the country with less than 10% hospital utilization, staff layoffs, hours cut, etc. Such a waste of resources
A good article, but infections and new cases are not the problem. Death rate of infections is the metric we should be concerned with, especially those outside the known risk pool. In these graphics, a highly populated state with no lock-down, with a large percentage of young, healthy people, with a lot of infections would score poorly, but would in fact, not be a true representation of situation there. Moreover, a State my have an early detection plan, with a lot of testing, to identify and treat infected people – accepting higher infections, but dealing with the cases properly, reducing death rates. That State would also score poorly here.
Death rate of infections is the only real way to score the effectiveness of a State’s recovery plan.
You need to take into consideration those people that have recovered.
Also maybe curves have flattened and some states are still at the top of the flat part.
Jim, Have you taken into account the degree to which the states have ramped up testing? Illinois has added many drive-up testing sites and is testing lots more people. That is bound to turn up lots more cases. Because a testing site opened in Waukegan two weeks ago, Waukegan has gone from1025 cases on April 29 to 1964 cases on May 16. The infection rate for Latinos is around 44%.
[Here is what my friend wrote me, somewhat angrily, on Facebook. I feel chided…but I have no idea if he is correct. Anyone care to tell me if he is right or wrong?]
This guy clearly doesn’t understand the graphs he has created, and what they mean, and his attempt to create a grading system fails in every respect actually obscuring information rather than clarifying it. You would have probably been better served linking directly to the visualization tool which he cited as the source for his graphs. I did find that tool useful in terms of being able to visualize the status of the epidemic on a global, country and state basis. For me source data is always more valuable than data wrapped in a narrative as I suspect it is for most people. We are six months out from any chance, in most cases for we the people to “hold responsible” people in government for their Covid 19 decisions and responses be they heroic, good, bad,
The worst place in the country with regard to catching and dying from Covid 19 is within 100 miles of NYC. Period. simple fact. Not even a close call. By any measure. The placement of New York and New Jersey as the sole B grade states should tip anybody off that the grading scale was seriously flawed.
All Politics – NO SCIENCE. Death Rate per Capita 100K in NY = 150 which is 10X normal Influenza. This is 2000% higher than the Death Rate in California. Say it again 20X higher than California. Where would you rather have your Parents or Grandparents living in NY or CA? NJ or TX? The only reason the curve may have dropped in your A grade states is because the Govornors and Mayors may have already killed off the Comorbid and Older most susceptible people.
I think it would be more accurate to look at how far a state is from the peak. This outbreak didn’t start in each state at the same time.
Brilliant Jim. A major contribution to the conversation.
This is absolute crap. New Jersey gets a B and has the same number of cases as Texas who gets an F. Even though Texas has more than 3 times the population.
I hate it when people try to shew math to present a false narrative.
Jim,
I think your aim of trying to push back on the NYT data was good, but, as you can see, there are bunch of problems comparing/grading the states.
These include:
MN is doing terrible because we have exploded the number of tests we are doing
Unfortunately, President Donny is right, fewer tests, fewer cases. More tests, more cases.
So, this data would be great IF it were scaled both by infections per 10,000 people, then crossed with a different table indicating number of people tested per 10,000.
Those two could then be used to grade the curve, perhaps?
Not sure.
Thank you! I had been seriously frustrated by the NYTimes’ categorization of states. Thank you SO MUCH. (And, yay for psci phds!)
How do you grade South Dakota?
It seems like California has acted to stem gross involvement but must, necessarily, manage an incidental 30 new cases where New York has quit moving on hundreds of new cases per day, yet you rate their success equivalently. I want Wisconsin to emulate California but definitely want nothing more to do with New York’s approach. You are being seduced by the shape of curves you constructed.
No, he’s saying that NY is better, drastically better! Sure they’re doing much better than they were, but there was more room for improvement there. I think “where would you want your elderly parents to live” is a good measure of who is doing well and who isn’t.
Nice try correcting some of the NYT problems but you created at least on new one. Oregon is ranked as a D+ but has basically always had low numbers. By comparing Oregon to Oregon’s low peak, Oregon looks bad but that is simply not the case.
The absolute number of cases might make sense when all other things are equal but they most definitely are not and therefore this metric is very misleading except for the trivial point that fewer cases are better than more. A downward trend in cases may or may not be evidence of an effective strategy for dealing with an outbreak or it may not. A downward trend in deaths or hospitalizations is far more meaningful and is a less ambiguous metric given the nature of this virus and its potential impact on those infected. But even here any analysis needs to also look at excess deaths because of problems attributing death to the virus as opposed to pre-existing conditions.
Hopefully, this experience will lead to significant changes in the way we collect and document public health risks.
It has been reported that some states have aggregated positive, current infections, and positive antibody tests (see ). Unfortunately, in many of those data sets, it isn’t always possible to disaggregate those numbers. Until there is a truly common set of data types the numbers are not comparable.
I live in GA. I didn’t expect a good score, but I wasn’t expecting a D-. Comparing GA to other states in the D- category, our graph looks much better. How is Wisconsin better than GA? I’m looking at trend lines and over all decrease in daily cases. GA is 27.7% down from its peak currently and the trend line is pointing down. Is it possible there was a little bias in that ranking due to GA being the first to open?
I like your analysis, but I think you leave one important criteria unaccounted for – you only account for trend and not the raw magnitude of the numbers. I live in Oregon and I feel Governor Brown has done a good job in calling for early shutdown of non- essential activities.
Our metro areas are fairly densely populated; our new infection rate is about 80 people per day; yet we get lumped together with states with a D+ grade that have 800 new infections per day. It’s granted that a decreasing trend is better than a steady or rising trajectory, but I think it can be argued that a low-level sustained rate is much better than a high-level sustained rate.
Oregon’s overall infection and death rate is much lower than states with comparable population.
Being a resident here, I’m truly grateful for Governor Brown’s early actions.