Less ink, more think

Occasionally I am asked to give a lecture on how to draw good graphs. While I am always tempted to drone on interminably about abstract principles such as minimalism, balance, and consistency, I have discovered that it is much more fun to criticize bad graphs, and to show how they can be improved. But how to quire a truly bad graph? Easy! Just use the defaults in Microsoft Excel!

Here is an actual example of a graph drawn with those defaults. The data are fictitious, but the ugliness is breathtakingly real. It is sometimes said (unfairly) that engineers lack all sense of graphical design, but I think they must have hired specialists to create something so painfully wrong. vss2011workshop.010 But what specifically is wrong, and how can we make it better? Michelangelo once said “I saw the angel in the marble and carved until I set him free.”  So here too we will chip away at the obscuring excess, to reveal the beauty that Microsoft tried to hide.

First of all, what purpose is served by the heavy black rectangle that surrounds the graph? It serves two purposes: 1) to obscure useful information, and 2) to waste ink. Let’s remove it.

vss2011workshop.011

Better, but still bad. Next we note that the quantity being plotted is identified in three separate places: the vertical axis label, a title above the plot, and a key to the left. Is this really necessary? I think not. Lets get rid of two of them. Of course a key can be useful when several quantities are plotted together, but not when there is only one. Likewise labels above a plot have their uses, but should be avoided when they are redundant with other information, such as the axis label. We remove the key and the title. Apart from reducing clutter, this substantially increases the area available for the useful parts of the graph.

vss2011workshop.012

Now we ask the question: what purpose is served by the gray background? It serves two purposes: 1) to reduce the contrast and thus visibility of the data points, and 2) to waste ink. Get rid of it!

vss2011workshop.014

Aaah…so much more cheerful and relaxing to look at! But a few troubling questions remain. For example, what purpose is served by those shadows behind each data point? Do they indicate some exciting three dimensional aspect to the data? Of course not. But they do serve two purposes: 1) to render ambiguous the actual locations of the data points, and 2) to waste ink! Please people, can all just agree to never, never, use little shadows to suggest that our data are floating above the page? Thank you. The corrected graph is below. We have removed the shadows and also changed the diamonds to discs for the very important reasons that 1) they are simpler, and 2) I like them better.

vss2011workshop.015

Next we note that graphs are usually employed to show a pattern or trend. This pattern is not communicated well by a set of individual points floating out there, each an island, entire of itself. Only connect! A line drawn between the points aids enormously in conveying the visual sense of the data.

vss2011workshop.016

Next we correct an obvious (except to the Microsoft designers) flaw: the axis number labels running through the middle of the graph. We move them where they belong: to the axis, outside the graph.

vss2011workshop.017

Now we are getting somewhere. It almost looks ok. But we can do better. Gridlines can serve a purpose – for example, to let the reader easily judge approximate values – but there is never a reason for them to be dark and heavy, and to mask the useful information in the figure. Lighten up! In fact, the gridlines should generally be as light as possible, and still be visible. In this example, we make one gridline a bit darker than the others, to identify the y = 0 line.

vss2011workshop.018Now we see that the data really stand out. But we can do better still. What remains to distract the eye from the data? Well we could try removing the gridlines altogether, and then there is no need for the top and right borders of the frame.

vss2011workshop.019

Next we ask: what is the purpose of the bold font on the axis labels? Of course, it is to waste ink. Using a bold font for your labels is like writing your emails in all upper case. It is the digital equivalent of shouting. Don’t do it. Use your indoor voice.

vss2011workshop.020

And finally (yes, finally) we can reduce the line weight of the remaining axes. All we really need is enough weight to see them, and note their positions.

vss2011workshop021

Thus we arrive at our final graph. It is not particularly exciting, but the data are clear, the trends are evident, and there is little to distract the eye from the essential information. Clearly, not all graphs are this simple, and there are often reasonable justifications for more elaborate presentations. But it is often a good idea to start with the simplest possible presentation, and elaborate from there. 

We conclude with the motto of this presentation, and indeed of this entire blog:

“Less ink, more think.”

Advertisements

Size matters, but only as a ratio

The other day I received yet another breathless email demanding my urgent attention. This one trumpeted the outcome of a recent poll for “the generic Congressional ballot.” Oh, yeah, that ballot. I hope you have mailed yours in by now.

The poll outcome was illustrated with the graphic below.

In any case, the democratic advantage looked pretty impressive until my eye drifted over to the left hand edge, where I noticed that the bars began at 30%. Why start at 30%, rather than zero? TOO MAKE THE DIFFERENCE LOOK BIGGER! Forgive me for shouting but this is such an elementary error, or transparent subterfuge, that I can’t help but be exasperated.

The principle here is that you cannot appreciate the size of the difference between the two bars without knowing the absolute size of each bar. The difference between them is meaningful only as a fraction of the total. This is why a scale extending to zero is called a “ratio scale.”

But lest your eyes glaze over in anticipation of a boring lecture, let me illustrate the idea with a few more graphs. We take the same data shown above, and plot it several times, in each case changing only the starting point of the bars.

Which is “correct?” They all show the same data. The first one reproduces the original figure, starting at 30. But why not start at 40 (second graph) or even 42 (third graph)? That appears to show a gargantuan advantage for the blue party, but only because we can’t see the total lengths of the bars. The correct depiction is the last, starting at zero, which visually presents a much more accurate, and less impressive picture.

When should you use a ratio scale? The question has some depth to it, which we will not fathom today, but it is always the case that percentages should be plotted on a ratio scale.

Reference:

Email from Democratic Congressional Campaign Committee

Received: June 22, 2012

Pump up the volume!

One of the most egregiously deceptive practices in graphology is what we might call “dimension boosting.” Like the use of a performance drug in sports, it is an effort to gain un unfair advantage by playing outside the rules. Usually this crime consists of using the  width of a two-dimensional figure, such as a circle or a square, to depict a one-dimensional quantity. But as the width increases, the area, which is what we perceive, grows as the square of the width. With this device, a small difference can be made to look much larger. If the plotted quantities differ by only a factor of two, their areas will differ by a factor of four.

That is bad enough, but sometimes the criminal decides to do all the way, and throw in not one but two extra dimensions! In other words, they depict a one dimensional quantity with a three-dimensional object. Below is an example from a recent edition of the Sunday New York Times Magazine. It illustrates the decline of drinking among American teenagers over the last three decades.

Now we will perform a little test. Quickly, without looking at the axes, look at the two images at the beginning and end of the time interval and tell me by what factor drinking declined over that period. Got your answer? OK. Lets review. Well… in 1980 it looks like a 1.5 liter jug, while in 2010 they evidently had one shot glass (3 ounces?). You can fit about 17 shot glass servings in a 1.5 liter bottle. A 17x decline! Wow! Those kids sure have cut back!

But suspecting that todays teens are not quite so abstemious, and having been burned by criminal graphologists before, we examine the plot more carefully. First, we notice that even though the little bottles and glasses vary in not one, not two, but three dimensions, the axis on the left is a simple linear scale. Presumably the top of each vessel is the relevant aspect. Also, the axis is labeled in %. On that basis we realize that  the incidence of drinking has only declined from 70% of teens to 40%, a decline of only 1.75x. An impressive decline, but not 17x.

Now that this graph has been caught red-handed, and we have it in a holding cell while it calls its lawyer, we can investigate further. Notice that the vertical axis only goes down to 40%? That is another devious trick to exaggerate the magnitude of a difference. If the axis had extended all the way to zero, the difference between 1980 and 2010 would not seem quite so impressive. (that would provide what we call a “ratio scale,” for the technically inclined). And since we are plotting a fraction of teenagers, maybe it would be fair to extend that axis all the way from 0 to 100%, further reducing the apparent magnitude of the change.

And another thing: why are the bottoms of the bottles and glasses jumping all over the place? If the top is meant to indicate the value, it would only be fair to keep the bottom stationary.

And while it feels like piling on, what is going on with the horizontal position of the containers? Their positions seem to jump around a bit, and there are different numbers in each decade. Did they forget to make the measurement is certain years? Or is the artist just exploiting their “artistic license?”

This graph is an instance of what is often called an “infographic.”  An infographic is to a graph what an infomercial is to information. A bastard form in which information takes second place to entertainment or marketing. Look! Little bottles! What fun! One could imagine a form in which entertainment was provided, but truth was retained, but regrettably that is rarely to be seen.

In the printed version of the magazine, this graph is attributed to O.o.p.s. They should be ashamed. But the Times cannot escape the blame for this many-count indictment of graphical crime.

For completeness, we show a less entertaining but more accurate plot of the same data. It shows the full range from a fractions from zero to one, and does not introduce extraneous dimensions. The change in teenage behavior is significant, but not exaggerated by multidimensional trickery.

Reference:

New York Times

Well: The Kids Are More Than All Right

By TARA PARKER-POPE

Published: February 2, 2012

http://well.blogs.nytimes.com/2012/02/02/the-kids-are-more-than-all-right/

Attack of the little people

Where did they come from, the little people? Like a horde of replicants they have streamed forth to cover the world of infographics. No trendy depiction of any statistic related to humans is complete without the little people. Consider todays freshly populated example, from our favorite whipping boy, the New York Times.

The graphic is an attempt to put “into perspective” the numbers of people in poverty in the US. It does this by rounding up a bunch of little people, and penning them in various corrals that seem to have something to do with states or demographic groups. Hard to tell, since it is an expository jumble.

Let us ask a few questions of this graphic. First, the question that we ask of every such graphic: does the point leap out at you, in a flash of effortless cognition? Uh…lets see, half the people in poverty live in New York, and half in Texas? Fail!

Some more questions. If the orange little people are women and girls, why are they all wearing men’s business suits, albeit in a saucy feminine color? And do all the impoverished women and girls live in Texas? Rick Perry, are you aware of this? The state could at least provide more appropriate apparel for those in need. If you are a woman or girl, going to a job interview in an orange men’s business suit is not advisable, especially in Texas.

There seem to be a lot of impoverished white people (31.7 million), but amazingly, none of them live in Texas or New York. And if you think that is amazing…wait for it…none of them are men, boys, women, or girls. Maybe they are little people.

Ok, but here is where it really gets crazy. There are 16.4 million aged 17 or younger in poverty. But evidently none of them are girls or boys!

What is the lesson? The little people are no substitute for clarity of expression. The artist is to be commended for attempting to make the numbers more meaningful, but the exercise is doomed from the start. First of all, there is a fundamental difficulty in trying to carve up a total population (those in poverty) into a large number of overlapping sets. To be an accurate depiction, the corrals (technically, we call these Venn diagrams) should contain the correct number of little people, but so also should the intersections between two or more corrals (e.g., Asian and male and living Texas). Easier said than done (and it wasn’t that easy to say). Second, comparisons with state populations are problematic, since most americans have only a dim sense of the population of any state, even their own.

As is so often the case, traditional methods of data representation are perfectly adequate, and much clearer than the sad corrals of little people. Below is my quick draft of a bar chart of the same data. I have used different colors to group the different sorts of comparisons (gender, age, ethnicity), and as sop to the New York Times, included horizontal lines indicating populations of a few states (source http://quickfacts.census.gov/qfd/index.html).

I hope you will agree that though my chart may be conventional, it is clear, and allows the viewer to make the comparisons that the Times felt were important.

The lesson? Beware the invasion of the little people. They look cute, and you figure they are so small they can’t do any harm. But invite them into your graphic, and they can create havoc. Advanced lesson: Venn diagrams are tricky to depict when many categories are involved.

Sources:

New York Times

The Impoverished States of America

By TOM KUNTZ and BILL MARSH

Published: September 17, 2011

http://www.nytimes.com/2011/09/18/sunday-review/the-impoverished-states-of-america.html

State populations in my chart:  http://quickfacts.census.gov/qfd/index.html).

All that glitters is not Silver

“Love is blind.”

So begins a teasing article in the New York Times Sunday Magazine, by Nate Silver, the current wunderkind of popular statistics.  “Popular statistics,” now that I think about it, is almost the definition of an oxymoron, and it is to Nate’s credit that he has made it possible to utter such a phrase without puzzlement. The gist of the article is that in the dating game you are more likely to get lucky on a wednesday night than on any other night of the week. The article is accompanied by a massive “infographic” that occupies more than half of a page.

Debate has raged over the years about “decoration” of graphs, and while I am obviously  firmly in the minimalist camp, I am not a wild-eyed fundamentalist. A little furbelow here and there is harmless, provided that it does not obscure or distort the data.

Regrettably, young Nate has been kidnapped by the graphic artistes at the Times, who have never met a graph that could not be obscured or distorted. Witness below their artsy creation.

Note that there is an overall graph, for the days of the week, and within each day, a graph for hours of the evening. From a visual point of view, the most prominent effect is the trend over days. What exactly is plotted by this larger graph, for days of the week? A little scrutiny will reveal that it plots: nothing! The top of each bar is offset from the actual data, for any hour, by bizarrely random amounts. This is not decoration, it is desecration.

But suppose we extract the data, and plot them correctly. For days of the week, which is the primary focus of the article, it might be sensible to take the average “score” over the evening hours, and plot that. If we do so, we get the graph below.

Wow! No wonder they call it hump-day! Look at that massive effect! Except of course, that a glance at the scale reveals that the needle, so to speak, has barely budged. A more correct rendition of the data, showing the variation as a fraction of the total score (a ratio scale), is shown below.

Umm…never mind.  For all practical purposes, every night is the same. The main point of the article is, how shall we put it, nonsense.

And what about the numbers for the different hours of the evening? Even though they are hard to see, at least they are big effects, right? Of course not. Here is the average score for the various hours of the evening, plotted on a ratio scale.

I don’t want to be Miss Grundy, and I know even serious statistics wonks need a night out every once in a while, but even if “love is blind,” Nate really ought to reconsider the artsy types he hangs out with. Whichever night it was, he didn’t get lucky.

New York Times MAGAZINE

Wednesday Night Is All Right for Loving

By NATE SILVER

Published: June 3, 2011

Approaching the singles scene statistically.

http://www.nytimes.com/2011/06/05/magazine/nate-silver-wednesday-night-is-right-for-loving.html

The direction of time’s arrow

Once again, the target of our arrow of criticism is the estimable New York Times, and their estimable Charles M Blow, whose op-ed contributions are always interesting but almost equally often decorated with sadly defective graphics. In this example, we have a graph that is wrong in at least five ways. Can you spot them? Here is the graph.

The subject of the graph is the change in approval rating of President Obama following the killing of Osama bin Laden, for various selected groups. It is certainly possible to extract the information for any given group from the chart, especially because the artist kindly prints all the numbers, but in this regard it is little better than a table. And a graph should be more than a table, it should use your native perception of form to make a point.

The first error is the use of space. As is often the case with Mr Blow, the graph occupies a remarkable amount of vertical space, considering the modest data it contains. For this reason, you may have to expand the graph just to be able read its contents. As we will show, these data can be plotted in much less space, with an increase in clarity.

The second thing that is wrong with the chart is the selection of colors. Since before and after are depicted with color, we would like a strong contrast between the two. Instead we get a weak difference in brightness and saturation of two greens. Quick, tell me whether any subgroup showed a decrease in approval! I suspect you had to scrutinize each pair of bars, carefully ensuring that the darker one was shorter.

The third thing that is wrong is that the bar depicting “after” is about twice as wide as the “before” bar. Thus the area of the “after” bar is much larger, even if there were no change in approval. This is potentially confusing, ad certainly biased against the before figure.

The fourth thing that is wrong is that the bars are overlapping. This makes it harder to see the length of the “before” bar.

The fifth thing that is wrong is that the graph fails to exploit our native sense of how to depict an increase over time. By convention, in graphs time is always shown as proceeding from left to right. And positive quantities are always shown as increasing from bottom to top. The horizontal arrangement of the chart, and the overlapping of the before and after bars, fails to observe either of these conventions.

Another way to be absolutely sure that the viewer understands the direction of time is to actually show it as an arrow. This is especially appropriate when only two points in time are involved.

Correcting all of these errors, we produce the following chart.

While this chart should require no explanation, I will make a few comments on design. First, unlike the New York Times, I do not have an army of graphologists to tweak my product to perfection. This is a first draft, created in a couple of minutes, and could doubtless be improved. But it clearly shows that every group showed an increase, and the relative size of each increase. In each bar, time goes left to right, and approval increases from bottom to top, just as we expect. The arrows reinforce each trend with a strong graphic element, while the single green bar shows the absolute values of approval, and ties each arrow to its group name. We omit the actual numbers, but provide a 50% line for guidance.

My chart makes all the essential points, and does so in a way that is immediate and transparent. Mr. Blows chart has a certain graphical panache to it, and that is not a bad thing. But panache should never replace clarity.

Reference:

 New York Times
The Bin Laden Bounce
By CHARLES M. BLOW
Published: May 6, 2011
http://www.nytimes.com/2011/05/07/opinion/07blow.html

Pie is a continued fraction

The pie chart is a venerable and effective way of showing how some total, say the federal budget, is divided up into its constituent elements, each represented by a “slice” of appropriate angular size. Of course, often these slices will change over time, and it is tempting to portray that trend in a series of pie charts. To paraphrase Richard Nixon, “you could do that, but it would be wrong.”

It would be wrong because it fails to make trends in the data effortlessly and immediately evident to the viewer, because it fails to exploit the human visual systems automated mechanisms for perceiving trends. As we have noted with tiresome regularity, the eye is tuned to see contours, and to judge their orientation (look! the market is going up!) but not to quickly judge areas of complex shapes depicted in separate, unconnected parts of a figure.

Here is an example, taken from a New York Times article on how medical device companies bribe doctors to use their products, rather than their competitors, irrespective of the value to the patient. (This practice would be outlawed under ObamaCare, but perfectly ok under BoehnerCare). The series of pie charts attempts to show how one company (Biotronik) rapidly achieved near-complete market dominance for its pacemaker at one Nevada Hospital, after paying the hospitals cardiologists for “consulting.”

As always, I ask you to take a quick look at the chart, and see what pops out at you. I think the sad answer is: nothing. It requires careful scrutiny, with endless searching back and forth between pies, and between labels and slices of pies, until the presumed point is made: Biotronik went up, suddenly, and everyone else went down, suddenly, to almost nothing. And in fact, everyone else is primarily one company: Boston Scientific. So the point is equally well made by just showing the two companies.

That is what we have done in the following graph.

The trends in the two companies fortunes is perceived immediately and effortlessly. And because the graph shows percent market share, and Biotronik is almost at 100%, it is clear they have achieved a near monopoly. There is no need to plot “Other.”

The use of filling in this chart (coloring in the areas below each line) is a judgement call. While filling uses more ink, it can convey a contour better than a line. And since this graph is showing market share, it feels appropriate.

This graph could have been shown in color, but since there are only two categories, and since their trends are so clear, there is no need. Nonetheless we show an example here. As we note below, when many categories are involved, it is helpful to use color as a linking device.

One of the problems in attempting to show trends over a series of pie charts is that the categories within the several charts must be linked. To use the current graph as an example, we need to know which slice in each pie belongs to “Biotronik.” In the Times graphic, two strategies are used to link categories: shades of gray, and text labels.

The use of shades of gray to link the categories in the four pies is particularly weak. As any vision scientist will tell you, the human eye is very bad at remembering or identifying particular shades of gray. You have to remember, because you have to move your eye from one pie to the next. Colors, such as red or blue, suffer from no such weakness. We say they are perceived “categorically.”

Text is also a poor way of identifying the categories. It is unambiguous, to be sure, but requires reading, and moving the eyes back and forth from the label to the slice, all of which disrupts what should be an immediate, effortless “grokking” of the categories.

Consider also the additional clutter introduced by all of the labels. The words “Boston Scientific” are printed out in full three times, as are the words “Biotronik,” while the word “Other” gets repeated four times. The last is particularly ironic, since the category “Other” contributes almost nothing to the discussion.

It might be argued that the timeline, and paragraphs of text, that float above the graph provide historical context. In this case, not much context is really required. In any case, the graph that we have provided can be stretched to suit, or the text, which is really supplementary material, could be attached to the graph through arrows marking significant events in the chronology.

In the present example, there are really only two categories of interest. But often there will be more, in which case another method of plotting might be considered. This is the so-called “stacked,” or as I prefer, “accumulated” graph. In this variant, we add the values of each series on top of each other, so the data for each category is represented by the vertical extent that it occupies. Here is our current example, rendered in this way.

This chart has the advantage that, like a pie chart, it slices up the total into all of its constituents. The share of each category is immediately evident by the share of the vertical height it occupies. But in contrast to the series of pie charts, the trend is immediately and effortlessly evident, because the “slices” are connected.

There are problems with this form of presentation. One must choose the order in which to place the elements, top to bottom. There is no correct order, and this introduces an opportunity for bias or inadvertent misrepresentation. For example here, it seems natural to put the “Other” category at the bottom, but what about the other two? In the example above, placing Biotronik in the middle causes the upper edge of its share to climb precipitously, which matches its growth in share. But this has the unfortunate consequence of causing the lower edge of Boston Scientific’s share to also climb, a visual cue that is contrary to its declining share. Plotting in the reverse order, as shown below, merely reverses the problem.

So caution should be used when employing stacked graphs, and only use them when there are more than two significant categories, and the data cannot be better shown with simple line plots as in our first figure above.

The lesson here is: don’t use a series of pie charts to show a trend. It doesn’t work. A line graph is always better.

A subsidiary lesson is to use caution when using stacked graphs. The arbitrary order of stacking can convey different impressions.

Reference:

New York Times

Tipping the Odds for a Maker of Heart Implants By BARRY MEIER Published: April 2, 2011

http://www.nytimes.com/2011/04/03/health/03implant.html

%d bloggers like this: