Monday, December 11, 2017

Model Decay, Causal Mechanisms and Townhallophobia

[Now with title.]

I know I've been going to the well a bit too often recently in terms of old posts. I'll admit there's an element of laziness here, but I also think it's useful and even important to review old arguments in terms of new developments.

This also gives us a chance to dig a little deeper into pertinent related topics. For instance, despite living in a data obsessed age when every journalists feels the need to pepper reporting with graphs and statistics and models and algorithms (and frequently models mislabeled as algorithms), there are still a number of fundamental analytic concepts that generally evade even the more sophisticated data journalists.

A notable example is model decay. With the exception of the physical sciences, even the best models tend to lose explanatory and predictive power over time (though in the life sciences, the changes can be very slow). This is particularly true in the social sciences where people quickly adapt to conditions, sometimes specifically in response to awareness of the very models in question.

Model decay can happen slowly or suddenly and there's no simple procedure for seeing how well things are standing, but there are red flags you can look out for.

Though I may get some pushback on this, I think most good modelers and researchers at least implicitly have a set of plausible mechanisms in mind when they propose a causal relationship. (If a researcher can't think of any plausible mechanisms that might cause the relationship he or she is seeing, that's probably cause for concern.) For example, let's say a study found that people who purchased a gym membership and did not make serious use of it saw their body fat tend to increase. The researcher might hypothesize that the subjects were rationalizing bad dietary choices based on the belief that they would work off the pounds.

Though these plausible mechanisms are just good guesses, it is a good idea to keep them in mind when considering model decay. If we have reason to believe that the mechanisms for a particular model no longer function in the same way, we also have reason to believe that the model may no longer be viable. Going back to our previous example, imagine that a certain unnamed financial institution buys a health club chain and starts signing up customers without their knowledge. We definitely have reason to believe that the relationship between unused gym memberships and body fat will break down.

With the rise of data-based political reporting, we've also seen an uptick in simplistic A-then-B models. Writers will often explicitly acknowledge that conditions are completely different then turn around and insist that the same relationships that held before hold now.

Consider the incumbency advantage. There's obviously a relationship here, but what are the plausible mechanisms behind it? We could always reach out to political scientists or dig into relevant studies and polling data, but we can also make some pretty good common sense guesses. It's entirely reasonable to suppose that the advantage is at least partially dependent on incumbents being responsive to their constituents' wants and needs and maintaining a strong connection with the people back home through regular contact via multiple channels.

When politicians take stands that are unpopular with their constituents and then abruptly and publicly cut off these channels of communication, there is reason to believe that assumptions about the incumbency advantage may no longer hold.

Here's what we were saying about this ten months ago.

Though, to be perfectly fair, Tennessee has always been a hotbed of leftist radicals


We have all heard the statistics about how difficult it is for a Congressional representative to lose his or her job. This is partially because of things like gerrymandering and spigots of campaign cash, but it also reflects a process that does a pretty good job allowing a reasonably competent and dedicated legislator to keep the constituents fairly happy in his or her district. A big part of that process is the maintaining of good relationships and lines of communication with voters and communities. Many political career has ended when voters felt someone had "lost touch with the people back home."

In this context, stories like the following from Talking Points Memo's Allegra Kirkland take on a special significance.
Constituents requesting that Rep. Jimmy Duncan Jr. (R-TN) hold a town hall on repealing the Affordable Care Act aren't being met with a polite brushoff from staffers anymore. Instead, Duncan's office has started sending out a form letter telling them point-blank that he has no intention to hold any town hall meetings.

“I am not going to hold town hall meetings in this atmosphere, because they would very quickly turn into shouting opportunities for extremists, kooks and radicals,” the letter read, according to a copy obtained by the Maryville Daily Times. “Also, I do not intend to give more publicity to those on the far left who have so much hatred, anger and frustration in them.”

In the first weeks of the 115th Congress, elected officials dropping by their home districts were surprised to find town halls packed to the rafters with concerned constituents. Caught off guard and on camera, lawmakers were asked to defend President Donald Trump’s immigration policies and provide a timeline on repealing and replacing the Affordable Care Act.

Now, many of them are skipping out on these events entirely. Some have said large meetings are an ineffective format for addressing individual concerns. Many others have, like the President himself, dismissed those questioning their agenda as “paid protesters” or radical activists who could pose a physical threat.

Voters turning out to town halls are pushing back hard on this characterization, arguing that they represent varied ideological backgrounds and have diverse issues to raise. Constituents unable to meet with their elected officials over the weekend told TPM that they’re not attending town hall events to make trouble. Instead, they say they want accountability from the people they pay to represent them.

Kim Mattoch, a mother of three and event planner, told TPM that she tried to go to a Saturday town hall in Roseville, California with GOP Rep. Tom McClintock but couldn’t make it in. The 200-seat theater hosting the event was quickly filled to capacity, leaving hundreds waiting outside.

“I’m a constituent of McClintock and a registered Republican in a very Republican district—though I don’t really align very well these days with the Republican Party,” Mattoch said in a Monday phone call. “So I wanted to go to the town hall because I legitimately had questions for the congressman.”

Mattoch said the protesters waiting outside had a wide range of “legitimate concerns.” She personally hoped to ask her representative about how the GOP was progressing on repealing and replacing the ACA and why House Republicans last week voted to kill a ruling aimed at preventing coal mining debris from ending up in waterways.

Yet McClintock told the Los Angeles Times that he thought an “anarchist element” was present in the crowd outside his event, and said he was escorted to his car by police because he’d been told the atmosphere was “deteriorating.”

Ramon Fliek, who attended the McClintock event with his wife, told TPM on Monday that police “were kind enough to block the whole road” to make space for the overflow crowd, and that he overheard protesters thanking law enforcement for “doing their jobs.”

“If you look at the videos from the event, you can’t get any notion that it was aggressive,” he said. “There was an older woman with a poodle that ran after him and it’s like, okay, the older lady with the poodle is not going to threaten you. I understand that he might want to give that impression, but it was very pleasant.”
Admittedly, it is a long time until midterms, but possibly not long enough to repair this kind of damage.

Saturday, December 9, 2017

Consequences of budget deficits

This is Joseph

Paul Krugman via Mark Thoma:
For budget deficits are going to soar thanks to Republican legislation — probably by even more than the official scorekeepers say, because the legislation creates so many new loopholes. And offsetting those deficits will require going after the true big-ticket programs, namely Medicare and Social Security.
Oh, they’ll find euphemisms to describe what they’re doing, talking solemnly about the need for “entitlement reform” as an act of fiscal responsibility — while their huge budget-busting tax cut for the rich gets shoved down the memory hole. But whatever words they use to cloak the reality of the situation, Republicans have given their donors what they wanted — and now they’re coming for your benefits.
I am actually a big opponent of "deficits don't matter", even when spouted by the left.  I totally understand the need to invest and that low interest rates that right now is a fine time to run deficits if there is a good reason.  But, in the long run, deficits either need to be paid or are going to act in an inflationary way.  What is saving us, for now, is that the rich are so rich that there are a shortage of secure investment opportunities so rates are low. 

But, in the long run, to spend is to tax.  All that reducing taxes now is doing is moving this taxing off into the future.  Or shifting the tax burden around, by future defaults on bonds (for example), which might target specific segments of the population. 

What I find hard to swallow is how the deficit language is swallowed by the media as if they are not able to follow the basics of the structure of the arguments.  If we use the (imperfect) household analogy, we are saying that we should reduce our income (taxes) and then discover that we need to reduce expenses, in world where there is plenty of income available. 

Now you could make a first principles argument for why social security and medicare are bad ideas (or perhaps defense, the other large item).  But that should be done in parallel with the cuts in income and not afterwards because of the cuts.  If a family decides that they no longer need a car, and thus can work less, the correct order is deciding to ditch the car and then deciding to cut back on work.  Not waiting for a crisis. 

I really wish that the reporting on this issue was more explicit. 

Friday, December 8, 2017

Adam ruins lab nice

Given the number of readers with relevant experience in this field, perhaps someone would like to weigh in on this.



Thursday, December 7, 2017

Agon

I'm a big fan of abstract strategy games, particularly those played on hexagonally-tiled boards (such as the ones you can purchase here). One of my favorites is Agon. It's a challenging but easy to learn game that's deserves a much bigger following. Here is the write-up I did of the rules a few years ago.

Agon may be the oldest abstract strategy game played on a 6 by 6 by 6 hexagonally tiled board, first appearing as early as the late Eighteenth Century in France. The game reached it greatest popularity a hundred years later when the Victorians embraced it for its combination of simple moves and complex strategy.

The Pieces: Each player has one queen and six pawns a.k.a. guards placed in the pattern indicated below
Agon Start

The Objective: To place your queen in the center hexagon and surround her with all six of her guards.(below)
win


concentric

Moves: Think of the Agon board as a series of concentric circles (see above). Pieces can move one space at a time either in the same ring or the ring closer to the center. In the figure on the left, a piece on a hexagon marked 4 could move to an adjacent hexagon marked either 4 or 3. Only the queen is allowed to move into the center hexagon. The figure below shows possible moves.

moves

Capturing: A piece is captured when there are two enemy pieces on either side of it. The player with the captured piece must use his or her next move to place the captured piece on the outside hexagon.
captures
If the captured piece is a guard, the player whose piece was captured can choose where on the outer hex to place the piece. If the piece is a queen, the player who made the capture decides where the queen should go.

If more than one piece is captured in one turn, the player whose pieces were captured must move them one turn at a time.

If a player surrounds the center hexagon with guards without getting the queen into position, that player forfeits the game.

Dilation and contraction.

Our standard narrative is deeply invested in the idea that progress is ever accelerating and that we are always on the cusp of an unimaginable leap forward. One of the favorite devices used to sustain this belief is dilation/contraction. The rate of technological change and its impact on society in the past is underestimated (particularly when describing the periods of the late 19th/early 20th centuries and the postwar era) while the current rate of change is overestimated, often wildly so when it comes to near-future predictions.

The following by Jim Hoagland is a good example.
Driverless cars and trucks rule the road, while robots “man” the factories. Super-smartphones hail Uber helicopters or even planes to fly their owners across mushrooming urban areas. Machines use algorithms to teach themselves cognitive tasks that once required human intelligence, wiping out millions of managerial, as well as industrial, jobs.

These are visions of a world remade — for the most part, in the next five to 10 years — by technological advances that form a fourth industrial revolution. You catch glimpses of the same visions today not only in Silicon Valley but also in Paris think tanks, Chinese electric-car factories or even here at the edge of the Sahara.

Technological disruption in the 21st century is different. Societies had years to adapt to change driven by the steam engine, electricity and the computer. Today, change is instant and ubiquitous. It arrives digitally across the globe all at once.

It is essential to note that, despite decades of serious and aggressive research and development, none of these technologies currently exist (at least not at the levels implied here) and, with the exception of autonomous vehicles, they probably won't exist a decade from now. Even with AVs, ruling the road will probably take decades unless we start heavily regulating non-autonomous cars.

(As we've pointed out before, there is an important distinction between driverless cars and driverless trucks. While both are coming, the current economic case for autonomous trucking makes far more sense and the technological challenge of driving a relatively small number of routes is considerably less. Long haul truck driving has the potential to go away quite suddenly.)

It's true that outside factors often slowed implementation in fields like electricity in the past. Factories had to be reconfigured. Power lines had to be laid. This meant that the adoption of certain technologies was delayed, particularly in certain localities, but even with this, the rate of technological change around the turn-of-the-century and, to a somewhat lesser extent, in the postwar era, was stunning.

More importantly in this case, (barring the special case where a new technology can be uploaded without any kind of change or upgrade of hardware) the same basic issues still apply. Even with AV's, there's still development time, infrastructure and adoption to consider.

The day is coming when you'll be able to say an address into your phone and have your car on its way without ever leaving the comfort of your couch. The prognosis for Hoagland's other just-around-the-corner innovations is considerably less promising. There are daunting problems involved with using AI to take over complex, badly understood tasks that largely lack reliable and agreed-upon metrics of success. While as for flying cars, the obstacles are huge and we have a history of failed promises going back literally a century. Just because some compulsive liar whose primary accomplishment has been getting gullible venture capitalists to write him ginormous checks makes this particular promise doesn't mean you should report it in the pages of the Washington Post as a done deal.

But even allowing for the vanishingly small chance that all of these things do come to pass roughly in the time frame given, we would still be nowhere near the level of technological change that people in the late 19th and early 20th centuries had to adapt to in virtually every aspect of their world, from day-to-day living to industry to transportation to mass media to telecommunications to medicine to agriculture and to all the others that I have invariably left off the list.

Wednesday, December 6, 2017

Insert obligatory what's-a-newspaper? joke here

Another small piece (or, more accurately, medium-size piece of a big piece) of the late 19th/early 20th century technology spike.

Media innovation was a big part of this story, not in the same league as internal combustion and electricity, but still a big player marked with a number of stunning advances.Most of the attention here tends to go to the then completely new developments such as recorded sound and moving pictures, but in terms of impact, advances in printing may have had more of an effect on the period.

I have been going through some old Scientific Americans for material and came across this excellent example from 1896. In the middle of the century, putting out a major metropolitan daily required the equivalent of a medium-size, well staffed factory.




By the last decade of the century, the presses were five times as fast, a fraction of the size, and largely automated.





This and other technological innovations (such as color printing) made publications like the Strand magazine and journalistic events like the Hearst/Pulitzer circulation wars and the rise of the comic strips possible. All of this had an enormous impact on American culture and politics.


Tuesday, December 5, 2017

Non-belated Tuesday Tweets












Monday, December 4, 2017

Want to see a 19th Century polar exploration balloon? Thought so...

Besancon and Hermite never actually made the trip, of course, but, damn, the illustrations are cool.



















Friday, December 1, 2017

Yes, I know I've reposted this, but it keeps getting more relevant







Tuesday, August 22, 2017

The Republicans' 3 x 3 existential threat

I've argued previously that Donald Trump presents an existential threat to the Republican Party. I know this can sound overheated and perhaps even a bit crazy. There are few American institutions as long-standing and deeply entrenched as are the Democratic and Republican parties. Proposing that one of them might not be around 10 years from now beggars the imagination and if this story started and stopped with Donald Trump, it would be silly to suggest we were on the verge of  a political cataclysm.

But, just as Trump's rise did not occur in a vacuum, neither will his fall. We discussed earlier how Donald Trump has the power to drive a wedge between the Republican Party and a significant segment of its base [I wrote this before the departure of Steve Bannon. That may diminish Trump's ability to create this rift but I don't think it reduces the chances of the rift happening. – – M.P.]. This is the sort of thing that can profoundly damage a political party, possibly locking it into a minority status for a long time, but normally the wound would not be fatal. These, however, are not normal times.

The Republican Party of 2017 faces a unique combination of interrelated challenges, each of which is at a historic level and the combination of which would present an unprecedented threat to this or any US political party. The following list is not intended to be exhaustive, but it hits the main points.

The GOP currently has to deal with extraordinary political scandals, a stunningly unpopular agenda and daunting demographic trends. To keep things symmetric and easy to remember, let's break each one of these down to three components (keeping in mind that the list may change).


With the scandals:

1. Money – – Even with the most generous reading imaginable, there is no question that Trump has a decades long record of screwing people over, skirting the law, and dealing with disreputable and sometimes criminal elements. At least some of these dealings have been with the Russian mafia, oligarchs, and figures tied in with the Kremlin which leads us to…

2. The hacking of the election – – This one is also beyond dispute. It happened and it may have put Donald Trump into the White House. At this point, we have plenty of quid and plenty of quo; if Mueller can nail down pro, we will have a complete set.

3. And the cover-up – – As Josh Marshall and many others have pointed out, the phrase "it's not the crime; it's the cover-up" is almost never true. That said, coverups can provide tipping points and handholds for investigators, not to mention expanding the list of culprits.


With the agenda:

1. Health care – – By some standards the most unpopular major policy proposal in living memory that a party in power has invested so deeply in. Furthermore, the pushback against the initiative has essentially driven congressional Republicans into hiding from their own constituents for the past half year. As mentioned before, this has the potential to greatly undermine the relationship between GOP senators and representatives and the voters.

2. Tax cuts for the wealthy – – As said many times, Donald Trump has a gift for making the subtle plain, the plain obvious, and the obvious undeniable. In the past, Republicans were able to get a great deal of upward redistribution of the wealth past the voters through obfuscation and clever branding, but we have reached the point where simply calling something "tax reform" is no longer enough to sell tax proposals so regressive that even the majority of Republicans oppose them.

3. Immigration (subject to change) – – the race for third place in this list is fairly competitive (education seems to be coming up on the outside), but the administration's immigration policies (which are the direct result of decades of xenophobic propaganda from conservative media) have already done tremendous damage, caused great backlash, and are whitening the gap between the GOP and the Hispanic community, which leads us to…



Demographics:

As Lindsey Graham has observed, they simply are not making enough new old white men to keep the GOP's strategy going much longer, but the Trump era rebranding of the Republican Party only exacerbates the problems with women, young people, and pretty much anyone who isn't white.

Maybe I am missing a historical precedent here, but I can't think of another time that either the Democrats or the Republicans were this vulnerable on all three of these fronts. This does not mean that the party is doomed or even that, with the right breaks, it can't maintain a hold on some part of the government. What it does mean is that the institution is especially fragile at the moment. A mortal blow may not come, but we can no longer call it unthinkable.

Lots of threads converging on this one

And all of them tremendously important. Corruption and regulatory capture. An almost mystical faith in the magic of markets and private-enterprise approaches (even if they end up being all but completely government-subsidized). An inexhaustible tolerance for concentration of wealth and power. Internalized Randianism complete with the belief that the rest of us owe a debt to the rich and powerful that we can never fully repay. Regulatory capture and occasionally good old-fashioned corruption.

Thank God for John Oliver.


Thursday, November 30, 2017

I'm running behind again so...

Here are two beautiful French planes from 1909. Back tomorrow with actual prose.



And a few impressive airships.



Tuesday, November 28, 2017

Belated Tuesday Tweets


















Kruzno

A few years ago, when I was teaching high school, I designed some games to illustrate mathematical concepts and develop strategic thinking and problem-solving skills. The clear winner in terms of player response was a game called Kruzno.



Despite the pieces (I decided to go with something off the shelf for the prototype), it's not a chess variant. Instead, it's a capture-and-evade abstract strategy game where the rank of the pieces is non-transitive. The rules are simple enough for a small child to play but challenging enough to keep reasonably competent chess players on their toes. It is also the official game of a small village in Slovakia, but that's a story for another time.


Between now and Christmas, I'll be writing some posts on game design and possibly sharing some thoughts and cautionary tales about jumping into a small business with no relevant experience. In the meantime, I still have some of that first run up for sale on Amazon. Go by and check it out.

Monday, November 27, 2017

Another Tesla article from the Scientific American archives (along with some very cool train pictures).

19th Century drawing of a Geissler tubes.






This story mainly stands on its own but there are a couple points I want to emphasize. First, it's a nice example of that distinctive Tesla combination: important innovator; savvy showmen; extravagantly over-promising flake. That's not just something that shows in retrospect; you can see it coming through contemporary accounts. Sober observers were well aware of all of these facets. The man was capable of amazing advances; he also reported getting messages from Mars.

I also like the way the story knocks down a false but persistent trope. There's a tendency to treat earlier generations as innocent and unimaginative, often oblivious to the extraordinary events that were starting to unfold. Call it the "little did they imagine" genre. The trouble is it's almost entirely wrong. Particularly in America, people in the late 19th and early 20th centuries saw themselves as living in a wondrous time and they tended to greet new developments with great excitement, if anything, often overestimating the degree to which their world was about to change.

Reading through these old articles, one thing that strikes me is that wildly ambitious ideas were not accepted unskeptically, but they weren't treated dismissively either. People seemed to get that an idea could fail miserably and still be of great value.
























































Friday, November 24, 2017

This is what a good job market looks like.

In the postwar era, employers (particularly what we would now call the stem fields) were genuinely hungry. They were willing to be flexible about qualifications and creative about recruiting.

What's My Line? (1950–1967)
Episode dated 20 February 1955 






Thursday, November 23, 2017

"As God as my witness..." is my second favorite Thanksgiving episode line [Repost]



If you watch this and you could swear you remember Johnny and Mr. Carlson discussing Pink Floyd, you're not imagining things. Hulu uses the DVD edit which cuts out almost all of the copyrighted music. .

As for my favorite line, it comes from the Buffy episode "Pangs" and it requires a bit of a set up (which is a pain because it makes it next to impossible to work into a conversation).

Buffy's luckless friend Xander had accidentally violated a native American grave yard and, in addition to freeing a vengeful spirit, was been cursed with all of the diseases Europeans brought to the Americas.

Spike: I just can't take all this mamby-pamby boo-hooing about the bloody Indians.
Willow: Uh, the preferred term is...
Spike: You won. All right? You came in and you killed them and you took their land. That's what conquering nations do. It's what Caesar did, and he's not goin' around saying, "I came, I conquered, I felt really bad about it." The history of the world is not people making friends. You had better weapons, and you massacred them. End of story.
Buffy: Well, I think the Spaniards actually did a lot of - Not that I don't like Spaniards.
Spike: Listen to you. How you gonna fight anyone with that attitude?
Willow: We don't wanna fight anyone.
Buffy: I just wanna have Thanksgiving.
Spike: Heh heh. Yeah... Good luck.
Willow: Well, if we could talk to him...
Spike: You exterminated his race. What could you possibly say that would make him feel better? It's kill or be killed here. Take your bloody pick.
Xander: Maybe it's the syphilis talking, but, some of that made sense.

Wednesday, November 22, 2017

Igon values, superstar coin flippers, and the Gladwell problem

Malcolm Gladwell has started coming up in quite a few major threads and larger pieces, so I decided I needed to get up to speed on some of the controversies involving the author. Some of the more substantial have centered around what Steven Pinker has called the Igon value problem

From Pinker's review of "What the Dog Saw"
An eclectic essayist is necessarily a dilettante, which is not in itself a bad thing. But Gladwell frequently holds forth about statistics and psychology, and his lack of technical grounding in these subjects can be jarring. He provides misleading definitions of “homology,” “sagittal plane” and “power law” and quotes an expert speaking about an “igon value” (that’s eigenvalue, a basic concept in linear algebra). In the spirit of Gladwell, who likes to give portentous names to his aperçus, I will call this the Igon Value Problem: when a writer’s education on a topic consists in interviewing an expert, he is apt to offer generalizations that are banal, obtuse or flat wrong.


Gladwell got the best of the follow up exchange, dismissing “igon value” as a spelling error while getting Pinker sucked into a bunch of secondary or even tertiary arguments. (One of the best indicators of intelligence is the ability to avoid discussions about the heritability of intelligence.)

The spelling error defense is technically correct but it misrepresents the main point of the criticism. First off, on a really basic level, this indicates poor fact checking on the part of Mr. Gladwell and the New Yorker. Even working under the relatively low standards of the blogosphere, I always try to Google unfamiliar phrases before quoting them. You'd think that the editors of America's most distinguished magazine would do at least that much.

More importantly, spelling errors fall in two basic categories. The first does not tell us anything substanitive about the writer. Given the ghoti insanity of the English language, being a bad speller does not necessarily imply a weak vocabulary or poor mastery of the language (put another way, not knowing whether it's double C or double S in "necessarily"does not necessarily suggest that you don't know what "necessarily" means). There are, however, cases (particularly involving transcription) where spelling errors can indicate that the writer is unfamiliar with the words in question. That appears to be the case here.

Pinker's central criticism largely boils down to phonetic reporting. Gladwell often goes into stories with a weak grasp of the field in question, as a result he frequently makes serious mistakes, constantly misses important subtleties, and is almost completely dependent on his subjects for understanding and context. Add to this poor fact checking and a disturbing nonchalance about getting the story right, and things can get ugly quickly.

Somewhat ironically, Gladwell hit back at Pinker for employing one of the same techniques which Gladwell is so proud of, picking a detail that told a good story and memorably illustrated a larger idea. The Igon Value Problem worked beautifully on those terms but it was far from the most serious or conclusive example available, even if we limit ourselves to the single article in question, "Blowing Up."

For example, the piece is very much invested in the idea of Taleb as Wall Street revolutionary. We could quibble about just how radical the Black Swan ideas and strategies are, but it is an entirely defensible interpretation. Unfortunately, Gladwell doesn't really understand which ideas are debatably new and which are familiar to anyone in finance. Here's an excerpt (starting and ending mid-paragraph):

There was just one problem, however, and it is the key to understanding the strange path that Nassim Taleb has chosen, and the position he now holds as Wall Street's principal dissident. Despite his envy and admiration, he did not want to be Victor Niederhoffer -- not then, not now, and not even for a moment in between. For when he looked around him, at the books and the tennis court and the folk art on the walls -- when he contemplated the countless millions that Niederhoffer had made over the years -- he could not escape the thought that it might all have been the result of sheer, dumb luck.

Taleb knew how heretical that thought was. Wall Street was dedicated to the principle that when it came to playing the markets there was such a thing as expertise, that skill and insight mattered in investing just as skill and insight mattered in surgery and golf and flying fighter jets.

...

For Taleb, then, the question why someone was a success in the financial marketplace was vexing. Taleb could do the arithmetic in his head. Suppose that there were ten thousand investment managers out there, which is not an outlandish number, and that every year half of them, entirely by chance, made money and half of them, entirely by chance, lost money. And suppose that every year the losers were tossed out, and the game replayed with those who remained. At the end of five years, there would be three hundred and thirteen people who had made money in every one of those years, and after ten years there would be nine people who had made money every single year in a row, all out of pure luck.

But of course, Taleb didn't have to "do the arithmetic in his head" because, like virtually everyone else on Wall Street, he had probably read almost the same analogy in a famous passage from A Random Walk Down Wall Street by Burton Malkiel [transcribed via Dragon so beware of homonyms]:

Perhaps the laws of chance should be illustrated. Let's engage in a coin flipping contest. Those who can consistently flip heads will be declared winners. The contest begins and 1,000 contestants flip coins. Just as would be expected by chance, 500 of them flip heads and these winners are allowed to advance to the second stage of the contest and flip again. As might be expected, 250 flip heads. Operating under the laws of chance, there will be 125 winners in the third round, the three in the fourth, 31 in the fifth, 16 in the sixth, and 8 in the seventh.

By this time, crowds start to gather to witness the surprising ability of these expert coin-flippers. The winners are overwhelmed with adulation. They are celebrated as geniuses in the art of coin-flipping, their biographies are written, and people urgently seek their advice. After all, there were 1000 contestants and only eight could consistently flip heads. The game continues and some contestants eventually flip heads nine and ten times in a row. [* If we had let the losers continue to play (as mutual fund managers do, even after a bad year), we would have found several more contestants who flipped eight or nine ads out of 10 and were therefore regarded as expert coin-flippers.] The point of this analogy is not to indicate that investment-fund managers can or should make their decisions by flipping coins, but that the laws of chance do operate and that they can explain some amazing success stories.


(I love that second paragraph. Pretty much any time I flip past CNBC it comes flooding back to mind.)

Malkiel published this book in 1973 and though it more than ruffled a few feathers, it quickly became one of the seminal books on investing. Malcolm Gladwell's New Yorker piece came out 25 years later.

None of this is meant to imply any kind of deliberate plagiarism. Quite the opposite. I very much doubt that Gladwell realized he was paraphrasing a well-known passage. What I strongly suspect happened was that Taleb cited this in an interview as a standard example that everyone would be familiar with, sort of like describing a situation as a "frog in boiling water."

Gladwell's unacknowledged paraphrase is yet another indication that he didn't understand the strange role that economic theory and particularly market efficiency (in this case the semi-strong variety) plays on Wall Street, a role that was central to his narrative. This would be bad enough if he was just shooting for a straightforward profile, but Gladwell insists on playing the deep thinker, making pseudo-profound points, even closing with a grand sweeping moral about human nobility:

“That is the lesson of Taleb and Niederhoffer, and also the lesson of our volatile times. There is more courage and heroism in defying the human impulse, in taking the purposeful and painful steps to prepare for the unimaginable.”

Gladwell loves to tell what Christopher Chabris has termed "just-so stories," cute little fables counterintuitive and surprising enough to catch the eye but neat and simple enough to go down easy. Paradoxically, pulling off that sort of simplicity requires that the writer have a deep and subtle understanding of his or her subject. Simplifying a subject you don't understand never goes well.

Tuesday, November 21, 2017

Tuesday Tweets

One of my great, ongoing problems is what to do with ideas that I'm not ready to write up or which are not ready to be written. One of the wonderful things about a blog is that it's a welcoming environment for these not-ready-for-prime-time pieces. It's low-stakes and flexible both in terms of length and format. In other words, a great place to pin notions and notes.

That said, there are things too insubstantial even for a blog. That and the need for self-promotion are the main reasons I've taken to Twitter. The bar for a tweet is not so much low, as it is sitting on the ground. Nonetheless, there are a few that feel like they might be worth sharing, either because I liked how they turned out or in parentheses far more frequently) they cite some article that might be of interest to our regular readers.




















Monday, November 20, 2017

Australia citizenship furore

This is Joseph

Here is a case where they missed the main argument.  In Australia a number of MPs have been disqualified because the constitution prohibits dual citizens from running for office.  The referenced article asks why MPs will not be forced to repay their salaries when welfare debts are collected all of the time.  Now I am not necessarily a great fan of putting in place debt collection on the most vulnerable.  But that's not the real issue here.

The real issue is that you don't want to raise the stakes any higher.  As it is, being disqualified is a terrible process for a politician to go through.  Forcing the unexpectedly unemployed person to pay back hundreds of thousands of dollars can well add destitution to the consequences. 

And, in some cases, this may be out of proportion to the crime.  In some cases MPs were either sloppy, cavalier, or deceptive and this makes sense.  These are not traits one would want.  But how can you be sure you don't have citizenship somewhere?  After all, many people have overseas links and it is often possible that a parent has taken actions that you are unaware of (estrangement does happen).  Or, even more likely, is ambiguity like in the case of Susan Lamb.  She might have UK citizenship but cannot prove it and so cannot renounce it. 

In the case of these types of complex administrative cases, being punitive only makes a bad case worse.  I am all in favor of rooting out corruption but this would appear to run some risks of making defiance the most viable strategy and I am not sure that ends well. 


Friday, November 17, 2017

That sense of living in a science fiction novel

As previously mentioned, I'm working on a project about technology. One of the central points is the ways we think and feel about technology -- the language, the attitudes, the mental frameworks -- were largely in place by the early 20th century, and became set in the post-war era.

When people internalize their reactions to abnormal conditions, unconsciously continue to respond to new things in old ways, The result is seldom good. We'll delve more into that later. For now, we're still in the anecdote and data gathering stage of the process, so here's an excellent example of how even the most sober observers of the period were so conscious of how astounding the times were that they explicitly described the advances of the time in terms of science fiction novels (even before the term "science fiction" had been coined).


From Scientific American May 17, 1890
We know that at the time of the first official experiment, the two navigators of the Goubet remained  eight full hours under water without any other communication with the outside world than the telephone wire that they used to give their impressions, which, by the way, were of a cheerful character. When they made their appearance, they were fresh, well, and lively, having been able to attend to all the functions of life, and being ready to begin again. There was still in the tubes enough oxygen to last twenty hours.

It has been said of the Goubet that it is a realization of the dream conceived by Jules Verne in his "Twenty Thousand Leagues under the Sea;" but the Goubet is better than that. It is not only one romance, but it is rather two of the great amuser's romances amalgamated. It is both "Twenty Thousand Leagues under the Sea" and " Doctor Ox" in action!









Thursday, November 16, 2017

Tax policy

This is Joseph

There are two pieces of journalism by Kevin Drum that are worth not missing.

One, is his noting that some of the current tax reform plans are absurdly generous to inherited wealth:
If you’re not following what this means, here’s an example. Suppose you’re uber-rich and you buy $1 billion in Apple Stock. By the time you die it’s worth $3 billion. Your heirs, lucky ducks that they are, don’t have to pay estate tax on that $3 billion. But they do have to pay normal capital gains on the $2 billion appreciation in the Apple stock. At 20 percent that comes to $400 million.
However, the Republican bill eliminates that too. Not only does it eliminate the estate tax completely, but it allows you to “step up” the value of the estate and avoid capital gains taxes entirely. In our example, you literally get $3 billion free and clear, and you owe taxes in the future only on the appreciation above $3 billion.
Now it is possible to decide that there should not be capital gains taxes.  This is actually an area of policy discussion as Canada has had a lifetime capital gains exemption.  But if this is what you would like to see happen then you should do it for all capital gains and not focus all of the benefit on the wealthy.  And if you don't exempt these assets from capital gains then the paperwork can get messy.  So, in a sense, the estate tax is a nice way to give the gift of less paperwork to recently bereaved families (not a bad thing).

But turning tax policy into a way to maximize the benefits of inherited wealth seems to an unnecessary enhancement, above and beyond the paperwork simplification.  There should be one or the other. 

Two, is his piece on how tax havens actually make income taxes regressive in Scandinavia.  It's a good item for considering how much real tax burden there is among the elite (the one's who are getting a large tax cut, or potentially getting a large tax cut at least).  You can make moral arguments about what is the right level of obligation that people have to support the nations and laws that make their wealth possible.  It's a complicated area.  But this does somewhat undermine the narrative that taxes at the topic are actually crushing.

Wednesday, November 15, 2017

What if shame works?

I've been meaning to do multiple posts on Patrick Radden Keefe's extraordinary article on how the Sackler family built a fortune by largely engineering the opioid crisis. You should definitely read the whole thing, but there are a number of interesting points that are worth singling out.

For instance, the role of social pressure in curbing bad corporate behavior.


Mike Moore, the former Mississippi attorney general, believes that the Sacklers will feel no pressure to emulate this gesture until more of the public becomes aware that their fortune is derived from the opioid crisis. Moore recalled his initial settlement conference with tobacco-company C.E.O.s: “We asked them, ‘What do you want?’ And they said, ‘We want to be able to go to cocktail parties and not have people come up and ask us why we’re killing people.’ That’s an exact quote.” Moore is puzzled that museums and universities are able to continue accepting money from the Sacklers without questions or controversy. He wondered, “What would happen if some of these foundations, medical schools, and hospitals started to say, ‘How many babies have become addicted to opioids?’ ” An addicted baby is now born every half hour. In places like Huntington, West Virginia, ten per cent of newborns are dependent on opioids. A district attorney in eastern Tennessee recently filed a lawsuit against Purdue, and other companies, on behalf of “Baby Doe”—an infant addict.

There is something both sad and hopeful in this passage.

One of the overarching themes of this article is the way we are replaying the gilded age in increasingly obvious way. Specifically, the piece compares 19th century robber barons buying respectability with libraries and universities and today's billionaires doing much the same (though I think the comparison may be somewhat unfair to Carnegie and Stanford). This certainly applies to the Sackler family.

But if vanity and the pursuit of respectability can persuade plutocrats to give large chunks of money to good causes (or at least causes with the appearance of good), is it possible that fear of shame might dissuade these same people from doing bad things?

Tuesday, November 14, 2017

If you believe in n-rays, believing in mental radio is not that heavy a lift

Another visit to the Scientific American collection of the Internet Archive.

Having spent a big chunk of the past few weeks reading over contemporary accounts of science and technology from the late 19th and early 20th centuries, I've come to the conclusion that, from our 21st century perspective, we tend to overestimate the credulousness of people at the turn-of-the-century while greatly underestimating our own gullibility

The key here is context, belief in something (or, more precisely, belief in the possibility of something) that turns out not to exist has to be judged based on the amount of evidence (or, more precisely, the amount of non-evidence) that had accumulated at the time. 150 years ago or so, sending expeditions to look for legendary mega-fauna was an entirely reasonable type of scientific investigation. 50 years ago, not so much. Today, not at all. Every time a researcher, preferably one who very much wants to find a phenomenon in question, comes up empty, the lack of result adds to the evidence that the phenomenon simply doesn't exist. Eventually, enough would be believers dig enough dry wells to make the nonexistence a confirmed scientific fact.

But there's another important difference between 1900 and 2017 that needs to be taken into account. Without understating the steady and impressive current flow of new ideas and discoveries, we now generally work under a relatively stable intellectual framework. We reject the possibility of telepathy in part because so many motivated researchers have failed to find it, but also because no one has proposed a plausible mechanism for it consistent with the other things we know with great confidence about the world.

The close of the 19th century was a time of great optimism and discovery, but as a consequence not one of great certainty. Science and technology were causing cataclysmic shifts in the way people viewed the world. When Einstein endorsed a book on mental telepathy and Tesla speculated that he had just received communications from Martians, they were working under a framework that was assumed to be changing and incomplete.

Coming shortly after the discovery of x-rays, n-rays did not initially seem all that implausible, and once you've accepted the possibility of a new kind of radiation that was related to biological and even mental activities, accepting ideas like "mental radio" and even spiritualism isn't that much of a reach. N-rays themselves were debunked fairly quickly (see below for the fascinating details), but the sense that everything you know can suddenly change remained a key part of the mentality of the period.



June 17, 1905



The debunking of n-rays is, itself a fascinating story. From Wikipedia.

The "discovery" excited international interest and many physicists worked to replicate the effects. However, the notable physicists Lord Kelvin, William Crookes, Otto Lummer, and Heinrich Rubens failed to do so. Following his own failure, self-described as "wasting a whole morning", the American physicist Robert W. Wood, who had a reputation as a popular "debunker" of nonsense during the period, was prevailed upon by the British journal Nature to travel to Blondlot's laboratory in France to investigate further. Wood suggested that Rubens should go since he had been the most embarrassed when Kaiser Wilhelm II of Germany asked him to repeat the French experiments, and then after two weeks Rubens had to report his failure to do so. Rubens, however, felt it would look better if Wood went, since Blondlot had been most polite in answering his many questions.

In the darkened room during Blondlot's demonstration, Wood surreptitiously removed an essential prism from the experimental apparatus, yet the experimenters still said that they observed N rays. Wood also stealthily swapped a large file that was supposed to be giving off N rays with an inert piece of wood, yet the N rays were still "observed". His report on these investigations were published in Nature, and they suggested that the N rays were a purely subjective phenomenon, with the scientists involved having recorded data that matched their expectations. There is reason to believe that Blondlot in particular was misled by his laboratory assistant, who confirmed all observations. By 1905, no one outside of Nancy believed in N rays, but Blondlot himself is reported to have still been convinced of their existence in 1926

Monday, November 13, 2017

Cutting-edge solar power circa 1904


Came across this while researching something else. It struck me as an interesting glimpse into the early days of solar power. More to the point, it had a cool picture.

  
 









Friday, November 10, 2017

Why do we still have cities?

Following up on "remembering the future."

Smart people, like statisticians' models, are often most interesting when they are wrong. There is no better example of this than Arthur C Clarke's 1964 predictions about the demise of the urban age, where he suggested that what we would now call telecommuting would end the need for people to congregate around centers of employment and would therefore mean the end of cities.







What about the city of the day after tomorrow? Say, the year 2000. I think it will be completely different. In fact, it may not even exist at all. Oh, I'm not thinking about the atom bomb and the next Stone Age; I'm thinking about the incredible breakthrough which has been made possible by developments in communications, particularly the transistor and above all the communications satellite. These things will make possible a world where we can be in instant contact with each other wherever we may be, where we can contact our friends anywhere on earth even if we don't know their actual physical location. It will be possible in that age, perhaps only 50 years from now, for a man to conduct his business from Tahiti or Bali just as well as he could from London. In fact, if it proved worthwhile, almost any executive skill, any administrative skill, even any physical skill could be made independent of distance. I am perfectly serious when I suggest that someday we may have rain surgeons in Edinburgh operating on patients in New Zealand. When that time comes, the whole world will have shrunk to a point and the traditional role of a city as the meeting place for man will have ceased to make any sense. In fact, men will no longer commute; they will communicate. They won't have to travel for business anymore; they'll only travel for pleasure. I only hope that, when that day comes and the city is abolished, the whole world isn't turned into one giant suburb.


Clarke was working with a 20 to 50 year timeframe, so it's fair to say that he got this one wrong. The question is why. Both as a fiction writer and a serious futurist, the man was remarkably and famously prescient about telecommunications and its impact on society. Even here, he got many of the details right while still being dead wrong on the conclusion.

What went wrong? Part of this unquestionably has to do with the nature of modern work. Clarke probably envisioned a more automated workplace in the 21st century, one where stocking shelves and cleaning floors and, yes, driving vehicles would be done entirely by machines. He likely also underestimated the intrinsic appeal of cities.

But I think a third factor may well have been bigger than either of those two. The early 60s was an anxious but optimistic time. The sense was that if we didn't destroy ourselves, we were on the verge of great things. The 60s was also the last time that there was anything approaching a balance of power between workers and employers.

This was particularly true with mental work. At least in part because of the space race, companies like Texas Instruments were eager to find smart capable people. As a result, employers were extremely flexible about qualifications (a humanities PhD could actually get you a job) and they were willing to make concessions to attract and keep talented workers.

Telecommuting (as compared to off shoring, a distinction will need to get into in a later post) offers almost all of its advantages to the worker. The only benefit to the employer is the ability to land an otherwise unavailable prospect. From the perspective of 1964, that would have seemed like a good trade, but those days are long past.

For the past 40 or so years, employers have worked under (and now completely internalized) the assumption that they could pick and choose. When most companies post jobs, they are looking for someone who either has the exact academic background required, or preferably, someone who is currently doing almost the same job for a completely satisfied employer and yet is willing to leave for roughly the same pay.

When you hear complaints about "not being able to find qualified workers," it is essential to keep in mind this modern standard for "qualified." 50 or 60 years ago it meant someone who was capable of doing the work with a bit of training. Now it means someone who can walk in the door, sit down at the desk, and immediately start working. (Not to say that new employees will actually be doing productive work from day one. They'll be sitting in their cubicles trying to look busy for the first two or three weeks while IT and HR get things set up, but that's another story.)

Arthur C Clarke was writing in an optimistic age where workers were on an almost equal footing with management. If the year 2000 had looked like the year 1964, he just might have gotten this one right.

Thursday, November 9, 2017

Hyperloops, red flags, deep dark tunnels, and the state of American journalism



The Sigafoos Tunneling Machine, 1909


At this point, I'd imagine most of our regular readers are getting fairly burned out on the Hyperloop. I more than sympathized. In terms of technology, infrastructure, transportation, and public policy, we've pretty much exhausted the topic.

The one area, however, where the Hyperloop remain somewhat relevant, is as an example of certain dangerous journalistic trends, such as the tendency to ignore red flags, warnings that there's something wrong with the story as being presented be it lies or faulty data were simply a fundamental misunderstanding.

This recent article from the Baltimore Sun provides a perfect case study [emphasis added]:
Maryland has given transportation pioneer Elon Musk permission to dig tunnels for the high-speed, underground transit system known as a hyperloop that Musk wants to build between New York and Washington.

Hogan administration officials said Thursday the state has issued a conditional utility permit to let Musk’s tunneling firm, The Boring Co., dig a 10.3-mile tunnel beneath the state-owned portion of the Baltimore-Washington Parkway, between the Baltimore city line and Maryland 175 in Hanover.
...

The Boring Co. aims to reduce traffic congestion by creating a low-cost, efficient system of tunnels. The company has developed tunneling machines it says will drill quickly through soft soils at a fraction of the cost of traditional tunneling.

 
Before we get to our main point, there are a few important but secondary issues we need to get out of the way.

When virtually every independent authority who weighed in lambasted Elon Musk for unrealistic cost estimates, those experts were talking about a much cheaper version of the Hyperloop. It is far more expensive to elevate a train than it is to build it on the ground, and it is far more expensive to build a train underground than it is to elevate it. Though there are other expenses associated with underground construction, an 80% or 90% reduction in tunneling costs might put the underground Hyperloop on a comparable budget with the elevated version, but that just gets you back to prohibitively expensive.

These points are important for context but this is not another what's-wrong-with-the-Hyperloop post. We've been over that a number of times and I don't have much to add. Besides, that was never really the important part of the story. The important part was the dysfunctional way we discuss this and other dubious technological and business proposals.

One of the major causes of this dysfunction is the willingness to downplay or even completely ignore massive warning signs. Take the claim (if in fact it was a claim, but more on that later) that Elon Musk has reduced tunneling costs by a factor of 5 to 10.

A bit more context. The dream of a cheap, fast tunneling machine has been something that the world's best engineering minds have actively pursued since the 19th century. The revolutionary potential of such a machine was obvious to everyone from the beginning. It could completely rewrite the rules of construction and infrastructure, and, in the process, make its developers fantastically wealthy.

Based on previous statements, I suspect that Elon Musk and his spokespersons didn't actually claim (as the article suggests) that they had a tunneling machine capable of this; they probably said something vague about being in the process of getting such a machine and left it to the journalists to fill in the details. Either way, we have two reasons to believe that the Boring company does not currently have this technology.

First, this is a mechanical engineering and materials science problem that has been intensely examined by the smartest people in the field for well over 100 years. It is not an easy nut to crack, and by his own account, Elon Musk and his team only came up on the problem recently and did so with no ideas on how to proceed. It beggars the imagination to think that a small and inexperienced group of researchers took a look at this extraordinarily difficult problem and immediately made a breakthrough.

It is even more difficult to imagine that Musk would let a piece of revolutionary technology easily worth tens of billions of dollars sit essentially unused in some Hawthorne storage building. It's as if he had announced that he had developed cold fusion, but wasn't going to do anything with it until the next Tesla model came out.

Put bluntly, the only reasonable conclusions are that the article misstated a claim, or the claim was faults. It is the job of journalists to spot these red flags and follow up. The failure to do so is proving increasingly costly, not just in technology reporting, but in business, public policy, and politics

Wednesday, November 8, 2017

The Sigafoos Tunneling Machine




With all this discussion of hyperloops and tunneling technology, I thought this would provide some interesting context (and an excuse to post a couple of cool pictures). I found the last paragraph particularly interesting. Notice how even more than a hundred years ago, people recognized the tremendous potential of fast, cheap tunneling.




Like the poor, new ideas for tunneling through rock and doing away with drilling and powder and dangerous blasting, are forever with us. Since 1853 there have been no less than sixty-nine patents granted on tunneling machines, and of this number but three have progressed beyond the blue-print stage. One was constructed and used with some slight success in the East, but owing to lack of funds or disputes among its builders, all progress was stopped. The second was built in Colorado, and at the present time is installed in a tunnel near Boulder. This machine does the work claimed for it, but the cutting is very irregular, numerous breakdowns are constantly happening, and in the course of over six months the machine has penetrated but a few hundred feet.

The third machine, here illus­trated, was invented by Mr. Sigafoos, of Denver, long associated with many eastern manufacturers until of late, when he turned his mind and labor toward western mining fields. Mr. Sigafoos built his first model three years ago, and until the present day it is on exhibition in his offices. Even this little working model, barely two feet long, has eaten through solid granite quite as easily and determinedly as a hungry earthworm.

Early in January of this year the first regular-sized machine was constructed in the East and shipped complete to Georgetown, Col., where the first contract was let and its behavior eagerly watched. The utmost secrecy was observed, for the first trial, and the author was extremely fortunate in being allowed to witness the test. In every instance the rotary proved its value, and came up to the highest expectations. Mr. Sigafoos stands ready to take contracts with his machine, in any and all rock, and will guarantee to cut five feet an hour, twenty-four hours a day.
...

It may not be amiss to state that the famous Moffat road will probably use these large rotaries in cutting its great tunnel through the mountains. In places today where the road ascends and descends mountains, it is expected within a short time to eventually bore through them, cutting down the time from coast to coast fully twenty-four hours. The contractors, before learning of the new machine, allowed ten years for the completion of this gigantic undertaking; but today, with a sufficient number of tunnel rotaries at work, two years will not be an impractical limit. The immediate uses to which this machine can be put to work are innumerable. Subways that formerly took five years to construct can now be run for half the expense in one-tenth the time. Water in unlimited quantities can be brought through the mountain walls, and the vast arid areas of the deserts will be made to blossom as a wonderful garden. If the claims made for it continue to be substantiated in practice, Mr. Sigafoos may well be considered a world's benefactor in giving us this marvelous rotary tunnel machine. 




Repost -- we're letting this one stand





Thursday, March 2, 2017

There will be safe seats. There are no safe seats.

In 2017, we have a perfect example of when not to use static thinking and naïve extrapolation.

Not only are things changing rapidly, but, more importantly, there are a large number of entirely plausible scenarios that would radically reshape the political landscape and would undoubtedly interact in unpredictable ways. This is not "what if the ax falls?" speculation; if anything, have gotten to the point where the probability of at least one of these cataclysmic shifts happening is greater than the probability of none. And while we can't productively speculate on exactly how things will play out, we can say that the risks fall disproportionately on the Republicans.

Somewhat paradoxically, chaos and uncertainty can make certain strategic decisions easier. Under more normal (i.e. stable) circumstances it makes sense to expend little or no resources on unwinnable fights (or, conversely,  to spend considerable time and effort deciding what's winnable). The very concept of "unwinnable," however, is based on a whole string of assumptions, many of which we cannot make under the present conditions.

The optimal strategy under the circumstances for the Democrats is to field viable candidates for, if possible, every major 2018 race. This is based on the assumption not that every seat is winnable, but that no one can, at this point, say with a high level of confidence what the winnable seats are.