Saturday, April 22, 2017

How Speech Gets Computed, Maybe

Speech is just sounds, so why does it not sound like noise?

Even wonder how we learn a language? How we make the transition from hearing how a totally foreign language sounds- like gibberish- to fluent understanding of the ideas flowing in, without even noticing their sound characteristics, or at best, appreciating them as poetry or song- a curious mixture of meaning and sound-play? Something happens in our brains, but what? A recent paper discusses the computational activities that are going on under the hood.

Speech arrives as a continuous stream of various sounds, which we can split up into discrete units (phonemes), which we then have to reconstruct as meaning-units, and detect their long-range interactions with other units such as the word, sentence, and paragraph. That is, their grammar and higher-level meanings. Obviously, in light of the difficulty of getting machines to do this well, it is a challenging task. And while we have lots of neurons to throw at the problem, each one is very slow- a mere 20 milliseconds at best, compared to about well under a nanosecond for current computers, about 100 million times faster. It is not a very promising situation.

An elementary neural network, used by the authors. Higher levels operate more slowly, capturing broader meanings.

The authors focus on how pieces of the problem are separated, recombined, and solved, in the context of a stream of stimulus facing a neural network like the brain. They realize that analogical thinking, where we routinely schematize concepts and remember them in relational ways that link between sub-concepts, super-concepts, similar concepts, coordinated experiences, etc. may form a deep precursor from non-language thought that enabled the rapid evolution of language decoding and perception.

One aspect of their solution is that the information of the incoming stream is used, but not rapidly discarded as one would in some computational approaches. Recognizing a phrase within the stream of speech is a big accomplishment, but the next word may alter its interpretation fundamentally, and require access to its component parts for that reinterpretation. So bits of the meaning hierarchy (words, phrases) are identified as they come in, but must also be kept, piece-wise, in memory for further Bayesian consideration and reconstruction. This is easy enough to say, but how would it be implemented?

From the paper, it is hard to tell, actually. The details are hidden in the program they use and in prior work. They use a neural network of several layers, originally devised to detect relational logic from unstructured inputs. The idea was to use rapid classifications and hierarchies from incoming data (specific propositional statements or facts) to set up analogies, which enable drawing very general relations among concepts/ideas. They argue that this is a good model for how our brains work. The kicker is that the same network is quite effective in understanding speech, relating (binding) nearby (in time) meaning units together even while it also holds them distinct and generates higher-level logic from them. It even shows oscillations that match quite closely those seen in the active auditory cortex, which is known to entrain its oscillations to speech patterns. High activity at the 2 and 4 Hz bands seem to relate to the pace of speech.

"The basic idea is to encode the elements that are bound in lower layers of a hierarchy directly from the sequential input and then use slower dynamics to accumulate evidence for relations at higher levels of the hierarchy. This necessarily entails a memory of the ordinal relationships that, computationally, requires higher-level representations to integrate or bind lower-level representations over time—with more protracted activity. This temporal binding mandates an asynchrony of representation between hierarchical levels of representation in order to maintain distinct, separable representations despite binding."

This result and the surrounding work, cloudy though they are, also forms an evolutionary argument, that speech recognition, being computationally very similar to other forms of analogical / relational / hierarchical thinking, may have arisen rather easily from pre-existing capabilities. Neural networks are all the rage now, with Google among others drawing on them for phenomenal advances in speech and image recognition. So there seems to be a convergence from the technology and research sides to say that this principle of computation, so different from the silicon-based sequential and procedural processing paradigm, holds tremendous promise for understanding our brains as well as exceeding them.


Sunday, April 16, 2017

Are You a Turtle?

The many types of personality types. Review of "The Five Elements", by Dondi Dahlin.

It may be trite to say that people are different. Yet it bears repeating, as we routinely lose sight of that simple fact in our private echo chambers of righteous self-regard. We routinely assume that everyone is on some level the same as we are, and then are left befuddled and shocked when an election turns that idea on its head. In our personal lives, we routinely assume that others see what we see, and care about what we care about, and get angry when they blithely ignore that sock on the floor, or the monthly budget. Thus it is perennially important to raise consciousness about the depth and durability of human differences, so that we can live together more harmoniously, with greater understanding.

Personality and temperament have been an ongoing theme on this site, so it was a pleasure to find a new book on the subject that is as compelling as anything I have read. There are many systems that try to schematize the protean landscape of human personality, from the ancient (the four humors) to the pseudo-scientific (Jung's) and more systematic (Myers-Briggs), and the more frankly scientific (the five factor model). Dahlin is a California New-Age scion whose family has long used the Chinese five-element system to understand themselves and others.

Who are you? Who am I?

There are virtues to using such a simple system. On the surface, it captures extreme personality types, which can then be combined to come up with a more nuanced model of a particular person's style. To recount them very briefly, the Water type is introverted, slow, inner-directed, and creative. The Wood type is type-A, always doing, with a task list in hand, and impatient with ritual. The Fire type is the happy-go-lucky, life-of-the-party type, intensely interested and charming one minute, then off to something new the next. The Earth type is the home and family-oriented type, the human doormat, always serving others. And the Metal type is cool, idealistic, minimal. They subscribe to Dwell magazine.

There is a great deal more to say, of course, and Dahlin says it very well, with warmth, verve, and fascinating anecdotes. Each type has its spiritual approaches, its physical and relationship needs, its manifestations in childhood. Dahlin even supplies special exercises for each type, drawn from her experience as a dancer. This book, more than the others of this genre that I have read, prompted me to think about other people in my life through its lens, which brings out particuliarities and deeper themes that tend to get lost in the hubbub of normal relations.

Are these personality types genetic? Dahlin's mother writes in the forward about the experience of being pregnant with children of different elements. Their personalities were markedly different well before birth. Dahlin herself notes, however, that personalities can change with time, during childhood and after trauma, which suggests that they are canalized styles of expression, solutions to the problems of life, which are not entirely determined. Indeed, the genetics of twins indicates that stable personality traits are determined only about half genetically, and half by other influences, whether environmental or stochastic.

Dahlin notes that most couples are of different types- opposites really do attract, and often complement each other as well. This suggests two things- that the insofar as personality type is genetic, the stable population distribution of different types is a matter of balanced selection. This should not come as a surprise, as many other species show personalities and stable population distributions of different types, down to ants and their castes, as an extreme form. The second implication is that appreciation of different types (using whatever system you like) is highly important at all levels- to our personal lives as well as our public and work lives, to promote understanding and lift a veil of mystery from our often unthinking view of the "other".


  • Fairness, inequality and just deserts.
  • We are still in a zero world, and need more fiscal support.
  • United uses a complex market to get passengers into its seats, but when it wants them out, it calls the police.
  • Where is the GOP going with Social Security? Not towards security.
  • Tax cuts serve the rich, unsurprisingly.
  • The disaster of privatization.
  • The disaster of deregulation. Who can call this populist?
  • We are going Byzantine.
  • Remember when Trump promised everyone even better health care?
  • An economist on carbon taxes.

Saturday, April 8, 2017

American Exceptionalism Rests on the Middle Class

What happens to our country when the middle class disappears? Review of Ganesh Sitaraman's "The Middle Class Constitution".

We are in a crisis right now in the US. Our government has been taken over by plutocrats who are busily reversing every progressive and public-spirited policy they can get their hands on. The worst nightmare of the nation's founders, revolving around demagogues, rule by extreme wealth, deepening political corruption, and mass immiseration, is coming to pass. Thomas Piketty's work on the nature and growth of economic inequality was the first major intellectual shot against this development, from an econometric perspective. Sitaraman's new book is the second, putting our economic inequality and its political consequences into historical and legal perspective, as a far greater matter than mere economics- rather, as a constitutional crisis.

The US has been since its founding exceptional on the world stage, not for our apple pie or iPhones, but for our democratic social system. Now the world is filled with democracies, so our position as one of the more mediocre ones in terms of governance, public services, health status, and overall happiness is perhaps not such a surprise. But at the beginning, we were truly revolutionary, and Lincoln was not far off when he posed the Civil War as a test whether government by the people would survive globally. (Switzerland, however, probably would have pressed on with its own experiments).

Why did this experiment happen here? According to Sitaraman, and to Toqueville, and to many of the founders themselves and other observers, it happened because the economic condition of Americans was substantially one of equality. Immigration put most people on a similar footing, which is to say, poor. The availability of land to all who wanted to work it provided a base of subsistence to these immigrants, and a foundation of modest wealth. The ideology of the early colonies and Republic was one of severe allergy to class distinctions, titles, and the like. There were some wealthy people, but nothing like the disparities that we would see later, either in the guilded age, or today.

Our constitution reflects these origins, and depends on continued rough equality with a predominant middle class. Sitaraman makes it clear that historically, constitutions have always reflected the economic conditions they sought to rule. Where feudalism and serfdom was the rule, so was autocracy. And oligarchy or autocracy have been the rule through the vast majority of human history, with democracy only possible at the lowest levels of rural and small-town society, where equality was likewise occasionally possible. Golden-age Greece, our democratic ideal, was more like an oligarchy of the free men of the city. Karl Marx had a point when he maintained that ownership of the means of production controls the nature of the class system, and thence the political system. Even Aristotle recognized, that while a true middle class polity was impossible in the power-economics setting of the ancient world, it would be the ideal polity, composed of that class of people who are neither so greedy and power-mad that they would factionalize and corrupt the system, nor so poor that their only wish would be to revolt and overthrow it.

Our rough equality continued through the frontier era, when anyone could pull up stakes and get land out west. Conditions were increasingly difficult the farther west one went, until the coastal paradise of California and Oregon gave one last great frontier of fertile land. This availability of land amounted to a job guarantee for its day, allowing any willing person to work and make money regardless of the willingness of an employer to hire. Farming certainly had its own terrible risks, between the weather, pests, and fluctuating markets. But no able person had to be destitute.

Now, we are neither an agricultural society, nor have any frontiers left. We exist in a new form of feudalism where employers have no governance responsibility to their employees. Employees are purely at will, and may be fired at any time for any reason. It is ironic that we so fetishize political freedom and legal equality, while practicing, in our corporate culture, the most retrograde feudalism. For all the regulations that modestly ameliorate this condition, most workers are in a dramatically assymetric and powerless position. It wouldn't be so bad if the economic system were not also unstable, experiencing crises of demand and investment that force millions of workers into destitution. Additionally, the closing of the frontier and other restrictions have made land, housing, and rents increasingly subject to scarcity costs far above their construction costs. This makes destitution a far more probable and chilling prospect.



Sitaraman cites disturbing research into our political system that makes the case that we are already living in an oligarchy/plutocracy. The fact is that each of our politicians is for sale, the media system is run by corporations, and much of the governmental regulatory apparatus that was supposed to protect us from the predatory special interests have instead been captured by them. This latest research shows that the actions of our political system accord essentially none of the time with the preferences of the lower 90% of the population, and all of the time with the preferences of the rich and super-rich. Our so-called leaders, thanks to various forms of legal corruption, are steeped in the social melieu of the rich, its super-PACs, and its propaganda organs, from cable channels, to think tanks, to lobbyists. The rest of us hardly stand a chance. Even at state and local levels, the rich have been funding dramatic new advances in corruption and propaganda, which have turned our nation into a sea of red.

This is how wealth translates to power, and the US may have entered a terminal loop where the plutocracy is so entrenched and so shameless about leveraging that power into yet more power, that there is nothing further to do, short of revolution. The breathtaking nepotism, incompetence, greed, and immorality of our current administration merely puts an exclamation point on a decades-long process that has not only reshaped the political system into a frank plutocracy, but reshaped the economic system as well into one that freezes out the middle class, by dramatically lowering taxes on the rich and reducing public facilities and services, among many other policies.

Middle classes do not happen by accident. The natural course of events, given the Malthusian pressures documented so dryly by Thomas Piketty and many others, is towards competitive differentiation, with winners gathering more power and wealth, which, once it reaches a high level, grows by natural accretion and compounding (where it is not more actively leveraged) far beyond anyone's needs, and losers finding it ever more difficult to find a way into a brutally rigged system. Classically, this was expressed by ownership of land, to which the answer has been land reform, which is to say, expropriation. Sitaraman provides a fascinating aside on the sequel to the Civil War.
"The most eloquent advocate for confiscation and redistribution was Thaddeus Stevens. The clubfooted Pennsylvania congressman proposed confiscating the estates of the top 10% of wealthy rebel planters, which at the time amounted to those with more than $10,000 or more than two hundred acres of land. With that land, which he pointed out would leave 90 percent of southerners untouched, every freedman could be given forty acres. The remainder wold be sold at auction and used to fund veteran pensions, compensate the injured, and retire the war debt. Steven's reasoning acknowledge that this ction would be revolutionary, but he also deemed it necessary for preserving republican government. "The whole fabric of Southern society must be changed," he declared in a speech to constituents in 1865: 
"Without this, this government can never be, as it has never been, a true republic. Heretofore, it had more the features of aristocracy than of democracy. The Southern States have been despotisms, not governments of the people. It is impossible that any practical equality of rights can exist where a few thousand men monopolize the whole landed property. ... If the South is ever to be made a safe republic, let her lands be clutivated by the toil of the owners, or the free labor of intelligent citizens. This must be done even though it drive her nobility into exile. If they go, all the better."

The book ends with a relatively standard plea, in the Bernie Sanders/Elizabeth Warren frame of argument, for higher consciousness of these facts and trends, in order to frame a new phase of social activism. Sitaraman has great respect for the Progressive era of the late 19th century, which brought us the progressive income tax, anti-trust, and the regulatory state, among other innovations. Thinkers and movements of this time took a fundamental look at our system and recognized that one more agency would not be enough- we needed constitutional amendments and deep reform. Today, with the Supreme Court wielding the First Amendment like bludgeon against the very citizens it was designed to protect, drowning them in corporate doublespeak, and with our politico-economic power system declining into bannana republic levels of dysfunction and disparity, it is time again to crank up the volume of protest, and direct it to fundamental and radical aims, such as taking money out of politics, remaking corporate governance on a more democratic model, and restoring a tax and public financing policies that sustainably strengthen the middle class.


  • A clown is in charge of our economic policy.
  • Pay should not be a big secret.
  • Another lie ... and another.
  • We need a new privacy regime.
  • China is the last country to want any change in North Korea.
  • They really are better than everyone else.
  • Corrupt enrichment.
  • Health care is one of the bigger drivers of inequality.
  • Economic graph of the week. Corporations are saving more, growing fatter, while everyone else grows thinner.


Saturday, April 1, 2017

Con Men Bring Us Hope

Herbalife, Christianity, Scientology, and the Trump Network. Only I can connect the dots!

The New Yorker carried a remarkable article recently, ostensibly about Herbalife. That is the network or multilevel marketing company that makes its money luring would-be entrepreneurs to new careers selling a powdered food-related product, and charging them for every step towards their doom. The story was structured around a short that hedge fund star Bill Ackman attempted against Herbalife, thinking that a good dose of negative publicity could cause the pyramid scheme to collapse.

But the short ended in tears, as the stock recovered robustly and still trades at relatively high levels, with a respectable P/E of 19. It turned out that investors all knew the company was a scam. They didn't need to be pursuaded with flashy slides and conference calls. But getting the message to Herbalife's actual customers / marks was another matter, far beyond Ackman's PR capacity. The beauty of network marketing is that every person in the operation is motivated to spread the gospel. It is a theo-capitalist virus, made for growth on the back of desperate people's dreams.

Where were the regulators in all this? Well you may ask. The current Republican regime doesn't think much of any kind of regulation, representing, as they do, the predators of the system. The story brings up the extremely interesting tale of Donald Trump's own foray into network marketing, a window into his character like no other. This was an even more tenuous proposition, claiming to convey Trump's valuable business insights to its entrepreneur members, thereby allowing them to become rich in the wake of the 2008 recession. The scheme naturally ended up in court, then sold off to some other paragon of capitalism. What interest would such a person have in the victims of his form of capitalism? What credence should his voters have in his promises to help the little people?

Trump's character on display- always looking for a sucker.

Anyhow, the FTC did finally look into Herbalife, and issued a fine as well as directives to clean up its business. The fine was apparently sent to Herbalife distributors (who are also customers) who have been using it to buy more Herbalife products. The re-organization forced Herbalife to reclassify thousands of its distributors as customers so that it would have some customers actually buying products, instead of a multilevel network of distributors all selling down-line to suckers who thought they were also running a business. But naturally Herbalife found a way around this as well, spinning the whole thing as an FTC endorsement of their business model, and happy to putatively do the re-organization. Shares went up and all is well.
Herbalife is robust to whistleblowers, truth-tellers, the FTC, short sellers, and bad publicity in general.

The lesson from all this is that con men are virtually ineradicable, especially when we have toothless regulators. People are naturally, and necessarily, optimistic. America runs on optimism, yet optimism breeds the deceit and lies that are so easily passed off by psychopaths like Mr. Trump and the many other bottom-feeders of our competitive, capitalist system. Without regulation, a Gresham dynamic takes hold, honest business people can't make a dollar, and the entire system descends into a mire of curruption. Just look at the business climate in a country like Russia or Afghanistan.

But what if there are no regulators, because the con is not (ostensibly) business-related? What if a religion is being sold, with the most absurd visions of heavenly bliss (if you believe) and personal empowerment? Scientology started out as a sort of new-age psychotherapy. But when it met with derision from the from professional psychological community, and official sanction and bankruptcy for offering medicine (and selling E-meters) without a license, it was re-organized as a religion. Presto- not only could it continue as a quack medical cult, but it became tax-exempt as well! Now its members can believe in their hidden thetan abilities, purified "clear" states of consciousness, and extraterrestrial origins without skeptical questions from official bodies.

Religious cons are unimaginably more ornate and bold than business cons. Just think of the Jesus story. Who could have come up with such a thing? But given enough time, and enough idle dreaming and theological imagination, and, well, anything is possible. Now the entire world is saved, except for the non-believers who are all going to hell. We are so crushed by reality, so filled with hope that life should be better, and can be better if we only dream hard enough, that those who have crossed the line between reality and imagination have only to assert their confidence, for someone to believe them.

So the con men, mystics, and preachers, whether cynical, self-deluded, or psychopathic, make a matched set with the credulous, lost, and desperate. And aren't we all desperate, at some level? Desperate that this life, filled as it is with trials and pain, should be so difficult and end so definitively? What can the truth say about that? It is a perennial question, whether we put ourselves into the hands of yet another purveyor of hope, or practice moderation in this, as in so many other things.