One of the people I worked with at CMU was a "Herbert A. Simon Professor of Human-Computer Interaction". I hadn't heard of Herbert Simon before, but his name came up again recently at the end of "Skin in the Game". Turns out Simon was a pretty interesting guy, and had a lot to say about Rationality.

Simon was an academic, and mostly wrote for academic audiences. This is his least academic book, and still presents a dry read. The first essay, called "Visions of Rationality", describes three flavors of rationality: SEU, Bounded Rationality, and Intuitive Rationality. I skipped the second essay because it was too boring. The third essay was about rationality in organizations, which is Simon's home turf, being a key figure in Organizational Behavior.

Critically, Simon treats reason as "wholly instrumental. It cannot tell us where to go; at best it can tell us how to get there. Thus, whereas reason may provide powerful help in finding means to reach our ends, it has little to say about the ends themselves".

Subjective Expected Utility (SEU) is the God-like “Olympian model”, but is not a suitable model for the mind of man. That said, Rationalists tend to elevate this model:

1. Well defined utility function.
2. Well defined set of alternatives to choose from.
3. Can assign a consistent joint probability distribution to all future sets of events.
4. Maximize the expected value of the utility function.

Bounded rationality postulates that our minds are limited, bounded by computational powers. The model more closely reflects human ability:

1. A mechanism to focus attention. "Rationality could focus on dealing with one or a few problems at a time".
2. A mechanism capable of generating alternatives.
3. A capability for acquiring facts about the environment, the ability to draw inferences from facts.

Intuitive rationality recognizes that thought is often affected by emotion, which can usefully "distract you from your current focus of thought, and to call your attention to something else that presumably needs attention right now.". There is no contradiction between SEU/Bounded models and intuitive rationality.

I really liked Herb Simon’s take on novels over textbooks for teaching social science. This “hot cognition” is what makes Ted Chiang’s stuff so compelling.

“Most humans are able to attend to issues longer, to think harder about them, to receive deeper impressions that last longer, if information is presented in a context of emotion, a sort of hot dressing, than if it’s presented wholly without effect.”

Examples:

• "Darkness at Noon" vs. a history book or transcripts of Stalin's purges?
• "War and Peace" vs. a treatise on military sociology?
• Proust and Chekhov versus a textbook on personality?

People respond well to story, but this leaves us with an important responsibility: “if we are to learn our social science from novelists, then the novelists have to get it right. The scientific content must be valid.” Example: Freud’s stuff still permeates novels despite very few orthodox Freudians left in psych today.