Against Rationalism, For Meta-Rationality, Evidence, and Experience, Part 1

Foundations, and, why is Eliezer Yudkovsky fat and the Zizians nutty?

In this three Part series, I will discuss “Rationalism” as it is known with the recent movement of that name in particular and then more broadly as more commonly described in psychological discourse as “Theory Crafting”. The first part will discuss the philosophical principles associated with the necessity of appropriately constraining this notion as well as a couple of illustrative examples from the modern movement. Part 2 will discuss the utility of, in contrast to standard practice, possibly contradicting ideas at different layers of abstraction, with examples in psychotherapy and political commentary. Finally, Part 3 will present my theory that the Frankfurt School and French post-structuralists actually did a comprehensive troll job that ultimately led to some unfortunate developments within the intelligentsia that we see in the present day (and my perennial beating drum of how the working class has come to detest the intelligentsia).

To begin with, let me discuss some foundational problems with respect to “Rationalism”. The basic thesis of the movement is that one should use “reason” as a guiding force in decision making, taking care to avoid pitfalls of cognitive biases or any emotional or social norm driven limitations as far as the set of feasible actions. In order to properly frame the discussion I need to review some necessary philosophical background literature.

Background

First: there is a serious limitation with respect to what pure rational deduction can do for you. Reviewing Kant’s A Critique of Pure Reason is like going to the dentist: not a particularly pleasant experience, but chewing on ideas feels cleaner afterwards. It’s really something the Rationalists should do more. Kant pains to locate forms of thought that can be described as pure reason, presenting a rather tight tether as far as forms of inquiry one can pursue that can be characterized as such. His findings, or assertions as to the categories of cognition that can be enshrined with this holy space of a priori knowledge are limited to, essentially, mathematics, with arithmetic being the embodied cognition encoding time and geometry encoding space, together with logic, as well as what could be referred to in the modern day as systems thinking or praxeology (see Austrian Economics)

Moving forward, let us recall the seminal work of Quine, The Two Dogmas of Empiricism. He demonstrates how there is no crisp demarcation between a priori and empirical knowledge. Rather, knowing is a web, with a tight dense center, much more difficult to dislodge, such as that fire is hot, spreading outwards and gradually thinning in entanglement towards a looser outer rim, such as your friend Joe being good at poker. Whereas a couple of sloppy games can change your understanding of Joe’s poker abilities, your whole world will be spinning in confusion if you approach what you know to be a fire and as you get closer to it the air becomes cooler. This also fits developments in the Philosophy of Science, from Kuhn to Lakatos’ synthesis of Popper and Kuhn – there is a hard core of foundational knowledge that takes a scientific revolution to dislodge, and experiments just fill out details are performed in the contexts of models defined by the hard core.

Next: except physics, everything else is emergent phenomenon. Biology, sociology, etc. is completely impossible to devise a model with comprehensive negligible prediction error across all environments and scenarios. The reason is that it’s computationally impossible to define and fit a model that incorporates every component in the system, and the interesting features are emergent. For instance your health is ultimately governed by your biochemical reactions, but you can’t possibly know and identify them all. Moreover it is an open system (formally, it is not causally closed, or causal sufficiency does not hold) meaning complicated mutual interactions with unidentified aspects of the outer environment are myriad and only add to complications. For more details, see my draft, assisted by colleagues, on the epistemology of causality.

Finally: recall G.E.Moore’s Principia Ethica: there is no form of logical deduction by which an “Ought” statement follows an “Is” statement. This has essentially mired the discipline of Meta Ethics - that is, what Ethical statements even mean in the first place, to a standstill in the early 20th century, with the interest arising in the second half for a descriptive understanding of Ethics, that is, what Ethics is, and what its role and function is, as far as a language game embedded in a tribal mythical meta-narrative. Within Rationalist discourse itself, Normative Ethics games take place, e.g., endless variations of the trolley problem and demarcating “true believers” of utilitarianism versus alternatives, but these are purely academic in being far removed from the real dilemmas associated with the choices that people make.

The Ought-Is cleavage also has a neuroscience correspondence: broadly speaking our prefrontal cortex says “how” and our mid and hind brain say “what”. Natural experiments from catastrophic accidents resulting in disconnection of the latter from the former didn’t lead to a dispassionate efficient Spock-like successful human being, but rather someone who has significant trouble making any decisions at all, constantly in hesitation and passivity.

Implications

Let us turn to cognitive biases. We have to truly consider the possibility that since these arose from millions of years of evolution, they are a feature and not a bug. The important insight here is that the point and function of hominid capacity to reason is not to be as accurate as possible (recall above, that’s impossible for almost everything), it’s to reason in an instrumentally useful way, that is, what information is most salient and relevant and what model and paths of deliberation of heuristics to apply in order to obtain the best possible expected outcome, modulo any risk considerations, in some scenario.

We can see a bridge here to my homeboy Wittgenstein. It is at this stage that we can define what reason truly means, as far as language and discourse in the narrative and argumentation of ideas. Reasoning is proper execution of the rules of specific language games in order to maximize the utility of their corresponding domain.

One thing one can see is that with Rationalist discourse, there is often an implicit assumption: “It is rational to do X” means “It is rational to do X in order to achieve Y, where Y is a near-universally agreed desirable end” (such as health) In this case, there is little to be concerned with.

However, as soon as more abstract matters arise, such as the maximization of “Altruism”, the fundamental indeterminacy of any model, and thus any “Rationalist” line of reasoning, becomes increasingly noisy to the point of becoming nonsense, while also as embodied in intellectual language games, present a myriad of possible crevasses to fall into.

Much of this reasoning does not follow specific scientific processes of inquiry, but a meandering string of ideas and merely plausible sounding assertions. Ultimately they form complex and grandstanding meta-narratives. Nonsense and absurdity can be demonstrated readily with the Semiotic tools of Socratic Questioning and Derrida’s deconstruction. Each new word presents a myriad of interpretations which surreptitiously smuggle their symbolic imagery, leading to statements made in confidence that lie on a seemingly comfortable bed of a web of signifiers, with both instrumentality and prediction error effectively chucked out the window as serious concerns. By merely inquiring what each term precisely refers to, and the source of evidence for each assertion, with militant persistent, pretty much any meta-narrative structure can be broken down, as far as its claim to being “pure reason”.

I do not contend that pure theory is entirely useless. Rather, some literature of pure theory is to be read to consider how a network of theoretical scaffolding “tastes”. Then it can be cataloged, to be used later for some practical purpose in application, when the theory’s foundations are an appropriate model representation for the phenomenon of interest. This is sort of like adjudicating pure math results by their elegance, which facilitates interpretive transparency. Prove a beautiful Theorem and 50 years later it can be used for some new technology. But certainly the quantity of pure theory out there, at least as far as the verbally defined kind, is superfluously excessive. And one must be careful not to take extensive application of such “reasoning” too seriously. In fact we have a name for the most creative honed untethered theory crafters of comprehensive enterprises who then take their complex theories far too seriously - Schizophrenics.

Beyond simply generating potential nonsense, there is another hazardous element – the potential for ready introduction of power within the context of a stream of verbal argument. In Enfield’s book Language vs. Reality: Why Language is Good for Lawyers and Bad for Scientists, drawing from definitive sources from the scientific literature surrounding the evolutionary emergence of language in the hominids and its anthropological properties, it is comprehensively demonstrated that the function is language is more strategic and tactical in the way of convincing individuals of myths and their suggested associated behaviors. Of course, that Foucault is the most cited individual of all time is a testament to how compelling the observation of language’s use as power truly is. And so we can see that Rationalist leaders may be simply better at what the Greek Philosophers called Rhetoric, in contrast to Philosophy. Daniel Kahneman observed that when reasoning is done through verbal debate within a live group context, as opposed to generated privately and then compared, they become more extreme and inaccurate, and in the service of the most vocal or high status individual in the group. So when there is an opportunity for social reasoning to be subject to dynamic real time social pressure, power takes shape and leads human reason astray. Another excellent resource on how endemic such language is, see Harry Frankfurt’s essay On Bullshit.

As an exercise, one can consider a generic phrase of “we/you should do X” or even “we/you should do X for Y” where Y is lifted abstraction with multiplicity of deconstructions (e.g., “harmony” or “justice”) – is likely to be an exercise power. “We/you should do X in order to have more/less of Y” where Y is at least close to a precise objective metric (such as GDP) or a statement of physical sciences (lung health), is, by contrast, more properly in good faith. Otherwise if it is not naturally and easily agreeable, you can be sure that any statement of how one should spend one’s time is either deliberate power or simply NPC memetic talking points of infected power is on display.

Finally, there is a potential risk of self delusion through identification as a Rationalist. It can happen that a “Rationalist” would have an emotional impulse to do something, then think “I am a rational person so this impulse must be correct.” And so, ironically, the Rationalist will act on that emotional impulse without comprehensive reflection on its validity. Rationalism becomes an enabler of narcissistic behavior, that is behavior done through a deluded belief in flawlessness, towards one’s cognition.

My Proposed Alternative

Let me now posit that the appropriate Epistemological Decision Protocol is defined by a triumverate of Meta-Rationality, Evidence, and Experience:

Meta-Rationality – Rationality itself is really a toolkit. The toolkit is not meant to be chosen by impulse, but rather selected instrumentally. One considers a possible outcome, such as improved physical health or lowering a country’s debt to GDP ratio. Then one considers the best tools to use for the occasion, based purely from personal experience, if it exists, and ideally statistical, but a collection of anecdotal suffices, examples of people using these tools to successfully achieve the outcome you are interested in. Observe that I wrote tools here in the plural, any and all forms of technical disciplines should be considered even simultaneously, even if they are contradictory in their cosmology/ontology.

Evidence – I am being Bayesian with “Evidence” here. Recall Bayes Rule, with p(m) your prior on models of the world and p(D|m) is the likelihood giventhe data D. The posterior for a particular model M given the data is:

p(M|D) = p(D|M)p(M) / p(D)

Here p(D) is the Evidence. But how can you know the probability distribution of the data, wouldn’t that being knowing everything about the population distribution mean not needing to do any modeling?

Computationally, samplers don’t require knowing the normalizing constant. But we don’t have Metropolis running in our heads, and really the question is what is p(D) epistemologically?

Well, consider that the set of possible models m is finite, then the evidence is also customarily written as:

p(D) = sum(p(D|m)p(m)) .

and with an integral for set of models parametrized on a measure space. Thus Data as Evidence ONLY makes sense in the context of a set of models. This fits Quine’s web and Lakatos’ hardcore, as defining what the set of models even are (or higher level modeling abstractions that yield these models, i.e., hyperpriors) that you use to understand the phenomenon of interest in the first place. “The Map is Not the Territory”, but you’d be entirely lost without any map at all.

Experience – From the background we can understand that reality always messy. Formally you can say that the theory’s underlying model to its observable map is nonlinear and noisy and often is known to involve components with Knightian uncertainty. However, the human brain does have the capacity to subconsciously learn patterns, as far as useful patterns, and develop intuition in some clinical/artisan practice. And so it is necessary, for good faith reasoning, to engage in consistent and regular reality testing of your model and decision making process. Its application in the real world should be attempted as comprehensively as possible, observing how the guidance given by any rational depiction of optimal decisions actually turns out. There are a number of facts that were long folk knowledge before they were scientifically confirmed, such as the mental health benefits of meditation and yogic practice long before clinical trials’ confirmation of a benefit to mindfulness, and the use of anabolic steroids by early Olympian athletes and professional bodybuilders before evidence as to their effects on strength and muscle hypertrophy were published in the scientific literature.

Of course, as with anything, there are good contributions from Rationalism, as far as the willingness of dropping constraints of politeness as well as habit in performing one’s reasoning. Attempting to drop some of the historical contingencies of language through originality of exploration in reasoning streams can present greater clarity, and more significantly, theory scaffolding innovation, through cleaning signifiers of their thick film of Derrida’s tracers of their historical use. As short bursts to work through an issue with explicit formal crisp constraints and clearly defined objective criteria of what one wishes to achieve, its principles can be of service. When Rationalism becomes a problem is when very long streams of pure Rationalist discourse arguing for a particular decision or course of action, with no interplay across domains, explicit formulation of objective criteria, means of ascertaining and judging success, etc. These easily lead to, in many cases, nonsense, and, unfortunately, can easily turn wayward into dysfunction, as described in the examples, now.

Illustrative Examples

If Eliezer is so Rational then why is he fat?

The originator of the Rationalist movement, Eliezer Yudkowsky, is noticeably overweight. Not morbidly obese, but certainly not a healthy weight. He appears to have been this way throughout his adulthood, to the best of my knowledge.

This is not to just pick on him, although his community already has: beyond this, I have often wondered why PhDs are often not particularly physically fit (obviously with exceptions). I even remember being at a conference for a project studying lifestyle and noncommunicable disease, and in the main hotel for the conference, I was the only one in the hotel gym every day the morning before the conference started.

I have had a visible six pack since my early 20s, and to me maintaining a solid regime of training and nutrition is an obvious thing to do.

First: nutrition and fitness is not THAT noisy biologically. There is a large range of nutrition and exercise protocols that will almost certainly improve their physical condition, it’s really just execution.

Now if someone has problems with execution, then Rationalism is a poor model for that person’s own behavior. If you can’t even exercise your own prefrontal cortex defined processes, then you need to understand how your mind functions, which betrays the real influence of emotionally motivated reasoning, etc., which should show you that the whole enterprise is potentially quite useless.

Second, we can see that the standard implicit Rationalist assumption applies, i.e., that being of a good level of physical fitness is something that helps things almost everyone would want.

It’s clear that being fit, especially relative to being overweight, is beneficial to both longevity and healthspan – there is compendium of evidence to the point of common wisdom that anything related to Syndrome X (obesity, heart disease, diabetes) has severe consequences for morbidity and mortality.

Moreover, most recent evidence for lifespan and healthspan collated by Peter Attia suggests that there is a 4-fold hazard reduction in morbidity and mortality when one has an athletic level versus a median 1) grip strength and 2) VO2 Max (the rate amount of oxygen your lungs can take in). The latter especially is clearly contradictory with lots of excess body fat.

There are also various other benefits to being healthy and with an athletic body, including professional and social benefits from a halo effect (and lower body fat % leans out the face, as well) as well as sex appeal.

Even instrumentally, consider that Eliezer’s priority for his life is dedication towards his version of e/acc AI Alignment, or even being in general an active and effective steward of AI. For that goal, the longer his healthspan/lifespan, the more total time and energy that he can devote to it, and the more effective he would be.

So why Eliezer fat? Untethered theory crafting. Taking “we must consider the most critical risks to mankind!” (even if low probability) through a serpentine prolific canon exploring the farthest contours of his methodological discovery. Rather than truly considering all of the components for a thriving for his own life, as well as fulfilling as far as legacy and contribution to his goals, as would be done under Meta-Rationality, and how they interact, and then proceeding with each one as according to the relevant domain of science or practice.

In these meandering paths of theory crafting (that are impressively intelligently creative, mind you!), when he stumbles upon a fanciful but in some narrative piece by piece structure plausible if tenuous warning about the world being taken over by robots, here emotion rears its ugly head. Neuroscientifically, it is known that even if it’s an imaginary scenario, humans are capable of taking an image purely in their mind as real, and so the various catastrophes that lie at the end of a long voyage of theory crafting feels stark and critical, and takes mental priority. Without demarcations of language games of what one is doing, and the appraisal of streams of thoughts as tools for a specific occasion only, the Rationalist is ultimately still trapped in being Human, All too Human, as in the aptly titled Nietzsche’s work on the challenge and rarity in living to courageously definitive good faith of will.

The Zizians

The Zizians are a Rationalist cult that murdered several people. Yes, the Computer Science students and professionals discussing esoteric science fiction, technology, and developing your LLMs. If you don’t know the story, it is worth reading, they are utterly bizarre. A movement within the Rationalists, in form and style disturbingly not uncommon in the movement in general, developed into a cult performing all sorts of dysfunction and tragedy, and most recently political violence.

How did these “Rationalists” turn out to behave in a manner that is best described as batshit crazy?

In the history of this movement, there are clear indications of the use of Rationalist discourse as power. Rationalist leaders, through broader authority within the movement, controlled a meta-narrative that elevates them into a position of hegemonic control over their specific followers, i.e., behaved as cult leaders. We also see the frequent synergy between motivated reasoning with emotional impulse arising from mental disorder and altered states of mind leading to increasingly unstable and reckless behavior that was done with the full belief that, because it was performed in the milieu and traditions of Rationalism, these are the rational things to do.

Finally, the author of the article above on the Zizians provides a powerful statement towards the end of the article:

By the same token, the ability to dismiss an argument with a “that sounds nuts,” without needing recourse to a point-by-point rebuttal, is anathema to the rationalist project. But it’s a pretty important skill to have if you want to avoid joining cults.

One way to know that some argument is “nuts” is that there being multiple lines of reasoning across disparate domains contradicting its claims. That is, rather than take the argument purely within its own context, trying to pick apart the individual pieces, consider its implications in grounded reality. This can be both the conclusion as well as potential implications of intermediate statements. Do they harmonize or are they in contradiction with other known facts about the phenomenon of interest in that grounded reality?

Conclusion

It is often observed and lamented that there is a sizable portion of those with a high IQ that do not appear to achieve success, or even a pleasant life situation. This is unfortunate as they clearly have the cognitive horsepower to do so. Frustration with the games of human social organization play a significant role, and that these individuals are often socially challenged can make this a minefield to navigate.

And in their comfort among claimed intellectuals, these individuals can become easily exploited. The Dunning-Kruger appears in full force here - if one is understandably confused about many things in the world, and someone less intelligent appears to be confident in their narrative, the lost soul is easily duped through said influence.

In Part 2 I will explore, in more detail, the importance as well as practical utility of being clear and explicit in the language game one is playing, or tool of scientific understanding one is employing. Moreover, I discuss the understanding that just because they are associated with some theoretical cosmology or ontology, does not mean that one cannot conduct inquiry and reasoned decision making applying multiple domains and layers of abstraction simultaneously when their metaphysical narratives contradicting themselves. In obnoxious nerd internet lingo, one can say that an inability to hold in belief superficially contradicting ideas across different layers of abstraction is a good identifying feature of who can be described as a “midwit”.