Overfitting and Heuristics in Philosophy

In Overfitting and Heuristics in Philosophy, Timothy Williamson presents his latest set of tools for constructing intellectually respectable philosophical escape hatches when one’s preferred theory becomes trapped by inconvenient data. The chief innovation, which I will focus on here, is his invocation of two notions whose primary home is the empirical sciences rather than philosophy—as it says on the label, overfitting and heuristics. Briefly: In the empirical modeling literature, the danger of overfitting arises when one selects a model that trades away too much simplicity in order to cleave more closely to the most exacting particulars of one’s observations, thereby tracing more noise than signal; thus, good methodology often prescribes trade-offs in the reverse direction. And when a recalcitrant piece of data can be sourced to the psychological shortcut of a heuristic, then that might, under the right conditions, give us license to disregard that datum. Williamson fiercely advocates for importing these scientific methodological norms over into current philosophical practice.I agree very much with the spirit of what Williamson is offering us here—that much of current analytic methodology is too quick to break a theory on a small number of odd counterexamples, especially when the theory has other compensating virtues; and that we need to attend more closely to the potential limitations and foibles of our cognitive machinery, with meaningful methodological consequences. Nonetheless, I fear that Williamson has not yet reckoned with how radical his proposals truly are, and how much more we will need to draw upon scientific concepts and tools, if we are to achieve his proposed methodological revisions.Let’s start with heuristics. Although Williamson ably discusses such canonical psychologists of heuristics as Daniel Kahneman and Gerd Gigerenzer, it soon becomes clear that he has a much more inclusive notion in mind than they have. Most conspicuously, many processes that Williamson labels as “heuristics” do not rely on only a small number of cues that thereby ignore large bodies of potentially relevant information. In fact, at many points it appears that all it takes for a mental process to count as a heuristic is that it be fallible; and, in general, fallibility is what does the methodological heavy lifting for him. For example: “Normally, a semantic theory …is expected to explain the data by vindicating the assessments, predicting their correctness. But if the assessments are the outputs of imperfectly reliable heuristics, then they may be incorrect, in which case a semantic theory should not predict their correctness” (191). It is the imperfect reliability that is the crucial characteristic.Or here: “In practice, the ‘Why?’ principle is less than perfectly reliable. In recognition of that, we may rename it the ‘Why?’ heuristic” (149). This principle prescribes evaluating “A because B” by evaluating whether B seems a good answer to the question “Why A?” Yet our evaluations of answers to “why?” questions are not generally done by considering only a small number of cues. There are a great many factors that get pulled into our deliberations about whys and wherefores, including broad coherence considerations—poor candidates for counting as heuristics in the typical psychologist’s sense. But surely it is fallible, and thus can count as a Williamsonian heuristic. To be clear, this is not in itself any sort of objection to Williamson: “Heuristics” is a theoretical term, and different theoreticians can deploy it for distinct purposes. My concern, however, is that heuristics understood merely as fallible psychological processes cannot bear the methodological weight Williamson would place upon them.Given Williamson’s well-known epistemicist position on vagueness, it is unsurprising that he offers a heuristic-based move to discount the key premise in a sorites paradox, by introducing a “persistence heuristic,” that “small changes don’t matter” (11). Clearly the problem with such a heuristic isn’t its unreliability, since in a sorites situation it gets things right a vast majority of the time. Rather, he argues, since “our susceptibility to sorites paradoxes simply results from our reliance on the persistence heuristic in epistemically non-ideal conditions, it motivates no revision of classical logic or bivalent semantics” (15). But this threatens to overgeneralize terribly—not so much in terms of an everyday skepticism (though one may well ask when we are ever even in our ordinary lives in epistemically non-nonideal conditions) but in terms of its methodological impact in philosophical inquiry. For pretty much all contexts where we are at the edges of philosophical disputation are epistemically very nonideal. If heuristics sensu Williamson are just any sort of fallible cognition; and almost all our cognition is fallible, including much of what we employ when philosophizing; and almost any meaningful philosophical activity takes place under conditions of epistemic nonideality—then how does any such philosophical activity put rational pressure on anything, anywhere? It would be methodological suicide if philosophers were licensed to pick and choose at will which undesirable premises and arguments to simply dismiss. What was meant to be a targeted escape hatch threatens to scuttle our Neurathian boat altogether.Obviously Williamson would not abide such a state of anything-goesism (nor should any of us). Thus he offers a set of conditions that might block any attempt to use a heuristics critique to save JTB from Gettier counterexamples:(That last sentence strikes me as odd in context, since Williamson is offering the heuristic-based defense as a new move in philosophical methodology. We wouldn’t expect any promising attempts to have already been made yet, since the argumentative strategy was not hitherto available.)These conditions are certainly not trivial; one can easily devise hypothetical applications that would fail any number of them. But I worry that in the context of real philosophical dispute, they will almost always be satisfied. Let me show how easy it is to meet it for Gettier. Consider this heuristic, adapted straight from Alvin Goldman: “If S’s belief that p has not been causally connected in an appropriate way with the fact that p, then S does not have knowledge that p.” Putting aside whatever shortcomings it may ultimately have as part of an analysis of knowledge (and I agree with Williamson that we likely should not be looking for any such analyses 72), it is an excellent hypothesis for a fallible psychological rule we may generally follow in detecting failures of knowledge. The very features of it that made it attractive to Goldman are more than adequate both to provide independent evidence that we use it, and to explain how it gives the verdict that a large class of Gettier cases do not count as knowledge.Williamson’s fourth condition is perhaps somewhat opaque, and taken too literally it would improbably seem to indicate we need to settle the theoretical issues antecedently. But, looking at how he argues elsewhere in the book, it seems that what is needed is some at least prima facie independent motivation for the JTB theory itself, so the heuristics-based critique isn’t merely a case of special pleading against the apparent data. Yet such motivation is not hard to find, pace Williamson’s very on-brand railing against it, such as “JTB is not an elegant analysis; it just provides a list of three poorly related bullet points, of the kind which undergraduates like to write down in their notes” (72). That knowledge might at least crucially involve our beliefs being right, and in some specific right way of being right, has been attractive to many epistemologists; Chisholm, for starters, is no mere undergraduate. And putting aside JTB as a conceptual analysis of knowledge, considered as an empirical model of knowledge it is clearly a legit contender, capturing the vast majority of our attributions and denials of knowledge with just three parameters.I would suggest that similar moves will be available in a great many actual loci of dispute. The rules for heuristics-based critique will have to be made more stringent if they are to be fit for methodological service.There are similar worries for Williamson’s appeal to overfitting: Although there are numerous invocations of the idea of a trade-off between simplicity and capturing the data, we are only ever exhorted to trade the latter in favor of the former, with no concrete guidance as to when the trade should go the other way. In general, Williamson’s textual strategy seems to diagnose the problem with philosophy that it is just unaware that our philosophical data can be mistaken, and that one should frequently look to sacrifice the exact capture of such data in order to retain an elegant theory. He seems to think philosophers have just not considered heuristics as among the possible denizens of the mind, conjecturing “the absence of ‘heuristic’ from the traditional philosopher’s menu of options” (13) for how to understand what might generate intuitive principles. And he hypothesizes that philosophers have suffered from a misguided understanding of philosophical evidence as infallible, thereby operating with a “naïve falsificationist spirit” (55) that allows one apparent counterexample to capsize a theory. While I concur that our methodological norms are often problematic in this way, I am unsure about the diagnosis. Even just considering case verdicts as one among other forms of philosophical evidence, it has long been baked into the literature by many proponents of that form of evidence that these are fallible: It is part and parcel of reflective equilibrium, the motivation for much of Austin’s methodological strictures, and a commitment of Sidgwick’s method, and it is asserted and defended by such metaphilosophically distinct authors as Alvin Goldman and Joel Pust, George Bealer, Laurence BonJour, and Jennifer Nagel. The problem, I fear, isn’t that philosophers haven’t considered whether our data might be fallible. Rather, it’s that we don’t currently have good general tools for determining when to discount them, in a way that can be made to put rational pressure on those who would prefer, to the contrary, that such data be fully counted.There seem to me two clear methodological lessons to draw from the book, even if Williamson has not given us the tools needed to allow us principled and robust ways of cutting philosophical methodology free from too strict a demand to tithenai ta phainomena. First, in the first and explicitly methodological half of the book, he renders vivid just how desirable it would be to have such tools and points in highly plausible directions for their development. As I noted above, my contention is not that the overfitting and heuristic moves cannot be made disciplined but just that Williamson has not done so—at least, not yet.Second, Williamson’s substantive philosophizing in the book demonstrates just what philosophy stands to lose if we insist on a methodology that mandates that theories cleave closely to the phenomena right out of the gate. For, beyond the explicitly methodological substance of the book I have engaged with here, on the whole OHP presents the reader with a wealth of creative and insightful elaborations of what a strictly intensional approach in philosophy can yield, if we simply do not take ourselves to be bound to the odd bits and bobs of philosophical data that standardly are offered to motivate going hyperintensional. If the proof of the methodological pudding is in the philosophical eating, then one takeaway lesson of OHP is: Our methods need to be ones that (with apologies to Gen Z philosophers) let Williamson cook. As someone who was raised on a particular 1990s version of naturalistic philosophy of mind, I found Williamson’s final chapter particularly compelling, for he makes a powerful case for accommodating cognitive significance with explanatory resources outside of semantics, in a way that seems to me to converge intriguingly with views of folks like Fodor and Millikan—but built from utterly different materials. As another example, his parodic riff on “politeness logic” (89) is well worth the price of admission. Our profession will be better off if many of us can deploy our creativity and intellectual firepower more toward the development of theories that similarly color a bit more outside the lines, instead of ever more maddeningly complicated ways of trying to map the exact contours of those lines. OHP’s first-order philosophical content constitutes an excellent reason for us to strive to develop its still-nascent methodological ideas.And, in the end, I suspect we can only pay off the promise of the notions of overfitting and heuristics in our methodology if we appeal much more, and much more directly, to the sciences. For tightening up conditions for heuristics-based critique will likely require, at a minimum, heightening the evidential bar for pinning a heuristic on someone. I suspect we will also need more demanding conditions than mere fallibility, to avoid catastrophic methodological overgeneralization, and that those conditions may need actual empirical work to demonstrate when they do or do not obtain. And the trade-offs that try to steer the boat between both overfitting and underfitting will ultimately require truly quantitative methods in philosophy, enabling the application of the sorts of mathematical tools that scientific modelers use in assessing proposed trade-offs, such as the Bayesian information criterion. The methodological escape hatch that Williamson looks to open may be one that can only safely be deployed with the direct application of scientific tools. A final, and unintended, methodological lesson of Williamson’s OHP thus may well be: Drink deep, or taste not the Baconian spring.