When faced with a large enough potential payoff, a rational agent can be swayed to take an action even if the probability of success is extremely low. By making the promised outcome large enough, even a tiny probability of success can result in a high expected value, leading to counterintuitive decisions.
Blaise Pascal argued that believing in God is rational, even if one thinks it's unlikely that God exists. The potential infinite reward of eternal happiness in heaven outweighs a whole lifetime of effort or dedication to faith, making belief in God the most pragmatic choice. Nick Bostrom proposed a thought experiment, synthesized below.
In a dark alley, you find yourself approached by a mugger while carrying a wallet containing 10 coins. The unarmed mugger proposes a deal: hand over your wallet, and he'll give you 20 coins tomorrow. You refuse, deeming it highly improbable. The mugger then offers 20,000 coins, but you still decline. When asked about the chances of him being trustworthy, you estimate 1 in a million. The mugger then claims to possess magic powers, offering you an astounding 999 trillion coins and an equal number of maximally happy days in exchange for your wallet. Confused by the staggering offer and the mind-boggling mathematics involved, you hesitantly hand over your wallet, wondering if the mugger's improbable promises could possibly be true.
Pascal mugging happens when a person becomes excessively absorbed in abstract theories without engaging enough with their practical implications, probability of success. In order to counteract that one needs to firmly look for effects, execute steps that bring the theory to execution.
Another approach to mitigating Pascal's Mugging is to set a maximum on the potential positive outcome, preventing improbably high rewards from skewing the decision-making process. Another strategy is to engage with ideas that are above a threshold of probability of idea materializing and succeeding, below which one refuses to consider or engage with potential solutions, no matter how large the hypothetical payoff might be.
I wanted to capture this definition because I notice this tendency in myself and want to be better at spotting and avoiding it.